Changes

=INTRODUCTION TO SHELL SCRIPTINGUSING SED & AWK UTILTIES=

:* Explain the purpose of Use the '''she-bang linesed''' command to '''manipulate text''' contained at the top of in a shell scriptfile.

:* List rules for naming a Bash script fileand explain several '''addresses''' and '''instructions''' associated with the '''sed''' command.

:* Explain how to set Use the '''permissionssed''' and how to command as a '''executefilter''' a shell scriptwith Linux pipeline commands.

:* Explain Use the purpose of '''environmentawk''' and command to '''user-definedmanipulate text''' variablescontained in a file.

:* Explain List and explain '''comparison operators''', '''variables''' and '''actions''' associated with the purpose of '''control flow statementsawk'''command.

:* Explain the purpose of the '''$?''' exit status and Use the '''testawk''' command. :* Explain the purpose and usage of the '''if''' and '''if-else''' logic statements. :* Explain the purpose and usage of the as a '''forfilter''' loop statementwith Linux pipeline commands.

|colspan="2" |Course Notes'''Slides''':<ul><li>Week 11 Lecture 1 Notes:<br> [[Media:ULI101-Week11.1.pdf | PDF]] | [https://ictmatrix.senecacollege.ca/~murraychris.sauljohnson/uli101ULI101/ULI101-Week11.1.pptx PPTX]</li><li>Week 11 Lecture 2 Notes:<br> [[Media:ULI101-Week10Week11.2.pdf | PDF]] | [https://ictmatrix.senecacollege.ca/~murrayjason.saulcarman/uli101slides/ULI101-Week10Week11.2.pptx PPTX]<br></li></ul>

| style="padding-left:15px;" |Shell Scripting'''Text Manipulation:'''* [https://searchdatacenterwww.techtargetdigitalocean.com/definitioncommunity/shelltutorials/the-basics-of-using-the-sed-stream-editor-to-manipulate-text-in-linux Purpose of using the sed utility]* [https://www.digitalocean.com/community/tutorials/how-to-use-the-awk-language-to-manipulate-text-in-script linux Purposeof using the awk utility]

Variables* [https://opensource.com/article/19/8/what| style="padding-are-environment-variables Environment]* [httpsleft://www.linuxtechi.com/variables-in-shell-scripting/#:~:text=User%20Defined%20Variables%3A,like%20a%20real%20computer%20program. User Defined]* [http://osr600doc.xinuos.com/en/SDK_tools/_Positional_Parameters.html#:~:text=A%20positional%20parameter%20is%20a,up%20to%20nine%20positional%20parameters. Positional Parameters]15px;" |'''Commands / Techniques* [http://linuxcommand.org/lc3_man_pages/readh.html read]'''* [https://man7.org/linux/man-pages/man1/readonlysed.1p.html readonlysed]* [https://www.gnuman7.org/softwarelinux/bash/manual/html_node/Commandman-Substitution.html Command Substitution]| style="padding-left:15px;"|Control Flow Statements* [https:pages/man1/enawk.wikipedia1p.org/wiki/Control_flow Purpose]* [https://www.computerhope.com/unix/test.htm test command]* [https://ryanstutorials.net/bash-scripting-tutorial/bash-if-statements.php#:~:text=If%20statements%20(and%2C%20closely%20related,conditions%20that%20we%20may%20set. if statement]* [https://www.tutorialspoint.com/unix/if-else-statement.htm if-else statement]* [https://www.cyberciti.biz/faq/bash-for-loop/#:~:text=A%20'for%20loop'%20is%20a,files%20using%20a%20for%20loop. for loophtml awk]

 |colspan="1" style="padding-left:15px;" width="30%"|'''Brauer Instructional Videos:'''<ul><li>[https://www.youtube.com/watch?v=kxEP-KUhOSgnpU6S61AIko&list=PLU1b1f-2Oe90TuYfifnWulINjMv_Wr16N&index=5 Bash Shell Scripting - Part 114 Using the sed Utility]</li><li>[https://www.youtube.com/watch?v=cQepf9fY6cE Creating and Running a Shell ScriptOV3XzjDYgJo&list=PLU1b1f-2Oe90TuYfifnWulINjMv_Wr16N&index=13 Using the awk Utility]</li></ul>

[[Image:ipso.png|thumb|right|500px|An '''IPSO Diagram''' (INPUT, PROCESSING, STORAGE, OUTPUT) can be used to map-out and then list the sequence of steps to assist when coding your shell script.]]It is recommended to '''plan''' out on a piece of paper the purpose of the shell script.<br>You can do this by creating a simple '''IPSO''' diagram (stands for '''INPUT''', '''PROCESSING''', '''STORAGE''', '''OUTPUTUsage:''').

'''Using a Shebang LineHow it Works:'''

[[Image:shebang.png|thumb|right|200px|* The '''shebang line''' <u>must</u> appear on sed command reads all lines in the '''first line''' and at the '''beginning''' of input file and will be exposed to the shell script.]]If you are learning Bash scripting by reading other people’s code you might have noticedexpression<br>that the first (i.e. area contained within quotes) one line in the scripts starts with the <span style="font-family:courier;font-weight:bold">#!</span> characters and the path to the Bash interpreterat a time.* The expression can be within single quotes or double quotes.<i>This sequence of characters * The expression contains an address (match condition) and an instruction (#!operation) is called .* If the line matches the address, then it will perform the instruction.* Lines will display be default unless the '''shebang–n''' and option is used to tell the operating systemsuppress default display<br>which interpreter to use to parse the rest of the file. </i>Reference: https://linuxize.com/post/bash-shebang/ The '''shebang line''' <u>must</u> appear on the '''first line''' and at the '''beginningAddress:''' of the shell script,<br>otherwise, it will be treated as a regular comment and ignored.

* Can use a line number, to select a specific line (for example: '''5''')* Can specify a range of line numbers (for example: '''5,7'''Setting Permissions &amp; Running )* Regular expressions are contained within forward slashes (e.g. /regular-expression/)* Can specify a regular expression to select all lines that match a Shell Scriptpattern (e.g '''/^[0-9].*[0-9]$/''') * If NO address is present, the instruction will apply to ALL lines

'''Example:<br><br><span style="font-family:courier;">chmod u+x myscript.bash<br>./myscript.bash<br>/home/username/myscript.bash<br>~/myscript.bash</span>''' ===Using Variables in Shell Scripts=== '''Definition''' ''<b>Variables</b> are used to '''store information''' to be referenced and manipulated in a computer program.<br>They also provide a way of labeling data with a descriptive name, so our programs can be understood<br>more clearly by the reader and ourselves.''<br>Reference: https://launchschool.com/books/ruby/read/variables  '''Environment Variables''' [[Image:environmentsed.png|thumb|right|500px|Examples of using '''Environment''' and '''User Defined''' variables.]]Shell '''environment variablesInstruction:''' shape the working environment whenever you are logged in Common shell. Some of these variables are displayed via Linux commands in the diagram displayed on the right-side.<br>(you can issue the pipeline command *'''set | moreAction''' to view all variablestake for matched line(s) Placing a dollar sign ('''$''') prior *Refer to the variable name will cause the variable to expand to the value contained in the variable.  '''User Defined Variables''' ''<b>Usertable on right-defined variablesside for list of some</bbr> are variables which can be '''created by the usercommon instructions''' and exist in the session. This means that no one can access user-defined variables that have been set by another user,<br>and when the session is closed these variables expire.''<br>Reference: https://mariadb.com/kb/en/user-defined-variables/<br><br>Data can be stored and removed within a variable using an '''equal sign'''.<br><br>The '''read''' command can be used to prompt the user to enter data into a variable.<br>Refer to the diagram on the right-side to see how user-defined variables are assigned data.their purpose

[[Image:for-command-substitution.png|thumb|right|300px|Example of how a '''for loop with command substitution''' works.]]<i>'''Command substitution''' is a facility that allows a command<br>to be run and its output to be pasted back on the command line as arguments to another command.</i><br>Reference: https://en.wikipedia.org/wiki/Command_substitution<br><br> ''Usage:'' <span style="font-family:courier"><b>command1 $(command2)</b><br>or<br><b>command1 `command2`</b></span><br><br> ''Examples:'' <span style="font-family:courier;font-weight:bold">file $(ls)<br>mail -s "message" $(cat email-list.txt) < message.txt<br>echo "The current directory is $(pwd)"<br>echo "The current hostname is $(hostname)"<br>echo "The date is: $(date +'%A %B %d, %Y')"<br> ===Using Control Flow Statements in Shell Scriptsthe awk Utility===

The special parameter '''$?Usage:''' Is used to determine the '''exit status''' of the previously issued Linux command.The exit status will either display a '''zero''' (representing TRUE) or a '''non-zero''' number (representing FALSE). The test command can be used with control flow statements to control the sequence of a shell script.

You CANNOT use the '''<span style="color:blue;font-weight:bold;font-family:courier;">awk [-F] ''' or '''selection-criteria {action}’ file-name<''' symbols when using the test command since these are redirection symbols. Instead, you need to use options when performing numerical comparisons.Refer to the table below for test options and their purposes./span>

Refer to diagrams to the right involving some of the options used with the '''testHow It Works:''' command. Refer to the test man pages for a full list of options for the test command.

* The '''Logic Statementsawk''' command reads all lines in the input file and will be exposed to the expression (contained within quotes) for processing.*The '''expression''' (contained in quotes) represents '''selection criteria''', and '''action''' to execute contained within braces '''{}'''* if selection criteria is matched, then action (between braces) is executed.* The '''–F''' option can be used to specify the default '''field delimiter''' (separator) character<br>eg. '''awk –F”;”''' (would indicate a semi-colon delimited input file).<br>'''Selection Criteria'''

A * You can use a regular expression, enclosed within slashes, as a pattern. For example: '''logic statement/pattern/''' is used to determine which Linux commands to be executed based<br>on the result of * The ~ operator tests whether a condition (i.e. TRUE (zero value) field or FALSE (non-zero value))variable matches a regular expression. <table align="right"><tr valign="top"><td>[For example: '''$1 ~ /^[Image:logic0-1.png|thumb|right|250px|Example of using the 9]/'''* The '''if!~''' logic control-flow statementoperator tests for no match.]]<For example: '''$2 !~ /line/td'''* You can perform both numeric and string comparisons using relational operators ( '''><td''' , '''>[[Image:loop-1.png|thumb|right|250px|Example of using the ='''for, ''' looping control-flow statement.]]</td>''' , '''</table> There are several logic statements=''' , '''==''' , but we will just concentrate on the if statement.<pre style'''!="width:20%">if test condition then command(s''' ) .fi</pre> Refer to * You can combine any of the diagram immediately to the right for patterns using the Boolean operators '''if logic statement||''' with the (OR) and '''test&&''' command(AND). * You can use built-in variables (like NR or "record number" representing line number) with comparison operators.<br><br><br><br><br>For example: '''if-else statement:NR >=1 && NR <= 5'''

[[Image:if-else.png|thumb|right|300px|Example of how an '''if-else''' statement works.<br>Action (Image licensed under [https://creativecommons.org/licenses/by-sa/3.0/ cc]execution)]] Unlike using only an ''if'' statement, an '''if-else''' statement take '''two different sets of actions'''<br>based on the results of the test condition.<br><br>''How it Works:''<br>When the test condition returns a '''TRUE''' value, then the Linux Commands between<br>'''then''' and '''else''' statements are executed.<br>If the test returns a '''FALSE''' value, then the the Linux Commands between<br>the '''else''' and '''fi''' statements are executed.<br><br> ''Example:'' <span style="font-family:courier;font-weight:bold;">num1=5<br>num2=10<br>if test $num1 –lt $num2<br>then<br> &nbsp;&nbsp;&nbsp;echo “Less Than”<br>else<br>echo &nbsp;&nbsp;&nbsp;“Greater Than or Equal to”<br>fi</span><br><br>  '''Loop Statements''' ''A <b>loop statement</b> is a series of steps or sequence of statements executed repeatedly zero or more times satisfying the given condition is satisfied.''<br>Reference: https://www.chegg.com/homework-help/definitions/loop-statement-3 There are several loops, but we will look at the '''for loop''' using a '''list'''.

<pre style="width:20%">* Action to be executed is contained within braces '''{}'''* The '''print''' command can be used to display text (fields).* You can use parameters which represent fields within records (lines) within the expression of the awk utility.for item * The parameter '''$0''' represents all of the fields contained in list the record (line).do* The parameters '''$1''', '''$2''', '''$3''' … '''$9''' represent the first, second and third to the 9th fields contained within the record. command* Parameters greater than nine requires the value of the parameter to be placed within braces (sfor example: '''${10}''','''${11}''','''${12}''', etc.) done* You can use built-in '''variables''' (such as '''NR''' or "record number" representing line number)</prebr>eg. '''{print NR,$0}''' (will print record number, then entire record).

<span style=INVESTIGATION 1"color: CREATING A SHELL SCRIPT=red;">'''ATTENTION''': Effective '''May 9, 2022''' - this online tutorial will be required to be completed by '''Friday in week 11 by midnight'''<br>to obtain a grade of '''2%''' towards this course</span><br><br>

<br>In this sectioninvestigation, you will learn how to create and run a manipulate text using the '''simple Bash Shell scriptsed'''utility.

# '''Login''' to your matrix accountand confirm you are located in your '''home''' directory.<br><br># Issue a Linux command to create a directory called '''sed'confirm''<br><br># Issue a Linux command to <u>change</u> to the '''sed''' directory and confirm that you are located in your the '''homesed''' directory.<br><br>We want # Issue the following Linux command to create a Bash Shell script to welcome download the user by their usernamedata.txt file<br>Let('s first provide some useful tips in terms of selecting an appropriate name for the shell script''copy and paste''' to save time):<br><span style="color:blue;font-weight:bold;font-family:courier;">wget <nowiki>https://ict.senecacollege.ca/~murray.saul/uli101/data.txt</nowiki></span><br><br># Issue the following linux '''more''' command to quickly view the contents of the '''data.txt''' file.<br>When finished, exit the more command by pressing the letter <span style="color:blue;font-weight:bold;font-family:courier;">q</span>[[Image:sed-1.png|thumb|right|300px|Issuing the '''p''' instruction without using the '''-n''' option (to check if your intended shell script namesuppress original output) will display lines twice.]]<br><br>The '''p''' instruction with the '''sed''' command is already exists used to be run automatically from <br>'''print''' (i.e. ''display'') the contents of a text file.<br><br># Issue the Bash shellfollowing Linux command:<br><span style="color:blue;font-weight:bold;font-family:courier;">which hellosed 'p' data.txt</span><br><br>'''NOTE: You should notice that there is no output and therefore, this shell script name can be usedeach line appears twice'''.<br>On <br>The reason why standard output appears twice is that the other hand, if you wanted to create a file called sed, then command<br>(without the '''which sed-n option''' command) displays all lines regardless of an address used.<br><br>would indicate it is already being used by We will use '''pipeline commands''' to both display stdout to the shell screen and that save to files<br>for <u>confirmation</u> of running these pipeline commands when run a '''sedchecking-script''' wouldn't be an appropriate shell script name to uselater in this investigation.<br><br># Use a Issue the following Linux pipeline command:<br><span style="color:blue;font-weight:bold;font-family:courier;">sed -n 'p' data.txt | tee sed-1.txt</span><br><br>What do you notice? You should see only one line.<br><br>You can specify an '''address'''text editorto display lines using the sed utility<br>(eg. ''line #'' like , '''viline #s''' or range of '''line #s''nano').<br><br># Issue the following Linux pipeline command:<br><span style="color:blue;font-weight:bold;font-family:courier;">sed -n '1 p' data.txt | tee sed-2.txt</span><br><br>You should see the first line of the text file displayed.<br>What other command is used to create only display the text first line in a file called ?<br><br>[[Image:sed-2.png|thumb|right|500px|Using the sed command to display a '''range''hello'of lines.]]# Issue the following Linux pipeline command:<br><span style="color:blue;font-weight:bold;font-family:courier;">sed -n '2,5 p'data.txt | tee sed-3.txt</span><br><br>If What is displayed? How would you are using modify the sed command to display the nano text editor, refer line range 10 to 50?<br><br>The '''s''' instruction is used to notes on '''substitute''' text editing in <br>(a previous week similar to method was demonstrated in the course schedulevi editor in tutorial 9).<br><br># Enter Issue the following two lines in your shell script, replacing "your-username" with your actual nameLinux pipeline command:<br><span style="color:blue;font-weight:bold;font-family:courier;">echosed '2,5 s/TUTORIAL/LESSON/g' data.txt | tee sed-4.txt | more</span><br>echo "Hello $USER"<br>echoWhat do you notice? View the original contents of lines 2 to 5 in the '''data.txt''' file</spanbr>in another shell to confirm that the substitution occurred.<br><br>[[Image:sed-3.png|thumb|right|500px|Using the sed command with the '''NOTE:-q''' option to display up to a line number, then quit.]]The variable called '''USERq''' is an instruction terminates or '''environmentquits''' variable that contains the current userexecution of the sed utility as soon as it is read in a particular line or matching pattern.<br><br># Issue the following Linux pipeline command:<br><span style="color:blue;font-weight:bold;font-family:courier;">sed 's login name11 q' data.txt | tee sed-5.txt</span><br><Brbr>In this way if What did you wanted notice? How many lines were displayed<br>before the sed command exited?<br><br>You can use '''regular expressions''' to share your shell script with other usersselect lines that match a pattern. In fact, when they run the shell script<br>it will greet by the sed command was one of the <u>theirfirst</u> usernameLinux commands that used regular expression. Using <br><br>The rules remain the same for using regular expressions as demonstrated in '''tutorial 9'environment variables'' makes your shell script to <br>except the regular expression must be more contained within '''forward slashes'''<br>(eg. <span style="usablefont-family:courier;font-weight:bold;" by others.<br>/regexp/<br/span># Save your editing session and exit the text editor).<br><br>Instead of issuing [[Image:sed-4.png|thumb|right|400px|Using the sed command using regular expressions with '''bashanchors''' .]]# Issue the following Linux pipeline command followed by your shell script pathname as an ''argument'',:<br>let<span style="color:blue;font-weight:bold;font-family:courier;">sed -n 's simply run it by its filename/^The/ p' data. This is the most common method of running shell scriptstxt | tee sed-6.txt</span><br><br>What do you notice?<br><br># Issue the following linux Linux pipeline command to run your shell script in your current directory:<br><span style="color:blue;font-weight:bold;font-family:courier;">sed -n '/d$/ p' data./hellotxt | tee sed-7.txt</span><br><br>You should What do you notice an error indicating you don?<br><br>The '''sed''' utility can also be used as a '''filter'''t have permissions to run the file.manipulate text that<br>You need to first add execute permissions prior to running the shell scriptwas generated from Linux commands.<br><br>[[Image:sed-5.png|thumb|right|400px|Using the sed command with '''pipeline''' commands.]]# Issue the following linux Linux pipeline command to add execute permissions for your shell script:<br><span style="color:blue;font-weight:bold;font-family:courier;">chmod u+x hellowho | sed -n '/^[a-m]/ p' | tee sed-8.txt | more</span><br><br>What did you notice?<br><br># Re-run your shell scriptIssue the following Linux pipeline command: <br><span style="color:blue;font-weight:bold;font-family:courier;">ls | sed -n '/txt$/ p' | tee sed-9./hellotxt</span><br><br>Did What did you shell script runnotice?<br><br># Issue the following Linux command to run a checking script:<br><span style="color:blue;font-weight:bold;font-family:courier;">bash /home/murray.saul/scripts~uli101/week10week11-check-1</span><br><br># If you encounter errors, make corrections and '''re-run''' the checking script until you<br>until you receive a congratulations message, then you can proceed.<br><br>::In the next investigation, you will learn to create and run shell scripts that<br >use variables, positional and special parameters. You will also learn<br>how to add a '''she-bang line''' at the top of a shell script to force it to run in a specified shell.<br><br>

=INVESTIGATION 2: USING VARIABLES IN SHELL SCRIPTS = In this sectionthe next investigation, you will learn how to use variables, positional and special parameters to assist you in creating adaptable shell scriptsmanipulate text using the '''awk''' utility.<br><br>

# Confirm that Change to your '''home''' directory and issue a command to '''confirm'''<br>you are located in your '''home''' directory in your Matrix account.<br><br># Use Issue a text editor Linux command to edit the shell script create a directory called '''helloawk'''<br><br># Add the following line Issue a Linux command to the bottom of the file:<bru>change<span style="font-family:courier;">echo "The current shell /u> to the '''awk''' directory and confirm you are using is: $(ps -o cmd= -p $$|cut -dlocated in the '''awk'' ' -f1)"directory.<br></spanbr>Let's download a database file that contains information regarding classic cars.<br><br>'''NOTE:''' This # Issue the following linux command displays the ('''namecopy and paste''' of the ''shell'' that the shell scriptto save time):<br>is running within. The command within '''$( )''' uses a technique known as <span style="''command substitution''color:blue;font-weight:bold;font-family:courier;">wget <brnowiki>which you will learn about in '''week 12'''https://ict.senecacollege.ca/~murray.saul/uli101/cars.txt<br/nowiki><br/span><br><br># Issue the '''Savecat''' your editing changes and command to quickly view the contents of the '''exitcars.txt''' your text editorfile.<br><br># Issue the following linux The "'''print'''" action (command ) is the <u>default</u> action of awk to change to the print<br>all selected lines that match a '''Bourne Shellpattern''':.<br><span style="color:blue;font-weight:bold;font-family:courier;">sh</span>br>This '''action''' (contained in braces) can provide more options<br>such as printing '''specific fields''' of selected lines (or records) from a database.<br><br>You should notice your shell prompt change indicating you are in a different shell[[Image:awk-1.png|thumb|right|400px|Using the awk command to display matches of the pattern '''ford'''.<br><br>]]# Issue the following linux command all to run your shell script display all lines (i.e. records) in the ''Bourne Shell'cars.txt''' database that matches the pattern (or "make") called '''ford''':<br><span style="color:blue;font-weight:bold;font-family:courier;">awk '/ford/ {print}' cars./hellotxt</span><br><br>You should see that you are currently running the shell script in the We will use '''pipeline commands'Bourne'' shell.to both display stdout to the screen and save to files for <bru>confirmation<br/u>of running these pipeline commands when run a '''NOTE:checking-script''' Due later in this investigation.<br><br># Issue the following linux pipeline command all to display records<br>in the fact that shells (and their features) have '''evolvedcars.txt''' over a period of time,<br>an error may occur if you include a ''NEWER'' shell feature database that contain the pattern (i.e.g. make) '''ford'Bash Shell'') in your shell script,:<br>but run it in an <span style="color:blue;font-weight:bold;font-family:courier;">awk '/ford/'OLDER'' shell (ecars.g. ''Bourne Shell'')txt | tee awk-1.txt</span><br><br>What do you notice? You can add a should notice ALL lines displayed <u>without</u> using '''special commentsearch criteria''' to the BEGINNING of the FIRST line of your shell script to.<br><br>You can use ''builtin''force''' it to run in variables''' with the shell you want (for example: the Bash shell)'''print''' command for further processing.<br>We will discuss the following variables in this tutorial:<br> # Edit your '''hello''' shell script using a text editor.<br><br># Insert the following line at [[Image:awk-2.png|thumb|right|400px|Using the awk command to print search results by '''beginningfield number''' of the .]]'''first$0''' - Current record (entire line of your hello file:)<br><span style="font'''$1''' -family:courier;">#!/bin/bash</span>First field in record<br><br>This is referred to as a '''she$n''' -bang line'''. It forces this script to be run nth field in the record<br>'''Bash ShellNR'''.- Record Number (order in database)<br>When your Bash Shell script finishes execution, you are returned to your '''NF''' - Number of fields in current shell that you are usingrecord<br><br>(which in our case in MatrixFor a listing of more variables, is still the Bash shell)please consult your course notes.<br><br># Issue the following linux pipeline command to display the '''model''', '''Saveyear''' your editing changes and , '''exitquantity''' your text editor.<br>and price<br># While in the Bourne shell, issue the following linux command'''cars.txt''' database for makes of '''chevy''':<br><span style="color:blue;font-weight:bold;font-family:courier;">.awk '/hello<chevy/span><br><{print $2,$3,$4,$5}' cars.txt | tee awk-2.txt</span><br>You should notice <br>Notice that the shell name is running in the a '''bashspace''' shell.is the delimiter for the fields that appear as standard output.<br><br> # It is a good idea to rename your shell script to include an The '''extensiontilde character''' to<br>explain that this is '''Bash Shell script~''' file (referred is used to as search for a "''portable Bash shell script''")pattern or display standard output for a particular field. <br><br># Issue the following linux pipeline command to rename your shell script file:<br>display all '''plymouths''' ('''plym''')<span style="color:blue;font-weight:bold;font-family:courier;">mv hello hello.bash</spanbr>by '''model name''', '''price''' and '''quantity''':<br><br># Run your renamed shell script for confirmation by issuing:<br><span stylespan style="color:blue;font-weight:bold;font-family:courier;">.awk '$1 ~ /hello.bash<plym/span><br>{print $2,$3,$4,$5}' cars.txt | tee awk-3.txt</span><br># Enter the following linux command to return to your Bash shell: <span style="color:blue;font-weight:bold;font-family:courier;">exitbr>You can also use '''comparison operators''' to specify conditions for processing with matched patterns</spanbr>when using the awk command. Since they are used WITHIN the awk expression,<br>they are not confused with redirection symbols<br>Let's use some '''ENVIRONMENT variables''' in our Bash Shell script.<br><br># Use a text editor [[Image:awk-3.png|thumb|right|400px|Using the awk command to edit the shell script called display results based on '''comparison operators'''hello.bash]]'''<br><br># Add the following lines to the bottom of the file:''' &nbsp;&nbsp;&nbsp;&nbsp;Less than<br>'''<span style="font-family:courier''' &nbsp;&nbsp;">echoLess than or equal<br>echo "The current directory location is: $PWD"<br'''>echo "The current user home directory is: $HOME''' &nbsp;&nbsp;&nbsp;&nbsp;Greater than<br>echo</span'''>=''' &nbsp;&nbsp;Greater than or equal<br>'''==''' &nbsp;&nbsp;Equal<br># Save your editing changes and exit your text editor.'''!=''' &nbsp;&nbsp;&nbsp;Not equal<br><br># Run your renamed shell script for confirmation by issuing:<br><span style="color:blue;font-weight:bold;font-family:courier;">./hello.bash</span>Issue the following linux pipeline command to display display the '''car make''', '''model''', '''quantity''' and '''price''' of all vehicles whose '''prices are less than $5,000''':<br><brspan style="color:blue;font-weight:bold;font-family:courier;">Take time to view the output and the values of the environment variablesawk '$5 < 5000 {print $1,$2,$4,$5}' cars.txt | tee awk-4.txt</span><br><br># Issue What do you notice?<br><br># Issue the following linux pipeline command to add your current directory to the display display '''PATHprice''' environment variable:,<br><span style="color'''quantity''', '''model''' and '''car make''' of vehicles whose '''prices are less than $5,000''':blue;<br><span style="color:blue;font-weight:bold;font-family:courier;">PATH=awk '$5 < 5000 {print $5,$4,$2,$PATH:1}' cars.txt | tee awk-5.txt</span><br><br># Issue the following linux pipeline command to confirm that display the current directory "." has been added to the PATH environment variable:'''car make''',<br><span style="color'''year''' and '''quantity''' of cars that '''begin''' with the '''letter 'f'''':blue;font<br><span style="color:blue;font-weight:bold;font-family:courier;">echo awk '$1 ~ /^f/ {print $PATH1,$2,$4}' cars.txt | tee awk-6.txt</span><br><br># Issue [[Image:awk-4.png|thumb|right|400px|Using the following awk command to run your Bash shell script just by name:display combined search results based on '''compound operators'''.]]Combined pattern searches can be made<br><span style="color:blue;font-weightby using '''compound operator''' symbols:bold;font-family:courier;">hello.bash</span><br><br>Did your Bash shell script run?<br><br># Exit your Matrix session, and log back into your Matrix session.<br><br># Re-run the '''hello.bash&&'''' shell script by just using the name.<br>&nbsp;&nbsp;&nbsp;&nbsp;(and)<br>What did you notice?<br><br>The setting of the '''PATH||''' environment variable only worked in the current session only.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;(or)<br>If you exit the current Matrix session, then the recently changed settings for environment variables will be lost.<br>You will in a <u>future</u> tutorial how # Issue the following linux pipeline command to set environment variables in list all '''start-upfords''' files.<br>whose '''price is greater than $10,000''':<br><span style="color:redblue;"font-weight:bold;font-family:courier;">awk '$1 ~ /ford/ && $5 > 10000 {print $0}''ATTENTION:''' Students will get FRUSTRATED when performing their cars.txt | tee awk-7.txt</span><br><br># Issue the following linux command ('''assignment 3copy and paste''' when their Bash shell scripts have errors. One major cause is the the OUTPUT of their Bash shell script when run does not '''EXACTLY match''' the required output for the correct Bash shell scriptto save time):<br><span style="color:blue;font-weight:bold;font-family:courier;">wget <nowiki>https://ict.senecacollege.ca/~murray. This requires that you CAREFULLY '''read''' the requirements of your Bash shell script and create it to the EXACT specificationssaul/uli101/cars2.txt</nowiki></span><br><br># Issue the following Linux '''cat''' command to run a checking script:quickly view the contents of the '''cars2.txt''' file.<br><span style="color:blue;font-weight:bold;font-family:courier;"br># Issue the following linux pipeline command to display the '''year'''<br>bash /home/murray.saul/scripts/week10-check-2 | more</span><br><br># If you encounter errors, make corrections and and '''quantity''re-run' of cars that '''begin''' with the '''letter 'f'''' for the checking script until you<br>receive a congratulations message, then you can proceed.<br><br>Let's create a Bash shell script that contain '''user-created variablescars2.txt'''.database:<br><br># Use a text editor to create a Bash shell script called '''userspan style="color:blue;font-weight:bold;font-variables.bash'family:courier;">awk '$1 ~ /^f/ {print $2,$4}'cars2.txt | tee awk-8.txt<br/span><br># Add the following lines to the beginning of this file:<br><span style="font-family:courier;">#!/bin/bashWhat did you notice?<br>age=25<br>readonly age<br>read -p "Enter your Full Name" name<br>read The problem is that the '''cars2.txt''' database separates each field by a semi-p "Enter your age colon (in years''';'''): " age<bru>echo "Hello $name - You are $age years old"</spaninstead</u>of '''TAB'''.<br><br># Save your editing changes Therefore, it does not recognize the second and exit your text editorfourth fields.<br><br>You need to issue awk with the -F option to indicate that this file's fields are separated (delimited) by a semi-colorn.<br><br># Issue the following linux pipeline command to display the '''chmodyear''' command to add execute permissions for the user for <br>and '''quantity''' of cars that '''begin''' with the '''user-variables.bashletter 'f'''' file.<br><br># Issue for the following to run the user-variables'''cars2.bash Bash shell scripttxt''' database:<br><span style="color:blue;font-weight:bold;font-family:courier;">./userawk -variables.bashF";" '$1 ~ /^f/ {print $2,$4}' cars2.txt | tee awk-9.txt</span><br><br>What do did you notice when you try to change the age variable? Whythis time?<br><br># Use a text editor Issue the following to create run a file called '''parameters.bash'''<br><br># Add the following lines to the beginning of this filechecking script:<br><span style="color:blue;font-weight:bold;font-family:courier;">#!~uli101/binweek11-check-2</bash<brspan>echo \$0: $0<br>echo \$2: $2<br>echo \$3: $3If you encounter errors, make corrections and '''re-run''' the checking script until you<br><br>echo \$#: $#<br>echo \$*: $*<br><br>shift 2<br>echo \$#: $#<br>echo \$*: $*</span><br><br># Save your editing changes and exit your text editor.<br><br>Notice how the quoting character "'''\'''" is used to display positional parameters like "'''$2'''"<br>as opposed to the value stored in the <u>second</u> positional parameter.<br><br># Issue the '''chmod''' command to add execute permissions for the user for the '''parameters.bash''' file.<br><br># Issue the following to run the '''user-variables.bash''' Bash shell script:<br><span style="color:blue;font-weight:bold;font-family:courier;">./parameters.bash</span><br><br>What happened?<br><br>The values for the parameters may not be displayed properly since receive a congratulations message, then you did NOT provide any arguments when running the shell script.<br><br># Issue the following to run the user-variables.bash Bash shell script with arguments:<br><span style="color:blue;font-weight:bold;font-family:courier;">can proceed./parameters.bash 1 2 3 4 5 6 7 8</span><br><br>Take some time to view the results and how the parameters are changed when using the shift command. What do you notice?<br>

In the next investigationThe purpose of this section is to obtain '''extra practice''' to help with '''quizzes''', your '''midterm''', you will use control-flow statements to allow and your shell scripts to perform differently under different situations'''final exam'''.

'''Command SubstitutionPart A: Display Results from Using the sed Utility'''

Command Substitution is a method of running a Linux command that provides Note the contents from the following tab-delimited file called '''stdout'''<br>that is used as ~murray.saul/uli101/stuff.txt'''argument:(sthis file pathname exists for checking your work)''' for <u>another</u> Linux command.

In this section# <span style="font-family:courier;font-weight:bold">sed -n '3, you will learn how to use control6 p' ~murray.saul/uli101/stuff.txt</span><br><br># <span style="font-family:courier;font-flow statements to make your shell script behave differently under different situationweight:bold">sed '4 q' ~murray.saul/uli101/stuff.txt</span><br><br># <span style="font-family:courier;font-weight:bold">sed '/the/ d' ~murray.saul/uli101/stuff.txt</span><br><br># <span style="font-family:courier;font-weight:bold">sed 's/line/NUMBER/g' ~murray.saul/uli101/stuff.txt</span>

'''Perform Write a single Linux command to perform the Following Steps:'''specified tasks for each of the following questions.

Here is a link to the MS Word Document of ALL of the questions displayed below but with extra room to answer on '''Part C: Writing Linux Commands Using the document tosimulate a quiz:awk Utility'''

https://ict.senecacollege.ca/Note the contents from the following tab-delimited file called '''~murray.saul/uli101/uli101_week10_practicestuff.docxtxt''':(this file pathname exists for checking your work)

Your instructor may take-up these questions during class<pre>Line one. It This is up to the student to attend classes in order to obtain second line.This is the answers to the following questionsthird.This is line four. Your instructor will NOT provide these answers in any other form (egFive. e-mailLine six followsFollowed by 7Now line 8and line nineFinally, etc).line 10</pre>

'''Review QuestionsWrite the results of each of the following Linux commands for the above-mentioned file:'''

'''PART A# <span style="font-family: WRITE BASH SHELL SCRIPT CODE'''courier;font-weight:bold">awk ‘NR == 3 {print}’ ~murray.saul/uli101/stuff.txt</span><br><br># <span style="font-family:courier;font-weight:bold">awk ‘NR >= 2 && NR <= 5 {print}’ ~murray.saul/uli101/stuff.txt</span><br><br># <span style="font-family:courier;font-weight:bold">awk ‘$1 ~ /This/ {print $2}’ ~murray.saul/uli101/stuff.txt</span><br><br># <span style="font-family:courier;font-weight:bold">awk ‘$1 ~ /This/ {print $3,$2}’ ~murray.saul/uli101/stuff.txt</span><br><br>

# Write a Linux awk command to display all records for the file: '''~/cars''' whose fifth field is greater than 10000.<br><br># Write a Linux awk command to display the first and fourth fields for the file:'''Walkthru ~/cars''' whose fifth field begins with a number.<br><br>#1Write a Linux awk command to display the second and third fields for the file:'''~/cars''' for records that match the pattern “chevy”.<br><br># Write a Linux awk command to display the first and second fields for all the records contained in the file: '''~/cars'''<br><br>