1,234
edits
Changes
no edit summary
{{Admon/caution|DO NOT USE THIS VERSION OF THE LAB. This page will no longer be updated.|'''New version here:''' https://seneca-ictoer.github.io/ULI101/A-Tutorials/tutorial5<br />'''Andrew's students please go here:''' http://wiki.littlesvr.ca/wiki/OPS145_Lab_5}}
=REDIRECTION: STANDARD INPUT / STANDARD OUTPUT / STANDARD ERROR=
<br>
===Main Objectives of this Practice Tutorial===
:* xUnderstand and use the '''cut''', '''tr''', and '''wc''' Linux commands
:* xDefine the terms '''Standard Input''' (''stdin''), '''Standard Output''' (''stdout''), and '''Standard Error''' (''stderr'')
:* xUnderstand and use the '''>''', '''>>''', '''2>''', '''2>>''' symbols with Linux commands
:* xUnderstand the purpose of the the '''/dev/null''' file and the '''Here Document''' :* Define the term '''pipeline command''' and explain how a pipeline command functions :* Define the term '''filter''' and how it relates to pipeline commands :* Use the '''semicolon''' ";" and '''grouping''' "( )" symbols to issue multiple Unix / Linux commands on a single line :* Use the '''backslash''' "\" symbol to spread-out long Unix/Linux commands over multiple lines<br>
===Tutorial Reference Material===
|colspan="2" style="font-size:16px;font-weight:bold;border-bottom: thin solid black;border-spacing:0px;"|Course Notes<br>
|colspan="2" style="font-size:16px;font-weight:bold;border-bottom: thin solid black;border-spacing:0px;padding-left:15px;"|Linux Command/Shortcut Reference<br>
|colspan="1" style="font-size:16px;font-weight:bold;border-bottom: thin solid black;border-spacing:0px;padding-left:15px;"|YouTube Videos<br>
|- valign="top" style="padding-left:15px;"
|colspan="2" |Course Notes'''Slides:'''<ul><li>Week 5 Lecture 1 Notes:<br>[https://ictwiki.cdot.senecacollege.ca/~murrayuli101/slides/ULI101-5.1.pdf PDF] | https://wiki.cdot.senecacollege.ca/uli101/slides/ULI101-5.1.pptx PPTX]</li><li>Week 5 Lecture 2 Notes:<br> [https://wiki.cdot.senecacollege.saulca/uli101/slides/ULI101-Week55.2.pdf PDF] | [https://ictwiki.cdot.senecacollege.ca/~murray.saululi101/uli101slides/ULI101-Week55.2.pptx PPTX]<br></li></ul>
| style="padding-left:15px;" |'''Redirection:'''* [http://www.linfo.org/standard_input.html Standard Input (stdin)]* [http://www.linfo.org/standard_output.html Standard Output (stdout)]* [http://www.linfo.org/standard_error.html Standard Error (stderr)]* Piping (pipes)[http://www.linfo.org/pipe.html Pipeline Commands]<br><br>'''Multiple Commands:'''* [https://www.javatpoint.com/linux-semicolon Semicolon ]<br>* [https://www.gnu.org/software/bash/manual/html_node/Command-Grouping.html Grouping ( )]<br><br>| style="padding-left:15px;"|'''Redirection Filters:'''* [http://man7.org/linux/man-pages/man1/more.1.html more] , [http://man7.org/linux/man-pages/man1/less.1.html less]<br>* [http://man7.org/linux/man-pages/man1/head.1.html head] , [http://man7.org/linux/man-pages/man1/tail.1.html tail]* [http://man7.org/linux/man-pages/man1/sort.1.html sort]<br>* [http://man7.org/linux/man-pages/man1/uniq.1.html uniq]<br>* [http://linuxcommand.org/lc3_man_pages/grep1.html grep]<br>* [http://man7.org/linux/man-pages/man1/cut.1.html cut]<br>* [http://linuxcommand.org/lc3_man_pages/tr1.html tr]<br>* [http://man7.org/linux/man-pages/man1/wc.1.html wc]<br>* [http://man7.org/linux/man-pages/man1/tee.1.html tee]<br>
|colspan="1" style="padding-left:15px;" width="30%"|'''Brauer Instructional Videos:'''<ul><li>[https://www.youtube.com/watch?v=ocU34PcYn2U&list=PLU1b1f-2Oe90TuYfifnWulINjMv_Wr16N&index=4 Reading/Writing to Files<br>(echo, stdin, stdout, stderr, >, >>, 2>, cat, more, less, man, date, diff, diff -y, find, wc])<br></li></ul>
|}
= KEY CONCEPTS =
===Redirection (Standard Input, Standard Output, Standard Error)===
<i>''... 'Redirection''standard streams''' are preconnected can be defined as changing the way from where commands read input and to where commands sends output communication channels between a computer program and its environment when it begins execution. The three <br>You can redirect input/output (I/O) connections are called '''standard input''' ('''stdin'''), '''standard output''' ('''stdout''') and '''standard error''' ('''stderr'''). Originally I/O happened via a physically connected system console (input via keyboard, output via monitor), but standard streams '''abstract''' this. When of a command is executed via an interactive shell, the streams are typically connected to the text terminal on which the shell is running, but can be changed with '''redirection''' or a '''pipeline'''. '' Reference: https://en.wikipedia.org/wiki</Standard_streamsi>
Reference: https://www.javatpoint.com/linux-input-output-redirection
[[Image:stdin-symbol.png|thumb|right|250px|The '''standard input''' ('''stdin''') symbol that describes where a Unix/Linux command receives '''input''']]'''Standard input''' ('''stdin''') is a term which describes from where a command receives '''input'''.<br>With the examples belowThis would apply only to Unix/Linux commands that accept stdin input<br>(like ''cat'', standard input is being sent into the command from the text file''more'', ''less'', ''sort'', ''grep'', ''head'', ''tail'', ''tr'', ''cut'', ''wc'', etc.).<br>
''Examples:''
<table align="right"><tr><td>[[Image:stdout-symbol-1.png|thumb|right|250px|The '''standard out''' ('''stdout''') symbol with one greater than sign '''overwrites''' existing file content with command output]]</td><td>[[Image:stdout-symbol-2.png|thumb|right|250px|The '''standard output''' ('''stdout''') symbol with two greater than signs '''add''' command's output to '''bottom''' of existing file's contents.]]</td></tr></table>'''Standard output''' ('''stdout''') describes where a command sends itits '''s output'''.<br>In the examples below, output from a command is sent to the '''monitor''', unless it is sent to a '''regular text file'''.<br>We will discuss another redirection technique called '''pipelines''' later.
''Examples:''
<span style="font-family:courier;font-weight:bold">ls -l<br>ls -l > detailed-listing.txt<br>ls /bin >> output.txt</span>
''Examples:''
<span style="font-family:courier;font-weight:bold">ls | tee unsorted.txt | sort<br>ls | grep Linux | tee matched.txt | more<br>ls | head -5 | tee -a listing.txt</span>
<br><br>
===Multiple Commands Using Semicolon ; / , Grouping Commands ( ), and Backquotes===
=INVESTIGATION 1: BASICS OF REDIRECTION=
<brspan style="color:red;">In this section, you '''ATTENTION''': This online tutorial will learn how be required to ...be completed by '''Friday in week 6 by midnight''' to obtain a grade of '''2%''' towards this course</span><br><br>
In this investigation, you will learn how to redirect '''standard input''', '''standard output''' and '''standard error''' when issuing Unix / Linux commands.
'''Perform the Following Steps:'''
# x'''Login''' to your matrix account and issue a command to '''confirm''' you are located in your '''home''' directory.<br><br># Issue the following Linux command to create the following directory: <span style="color:blue;font-weight:bold;font-family:courier;">mkdir ~/redirect</span><br><br># Change to the '''~/redirect''' directory and confirm that you changed to that directory.<br><br># Use a text editor to create a file in your current directory called '''data.txt''' and enter the following text displayed below:<br><br><span style="font-family:courier;font-weight:bold">This is line 1<br>This is line 2<br>This is line 3</span><br><br># '''Save''' editing changes and '''exit''' the text editor.<br><br># Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">tr 'a-z' 'A-Z' < data.txt</span><br><br>What does this command do?<br><br># Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">tr 'a-z' 'A-Z' < data.txt > output.txt</span><br><br>What does this command do? What are the contents of the file ''output.txt''?<br><br># Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">tr 'a-z' 'A-Z' > output.txt < data.txt</span><br><br>What does this command do? Is there any difference in terms of this command and the previous command issued?<br><br># Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">tr 'a-z' 'A-Z' >> output.txt < data.txt</span><br><br>What happens to the content of the '''output.txt''' file? Why?<br><br># Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">tail -2 < data.txt > output.txt</span><br><br>What does this command do? Check the contents of the '''output.txt''' file to confirm.<br><br># Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">tail -2 > output2.txt < data.txt </span><br><br>Why does this command render the same results as the previous command?<br>Try explaining how the command works in terms of '''stdin''' and then '''stdout'''.<br><br># Issue the following Linux command to create a file: <span style="color:blue;font-weight:bold;font-family:courier;">cat > output3.txt </span><br><br># Enter the follow text displayed below:<br><br><span style="font-family:courier;font-weight:bold;">This is the file output3.txt</span><br><br># Press <span style="color:blue;font-weight:bold;font-family:courier;">ctrl-d</span> to exit the command.<br><br># Issue the '''cat''' command to view the contents of the file: '''output3.txt'''<br><br># Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">cp ~jason.carman/uli101/cars .</span><br><br># Issue the '''cat''' command to view the contents of the '''cars''' file.<br><br># Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">cut -c1-10 cars</span><br><br>What did this command do?<br><br># Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">cut -f5 cars > field5.txt</span><br><br>What did this command do?<br>Check the contents in the file '''field5.txt''' to see what happened.<br><br># Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">cut -f1-3 cars > field123.txt</span><br><br>What did this command do? (check file contents)<br><br># Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">cut -f1,5 cars > field15.txt</span><br><br>What did this command do? (check file contents)<br><br># Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">wc cars > count.txt</span><br><br>What information does the '''count.txt''' file contain?<br><br># Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">wc -l cars > count1.txt</span><br><br>What information does the '''count1.txt''' file contain?<br><br># Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">wc -w cars > count2.txt</span><br><br>What information does the '''count2.txt''' file contain?<br><br># Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">ls -l > listing.txt</span><br><br>What information does the '''listing.txt''' file contain?<br><br># Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">pwd > listing.txt</span><br><br>What happenned to the original contents of the file called '''listing.txt'''? Why?<br><br># Issue the following Linux command (use 2 greater-than signs): <span style="color:blue;font-weight:bold;font-family:courier;">date >> listing.txt</span><br><br>What information does the '''listing.txt''' file contain? Why?<br><br># Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">cat listing.txt cars > combined.txt</span><br><br>What information does the '''combined.txt''' file contain? Why?<br><br>'''NOTE''': The '''cat''' command stands for "'''concatenate'''" which means to '''combine''' contents of multiple files into a single file.<br>This is why the command is called "''cat''".<br><br># Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">cat listing.txt cars murray 2> result.txt</span><br><br>What is displayed on the monitor? What information does the '''result.txt''' file contain? Why?<br><br># Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">cat listing.txt cars murray > myoutput.txt 2> /dev/null</span><br><br>What is displayed on the monitor? What happened to the error message?<br><br># Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">cat listing.txt cars murray > myoutput.txt 2> result.txt</span><br><br>What is displayed on the monitor? what do those files contain? Why?<br><br>The '''Here Document''' allows you to redirect stdin from with the Linux command itself. Let's get some practice using the Here Document.<br><br># Issue the following Linux command:<br><span style="color:blue;font-weight:bold;font-family:courier;">cat <<+<br>line 1<br>line 2<br>line 3<br>+</span><br><br>What do you notice?<br><br># Issue the following Linux command:<br><span style="color:blue;font-weight:bold;font-family:courier;">grep 2 <<+<br>line 1<br>line 2<br>line 3<br>+</span><br><br>What do you notice? How does this differ from the previous command? Why?<br><br># Issue the following Linux command:<br><span style="color:blue;font-weight:bold;font-family:courier;">grep 2 > line2.txt <<+<br>line 1<br>line 2<br>line 3<br>+</span><br><br>What do you notice? What is contained in the file '''line2.txt'''? Why?<br><br>'''NOTE:''' You will now run a shell script to confirm that you properly issued Linux commands using redirection.<br><br># Issue the following Linux command to run a checking script:<br><span style="color:blue;font-weight:bold;font-family:courier;">~uli101/week5-check-1</span><br><br># If you encounter errors, make corrections and '''re-run''' the checking script until you receive a congratulations message, then you can proceed.<br><br># Issue the '''ls''' command to see all of the '''temporary files''' that were created as a result of redirection.<br><br>The problem with using these redirection symbols is that you create '''temporary text files''' that take up '''space''' on your file system.<br><br># Issue a Linux command (using '''Filename Expansion''') to '''remove''' those temporary text files in the current directory.<br><br># Issue the following Linux command to check that you removed ALL of those temporary text files:<br><span style="color:blue;font-weight:bold;font-family:courier;">~uli101/week5-check-2</span><br><br># If you encounter errors, make corrections and '''re-run''' the checking script until you receive a congratulations message, then you can proceed.<br><br>
:In the next investigation, you will ..be learning how to issue '''pipeline Linux commands''' which can<br>accomplish tasks <u>without</u> creating temporary files.<br><br>
=INVESTIGATION 2: REDIRECTION USING PIPES PIPELINE COMMANDS =
In this sectioninvestigation, you will learn how to ..issue '''pipeline commands''' to to accomplish tasks <u>without</u> having to generate temporary files.
'''Perform the Following Steps:'''
# xConfirm that you are still located in the '''~/redirect''' directory.<br><br>The '''problem''' with creating temporary files, is that they take up space on your server,<br>and should be removed. You actually did that in the previous investigation.<br><br>You will be issuing a '''pipeline command''' which will use the pipe symbol "|"<br>that will send the stdout from a command as stdin into another command<br><u>without</u> having to create temporary files.<br><br># Issue the follow Linux '''pipeline command''': <span style="color:blue;font-weight:bold;font-family:courier;">ls /bin | more</span><br><br>What happened? Press '''q''' to exit display.<br><br>In # Issue the next investigationfollowing Linux '''pipeline command''': <span style="color:blue;font-weight:bold;font-family:courier;">ls /bin | who</span><br><br>What happened? Although this pipeline command provides output,<br>it '''does <u>not</u> work''' properly as a pipeline command since the '''who''' command is<br>'''NOT''' designed to accept standard input.[[Image:pipe-diagram-1.png|thumb|right|350px|]]<br><br>'''NOTE:''' When issuing pipeline commands, commands to the right of the pipe symbol must be designed to <u>accept</u> '''standard input'''. Since the ''who'' command does not, you did NOT see the contents of the '''/bin''' directory but only information relating to the ''who'' command. Therefore, the '''order''' of which you build your pipeline command and the '''type of command''' that is used as a ''filter'' is extremely important!<br><br># Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">ls /bin/?? > listing.txt</span><br><br># Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">sort listing.txt</span><br><br># Issue the following Linux command to remove the listing file: <span style="color:blue;font-weight:bold;font-family:courier;">rm listing.txt</span><br><br># Issue the following Linux '''pipeline command''': <span style="color:blue;font-weight:bold;font-family:courier;">ls /bin/?? | sort </span><br><br>You should notice that the output from this pipeline command is the same output<br>from the command you issued in '''step #5'''.<br><br># Issue the following Linux '''pipeline command''': <span style="color:blue;font-weight:bold;font-family:courier;">ls /bin/?? | sort | more</span><br><br>What is difference with this pipeline command as opposed to the <u>previous</u> pipeline command? Press '''q''' to exit display.<br><br># Issue the '''ls''' command.<br><br>You should notice that '''no files have been created'''.<br>Let's get practice issuing more pipeline commands using commands<br>(previously learned or new) to be used as '''filters'''.<br><br># Issue the following Linux '''pipeline command''': <span style="color:blue;font-weight:bold;font-family:courier;">ls /bin/?? | sort | head -5</span><br><br>What did you notice?<br><br># Issue the following Linux '''pipeline command''': <span style="color:blue;font-weight:bold;font-family:courier;">ls /bin/???? | sort | grep r | tail -2</span><br><br>What did you notice? Could you predict the output prior to issuing this pipeline command?<br><br># Issue the following Linux '''pipeline command''': <span style="color:blue;font-weight:bold;font-family:courier;">ls /bin/???? | sort | grep r | cut -c1-6</span><br><br>Try to explain step-by-step each process in the pipeline command (including ''filters'')<br>to explain the final output from this pipeine command.<br><br># Confirm that you are still located in the '''~/redirect''' directory.<br><br># Issue the following Linux '''pipeline command''':<br><span style="color:blue;font-weight:bold;font-family:courier;">ls /bin/???? | tee unsort.txt | sort | tee sort.txt | grep r | tee match.txt | head </span><br><br># Issue the '''ls''' command to view the contents of this redirectory.<br><br>What did you notice?<br><br># View the <u>contents</u> of the '''text files''' that were created to see how the '''tee''' command<br>was used in the previous pipeline command.<br><br>What was the purpose of using the '''tee''' command for this pipeline command?<br><br>You will now run a shell script to confirm that you properly issued that Linux pipeline command<br>using the '''tee''' command and redirection.<br><br># Issue the following Linux command to run a checking script:<br><span style="color:blue;font-weight:bold;font-family:courier;">~uli101/week5-check-3</span><br><br>If you encounter errors, make corrections and '''re-run''' the checking script until you receive<br>a congratulations message, then you can proceed.<br><br># Change to <u>your</u> '''home''' directory.<br><br># Remove the '''~/redirect''' directory and its contents.<br><br>
In this investigation, you will learn how to issue multiple Unix / Linux commands in a single line or over multiple lines.
'''Perform the Following Steps:'''
# xConfirm you are located in your '''home''' directory in your Matrix account.<br><br># Issue the following Linux commands (using the ''semicolon'' character "''';'''" to separate <u>each</u> Linux command):<br><span style="color:blue;font-weight:bold;font-family:courier;">cal;pwd;date</span><br><br>Note the output as well as the <u>order</u> of what each Linux command results.<br><br># Issue the following Linux commands: <span style="color:blue;font-weight:bold;font-family:courier;">(cal;pwd;date)</span><br><br>Was there any difference in the output of this command as opposed to the previous command?<br><br>Let's see how grouping affects working with redirection.<br><br># Issue the following Linux commands: <span style="color:blue;font-weight:bold;font-family:courier;">cal;pwd;date > output.txt</span><br><br>What happened? Where is the output for the '''date''' command?<br>Why isn't the output for the '''cal''' and '''pwd''' commands are NOT contained in that file?<br><br># Issue a Linux command to view the contents of the file called '''output.txt'''<br><br>What do you notice?<br><br>Let's use '''grouping''' to make modification to the previous command<br><br># Issue the following Linux commands: <span style="color:blue;font-weight:bold;font-family:courier;">(cal;pwd;date) > output.txt</span><br><br>What did you notice?<br><br># Issue a Linux command to view the contents of the file called '''output.txt'''<br><br>What does ''grouping'' do when issuing multiple Linux commands (separated by a semi-colon ";") that uses redirection?<br><br># Issue the following Linux pipeline command (using \ at the end of most lines):<br><span style="color:blue;font-family:courier;font-weight:bold">echo "This will be split over multiple \<br>lines. Note that the shell will realize \<br>that a pipe requires another command, so \<br>it will automatically go to the next line" |tr '[a-z]' '[A-Z]'</span><br><br>Did the command work? What is the purpose of issuing a Linux command in this way?<br><br># Complete the Review Questions sections to get additional practice.<br><br>
= LINUX PRACTICE QUESTIONS =
simulate a quiz:
https://ictwiki.cdot.senecacollege.ca/~murray.saululi101/uli101files/uli101_week5_practice.docx
Your instructor may take-up these questions during class. It is up to the student to attend classes in order to obtain the answers to the following questions. Your instructor will NOT provide these answers in any other form (eg. e-mail, etc).
When answering Linux command questions, refer to the following Inverted Tree Diagram. The linux directory is contained in your home directory. Assume that you just logged into your Matrix account. Directories are <u>underlined</u>.
[[Image:week5-dir.png|thumb|left|300px|]]
<br><br><br><br><br><br><br><br><br>
'''Review Questions:'''
# xWrite a single Linux command to provide a detailed listing of all files in the '''/etc''' directory, sending the output to a file called listing.txt in the “'''projects'''” directory (append output to existing file and use a relative pathname)# x Write a single Linux command to redirect the stderr from the command:<br>'''cat a.txt b.txt c.txt''' to a file called '''error.txt''' contained in the “'''assignments'''” directory. (overwrite previous file’s contents and use only relative pathnames) # xWrite a single Linux command: '''cat ~/a.txt ~/b.txt ~/c.txt''' and redirect stdout to a file called “good.txt” to the “tests” directory and stderr to a file called “'''bad.txt'''” to the “'''tests'''” directory. (overwrite previous contents for both files and use only relative-to-home pathnames).# xWrite a single Linux command to redirect the stdout from the command:<br>'''cat a.txt b.txt c.txt''' to a file called wrong.txt contained in the “'''projects'''” directory and throw-out any standard error messages so they don’t appear on the screen (append output to existing file and use only relative pathnames).<br><br># xWrite a single Linux '''pipeline command''' to display a detailed listing of the '''projects''' directory but pause one screen at a time to view and navigate through all of the directory contents. Use a relative-to-home pathname.# xWrite a single Linux '''pipeline command''' to display the sorted contents (in reverse alphabetical order) of the “'''linux'''” directory. Use a relative pathname.# Assume that the text file called “'''.answers.txt'''” contains 10 lines. Write a single Linux pipeline command to only displays lines 5 through 8 for this file. Use only relative pathnames.# Write a single Linux '''pipeline command''' to only display the contents of the “'''assignments'''” directory whose filenames match the pattern “'''murray'''” (both upper or lowercase). Use an absolute pathname.# Write a single Linux '''pipeline command''' to display the number of characters contained in the file called “'''.answers.txt'''”. Use a relative-to-home pathname.# xWrite a single Linux '''pipeline command''' to display the number of lines contained in the file called “'''questions.txt'''”. Use a relative pathname.# xWrite a single Linux '''pipeline command''' to display only the first 10 characters of each filename contained in your current directory. Also, there is will be a lot of output, so also pause at each screenful so you can navigate throughout the display contents. Use a relative pathname.# Create a '''table''' listing each Linux command, useful options that were mentioned in the online assignment #1 and command purpose this tutorial for the following Linux commands: '''cut''' , '''tr''' , '''wc''' , and '''tee'''. _________________________________________________________________________________ Author: Murray Saul License: LGPL version 3Link: https://www.gnu.org/licenses/lgpl.html _________________________________________________________________________________
[[Category:ULI101]]