Changes

← Older edit

Tutorial5: Redirection

3,456 bytes added, 20:35, 4 September 2023

no edit summary

{{Admon/caution|DO NOT USE THIS VERSION OF THE LAB. This page will no longer be updated.|'''New version here:''' https://seneca-ictoer.github.io/ULI101/A-Tutorials/tutorial5 '''Andrew's students please go here:''' http://wiki.littlesvr.ca/wiki/OPS145_Lab_5}}

=REDIRECTION: STANDARD INPUT / STANDARD OUTPUT / STANDARD ERROR=

|- valign="top" style="padding-left:15px;"

|colspan="2" |~~Course Notes~~'''Slides:'''<ul><li>Week 5 Lecture 1 Notes: [https://~~ict~~wiki.cdot.senecacollege.ca/~~~murray~~uli101/slides/ULI101-5.1.pdf PDF] | https://wiki.cdot.senecacollege.ca/uli101/slides/ULI101-5.1.pptx PPTX]</li><li>Week 5 Lecture 2 Notes: [https://wiki.cdot.senecacollege.~~saul~~ca/uli101/slides/ULI101-~~Week5~~5.2.pdf PDF] | [https://~~ict~~wiki.cdot.senecacollege.ca/~~~murray.saul~~uli101/~~uli101~~slides/ULI101-~~Week5~~5.2.pptx PPTX] </li></ul>

| style="padding-left:15px;" |'''Redirection:'''

* [http://www.linfo.org/standard_input.html Standard Input (stdin)]

* [http://www.linfo.org/standard_output.html Standard Output (stdout)]

* [http://www.linfo.org/standard_error.html Standard Error (stderr)]

* [http://www.linfo.org/pipe.html Pipeline Commands]

'''Multiple Commands:'''

* [https://www.javatpoint.com/linux-semicolon Semicolon]

* [https://www.gnu.org/software/bash/manual/html_node/Command-Grouping.html Grouping ( )]

| style="padding-left:15px;"|'''Redirection Filters:'''

* [http://man7.org/linux/man-pages/man1/more.1.html more] , [http://man7.org/linux/man-pages/man1/less.1.html less]

* [http://man7.org/linux/man-pages/man1/head.1.html head] , [http://man7.org/linux/man-pages/man1/tail.1.html tail]

* [http://man7.org/linux/man-pages/man1/tee.1.html tee]

|colspan="1" style="padding-left:15px;" width="30%"|'''Brauer Instructional Videos:'''<ul><li>[https://www.youtube.com/watch?v=ocU34PcYn2U&list=PLU1b1f-2Oe90TuYfifnWulINjMv_Wr16N&index=4 Reading/Writing to Files (echo, stdin, stdout, stderr, >, >>, 2>, cat, more, less, man, date, diff, diff -y, find, wc]) </li></ul>

|}

<table align="right"><tr><td>[[Image:stdout-symbol-1.png|thumb|right|250px|The '''standard ~~input~~out''' ('''~~stdin~~stdout''') symbol with one greater than sign '''overwrites''' existing file content with command output]]</td><td>[[Image:stdout-symbol-2.png|thumb|right|250px|The '''standard ~~input~~output''' ('''~~stdin~~stdout''') symbol with two greater than signs '''add''' command's output to '''bottom''' of existing file's contents.]]</td></tr></table>

'''Standard output''' ('''stdout''') describes where a command sends its '''output'''. In the examples below, output from a command is sent to the '''monitor''', unless it is sent to a '''text file'''.

cat <<+ Line 1 Line 2 Line 3 +

===~~Piping (Using Pipes)~~Pipeline Commands===

[[Image:pipe-diagram-1.png|thumb|right|450px|A '''pipeline command''' sends a command's '''standard output''' directly to '''standard input''' of other command(s) without having to create temporary files.]]

'''Pipeline Command:''' Having commands send their '''standard output''' directly</u~~><br~~>to '''standard input''' of other commands WITHOUT having to use '''temporary''' files.

Pipes that are used in a '''pipeline command''' are represented by the '''pipe''' "|" symbol.

[[Image:tee-diagram-1.png|thumb|right|250px|The '''tee''' utility can be used to '''split''' the flow of information. For example to save in a file as well as display on a screen. (Image licensed under [https://creativecommons.org/licenses/by-sa/3.0/ cc])]]

~~The~~ The '''tee''' ~~utility~~ utility can be used to split the flow of ~~information~~'''standard output''' between a '''text file''' and the '''terminal screen'''. The '''tee ''' option '''-a''' can be used to add content to the '''bottom ''' of an existing file as opposed to ''overwriting '' the file's previous contents.

The reason for the name "'''tee'''" is that the splitting of the flow of information resembles a capital T.

Commands may also be '''spread-out over multiple lines''', making it easier (for humans) to interpret a long command. ~~You can add a~~ The '''~~backslash~~\''' symbol ~~"\" at~~ “''quotes-out''” the ~~end of a line, to get rid~~ meaning of the ~~special meaning~~'''ENTER''' key as text (i.e. ''new-line'' as instead of ~~newline (to end a~~ ''running'' the command ~~line~~).

=INVESTIGATION 1: BASICS OF REDIRECTION=

'''ATTENTION''': This online tutorial will be required to be completed by '''Friday in week 6 by midnight''' to obtain a grade of '''2%''' towards this course In this ~~section~~investigation, you will learn how to redirect '''standard input''', '''standard output''' and '''standard error''' when issuing Unix / Linux commands.

# Press ctrl-d to exit the command.

# Issue the '''cat''' command to view the contents of the file: '''output3.txt'''

# Issue the following Linux command: cp ~~~murray~~jason.~~saul~~carman/uli101/cars .

# Issue the '''cat''' command to view the contents of the '''cars''' file.

# Issue the following Linux command: cut -c1-10 cars What did this command do?

# Issue the following Linux command: cut -f5 cars > field5.txt What did this command do? Check the contents in the file '''field5.txt''' to see what happened. # Issue the following Linux command: cut -f1-3 cars > field123.txt What did this command do?(check file contents) # Issue the following Linux command: cut -f1,5 cars > field15.txt What did this command do?(check file contents)

# Issue the following Linux command: wc cars > count.txt What information does the '''count.txt''' file contain?

# Issue the following Linux command: wc =-l cars > count1.txt What information does the '''count1.txt''' file contain?

# Issue the following Linux command: wc -w cars > count2.txt What information does the '''count2.txt''' file contain?

# Issue the following Linux command: ls -l > listing.txt What information does the '''~~count2~~listing.txt''' file contain? # Issue the following Linux command: pwd > listing.txt What happenned to the original contents of the file called '''listing.txt'''? Why? # Issue the following Linux command (use 2 greater-than signs): date >> listing.txt What information does the '''listing.txt''' file contain? Why? # Issue the following Linux command: cat listing.txt cars > combined.txt What information does the '''combined.txt''' file contain? Why? '''NOTE''': The '''cat''' command stands for "'''concatenate'''" which means to '''combine''' contents of multiple files into a single file. This is why the command is called "''cat''". # Issue the following Linux command: cat listing.txt cars murray 2> result.txt What is displayed on the monitor? What information does the '''result.txt''' file contain? Why?

# Issue the following Linux command: cat listing.txt cars murray > myoutput.txt 2> /dev/null What is displayed on the monitor? What happened to the error message?

# Issue the following Linux command: cat listing.txt cars murray > myoutput.txt 2> result.txt What is displayed on the monitor? what do those files contain? Why? The '''Here Document''' allows you to redirect stdin from with the Linux command itself. Let's get some practice using the Here Document.

# Issue the following Linux command: cat <<+ line 1 line 2 line 3 + What do you notice?

# Issue the following Linux command: grep 2 <<+ line 1 line 2 line 3 + What do you notice? How does this differ from the previous command? Why? # Issue the following Linux command: grep 2 > line2.txt <<+ line 1 line 2 line 3 + What do you notice? What is contained in the file '''line2.txt'''? Why? '''NOTE:''' You will now run a shell script to confirm that you properly issued Linux commands using redirection. # Issue the following Linux command to run a checking script: ~uli101/week5-check-1 # If you encounter errors, make corrections and '''re-run''' the checking script until you receive a congratulations message, then you can proceed. # Issue the '''ls''' command to ~~view~~ see all of the ~~number of~~ '''temporary files ''' that were created as a result of redirection. The problem with using these redirection symbols is that you create '''temporary text files''' that take up '''space''' on your file system. # Issue a Linux command (using '''Filename Expansion''') to '''remove''' those temporary text files in the current directory. # Issue the following Linux command to check that you removed ALL of those temporary text files: ~uli101/week5-check-2 # If you encounter errors, make corrections and '''re-run''' the checking script until you receive a congratulations message, then you can proceed.

:~~The problem with using redirection to create files, you have these files taking up space, which requires you remove them.~~ In the next investigation, you will be learning how to issue '''pipeline Linux commands ''' which can ~~provide information by issuing several Linux commands~~ accomplish tasks without creating temporary files.

=INVESTIGATION 2: REDIRECTION USING PIPELINE COMMANDS =

In this ~~section~~investigation, you will learn to issue '''pipeline commands''' to ~~issue Linux commands~~ to accomplish tasks without having to generate temporary files.

'''Perform the Following Steps:'''

# Confirm that you are still located in the ~~'''~/redirect''' directory. # Issue the '''ls''' command to view the contents of your~~ '''~/redirect''' directory. The '''problem''' with creating temporary files, is that they take up space on your server, and should be removed.~~ # Issue~~ You actually did that in the following Linux command to remove all temporary files in your ''redirect'' directory: rm -r ~/redirect and confirm that you have removed this directory and its contentsprevious investigation. ~~'''NOTE:'''~~ You will be issuing a '''pipeline command''' which will use the pipe symbol "|" that will send the stdout from a command as stdin into another command without having to create temporary files. # Issue the follow Linux '''pipeline command''': ls /bin | more What happened?Press '''q''' to exit display. # Issue the following Linux '''pipeline command''': ls /bin | who What happened? ~~Why~~ Although this pipeline command provides output, it '''does not work''' properly as a pipeline command since the '''who''' command is ~~the result different than antipated?~~ '''NOT''' designed to accept standard input.[[Image:pipe-diagram-1.png|thumb|right|350px|]] '''NOTE:''' When issuing pipeline commands, commands to the right of the pipe symbol must be designed to accept '''standard input'''. Since the ''who'' command does not, you did NOT see the contents of the '''/bin''' directory but only information relating to the ''who'' command. Therefore, the '''order''' of which you build your pipeline command and the '''type of command''' that is used as a ''filter'' is extremely important!

# Issue the following Linux command: ls /bin/?? > listing.txt

# Issue the following Linux command: sort -r listing.txt

# Issue the following Linux command to remove the listing file: rm listing.txt

# Issue the following Linux '''pipeline command''': ls /bin/?? | sort -r You should notice that the output from this pipeline command is the same output from the command you issued in '''step #75'''. # Issue the following Linux '''pipeline command''': ls /bin/?? | sort -r | more What is ~~different~~ difference with this pipeline command as opposed to the previous pipeline command?Press '''q''' to exit display. # Issue the '''ls''' command. You should notice that '''no files have been created'''. Let's get practice issuing more pipeline commands using commands ~~using commands~~ (previously learned or new) to be used as '''filters'''. # Issue the following Linux '''pipeline command''': ls /bin/?? | sort -r | head -5 What did you notice? # Issue the following Linux '''pipeline command''': ls /bin/???? | sort -r | grep r | tail -2 What did you notice? Could you predict the output prior to issuing this pipeline command? # Issue the following Linux '''pipeline command''': ls /bin/???? | sort -r | grep r | cut -c1-36 Try to explain step-by-step each process in the pipeline command (including ''filters'') to explain the final output from this pipeine command. # Confirm that you are still located in the '''~/redirect''' directory. # Issue the following Linux '''pipeline command''': ls /bin/???? | tee ~~unsorted~~unsort.txt | sort -r | tee ~~sorted~~sort.txt | grep r | tee ~~unmatched~~match.txt | ~~grep r | tail -2~~head # Issue the '''ls''' command to view the contents of this redirectory. What did you notice? # ~~Check~~ View the contents of the '''text files ''' that were created to see how the '''tee''' command was used in the previous pipeline command. ~~# Change~~ What was the purpose of using the '''tee''' command for this pipeline command? You will now run a shell script to ~~your home directory~~confirm that you properly issued that Linux pipeline command using the '''tee''' command and redirection. # Issue the following Linux command to run a checking script: ~uli101/week5-check-3 If you encounter errors, make corrections and '''rmre-run''' ~~command~~ the checking script until you receive a congratulations message, then you can proceed. # Change to ~~only~~your ~~remove the files~~ '''~~unsorted.txt~~home''' , directory. # Remove the '''~~sorted.txt~~~/redirect''' , directory and ~~'''unmatched~~its contents.~~txt'''~~

:In the next investigation, you will learn various techniques to issue '''multiple Linux commands ''' on the same line, or ~~long~~ issue a '''single Linux ~~commands~~ command over multiple lines'''.

=INVESTIGATION 3: ISSUING MULTIPLE UNIX/LINUX COMMANDS=

In this ~~section~~investigation, you will learn how to issue multiple Unix / Linux commands in a single line or over multiple lines.

'''Perform the Following Steps:'''

~~# Issue a Linux command to change to your home directory. ~~# Confirm you are located in your '''home''' directoryin your Matrix account. # Issue the following Linux commands (using the ''semicolon '' character "''';'''" to separate each Linux command): cal;pwd;date Note the ~~from the~~ output as well as the order of ~~how~~ what each ~~of those commands were processed~~Linux command results.

# Issue the following Linux commands: (cal;pwd;date) Was there any difference in the output of this command as opposed to the previous command? Let's see how grouping affects working with redirection.

# Issue the following Linux commands: cal;pwd;date > output.txt What happened? Where is the output for the '''date''' command? Why isn't the output for the '''cal''' and '''pwd''' commands are NOT contained in that file? # Issue a Linux command to view the contents of the file called '''output.txt''' What do you notice? Let's use '''grouping ''' to make modification to the previous command # Issue the following Linux commands: (cal;pwd;date) > output.txt What did you notice? # Issue a Linux command to view the contents of the file called '''output.txt''' What does ''grouping'' do ~~you notice? What did grouping the three~~ when issuing multiple Linux commands do(separated by a semi-colon ";") that uses redirection? # Issue the following Linux pipeline command (using \ at the end of most lines): echo "This will be split over multiple \ lines. Note that the shell will realize \ that a pipe requires another command, so \ it will automatically go to the next line" |tr '[a-z]' '[A-Z]' Did the command work? What ~~does~~ is the purpose of issuing a Linux command in this ~~command do~~way? # ~~After you complete~~ Complete the Review Questions sections to get additional practice~~, then work on your '''online assignment 2''' and complete '''section3''' labelled: '''Redirection and Pipes'''~~.

simulate a quiz:

https://~~ict~~wiki.cdot.senecacollege.ca/~~~murray.saul~~uli101/~~uli101~~files/uli101_week5_practice.docx

Your instructor may take-up these questions during class. It is up to the student to attend classes in order to obtain the answers to the following questions. Your instructor will NOT provide these answers in any other form (eg. e-mail, etc).

'''Review Questions:'''

# Write a single Linux command to provide a detailed listing of all files in the '''/~~bin~~etc''' directory, sending the output to a file called listing.txt in the “'''projects'''” directory (append output to existing file and use a relative pathname)

# Write a single Linux command to redirect the stderr from the command: '''cat a.txt b.txt c.txt''' to a file called '''error.txt''' contained in the “'''assignments'''” directory. (overwrite previous file’s contents and use only relative pathnames)

# Write a single Linux command: '''cat ~/a.txt ~/b.txt ~/c.txt''' and redirect stdout to a file called “good.txt” to the “tests” directory and stderr to a file called “'''bad.txt'''” to the “'''tests'''” directory. (overwrite previous contents for both files and use only relative-to-home pathnames).

# Write a single Linux '''pipeline command''' to display only the first 10 characters of each filename contained in your current directory. Also, there is will be a lot of output, so also pause at each screenful so you can navigate throughout the display contents. Use a relative pathname.

# Create a '''table''' listing each Linux command, useful options that were mentioned in this tutorial for the following Linux commands: '''cut''' , '''tr''' , '''wc''' , and '''tee'''.

_________________________________________________________________________________

Author: Murray Saul

License: LGPL version 3

Link: https://www.gnu.org/licenses/lgpl.html

_________________________________________________________________________________

[[Category:ULI101]]

Andrew

Bureaucrats, Administrators

1,234

edits

CDOT Wiki β

Changes

Tutorial5: Redirection

CDOT Wiki ^β