Tutorial5: Redirection

REDIRECTION: STANDARD INPUT / STANDARD OUTPUT / STANDARD ERROR

Main Objectives of this Practice Tutorial

• Define the terms Standard Input (stdin), Standard Output (stdout), and Standard Error (stderr)

• Understand the purposes of redirection symbols >, >>, 2>, 2>>, | and define their purpose in terms of redirection

• Understand the purpose of the additional file manipulation commands: cut, tr, and wc

• Define the term pipeline command and explain how a pipeline command works in terms of redirection

• Define the term filter and how it relates to redirection using pipeline commands

• Understand how to use the semicolon symbol ";" to issue multiple Unix / Linux commands

• Understand how to use grouping using the symbols "( )" and how it is used to issue multiple Unix / Linux commands

Tutorial Reference Material

Course Notes		Linux Command / Shortcut Reference		YouTube Videos
Course Notes: PDF | PPTX		Redirection Standard Input (stdin) Standard Output (stdout) Standard Error (stderr) Piping (pipes) Multiple Commands Semicolon Grouping ( )	Redirection Filters more , less head , tail sort uniq grep cut tr wc tee	Brauer Instructional Videos: Reading/Writing to Files (echo, stdin, stdout, stderr, >, >>, 2>, cat, more, less, man, date, diff, diff -y, find, wc)

KEY CONCEPTS

Redirection (Standard Input, Standard Output, Standard Error)

... standard streams are preconnected input and output communication channels between a computer program and its environment when it begins execution. The three input/output (I/O) connections are called standard input (stdin), standard output (stdout) and standard error (stderr). Originally I/O happened via a physically connected system console (input via keyboard, output via monitor), but standard streams abstract this. When a command is executed via an interactive shell, the streams are typically connected to the text terminal on which the shell is running, but can be changed with redirection or a pipeline.

Reference: https://en.wikipedia.org/wiki/Standard_streams

The standard input (stdin) symbol that describes where a Unix/Linux command receives input

Standard input (stdin) is a term which describes from where a command receives input.
This would apply only to Unix/Linux commands that accept stdin input (like cat, more, less, sort, grep, head, tail, etc.).
With the examples below, standard input is being sent into the command from the text file.

Examples:

tr 'a-z' 'A-Z' < words.txt
cat < abc.txt
sort < xyz.txt

The standard input (stdin) symbol with one greater than sign overwrites existing file content with command output

The standard input (stdin) symbol with two greater than signs add command's output to bottom of existing file's contents.

Standard output (stdout) describes where a command sends it's output.
In the examples below, output from a command is sent to the monitor, unless it is sent to a regular file.
We will discuss another redirection technique called pipelines later.

Examples:

ls -l
ls -l > detailed-listing.txt
ls /bin >> output.txt

The standard error (sterr) symbol with one greater than sign overwrites existing file content with command's error message.

The standard error (stderr) symbol with two greater than signs add command's error message to bottom of existing file's contents.

Standard Error (stderr) describes where a command sends it's error messages.
In the examples below, error messages from a command are sent to a file or are redirected to a file that acts as a "garbage can".

Examples (remember, Unix/Linux is case sensitive and issuing PWD in caps will generate an error message):

PWD
PWD 2> error-message.txt
PWD 2 >> error-messages.txt
PWD 2> /dev/null

Additional File Manipulation Commands

There are some additional regular file manipulation commands that you can use with redirection
(in addition to the other regular file manipulation commands introduced in week 2).

These commands are displayed in the table below:

Linux Command	Purpose
cut	Used to extract fields and characters from records. The option -c option is used to cut by a character or a range of characters. The -f option indicates the field number or field range to display (this may require using the -d option to indicate the field separator (delimiter). Examples: cut -f2 filename - extract 2nd field from all records in file, using tab as delimiter (default) cut -d' ' -f2,5 filename - extract 2nd and 5th field, using space as delimiter cut -d' ' -f1-3,5 filename - extract 1st through 3rd and 5th fields, using space as delimiter cut -c3-5 filename - extract 3rd to 5th characters
tr	Used to translate characters to different characters. Examples: tr a A < filename - translate all characters "a" to "A" tr "[a-z]" "[A-Z]" < filename - translate lowercase "a" through "z" to uppercase tr "a-z" "A-Z" < filename - translate lowercase "a" through "z" to uppercase, different syntax (non-System V) tr ':' ' ' < filename - translate all colons to spaces tr ' ' '\n' < filename - translate all spaces to newline characters tr 'abc' 'A' < filename - translate 'a', 'b', and 'c' to 'A', the last character in the "to" string repeats
wc	Displays various counts of the contents of a file. Examples: wc -l filename - displays number of lines in file wc -c filename - displays number of characters in file wc -w filename - displays number of words in fil

The /dev/null file (sometimes called the bit bucket or black hole) is a special system file
that discards all data written into it. This is useful to discard unwanted command output.

Example:

find / -name "tempfile" 2> /dev/null

Piping (Using Pipes)

A pipeline command sends a command's standard output directly to standard input of other command(s) without having to create temporary files.

Pipeline Command: Having commands send their standard output directly to standard input of other commands without having to use temporary files.

A few simple commands can be combined to form a more powerful command line.

Pipes that are used in a pipeline command are represented by the | symbol.

Commands to the right of the pipe symbol are referred to as filters. They are referred to as filters since those commands are used to modify the stdin that was sent from the previous command. Many commands can be "piped" together, but these commands (filters) must be chained in a specific order, depending on what you wish to accomplish

Examples:
ls -al | more
ls | sort -r
ls | sort | more
ls -l | cut -d" " -f2 | tr 'a-z' 'A-z"
ls | grep Linux | head -5
head -7 filename | tail -2

The tee utility can be used to split the flow of information. The tee option -a can be used to add content to the bottom of an existing file as opposed to overwriting the file's previous contents.

The reason for the name "tee" is that the splitting of the flow of information resembles a capital T.

Examples: ls | tee unsorted.txt | sort
ls | grep Linux | tee matched.txt | more
ls | head -5 | tee -a listing.txt

Multiple Commands Using Semicolon ";" and Grouping Commands "( )"

Besides piping, there are other ways that multiple commands may be placed in one line: commands may be separated by semi-colons.

Example:

sleep 5; ls (NOTE: each command will be executed when the previous command has terminated)

commands may be grouped by using parentheses (echo "Who is on:"; w) > whoson commands may also be split over multiple lines, making it easier (for humans) to interpret a long command quote or "escape" the newline character at the end of a line, to get rid of the special meaning of newline (to end a command line) for example: echo "This will be split over multiple \ lines. Note that the shell will realize \ that a pipe requires another command, so \ it will automatically go to the next line" | tr '[a-z]' '[A-Z]'

INVESTIGATION 1: BASICS OF REDIRECTION

In this section, you will learn how to ...

Perform the Following Steps:

1. x
   
   

In the next investigation, you will ...

INVESTIGATION 2: REDIRECTION USING PIPES

In this section, you will learn how to ...

Perform the Following Steps:

1. x
   
   

In the next investigation, you will ...

INVESTIGATION 3: MULTIPLE COMMANDS / COMMAND GROUPING

In this section, you will learn how to ...

Perform the Following Steps:

1. x
   

LINUX PRACTICE QUESTIONS

The purpose of this section is to obtain extra practice to help with quizzes, your midterm, and your final exam.

Here is a link to the MS Word Document of ALL of the questions displayed below but with extra room to answer on the document to simulate a quiz:

https://ict.senecacollege.ca/~murray.saul/uli101/uli101_week5_practice.docx

Your instructor may take-up these questions during class. It is up to the student to attend classes in order to obtain the answers to the following questions. Your instructor will NOT provide these answers in any other form (eg. e-mail, etc).

When answering Linux command questions, refer to the following Inverted Tree diagram. The linux directory is contained in your home directory. Assume that you just logged into your Matrix account. Directories are underlined.

Review Questions:

1. Write a single Linux command to provide a detailed listing of all files in the /bin directory, sending the output to a file called listing.txt in the “projects” directory (append output to existing file and use a relative pathname)
2. Write a single Linux command to redirect the stderr from the command:
   cat a.txt b.txt c.txt to a file called error.txt contained in the “assignments” directory. (overwrite previous file’s contents and use only relative pathnames)
3. Write a single Linux command: cat ~/a.txt ~/b.txt ~/c.txt and redirect stdout to a file called “good.txt” to the “tests” directory and stderr to a file called “bad.txt” to the “tests” directory. (overwrite previous contents for both files and use only relative-to-home pathnames).
4. Write a single Linux command to redirect the stdout from the command:
   cat a.txt b.txt c.txt to a file called wrong.txt contained in the “projects” directory and throw-out any standard error messages so they don’t appear on the screen (append output to existing file and use only relative pathnames).
   
   
5. Write a single Linux pipeline command to display a detailed listing of the “projects “directory but pause one screen at a time to view and navigate through all of the directory contents. Use a relative-to-home pathname.
6. Write a single Linux pipeline command to display the sorted contents (in reverse alphabetical order) of the “linux” directory. Use a relative pathname.
7. Assume that the text file called “.answers.txt” contains 10 lines. Write a single Linux pipeline command to only displays lines 5 through 8 for this file. Use only relative pathnames.
8. Write a single Linux pipeline command to only display the contents of the “assignments” directory whose filenames match the pattern “murray” (both upper or lowercase). Use an absolute pathname.
9. Write a single Linux pipeline command to display the number of characters contained in the file called “.answers.txt”. Use a relative-to-home pathname.
10. Write a single Linux pipeline command to display the number of lines contained in the file called “questions.txt”. Use a relative pathname.
11. Write a single Linux pipeline command to display only the first 10 characters of each filename contained in your current directory. Also, there is will be a lot of output, so also pause at each screenful so you can navigate throughout the display contents. Use a relative pathname.
12. Create a table listing each Linux command, useful options that were mentioned in this tutorial for the following Linux commands: cut , tr , wc , and tee.