384
edits
Changes
import from 435
= LAB OBJECTIVES =
:0. Review SSH setup and SSH remote shell command execution
:1. Explore the Fabric Python library (Fabric API) and its command line tool "fab".
:2. Create Fabric scripts utilizing Fabric's API and environment objects to define tasks for the '''fab''' command.
:3. Use the '''fab''' command to execute fabric script to perform pre-defined tasks on remote Linux machines.
== Prerequisites ==
:# Has the user account named "student" and password for your VM with access port on myvmlab.senecacollege.ca
:# Regular account on Matrix.senecacollege.ca
== Overview ==
: Fabric is a Python library and command-line tool for streamlining the use of SSH for application deployment or system administration tasks. It has two major components:
:# a command-line interface program called "fab" which lets you execute arbitrary Python functions on local and remote machines.
:# a set of Python APIs that you can use and call in your Python functions to make executing shell commands over SSH much easier.
: We are going to use the Fabric API to define tasks and use the '''fab''' command to execute those tasks on one or more remote Linux machines.
== REFERENCE ==
:1. These links are helpful for learning more about Fabric's features:
{| class="wikitable" | style="margin-left:20px; border: 2px solid black;"
|- style="border: 2px solid black;font-weight:bold;text-align:center;"
| style="border: 2px solid black;" | Category
| style="border: 2px solid black;" | Resource Link
|- style="background-color:white;border:none;"
| style="border: 2px solid black;" valign="top"|
:Official Fabric tutorial
| style="border: 2px solid black;" valign="top"|
:[http://docs.fabfile.org/en/1.13/tutorial.html]
|- style="background-color:white;border:none;"
| style="border: 2px solid black;" valign="top"|
:Better Fabric tutorial
| style="border: 2px solid black;" valign="top"|
:[https://www.digitalocean.com/community/tutorials/how-to-use-fabric-to-automate-administration-tasks-and-deployments]
|- style="background-color:white;border:none;"
| style="border: 2px solid black;" valign="top"|
:Official '''Fabric''' website
| style="border: 2px solid black;" valign="top"|
:[http://www.fabfile.org/]
|}
: <font color='blue'>Please note that the version of Fabric currently installed on matrix.senecacollege.ca 1.14.0 and it supports only Python version 2.</font> The Fabric script files we are going to create in this lab have to meet Python version 2.x requirements. (e.g. print is a keyword, not a built-in function in Python 2.x)
:2. You should have some experience on the following topics in OPS235 and or OPS335. Please review them to prepare for the activities in this lab:
:* create and configure a regular user on a Linux system.
:* configure and manage sudo privilege for a regular user
:* Configure sudoers using the visudo command
:* using the yum command to install, remove, and update rpm packages
= INVESTIGATION 1: The Fabric Environment =
: The Fabric environment consists of the following components:
:# Controller workstation - the machine that has the Fabric package installed and runs the "fab" command
:## the Fabric Python Library (aka the fabric api)- the fabric package (already installed on matrix)
:## the Fabric command - '''fab''': runs Fabric script, name of the script is default to fabfile.py in the current working directory unless specified otherwise with the '-f' option.
:## Fabric script: contains fabric environment object value and Python functions (or tasks) to be executed by the '''fab''' command.
:# Remote machine: the target machine on which one or more Fabric tasks will be executed.
:## running the ssh server daemon
:## use public key (or password based) authentication for ssh connections
== PART 1 - Configure and test your controller workstation ==
: In this lab you will use your login account on matrix.senecacollege.ca as your Fabric controller workstation.
: The Fabric package version 1.14.0 has already been installed on matrix.senecacollege.ca. You should have access to the '''fab''' command on matrix. Login to matrix.senecacollege.ca and run the following command to confirm the version of the fabric package:<source lang='bash'>
fab --version
</source>
: Type the following command to get the command line options of the fab command:<source lang='bash'>
fab --help
</source>
:You should get something similar to the following:<source lang="bash">
Usage: fab [options] <command>[:arg1,arg2=val2,host=foo,hosts='h1;h2',...] ...
Options:
-h, --help show this help message and exit
-d NAME, --display=NAME
print detailed info about command NAME
-F FORMAT, --list-format=FORMAT
formats --list, choices: short, normal, nested
-I, --initial-password-prompt
Force password prompt up-front
--initial-sudo-password-prompt
Force sudo password prompt up-front
-l, --list print list of possible commands and exit
--set=KEY=VALUE,... comma separated KEY=VALUE pairs to set Fab env vars
--shortlist alias for -F short --list
-V, --version show program's version number and exit
-a, --no_agent don't use the running SSH agent
-A, --forward-agent forward local agent to remote end
--abort-on-prompts abort instead of prompting (for password, host, etc)
-c PATH, --config=PATH
specify location of config file to use
--colorize-errors Color error output
-D, --disable-known-hosts
do not load user known_hosts file
-e, --eagerly-disconnect
disconnect from hosts as soon as possible
-f PATH, --fabfile=PATH
python module file to import, e.g. '../other.py'
-g HOST, --gateway=HOST
gateway host to connect through
--gss-auth Use GSS-API authentication
--gss-deleg Delegate GSS-API client credentials or not
--gss-kex Perform GSS-API Key Exchange and user authentication
--hide=LEVELS comma-separated list of output levels to hide
-H HOSTS, --hosts=HOSTS
comma-separated list of hosts to operate on
-i PATH path to SSH private key file. May be repeated.
-k, --no-keys don't load private key files from ~/.ssh/
--keepalive=N enables a keepalive every N seconds
--linewise print line-by-line instead of byte-by-byte
-n M, --connection-attempts=M
make M attempts to connect before giving up
--no-pty do not use pseudo-terminal in run/sudo
-p PASSWORD, --password=PASSWORD
password for use with authentication and/or sudo
-P, --parallel default to parallel execution method
--port=PORT SSH connection port
-r, --reject-unknown-hosts
reject unknown hosts
--sudo-password=SUDO_PASSWORD
password for use with sudo only
--system-known-hosts=SYSTEM_KNOWN_HOSTS
load system known_hosts file before reading user
known_hosts
-R ROLES, --roles=ROLES
comma-separated list of roles to operate on
-s SHELL, --shell=SHELL
specify a new shell, defaults to '/bin/bash -l -c'
--show=LEVELS comma-separated list of output levels to show
--skip-bad-hosts skip over hosts that can't be reached
--skip-unknown-tasks skip over unknown tasks
--ssh-config-path=PATH
Path to SSH config file
-t N, --timeout=N set connection timeout to N seconds
-T N, --command-timeout=N
set remote command timeout to N seconds
-u USER, --user=USER username to use when connecting to remote hosts
-w, --warn-only warn, instead of abort, when commands fail
-x HOSTS, --exclude-hosts=HOSTS
comma-separated list of hosts to exclude
-z INT, --pool-size=INT
number of concurrent processes to use in parallel mode
</source>
<font color='green'><b>Please pay special attention and study the following command-line options for '''fab''' as they will be used in some of the activities in this lab:
:# -H,
:# -f,
:# -i,
:# -l,
:# --port
:# --user
:# --initial-sudo-password-prompt
</b></font>
== PART 2: Connect to VM in myvmlab.senecacollege.ca ==
: You should have received an email from ITS containing the following information:
:* account name: (usually 'student')
:* password: (let's assume it is 'P@ssw0rd' for the following instruction in this lab)
:* port number for SSH access via myvmlab.senecacollege.ca (e.g. 7200)
: This VM will be used as the remote Linux machine in our Fabric environment. Login to matrix and try the following SSH command to test the connectivity between matrix and your assigned VM:
<pre>
[raymond.chan@mtrx-node05pd lab8]$ ssh -p 7200 student@myvmlab.senecacollege.ca
student@myvmlab.senecacollege.ca's password: [type the password for the student user here]
Last login: Fri Jul 3 11:06:24 2020 from mtrx-node05pd.dcm.senecacollege.ca
</pre>
: Once you are on your VM, try the following commands: hostname, id, and df, and record the results for later comparison with the results of other commands:<source lang='bash'>
[student@centos7 ~]$ hostname
centos7
[student@centos7 ~]$ id
uid=1002(student) gid=1002(student) groups=1002(student),10(wheel)
[student@centos7 ~]$ df
Filesystem 1K-blocks Used Available Use% Mounted on
devtmpfs 878260 0 878260 0% /dev
tmpfs 889792 0 889792 0% /dev/shm
tmpfs 889792 9492 880300 2% /run
tmpfs 889792 0 889792 0% /sys/fs/cgroup
/dev/mapper/centos-root 38680112 1745524 36934588 5% /
/dev/sda2 1038336 331228 707108 32% /boot
/dev/sda1 204580 11296 193284 6% /boot/efi
/dev/mapper/centos-home 18880512 33160 18847352 1% /home
tmpfs 177960 0 177960 0% /run/user/1002
</source>
:Logout from your VM and get back to matrix.
:The previous SSH command when executed successfully, created a login shell on the remote machine. If the previous SSH command is followed by a specific bash command, it will be executed on the remote host instead of creating a login shell. Consider the following:<source lang='bash'>
[raymond.chan@mtrx-node05pd lab8]$ ssh -p 7200 student@myvmlab.senecacollege.ca 'hostname;id;df'
student@myvmlab.senecacollege.ca's password:
centos7
uid=1002(student) gid=1002(student) groups=1002(student),10(wheel)
Filesystem 1K-blocks Used Available Use% Mounted on
devtmpfs 878260 0 878260 0% /dev
tmpfs 889792 0 889792 0% /dev/shm
tmpfs 889792 9492 880300 2% /run
tmpfs 889792 0 889792 0% /sys/fs/cgroup
/dev/mapper/centos-root 38680112 1745608 36934504 5% /
/dev/sda2 1038336 331228 707108 32% /boot
/dev/sda1 204580 11296 193284 6% /boot/efi
/dev/mapper/centos-home 18880512 33160 18847352 1% /home
tmpfs 177960 0 177960 0% /run/user/1002
</source>
:The three shell commands: hostname, id, and df were executed sequentially. Compare the outputs above with the previous results when executing the corresponding commands in the login shell.
:Please note that your VM was configured to asked for the user's password for every SSH connection attemp. We are going to change that behaviour next.
==PART 3: Set up SSH login with public key authentication ==
: In order for you to run multiple tasks on multiple remote machines without typing the in the password for each connection, you need to configure your VM to accept SSH public key authentication in addtion to password authentication. You've done this in both OPS235 and OPS335, and here is a summary of how to do it between your account on matrix and your VM:
: Create a new SSH key pair (one private, and one public) under your account on matrix.senecacollege.ca.
: Once you have both keys, you can use the '''ssh-copy-id''' command to copy your public key to the student account on your VM, replace the port number with the correct value for your VM:<source lang='bash'>
ssh-copy-id -i ~/.ssh/id_rsa.pub -p 7200 student@myvmlab.senecacollege.ca
</source>
: The above command should add the contents of your pub key to ~/.ssh/authorized_keys under your student account on your VM. [Note: If you want to setup another controller workstation, you can either copy to '''private key''' to it, or generate another SSH key pair, and copy the '''public key''' to the VM.]
: Verify and confirm that your account on matrix can SSH to your VM as 'student' without being prompted for a password:<source lang='bash'>
[raymond.chan@mtrx-node05pd lab8]$ ssh -p 7200 student@myvmlab.senecacollege.ca
Last login: Fri Jul 3 12:46:19 2020 from mtrx-node05pd.dcm.senecacollege.ca
[student@centos7 ~]$ exit
logout
Connection to myvmlab.senecacollege.ca closed.
[raymond.chan@mtrx-node05pd lab8]$ ssh -p 7200 student@myvmlab.senecacollege.ca 'date;hostname;id'
Fri Jul 3 12:55:22 EDT 2020
centos7
uid=1002(student) gid=1002(student) groups=1002(student),10(wheel)
[raymond.chan@mtrx-node05pd lab8]$
</source>
: If you got similar result as shown above, you have successfully configured your controller workstation and your VM to use public key authentication.
=INVESTIGATION 2 - Running the fab command in ad-hoc mode =
: The fab command relies on SSH to make the connection to the remote machine before executing the intended commands. The fab command can run in ad-hoc mode:<source lang='bash'>
fab [options] -- [shell commands]
</source>
: When running the fab command in ad-hoc mode, it is very similar to running the SSH with commands attached at the end.
== PART 1: running non-privileged shell commands on remote machines ==
: In the following example, we use the '''fab''' to execute the "date", "hostname", and "id" command remotely on our VM. Try the following ad-hoc fab commands and record their results for later use, replace the port number with the correct value for your VM:<source lang='bash'>
[raymond.chan@mtrx-node05pd lab8]$ fab --host=myvmlab.senecacollege.ca --port=7200 --user=student -- 'date;hostname;id'
[myvmlab.senecacollege.ca] Executing task '<remainder>'
[myvmlab.senecacollege.ca] run: date;hostname;id
[myvmlab.senecacollege.ca] out: Fri Jul 3 13:05:39 EDT 2020
[myvmlab.senecacollege.ca] out: centos7
[myvmlab.senecacollege.ca] out: uid=1002(student) gid=1002(student) groups=1002(student),10(wheel)
[myvmlab.senecacollege.ca] out:
Done.
Disconnecting from myvmlab.senecacollege.ca:7200... done.
[raymond.chan@mtrx-node05pd lab8]$
</source>
: Note that there is no password prompting if you complete part 3 successfully, otherwise, the SSH server daemon on your VM will prompt you for a password. The output from the fab's ad-hoc mode is not much different from the SSH command with shell command attached at the end, however, please note that the additional information on the output from the fab command can be very useful for record keeping purpose - what has been done and whether the commands had been carried out successfully or not.
== PART 2: running privileged commands on remote machines ==
: <font color='red'><b>**WARNING** Run privileged commands incorrectly with sudo may cause irreparable damage to your remote machine.</b></font>
: We say that running an ad-hoc fab command is very similar to the SSH command with shell commands attached at the end. Let's try both with privileged commands, like the "yum" command.
=== Run the "yum" command on remote machine with SSH ===
: By default, your VM doesn't have the "tree" rpm package installed. You can verify this with the following SSH command (remember to replace the port number with the correct value for your VM):<source lang='bash'>
[raymond.chan@mtrx-node05pd lab8]$ ssh -p 7200 student@myvmlab.senecacollege.ca "yum list tree"
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: centos.mirror.colo-serv.net
* extras: centos.mirror.colo-serv.net
* updates: centos.mirror.ca.planethoster.net
Available Packages
tree.x86_64 1.6.0-10.el7 base
[raymond.chan@mtrx-node05pd lab8]$
</source>
: Please note that the tree package is "Available", but not yet installed.
: Let't try to install the "tree" package with the shell command "yum install tree -y":<source lang='bash'>
[raymond.chan@mtrx-node05pd lab8]$ ssh -p student@myvmlab.senecacollege.ca "yum install tree -y"
Loaded plugins: fastestmirror
You need to be root to perform this command.
</source>
: Using the "yum" command to query rpm package doesn't need special privilege, however, it does when you try to install or remove rpm packages.
: Your "student" account on your VM was configured to allow you to run the "sudo" command to perform software management using the "yum" command. Let's login to your VM and try the following "sudo" command to install and then remove the "tree" rpm package:<source lang='bash'>
[raymond.chan@mtrx-node05pd lab8]$ ssh -p 7200 student@myvmlab.senecacollege.ca
Last login: Fri Jul 3 16:51:07 2020 from mtrx-node05pd.dcm.senecacollege.ca
[student@centos7 ~]$ sudo yum install tree -y
[sudo] password for student:
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: less.cogeco.net
* extras: centos.mirror.colo-serv.net
* updates: mirror.calgah.com
Resolving Dependencies
--> Running transaction check
---> Package tree.x86_64 0:1.6.0-10.el7 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
========================================================================================================================
Package Arch Version Repository Size
========================================================================================================================
Installing:
tree x86_64 1.6.0-10.el7 base 46 k
Transaction Summary
========================================================================================================================
Install 1 Package
Total download size: 46 k
Installed size: 87 k
Downloading packages:
tree-1.6.0-10.el7.x86_64.rpm | 46 kB 00:00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : tree-1.6.0-10.el7.x86_64 1/1
Verifying : tree-1.6.0-10.el7.x86_64 1/1
Installed:
tree.x86_64 0:1.6.0-10.el7
Complete!
[student@centos7 ~]$
</source>
: Please note that when you run the "sudo" command the first time, it asks you for the user's password (i.e. user student's password). Let's now remove the "tree" package:<source lang='bash'>
[student@centos7 ~]$ yum remove tree -y
Loaded plugins: fastestmirror
You need to be root to perform this command.
[student@centos7 ~]$ sudo yum remove tree -y
Loaded plugins: fastestmirror
Resolving Dependencies
--> Running transaction check
---> Package tree.x86_64 0:1.6.0-10.el7 will be erased
--> Finished Dependency Resolution
Dependencies Resolved
========================================================================================================================
Package Arch Version Repository Size
========================================================================================================================
Removing:
tree x86_64 1.6.0-10.el7 @base 87 k
Transaction Summary
========================================================================================================================
Remove 1 Package
Installed size: 87 k
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Erasing : tree-1.6.0-10.el7.x86_64 1/1
Verifying : tree-1.6.0-10.el7.x86_64 1/1
Removed:
tree.x86_64 0:1.6.0-10.el7
Complete!
[student@centos7 ~]$
</source>
: The above tests confirmed that the student user is allowed to execute the sudo command to run the yum command to install and remove rpm packages. Now let's logout from the VM and go back to matrix. On matrix, try to run the sudo command using SSH:<source lang='bash'>
[student@centos7 ~]$ exit
logout
Connection to myvmlab.senecacollege.ca closed.
[raymond.chan@mtrx-node05pd lab8]$ ssh -p 7211 student@myvmlab.senecacollege.ca "sudo yum install tree -y"
sudo: no tty present and no askpass program specified
[raymond.chan@mtrx-node05pd lab8]$
</source>
: The above error indicated that you need a tty for the SSH session to prompt you for the sudo password. Please look up the ssh man page to find out the option which turn on a tty for the SSH session.
=== Run the privileged yum command on remote machine using ad-hoc fab command ===
: Let's try the corresponding ad-hoc fab command on your VM:<source lang='bash'>
fab --host=myvmlab.senecacollege.ca --port=7200 --user=student -- 'sudo yum install tree -y'
</source>
: Type in your user student's password when prompted for "sudo password", the '''yum install''' command should be executed successfully and install the '''tree''' rpm package. If the tree rpm package is already installed, you can remove it with the following ad-hoc fab command: <source lang='bash'>
fab --host=myvmlab.senecacollege.ca --port=7200 --user=student -- 'sudo yum remove tree -y'
</source>
= INVESTIGATION 3: Running the fab command in script mode =
: From investigation 2, we can see that running '''fab''' in ad-hoc mode is quick, straight forward, and easy. However, the rich output generated can not be easily captured and processed. If you have a need to capture and process the output generated by the commands executed on the remote machines, the solution is to run the '''fab''' command in script mode.
: The first step in running the '''fab''' command in script mode is to create a fabric script file.
: Let's start with a simple fabric script file to demonstrate some basic concepts that use the API from the Fabric python library.
: On matrix, cd to your lab8 directory and create a simple fabric script file named '''fabfile.py''' (this is the default filename used by the fab command when you invoke it without the '-f' optino):
== PART 1: Non-privileged task example ==
===Create non-privileged tasks: Getting the hostname of remote machines===
: Add the following contents to the default fabric script called "fabfile.py" in your lab8 directory:<source lang="python">
from fabric.api import *
# set the name of the user login to the remote host
env.user = 'student'
# Define the task to get the hostname of remote machines:
def getHostname():
name = run("hostname")
print("The host name is:",name)
</source>
: To check for syntax error in the fabric script, run the following command in the lab8 directory where it contains the fabric script named "fabfile.py":<source lang="bash">
fab -l
</source>
: you should get a list of tasks defined in your fabfile.py:<source lang="bash">
[rchan@centos7 lab8]$ fab -l
Available commands:
getHostname
</source>
: To perform the task of getHostname on your VM (replace with the actual port # for connecting to your VM), run the fab command on matrix:<source lang="bash">
[raymond.chan@mtrx-node05pd lab8]$ fab --hosts=myvmlab.senecacollege.ca --port=7200 getHostname
[myvmlab.senecacollege.ca] Executing task 'getHostname'
[myvmlab.senecacollege.ca] run: hostname
[myvmlab.senecacollege.ca] out: centos7
[myvmlab.senecacollege.ca] out:
The host name is: centos7
Done.
Disconnecting from myvmlab.senecacollege.ca:7200... done.
[raymond.chan@mtrx-node05pd lab8]$
</source>
: Notice that there is no need to specify the user name at the '''fab''' command line since we defined it in the fabric script file (env.user = 'student'). Also notice that we can capture the host name returned from the "hostname" command and print it out together with an descriptive text in a line.
:In the above executed '''fab''' command, the fab program imports the fabric script named "fabfile.py" and execute the getHostname function on the VM connect at port 7200 on myvmlab.senecacollege.ca. Note that the port number for your VM will likely has a different value.
: If you did all the setup right and you got a password prompt when execute the above command, read the prompt carefully and see who's password it was prompting you for. If it is not for the user student, verify that you have the following line in your fabfile.py and you can ssh to your VM as the user student without password:
:<source lang="python">env.user = 'student'</source>
: In the above output from the '''fab''' command, you have:
:* Lines with the FQDN of the remote machine you are working on.
:* Messages from the controller workstation (e.g. "Executing task...", and "run: ...").
:* Output from the remote machine ("out: ...")
:* Output generated on the controller workstation from your fab file (the print statement)
:You should get used to the above messages from the '''fab''' command. It's a lot of output but it's important to understand where each piece of information is coming from, so you are able to debug problems when they happen.
== PART 2: Privileged Tasks Examples ==
===Creat privileged tasks: install and remove rpm package on remote machines===
: Add the following two new functions to the end of the fabric script "fabfile.py" in your lab8 directory:<source lang='bash'>
def installPackage(pkg='dummy'):
cmd = 'yum install ' + pkg + ' -y'
status = sudo(cmd)
print(status)
def removePackage(pkg):
if pkg == '':
cmd = 'yum remove dummy -y'
else:
cmd = 'yum remove ' + pkg + ' -y'
status = sudo(cmd)
print(status)
</source>
: Note that both functions take one function argument in different ways. However, if no function argument is passed when calling the function, both will default to a string value of "dummy". Both functions call the sudo() from the fabric.api to execute the command contained in the "cmd" object on the remote machine via sudo.
: To check for any syntax error in your updated fabric script, run the following command in the same directory as the fabfile.py:<source lang='bash'>
fab -l
</source>
: You should get a list of tasks defined similar to the following:<source lang='bash'>
[raymond.chan@mtrx-node05pd lab8]$ fab -l
Available commands:
getHostname
installPackage
removePackage
[raymond.chan@mtrx-node05pd lab8]$
</source>
: If you only need to connect to the same remote machine, you can specify the host and port number in the fabfile.py to save some typing when executing the fab command. Add the following two lines after the env.user line in your fabfile.py:<source lang='bash'>
env.port = '7200' # <-- please replace with the actual value of your VM's port number
env.hosts =['myvmlab.senecacollege.ca']
</source>
: You can also store the user's password in this file so that it will respond to the "sudo password" prompt for sudo() call. It is not safe to do so as you can configure the sudo module on the remote machine not to ask for sudo password.
: Now you can run the fab command without the "--host" and "--port" option.
: Run the following two fab commands, note the results and compare their difference:<source lang='bash'>
fab installPackage
fab installPackage:tree
</source>
: Run the following two fab commands, note the results and compare their difference:<source lang='bash'>
fab removePackage
fab removePackage:tree
</source>
== Part 2: Create remote task for updating rpm packages ==
: Add a new function called "updatePackage" to your fabfile.py according to the following requirements:
:* Accept optional function argument as the rpm package name
:* If no function argument was given when called, default to all the packages installed
: The output of the updatePackage when executed, should produce similar output as shown below:
:1. Update a single package:<source lang='bash'>
fab updatePackage:tree
</source>
: Sample output:<source lang='bash'>
[raymond.chan@mtrx-node05pd lab8]$ fab updatePackage:tree
[myvmlab.senecacollege.ca] Executing task 'updatePackage'
[myvmlab.senecacollege.ca] sudo: yum update tree -y
[myvmlab.senecacollege.ca] out: sudo password:
[myvmlab.senecacollege.ca] out: Loaded plugins: fastestmirror
[myvmlab.senecacollege.ca] out: Loading mirror speeds from cached hostfile
[myvmlab.senecacollege.ca] out: * base: less.cogeco.net
[myvmlab.senecacollege.ca] out: * extras: centos.mirror.ca.planethoster.net
[myvmlab.senecacollege.ca] out: * updates: less.cogeco.net
[myvmlab.senecacollege.ca] out: No packages marked for update
[myvmlab.senecacollege.ca] out:
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: less.cogeco.net
* extras: centos.mirror.ca.planethoster.net
* updates: less.cogeco.net
No packages marked for update
Done.
Disconnecting from myvmlab.senecacollege.ca:7200... done.
[raymond.chan@mtrx-node05pd lab8]$
</source>
:2. Update all installed package:<source lang='bash'>
fab updatePackage:
</source>
: The following output had been trimmed, only showing the first few lines:<source lang='bash'>
[myvmlab.senecacollege.ca] Executing task 'updatePackage'
[myvmlab.senecacollege.ca] sudo: yum update -y
[myvmlab.senecacollege.ca] out: sudo password:
[myvmlab.senecacollege.ca] out: Loaded plugins: fastestmirror
[myvmlab.senecacollege.ca] out: Loading mirror speeds from cached hostfile
[myvmlab.senecacollege.ca] out: * base: less.cogeco.net
[myvmlab.senecacollege.ca] out: * extras: centos.mirror.ca.planethoster.net
[myvmlab.senecacollege.ca] out: * updates: less.cogeco.net
...
Verifying : systemd-219-73.el7_8.5.x86_64 53/54
Verifying : systemd-libs-219-73.el7_8.5.x86_64 54/54
Removed:
kernel.x86_64 0:3.10.0-862.el7
Installed:
kernel.x86_64 0:3.10.0-1127.13.1.el7
Updated:
bind-export-libs.x86_64 32:9.11.4-16.P2.el7_8.6 binutils.x86_64 0:2.27-43.base.el7_8.1
ca-certificates.noarch 0:2020.2.41-70.0.el7_8 device-mapper.x86_64 7:1.02.164-7.el7_8.2
device-mapper-event.x86_64 7:1.02.164-7.el7_8.2 device-mapper-event-libs.x86_64 7:1.02.164-7.el7_8.2
device-mapper-libs.x86_64 7:1.02.164-7.el7_8.2 kernel-tools.x86_64 0:3.10.0-1127.13.1.el7
kernel-tools-libs.x86_64 0:3.10.0-1127.13.1.el7 lvm2.x86_64 7:2.02.186-7.el7_8.2
lvm2-libs.x86_64 7:2.02.186-7.el7_8.2 microcode_ctl.x86_64 2:2.1-61.10.el7_8
net-snmp.x86_64 1:5.7.2-48.el7_8.1 net-snmp-agent-libs.x86_64 1:5.7.2-48.el7_8.1
net-snmp-libs.x86_64 1:5.7.2-48.el7_8.1 net-snmp-utils.x86_64 1:5.7.2-48.el7_8.1
ntp.x86_64 0:4.2.6p5-29.el7.centos.2 ntpdate.x86_64 0:4.2.6p5-29.el7.centos.2
python-perf.x86_64 0:3.10.0-1127.13.1.el7 rsyslog.x86_64 0:8.24.0-52.el7_8.2
selinux-policy.noarch 0:3.13.1-266.el7_8.1 selinux-policy-targeted.noarch 0:3.13.1-266.el7_8.1
systemd.x86_64 0:219-73.el7_8.8 systemd-libs.x86_64 0:219-73.el7_8.8
systemd-sysv.x86_64 0:219-73.el7_8.8 yum-plugin-fastestmirror.noarch 0:1.1.31-54.el7_8
Complete!
Done.
Disconnecting from myvmlab.senecacollege.ca:7200... done.
[raymond.chan@mtrx-node05pd lab8]$
</source>
= Optional Lab Exercise: Create a Fabric task called makeUser() =
: Study the Fabric API run(), sudo(), local(), and put() and utilize them to create a new Fabric task called makeUser()
: The makeUser() task should perform the following on a remote machine to:
:* create a new user called "ops445p" with home directory "/home/ops445p".
:* add the new user to the sudo group called "wheel".
:* from your instructor, get the ssh public key which is posted on the internet to your controller workstation.
:* add the ssh public key obtained from your instructor to the file named "authorized_keys" in the ~ops445p/.ssh directory on the remote machine. (Note: Make sure that you set the proper ownership and permissions on both the directory ~ops445p/.ssh and the file "~ops445p/.ssh/authorized_keys.)
:Add the makeUser() to your final version of fabfile.py.
:Run the new task makeUser() on your VM.
:Verify and confirm that your new makeUser() task is working correctly.
= LAB 8 SIGN-OFF (SHOW INSTRUCTOR) =
: Complete each part of the lab and upload the version of your fabfile.py to Blackboard by the due date.
[[Category:OPS445-Python]][[Category:rchan]]
:0. Review SSH setup and SSH remote shell command execution
:1. Explore the Fabric Python library (Fabric API) and its command line tool "fab".
:2. Create Fabric scripts utilizing Fabric's API and environment objects to define tasks for the '''fab''' command.
:3. Use the '''fab''' command to execute fabric script to perform pre-defined tasks on remote Linux machines.
== Prerequisites ==
:# Has the user account named "student" and password for your VM with access port on myvmlab.senecacollege.ca
:# Regular account on Matrix.senecacollege.ca
== Overview ==
: Fabric is a Python library and command-line tool for streamlining the use of SSH for application deployment or system administration tasks. It has two major components:
:# a command-line interface program called "fab" which lets you execute arbitrary Python functions on local and remote machines.
:# a set of Python APIs that you can use and call in your Python functions to make executing shell commands over SSH much easier.
: We are going to use the Fabric API to define tasks and use the '''fab''' command to execute those tasks on one or more remote Linux machines.
== REFERENCE ==
:1. These links are helpful for learning more about Fabric's features:
{| class="wikitable" | style="margin-left:20px; border: 2px solid black;"
|- style="border: 2px solid black;font-weight:bold;text-align:center;"
| style="border: 2px solid black;" | Category
| style="border: 2px solid black;" | Resource Link
|- style="background-color:white;border:none;"
| style="border: 2px solid black;" valign="top"|
:Official Fabric tutorial
| style="border: 2px solid black;" valign="top"|
:[http://docs.fabfile.org/en/1.13/tutorial.html]
|- style="background-color:white;border:none;"
| style="border: 2px solid black;" valign="top"|
:Better Fabric tutorial
| style="border: 2px solid black;" valign="top"|
:[https://www.digitalocean.com/community/tutorials/how-to-use-fabric-to-automate-administration-tasks-and-deployments]
|- style="background-color:white;border:none;"
| style="border: 2px solid black;" valign="top"|
:Official '''Fabric''' website
| style="border: 2px solid black;" valign="top"|
:[http://www.fabfile.org/]
|}
: <font color='blue'>Please note that the version of Fabric currently installed on matrix.senecacollege.ca 1.14.0 and it supports only Python version 2.</font> The Fabric script files we are going to create in this lab have to meet Python version 2.x requirements. (e.g. print is a keyword, not a built-in function in Python 2.x)
:2. You should have some experience on the following topics in OPS235 and or OPS335. Please review them to prepare for the activities in this lab:
:* create and configure a regular user on a Linux system.
:* configure and manage sudo privilege for a regular user
:* Configure sudoers using the visudo command
:* using the yum command to install, remove, and update rpm packages
= INVESTIGATION 1: The Fabric Environment =
: The Fabric environment consists of the following components:
:# Controller workstation - the machine that has the Fabric package installed and runs the "fab" command
:## the Fabric Python Library (aka the fabric api)- the fabric package (already installed on matrix)
:## the Fabric command - '''fab''': runs Fabric script, name of the script is default to fabfile.py in the current working directory unless specified otherwise with the '-f' option.
:## Fabric script: contains fabric environment object value and Python functions (or tasks) to be executed by the '''fab''' command.
:# Remote machine: the target machine on which one or more Fabric tasks will be executed.
:## running the ssh server daemon
:## use public key (or password based) authentication for ssh connections
== PART 1 - Configure and test your controller workstation ==
: In this lab you will use your login account on matrix.senecacollege.ca as your Fabric controller workstation.
: The Fabric package version 1.14.0 has already been installed on matrix.senecacollege.ca. You should have access to the '''fab''' command on matrix. Login to matrix.senecacollege.ca and run the following command to confirm the version of the fabric package:<source lang='bash'>
fab --version
</source>
: Type the following command to get the command line options of the fab command:<source lang='bash'>
fab --help
</source>
:You should get something similar to the following:<source lang="bash">
Usage: fab [options] <command>[:arg1,arg2=val2,host=foo,hosts='h1;h2',...] ...
Options:
-h, --help show this help message and exit
-d NAME, --display=NAME
print detailed info about command NAME
-F FORMAT, --list-format=FORMAT
formats --list, choices: short, normal, nested
-I, --initial-password-prompt
Force password prompt up-front
--initial-sudo-password-prompt
Force sudo password prompt up-front
-l, --list print list of possible commands and exit
--set=KEY=VALUE,... comma separated KEY=VALUE pairs to set Fab env vars
--shortlist alias for -F short --list
-V, --version show program's version number and exit
-a, --no_agent don't use the running SSH agent
-A, --forward-agent forward local agent to remote end
--abort-on-prompts abort instead of prompting (for password, host, etc)
-c PATH, --config=PATH
specify location of config file to use
--colorize-errors Color error output
-D, --disable-known-hosts
do not load user known_hosts file
-e, --eagerly-disconnect
disconnect from hosts as soon as possible
-f PATH, --fabfile=PATH
python module file to import, e.g. '../other.py'
-g HOST, --gateway=HOST
gateway host to connect through
--gss-auth Use GSS-API authentication
--gss-deleg Delegate GSS-API client credentials or not
--gss-kex Perform GSS-API Key Exchange and user authentication
--hide=LEVELS comma-separated list of output levels to hide
-H HOSTS, --hosts=HOSTS
comma-separated list of hosts to operate on
-i PATH path to SSH private key file. May be repeated.
-k, --no-keys don't load private key files from ~/.ssh/
--keepalive=N enables a keepalive every N seconds
--linewise print line-by-line instead of byte-by-byte
-n M, --connection-attempts=M
make M attempts to connect before giving up
--no-pty do not use pseudo-terminal in run/sudo
-p PASSWORD, --password=PASSWORD
password for use with authentication and/or sudo
-P, --parallel default to parallel execution method
--port=PORT SSH connection port
-r, --reject-unknown-hosts
reject unknown hosts
--sudo-password=SUDO_PASSWORD
password for use with sudo only
--system-known-hosts=SYSTEM_KNOWN_HOSTS
load system known_hosts file before reading user
known_hosts
-R ROLES, --roles=ROLES
comma-separated list of roles to operate on
-s SHELL, --shell=SHELL
specify a new shell, defaults to '/bin/bash -l -c'
--show=LEVELS comma-separated list of output levels to show
--skip-bad-hosts skip over hosts that can't be reached
--skip-unknown-tasks skip over unknown tasks
--ssh-config-path=PATH
Path to SSH config file
-t N, --timeout=N set connection timeout to N seconds
-T N, --command-timeout=N
set remote command timeout to N seconds
-u USER, --user=USER username to use when connecting to remote hosts
-w, --warn-only warn, instead of abort, when commands fail
-x HOSTS, --exclude-hosts=HOSTS
comma-separated list of hosts to exclude
-z INT, --pool-size=INT
number of concurrent processes to use in parallel mode
</source>
<font color='green'><b>Please pay special attention and study the following command-line options for '''fab''' as they will be used in some of the activities in this lab:
:# -H,
:# -f,
:# -i,
:# -l,
:# --port
:# --user
:# --initial-sudo-password-prompt
</b></font>
== PART 2: Connect to VM in myvmlab.senecacollege.ca ==
: You should have received an email from ITS containing the following information:
:* account name: (usually 'student')
:* password: (let's assume it is 'P@ssw0rd' for the following instruction in this lab)
:* port number for SSH access via myvmlab.senecacollege.ca (e.g. 7200)
: This VM will be used as the remote Linux machine in our Fabric environment. Login to matrix and try the following SSH command to test the connectivity between matrix and your assigned VM:
<pre>
[raymond.chan@mtrx-node05pd lab8]$ ssh -p 7200 student@myvmlab.senecacollege.ca
student@myvmlab.senecacollege.ca's password: [type the password for the student user here]
Last login: Fri Jul 3 11:06:24 2020 from mtrx-node05pd.dcm.senecacollege.ca
</pre>
: Once you are on your VM, try the following commands: hostname, id, and df, and record the results for later comparison with the results of other commands:<source lang='bash'>
[student@centos7 ~]$ hostname
centos7
[student@centos7 ~]$ id
uid=1002(student) gid=1002(student) groups=1002(student),10(wheel)
[student@centos7 ~]$ df
Filesystem 1K-blocks Used Available Use% Mounted on
devtmpfs 878260 0 878260 0% /dev
tmpfs 889792 0 889792 0% /dev/shm
tmpfs 889792 9492 880300 2% /run
tmpfs 889792 0 889792 0% /sys/fs/cgroup
/dev/mapper/centos-root 38680112 1745524 36934588 5% /
/dev/sda2 1038336 331228 707108 32% /boot
/dev/sda1 204580 11296 193284 6% /boot/efi
/dev/mapper/centos-home 18880512 33160 18847352 1% /home
tmpfs 177960 0 177960 0% /run/user/1002
</source>
:Logout from your VM and get back to matrix.
:The previous SSH command when executed successfully, created a login shell on the remote machine. If the previous SSH command is followed by a specific bash command, it will be executed on the remote host instead of creating a login shell. Consider the following:<source lang='bash'>
[raymond.chan@mtrx-node05pd lab8]$ ssh -p 7200 student@myvmlab.senecacollege.ca 'hostname;id;df'
student@myvmlab.senecacollege.ca's password:
centos7
uid=1002(student) gid=1002(student) groups=1002(student),10(wheel)
Filesystem 1K-blocks Used Available Use% Mounted on
devtmpfs 878260 0 878260 0% /dev
tmpfs 889792 0 889792 0% /dev/shm
tmpfs 889792 9492 880300 2% /run
tmpfs 889792 0 889792 0% /sys/fs/cgroup
/dev/mapper/centos-root 38680112 1745608 36934504 5% /
/dev/sda2 1038336 331228 707108 32% /boot
/dev/sda1 204580 11296 193284 6% /boot/efi
/dev/mapper/centos-home 18880512 33160 18847352 1% /home
tmpfs 177960 0 177960 0% /run/user/1002
</source>
:The three shell commands: hostname, id, and df were executed sequentially. Compare the outputs above with the previous results when executing the corresponding commands in the login shell.
:Please note that your VM was configured to asked for the user's password for every SSH connection attemp. We are going to change that behaviour next.
==PART 3: Set up SSH login with public key authentication ==
: In order for you to run multiple tasks on multiple remote machines without typing the in the password for each connection, you need to configure your VM to accept SSH public key authentication in addtion to password authentication. You've done this in both OPS235 and OPS335, and here is a summary of how to do it between your account on matrix and your VM:
: Create a new SSH key pair (one private, and one public) under your account on matrix.senecacollege.ca.
: Once you have both keys, you can use the '''ssh-copy-id''' command to copy your public key to the student account on your VM, replace the port number with the correct value for your VM:<source lang='bash'>
ssh-copy-id -i ~/.ssh/id_rsa.pub -p 7200 student@myvmlab.senecacollege.ca
</source>
: The above command should add the contents of your pub key to ~/.ssh/authorized_keys under your student account on your VM. [Note: If you want to setup another controller workstation, you can either copy to '''private key''' to it, or generate another SSH key pair, and copy the '''public key''' to the VM.]
: Verify and confirm that your account on matrix can SSH to your VM as 'student' without being prompted for a password:<source lang='bash'>
[raymond.chan@mtrx-node05pd lab8]$ ssh -p 7200 student@myvmlab.senecacollege.ca
Last login: Fri Jul 3 12:46:19 2020 from mtrx-node05pd.dcm.senecacollege.ca
[student@centos7 ~]$ exit
logout
Connection to myvmlab.senecacollege.ca closed.
[raymond.chan@mtrx-node05pd lab8]$ ssh -p 7200 student@myvmlab.senecacollege.ca 'date;hostname;id'
Fri Jul 3 12:55:22 EDT 2020
centos7
uid=1002(student) gid=1002(student) groups=1002(student),10(wheel)
[raymond.chan@mtrx-node05pd lab8]$
</source>
: If you got similar result as shown above, you have successfully configured your controller workstation and your VM to use public key authentication.
=INVESTIGATION 2 - Running the fab command in ad-hoc mode =
: The fab command relies on SSH to make the connection to the remote machine before executing the intended commands. The fab command can run in ad-hoc mode:<source lang='bash'>
fab [options] -- [shell commands]
</source>
: When running the fab command in ad-hoc mode, it is very similar to running the SSH with commands attached at the end.
== PART 1: running non-privileged shell commands on remote machines ==
: In the following example, we use the '''fab''' to execute the "date", "hostname", and "id" command remotely on our VM. Try the following ad-hoc fab commands and record their results for later use, replace the port number with the correct value for your VM:<source lang='bash'>
[raymond.chan@mtrx-node05pd lab8]$ fab --host=myvmlab.senecacollege.ca --port=7200 --user=student -- 'date;hostname;id'
[myvmlab.senecacollege.ca] Executing task '<remainder>'
[myvmlab.senecacollege.ca] run: date;hostname;id
[myvmlab.senecacollege.ca] out: Fri Jul 3 13:05:39 EDT 2020
[myvmlab.senecacollege.ca] out: centos7
[myvmlab.senecacollege.ca] out: uid=1002(student) gid=1002(student) groups=1002(student),10(wheel)
[myvmlab.senecacollege.ca] out:
Done.
Disconnecting from myvmlab.senecacollege.ca:7200... done.
[raymond.chan@mtrx-node05pd lab8]$
</source>
: Note that there is no password prompting if you complete part 3 successfully, otherwise, the SSH server daemon on your VM will prompt you for a password. The output from the fab's ad-hoc mode is not much different from the SSH command with shell command attached at the end, however, please note that the additional information on the output from the fab command can be very useful for record keeping purpose - what has been done and whether the commands had been carried out successfully or not.
== PART 2: running privileged commands on remote machines ==
: <font color='red'><b>**WARNING** Run privileged commands incorrectly with sudo may cause irreparable damage to your remote machine.</b></font>
: We say that running an ad-hoc fab command is very similar to the SSH command with shell commands attached at the end. Let's try both with privileged commands, like the "yum" command.
=== Run the "yum" command on remote machine with SSH ===
: By default, your VM doesn't have the "tree" rpm package installed. You can verify this with the following SSH command (remember to replace the port number with the correct value for your VM):<source lang='bash'>
[raymond.chan@mtrx-node05pd lab8]$ ssh -p 7200 student@myvmlab.senecacollege.ca "yum list tree"
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: centos.mirror.colo-serv.net
* extras: centos.mirror.colo-serv.net
* updates: centos.mirror.ca.planethoster.net
Available Packages
tree.x86_64 1.6.0-10.el7 base
[raymond.chan@mtrx-node05pd lab8]$
</source>
: Please note that the tree package is "Available", but not yet installed.
: Let't try to install the "tree" package with the shell command "yum install tree -y":<source lang='bash'>
[raymond.chan@mtrx-node05pd lab8]$ ssh -p student@myvmlab.senecacollege.ca "yum install tree -y"
Loaded plugins: fastestmirror
You need to be root to perform this command.
</source>
: Using the "yum" command to query rpm package doesn't need special privilege, however, it does when you try to install or remove rpm packages.
: Your "student" account on your VM was configured to allow you to run the "sudo" command to perform software management using the "yum" command. Let's login to your VM and try the following "sudo" command to install and then remove the "tree" rpm package:<source lang='bash'>
[raymond.chan@mtrx-node05pd lab8]$ ssh -p 7200 student@myvmlab.senecacollege.ca
Last login: Fri Jul 3 16:51:07 2020 from mtrx-node05pd.dcm.senecacollege.ca
[student@centos7 ~]$ sudo yum install tree -y
[sudo] password for student:
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: less.cogeco.net
* extras: centos.mirror.colo-serv.net
* updates: mirror.calgah.com
Resolving Dependencies
--> Running transaction check
---> Package tree.x86_64 0:1.6.0-10.el7 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
========================================================================================================================
Package Arch Version Repository Size
========================================================================================================================
Installing:
tree x86_64 1.6.0-10.el7 base 46 k
Transaction Summary
========================================================================================================================
Install 1 Package
Total download size: 46 k
Installed size: 87 k
Downloading packages:
tree-1.6.0-10.el7.x86_64.rpm | 46 kB 00:00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : tree-1.6.0-10.el7.x86_64 1/1
Verifying : tree-1.6.0-10.el7.x86_64 1/1
Installed:
tree.x86_64 0:1.6.0-10.el7
Complete!
[student@centos7 ~]$
</source>
: Please note that when you run the "sudo" command the first time, it asks you for the user's password (i.e. user student's password). Let's now remove the "tree" package:<source lang='bash'>
[student@centos7 ~]$ yum remove tree -y
Loaded plugins: fastestmirror
You need to be root to perform this command.
[student@centos7 ~]$ sudo yum remove tree -y
Loaded plugins: fastestmirror
Resolving Dependencies
--> Running transaction check
---> Package tree.x86_64 0:1.6.0-10.el7 will be erased
--> Finished Dependency Resolution
Dependencies Resolved
========================================================================================================================
Package Arch Version Repository Size
========================================================================================================================
Removing:
tree x86_64 1.6.0-10.el7 @base 87 k
Transaction Summary
========================================================================================================================
Remove 1 Package
Installed size: 87 k
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Erasing : tree-1.6.0-10.el7.x86_64 1/1
Verifying : tree-1.6.0-10.el7.x86_64 1/1
Removed:
tree.x86_64 0:1.6.0-10.el7
Complete!
[student@centos7 ~]$
</source>
: The above tests confirmed that the student user is allowed to execute the sudo command to run the yum command to install and remove rpm packages. Now let's logout from the VM and go back to matrix. On matrix, try to run the sudo command using SSH:<source lang='bash'>
[student@centos7 ~]$ exit
logout
Connection to myvmlab.senecacollege.ca closed.
[raymond.chan@mtrx-node05pd lab8]$ ssh -p 7211 student@myvmlab.senecacollege.ca "sudo yum install tree -y"
sudo: no tty present and no askpass program specified
[raymond.chan@mtrx-node05pd lab8]$
</source>
: The above error indicated that you need a tty for the SSH session to prompt you for the sudo password. Please look up the ssh man page to find out the option which turn on a tty for the SSH session.
=== Run the privileged yum command on remote machine using ad-hoc fab command ===
: Let's try the corresponding ad-hoc fab command on your VM:<source lang='bash'>
fab --host=myvmlab.senecacollege.ca --port=7200 --user=student -- 'sudo yum install tree -y'
</source>
: Type in your user student's password when prompted for "sudo password", the '''yum install''' command should be executed successfully and install the '''tree''' rpm package. If the tree rpm package is already installed, you can remove it with the following ad-hoc fab command: <source lang='bash'>
fab --host=myvmlab.senecacollege.ca --port=7200 --user=student -- 'sudo yum remove tree -y'
</source>
= INVESTIGATION 3: Running the fab command in script mode =
: From investigation 2, we can see that running '''fab''' in ad-hoc mode is quick, straight forward, and easy. However, the rich output generated can not be easily captured and processed. If you have a need to capture and process the output generated by the commands executed on the remote machines, the solution is to run the '''fab''' command in script mode.
: The first step in running the '''fab''' command in script mode is to create a fabric script file.
: Let's start with a simple fabric script file to demonstrate some basic concepts that use the API from the Fabric python library.
: On matrix, cd to your lab8 directory and create a simple fabric script file named '''fabfile.py''' (this is the default filename used by the fab command when you invoke it without the '-f' optino):
== PART 1: Non-privileged task example ==
===Create non-privileged tasks: Getting the hostname of remote machines===
: Add the following contents to the default fabric script called "fabfile.py" in your lab8 directory:<source lang="python">
from fabric.api import *
# set the name of the user login to the remote host
env.user = 'student'
# Define the task to get the hostname of remote machines:
def getHostname():
name = run("hostname")
print("The host name is:",name)
</source>
: To check for syntax error in the fabric script, run the following command in the lab8 directory where it contains the fabric script named "fabfile.py":<source lang="bash">
fab -l
</source>
: you should get a list of tasks defined in your fabfile.py:<source lang="bash">
[rchan@centos7 lab8]$ fab -l
Available commands:
getHostname
</source>
: To perform the task of getHostname on your VM (replace with the actual port # for connecting to your VM), run the fab command on matrix:<source lang="bash">
[raymond.chan@mtrx-node05pd lab8]$ fab --hosts=myvmlab.senecacollege.ca --port=7200 getHostname
[myvmlab.senecacollege.ca] Executing task 'getHostname'
[myvmlab.senecacollege.ca] run: hostname
[myvmlab.senecacollege.ca] out: centos7
[myvmlab.senecacollege.ca] out:
The host name is: centos7
Done.
Disconnecting from myvmlab.senecacollege.ca:7200... done.
[raymond.chan@mtrx-node05pd lab8]$
</source>
: Notice that there is no need to specify the user name at the '''fab''' command line since we defined it in the fabric script file (env.user = 'student'). Also notice that we can capture the host name returned from the "hostname" command and print it out together with an descriptive text in a line.
:In the above executed '''fab''' command, the fab program imports the fabric script named "fabfile.py" and execute the getHostname function on the VM connect at port 7200 on myvmlab.senecacollege.ca. Note that the port number for your VM will likely has a different value.
: If you did all the setup right and you got a password prompt when execute the above command, read the prompt carefully and see who's password it was prompting you for. If it is not for the user student, verify that you have the following line in your fabfile.py and you can ssh to your VM as the user student without password:
:<source lang="python">env.user = 'student'</source>
: In the above output from the '''fab''' command, you have:
:* Lines with the FQDN of the remote machine you are working on.
:* Messages from the controller workstation (e.g. "Executing task...", and "run: ...").
:* Output from the remote machine ("out: ...")
:* Output generated on the controller workstation from your fab file (the print statement)
:You should get used to the above messages from the '''fab''' command. It's a lot of output but it's important to understand where each piece of information is coming from, so you are able to debug problems when they happen.
== PART 2: Privileged Tasks Examples ==
===Creat privileged tasks: install and remove rpm package on remote machines===
: Add the following two new functions to the end of the fabric script "fabfile.py" in your lab8 directory:<source lang='bash'>
def installPackage(pkg='dummy'):
cmd = 'yum install ' + pkg + ' -y'
status = sudo(cmd)
print(status)
def removePackage(pkg):
if pkg == '':
cmd = 'yum remove dummy -y'
else:
cmd = 'yum remove ' + pkg + ' -y'
status = sudo(cmd)
print(status)
</source>
: Note that both functions take one function argument in different ways. However, if no function argument is passed when calling the function, both will default to a string value of "dummy". Both functions call the sudo() from the fabric.api to execute the command contained in the "cmd" object on the remote machine via sudo.
: To check for any syntax error in your updated fabric script, run the following command in the same directory as the fabfile.py:<source lang='bash'>
fab -l
</source>
: You should get a list of tasks defined similar to the following:<source lang='bash'>
[raymond.chan@mtrx-node05pd lab8]$ fab -l
Available commands:
getHostname
installPackage
removePackage
[raymond.chan@mtrx-node05pd lab8]$
</source>
: If you only need to connect to the same remote machine, you can specify the host and port number in the fabfile.py to save some typing when executing the fab command. Add the following two lines after the env.user line in your fabfile.py:<source lang='bash'>
env.port = '7200' # <-- please replace with the actual value of your VM's port number
env.hosts =['myvmlab.senecacollege.ca']
</source>
: You can also store the user's password in this file so that it will respond to the "sudo password" prompt for sudo() call. It is not safe to do so as you can configure the sudo module on the remote machine not to ask for sudo password.
: Now you can run the fab command without the "--host" and "--port" option.
: Run the following two fab commands, note the results and compare their difference:<source lang='bash'>
fab installPackage
fab installPackage:tree
</source>
: Run the following two fab commands, note the results and compare their difference:<source lang='bash'>
fab removePackage
fab removePackage:tree
</source>
== Part 2: Create remote task for updating rpm packages ==
: Add a new function called "updatePackage" to your fabfile.py according to the following requirements:
:* Accept optional function argument as the rpm package name
:* If no function argument was given when called, default to all the packages installed
: The output of the updatePackage when executed, should produce similar output as shown below:
:1. Update a single package:<source lang='bash'>
fab updatePackage:tree
</source>
: Sample output:<source lang='bash'>
[raymond.chan@mtrx-node05pd lab8]$ fab updatePackage:tree
[myvmlab.senecacollege.ca] Executing task 'updatePackage'
[myvmlab.senecacollege.ca] sudo: yum update tree -y
[myvmlab.senecacollege.ca] out: sudo password:
[myvmlab.senecacollege.ca] out: Loaded plugins: fastestmirror
[myvmlab.senecacollege.ca] out: Loading mirror speeds from cached hostfile
[myvmlab.senecacollege.ca] out: * base: less.cogeco.net
[myvmlab.senecacollege.ca] out: * extras: centos.mirror.ca.planethoster.net
[myvmlab.senecacollege.ca] out: * updates: less.cogeco.net
[myvmlab.senecacollege.ca] out: No packages marked for update
[myvmlab.senecacollege.ca] out:
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: less.cogeco.net
* extras: centos.mirror.ca.planethoster.net
* updates: less.cogeco.net
No packages marked for update
Done.
Disconnecting from myvmlab.senecacollege.ca:7200... done.
[raymond.chan@mtrx-node05pd lab8]$
</source>
:2. Update all installed package:<source lang='bash'>
fab updatePackage:
</source>
: The following output had been trimmed, only showing the first few lines:<source lang='bash'>
[myvmlab.senecacollege.ca] Executing task 'updatePackage'
[myvmlab.senecacollege.ca] sudo: yum update -y
[myvmlab.senecacollege.ca] out: sudo password:
[myvmlab.senecacollege.ca] out: Loaded plugins: fastestmirror
[myvmlab.senecacollege.ca] out: Loading mirror speeds from cached hostfile
[myvmlab.senecacollege.ca] out: * base: less.cogeco.net
[myvmlab.senecacollege.ca] out: * extras: centos.mirror.ca.planethoster.net
[myvmlab.senecacollege.ca] out: * updates: less.cogeco.net
...
Verifying : systemd-219-73.el7_8.5.x86_64 53/54
Verifying : systemd-libs-219-73.el7_8.5.x86_64 54/54
Removed:
kernel.x86_64 0:3.10.0-862.el7
Installed:
kernel.x86_64 0:3.10.0-1127.13.1.el7
Updated:
bind-export-libs.x86_64 32:9.11.4-16.P2.el7_8.6 binutils.x86_64 0:2.27-43.base.el7_8.1
ca-certificates.noarch 0:2020.2.41-70.0.el7_8 device-mapper.x86_64 7:1.02.164-7.el7_8.2
device-mapper-event.x86_64 7:1.02.164-7.el7_8.2 device-mapper-event-libs.x86_64 7:1.02.164-7.el7_8.2
device-mapper-libs.x86_64 7:1.02.164-7.el7_8.2 kernel-tools.x86_64 0:3.10.0-1127.13.1.el7
kernel-tools-libs.x86_64 0:3.10.0-1127.13.1.el7 lvm2.x86_64 7:2.02.186-7.el7_8.2
lvm2-libs.x86_64 7:2.02.186-7.el7_8.2 microcode_ctl.x86_64 2:2.1-61.10.el7_8
net-snmp.x86_64 1:5.7.2-48.el7_8.1 net-snmp-agent-libs.x86_64 1:5.7.2-48.el7_8.1
net-snmp-libs.x86_64 1:5.7.2-48.el7_8.1 net-snmp-utils.x86_64 1:5.7.2-48.el7_8.1
ntp.x86_64 0:4.2.6p5-29.el7.centos.2 ntpdate.x86_64 0:4.2.6p5-29.el7.centos.2
python-perf.x86_64 0:3.10.0-1127.13.1.el7 rsyslog.x86_64 0:8.24.0-52.el7_8.2
selinux-policy.noarch 0:3.13.1-266.el7_8.1 selinux-policy-targeted.noarch 0:3.13.1-266.el7_8.1
systemd.x86_64 0:219-73.el7_8.8 systemd-libs.x86_64 0:219-73.el7_8.8
systemd-sysv.x86_64 0:219-73.el7_8.8 yum-plugin-fastestmirror.noarch 0:1.1.31-54.el7_8
Complete!
Done.
Disconnecting from myvmlab.senecacollege.ca:7200... done.
[raymond.chan@mtrx-node05pd lab8]$
</source>
= Optional Lab Exercise: Create a Fabric task called makeUser() =
: Study the Fabric API run(), sudo(), local(), and put() and utilize them to create a new Fabric task called makeUser()
: The makeUser() task should perform the following on a remote machine to:
:* create a new user called "ops445p" with home directory "/home/ops445p".
:* add the new user to the sudo group called "wheel".
:* from your instructor, get the ssh public key which is posted on the internet to your controller workstation.
:* add the ssh public key obtained from your instructor to the file named "authorized_keys" in the ~ops445p/.ssh directory on the remote machine. (Note: Make sure that you set the proper ownership and permissions on both the directory ~ops445p/.ssh and the file "~ops445p/.ssh/authorized_keys.)
:Add the makeUser() to your final version of fabfile.py.
:Run the new task makeUser() on your VM.
:Verify and confirm that your new makeUser() task is working correctly.
= LAB 8 SIGN-OFF (SHOW INSTRUCTOR) =
: Complete each part of the lab and upload the version of your fabfile.py to Blackboard by the due date.
[[Category:OPS445-Python]][[Category:rchan]]
