Open main menu

CDOT Wiki β

Changes

OPS435 Python3 Lab 8

11,916 bytes added, 19:00, 4 July 2020
no edit summary
<font color='red'><u>'''This lab is currently being reviewed. The final version will be ready by November 11, 2019'''</u></font>
= LAB OBJECTIVES =
:0. Review SSH setup and remote shell execution:1. Use Explore the '''Fabric Python library and its command line tool "fab".:2. Create Fabric scripts utilizing Fabric''' program s API to execute administrative define tasks on remote host via Python functions under for the '''Fabricfab''' frameworkcommand.:23. Create python functions using Fabric API Use the '''fab''' command to execute fabric script to perform Linux system administrative pre-defined tasks on controlled remote Linux systemsmachines.
== Overview ==
: Completing this lab will give you Fabric is a taste Python library and command-line tool for streamlining the use of what is involved in automating remote SSH for application deployment or system/network administration tasks. It has two major components::# a command-line interface program called "fab" that lets you execute arbitrary Python functions :# a set of Python APIs that you can use and call in your Python functions to make executing shell commands over SSH much easier. : We will look at and are going use the Fabric API and its '''Fabricfab''' package command to define and execute Python functions (or tasks), to automate interactions with remote Linux machines in this lab. Using Fabric you can automate monitoring, deploying software, and updating many systems at the same time repeatedly.
== REFERENCE ==
| style="border: 2px solid black;" | Category
| style="border: 2px solid black;" | Resource Link
 
|- style="background-color:white;border:none;"
| style="border: 2px solid black;" valign="top"|
:Official '''Fabric''' website
| style="border: 2px solid black;" valign="top"|
:[http://www.fabfile.org/]
|- style="background-color:white;border:none;"
:[https://www.digitalocean.com/community/tutorials/how-to-use-fabric-to-automate-administration-tasks-and-deployments]
|- style="background-color:white;border:none;"
| style="border: 2px solid black;" valign="top"|
:Official '''Fabric''' website
| style="border: 2px solid black;" valign="top"|
:[http://www.fabfile.org/]
|}
: <font color='blue'>Please note that the version of Fabric installed on matrix.senecacollege.ca for this lab is 1.14 and it supports only Python version 2.</font> The Fabric script files we are going to create in this lab have to meet Python version 2.x requirements. (e.g. print is a keyword, not a built-in function in Python 2.x):2. You should have learned some experience on the following topics in OPS235 and or OPS335. Please review them to prepare for some of the tasks activities in this lab::* Configure create and allow configure a regular user to run the sudo commandon a Linux system. :* The man page on configure and manage sudoprivilege for a regular user:* Configure sudoers using the configuration file: /etc/sudoers.visudo command:* Managing critical system log files: /var/log/messagesusing the yum command to install, /var/log/maillogremove, /var/log/secureand update rpm packages
:* Retrieve current firewall setting using the iptables -L -n -v command
= INVESTIGATION 1: Extra VM Setup The Fabric Environment =
: In order to experience The Fabric environment consists of the following components: :# Controller workstation - the machine that has the Fabric package installed and runs the "fab" command:## the Fabric Python Library - the fabric package (already installed on matrix):## the Fabric API - fabric.api:## the Fabriccommand - '''fab''s features ': run Fabric script, name of the script is default to fabfile.py in a realistic way, wethe current working directory unless specified otherwise with the '-f're going option.:## Fabric script: contains fabric environment object value and Python functions (or tasks) to set up several virtual machines (You need at least be executed by the '''fab''' command.:# Remote machine: the target machine on which one or more VM)Fabric tasks will be executed. To begin with they are all going to have :## running the same configuration. Please make sure that each VM has direct network ssh server daemon:## use public key (or password based) authentication for ssh connection with other VMs you wish to control and configure.
== PART 1 - Set up Configure and test your controller workstation ==: In this lab you will use your login account on matrix.senecacollege.ca as your Fabric controller workstation.
: In this lab you will use your existing vm The Fabric package version 1.14.0 has already been installed on matrix.senecacollege.ca. You should have access to the '''centos7fab''' as a workstation command on matrix. Login to control other VMs which we'll call workersmatrix.senecacollege. Later in ca and run the lab, we will try following command to control and monitor your vm in myvmlab using confirm the version of the fabfile we are going to develop. fabric package: Install '''fabric''' using yum. Once it's installed you should have a '<source lang='bash'>fab''' command available. --version</source>: Type the following command to get the command line optionoptions of the fab command:<source lang='bash'>
fab --help
</source>
:You should get something similar to the following:<source lang="bash">
Usage: fab [options] <command>[:arg1,arg2=val2,host=foo,hosts='h1;h2',...] ...
number of concurrent processes to use in parallel mode
</source>
<font color='green'><b>Please note and study the following command-line options as they will be used in some of the activities in this lab::# -H, :# -f, :# -i, :# -l, and :# --port options.:# --user:# --initial-sudo-password-prompt </b></font>
== PART 2 : Connect to VM in myvmlab.senecacollege.ca ==: You should have received an email from ITS containing the following information::* account name: (usually 'student'):* password: (let's assume it is 'P@ssw0rd' for the following instruction in this lab):* port number for SSH access via myvmlab.senecacollege.ca (e.g. 7200): This VM will be used as the remote Linux machine in our Fabric environment. Login to matrix and try the following SSH command to test the connectivity between matrix and your assignment VM:<pre> [raymond.chan@mtrx-node05pd lab8]$ ssh -p 7200 student@myvmlab.senecacollege.ca student@myvmlab.senecacollege.ca's password: Last login: Fri Jul 3 11:06:24 2020 from mtrx-node05pd.dcm.senecacollege.ca</pre>: Once you are on your VM, try the following commands: hostname, id, and df, and record the results for later comparison with the results of other commands:<source lang='bash'>[student@centos7 ~]$ hostnamecentos7[student@centos7 ~]$ iduid=1002(student) gid=1002(student) groups=1002(student),10(wheel)[student@centos7 ~]$ dfFilesystem 1K-blocks Used Available Use% Mounted ondevtmpfs 878260 0 878260 0% /devtmpfs 889792 0 889792 0% /dev/shmtmpfs 889792 9492 880300 2% /runtmpfs 889792 0 889792 0% /sys/fs/cgroup/dev/mapper/centos- Create master Worker image root 38680112 1745524 36934588 5% //dev/sda2 1038336 331228 707108 32% /boot/dev/sda1 204580 11296 193284 6% /boot/efi/dev/mapper/centos-home 18880512 33160 18847352 1% /hometmpfs 177960 0 177960 0% /run/user/1002</source>:Logout from your VM and get back to matrix.:The previous SSH command when executed successfully, created a login shell on the remote machine. If the previous SSH command is followed by a specific bash command, it will be executed on the remote host instead of creating a login shell. Consider the following:<source lang='bash'>[raymond.chan@mtrx-node05pd lab8]$ ssh -p 7200 student@myvmlab.senecacollege.ca 'hostname;id;df'student@myvmlab.senecacollege.ca's password:centos7uid=1002(student) gid=1002(student) groups=1002(student),10(wheel)Filesystem 1K-blocks Used Available Use% Mounted ondevtmpfs 878260 0 878260 0% /devtmpfs 889792 0 889792 0% /dev/shmtmpfs 889792 9492 880300 2% /runtmpfs 889792 0 889792 0% /sys/fs/cgroup/dev/mapper/centos-root 38680112 1745608 36934504 5% //dev/sda2 1038336 331228 707108 32% /boot/dev/sda1 204580 11296 193284 6% /boot/efi/dev/mapper/centos-home 18880512 33160 18847352 1% /hometmpfs 177960 0 177960 0% /run/user/1002</source>:The three shell commands: hostname, id, and df were executed sequentially. Compare the outputs above with the previous results when executing the corresponding commands in the login shell.:Please note that you were asked to provide the user's password for every SSH connection.
==PART 3: Create a new virtual machineSet up SSH login with public key authentication ==: In order for your controller workstation to automate tasks execution on your VM, you need to configure your VM to SSH public key authentication instead of password authentication. You've done this in both OPS235 and OPS335, and allocate for here is a summary on how to do it 1GB of RAM between your account on matrix and 8GB of disk space. Install a Basic Web Server configuration of CentOS in that your VM using the same CentOS .iso file you used for your first machine in this course.:
: Make sure that::* The hostname of the system is '''worker1'''.:* It has a static IP address appropriate for your virtual network.:* Create a regular user using your Seneca email name as the user name: [seneca_id]. :* Add this new regular user to the '''wheel''' group using the following command:<source lang="bash"> usermod -a -G wheel [seneca_id]</source>This will allow the user to run the '''sudo''' command.:* After installation ensure that you can access '''worker1''' from your main vm using the static IP address you've assigned to it. === Set up SSH key login ===: In order for an automated system to be able to connect to your VM and administer it - you will need to be able to connect to it using SSH keys. You've done this in both OPS235 and OPS335. : Create a new SSH key pair (one private, and one public) under your account on your main VM with your '''regular user''' (don't do it under root)matrix.senecacollege.ca. : Once you have both keys, set things up so that:* your regular user on your you can use the '''controller VMssh-copy-id''' can SSH command to copy your public key to the worker student account on your VM as , replace the port number with the same regular user without prompting correct value for a password. (ie. add the contents of your pub key to VM:<source lang='bash'>ssh-copy-id -i ~/.ssh/authorized_keys)id_rsa.pub -p 7200 student@myvmlab.senecacollege.ca</source>:* your regular user on your '''controller VM''' can SSH to the worker VM as root without propmting for a password. (ie. The above command should add the contents of your pub key to /root~/.ssh/authorized_keys) == PART 3 - Clone the Workers ==: We're only simulating the real world where you'd have hundreds of VMs in one or more clouds, but you can just imagine that the VMs you're creating under your student account on your computer are actually being created on an Amazon or Microsoft serverVM. : '''** Optional **''' Make four clones of the master worker image you've just created. Then make sure that each of them has a unique IP address. That's all you're required to change manually. All the other configuration on the workers (inlcuding the hostnames) will be set by Fabric. Normally you would have some kind of automation doing all this cloning Verify and IP address assignment as well, but we don't have time for confirm that this semester. : Make snapshots of all your workers so that you account on matrix can easily restore them SSH to the original state after you modify them. = INVESTIGATION 2: Fabric practice =: We will start with some basics. Fabric runs python programs on the controller and the workers. You create an "instruction" file on your controller, and execute it on the controller using the VM as 'student'without prompting for a password:<source lang='fabbash''' program>[raymond. When you do that chan@mtrx-node05pd lab8]$ ssh - you specify which workers you want your instructions to be executed onp 7200 student@myvmlab.senecacollege.ca Last login: The instructions are stored in a python file. Let's start with a simple one named '''fabfile.py''' (the default filename used by fab without the '-f' optino)Fri Jul 3 12== PART 1: Simplest example =====Getting the hostname on the remote worker===46: <source lang="python">19 2020 from fabricmtrx-node05pd.dcm.senecacollege.api import *ca# set the name of the user on the remote hostenv.name = '[seneca_idstudent@centos7 ~]'$ exitlogout# Will get the hostname of this worker:Connection to myvmlab.senecacollege.ca closed.
def getHostname[raymond.chan@mtrx-node05pd lab8]$ ssh -p 7200 student@myvmlab.senecacollege.ca 'date;hostname;id'Fri Jul 3 12:55:22 EDT 2020centos7uid=1002(student): name gid=1002(student) groups= run1002("hostname"student) print,10(namewheel)[raymond.chan@mtrx-node05pd lab8]$
</source>
: To check for syntax errorIf you got similar result as above, run the following command in the same directory as you have successfully configure your fabfile.py:<source lang="bash">fab -l</source>: you should get a list of tasks stored in controller workstation and your fabfile.py:<source lang="bash">[rchan@centos7 lab8]$ fab -f fabfileVM to use public key authentication.py -lAvailable commands:
getHostname=INVESTIGATION 2 - Running the fab command in ad-hoc mode =: The fab command relies on SSH to make the connection to the remote machine before executing the intended commands. The fab command can run in ad-hoc mode:<source lang='bash'>fab [options] -- [shell commands]
</source>
: To perform When running the task of getHostname fab command in ad-hoc mode, it is very similar to running the SSH with commands attached at the end. == PART 1: running non-privileged shell commands on remote machines ==: In the worker machine 192.168.122.169following example, we run it use the '''fab''' to execute the "date", "hostname", and "id" command remotely on our VM. Try the following ad-hoc fab commands and record their results for later use, replace the port number with the controller machine like thiscorrect value for your VM:<source lang="'bash"'>[rchanraymond.chan@centos7 mtrx-node05pd lab8]$ fab -f fabfile.py -H 192host=myvmlab.168senecacollege.122.169. getHostnameca --port=7200 --user=student -- 'date;hostname;id'[192.168myvmlab.122senecacollege.169ca] Executing task 'getHostname<remainder>'[192.168myvmlab.122senecacollege.169ca] run: date;hostname;id[192myvmlab.senecacollege.168ca] out: Fri Jul 3 13:05:39 EDT 2020[myvmlab.122senecacollege.169ca] out: c7-rchancentos7[192myvmlab.168senecacollege.ca] out: uid=1002(student) gid=1002(student) groups=1002(student),10(wheel)[myvmlab.122senecacollege.169ca] out:
c7-rchan
Done.
Disconnecting from 192myvmlab.168senecacollege.122ca:7200.169..done. done[raymond.chan@mtrx-node05pd lab8]$
</source>
:All this has done Note that there is get no password prompting if you complete part 3 successfully, otherwise, the hostname of the worker and print it (SSH server daemon on the controller)your VM will prompt you for a password.:In The output from the command above wefab're using s ad-hoc mode is not much different from the fab program to import SSH command with shell command attached at the file fabfile.py and execute end, however, please note that the getHostname function additional information on the worker 192.168.122.169. Note that output from the IP address of your first worker will likely fab command can be differentvery useful for record keeping purpose - what has been done and whether the commands had been carried out successfully or not.
== PART 2: If you did all the setup right and you get a password prompt when execute running privileged commands on remote machines ==: We say that running an ad-hoc fab command is very similar to the above SSH command, read with shell commands attached at the prompt carefully and see whoend. Let's password it prompted you for. If it is not the same as your [seneca_id]try both with privileged commands, verify that you have like the following line in your fabfile and you can ssh to your worker vm without password:"yum" command.
=== Run the "yum" command on remote machine with SSH ===: By default, your VM doesn't have the "tree" rpm package installed. You can verify this with the following SSH command (remember to replace the port number with the correct value for your VM):<source lang='bash'>[raymond.chan@mtrx-node05pd lab8]$ ssh -p 7200 student@myvmlab.senecacollege.ca "pythonyum list tree"Loaded plugins: fastestmirrorLoading mirror speeds from cached hostfile * base: centos.mirror.colo-serv.net * extras: centos.mirror.colo-serv.net * updates: centos.mirror.ca.planethoster.netAvailable Packagestree.x86_64 1.6.0-10.el7 base[raymond.chan@mtrx-node05pd lab8]$</source>env: Please note that the tree package is "Available", but not yet installed.user : Let't try to install the "tree" package with the shell command "yum install tree -y":<source lang= 'bash'>[seneca_idraymond.chan@mtrx-node05pd lab8]$ ssh -p student@myvmlab.senecacollege.ca "yum install tree -y"Loaded plugins: fastestmirrorYou need to be root to perform this command.</source>: Using the "yum" command to query rpm package doesn't need special privilege, however, it does when you try to install or remove rpm packages. : Your "student" account on your VM was configured to allow you to run the "sudo" command to perform software management using the "yum" command. Let's login to your VM and try the following "sudo" command to install and then remove the "tree" rpm package:</sourcelang='bash'>[raymond.chan@mtrx-node05pd lab8]$ ssh -p 7200 student@myvmlab.senecacollege.caLast login: Fri Jul 3 16:51:07 2020 from mtrx-node05pd.dcm.senecacollege.ca[student@centos7 ~]$ sudo yum install tree -y[sudo] password for student:Loaded plugins: fastestmirrorLoading mirror speeds from cached hostfile * base: less.cogeco.net * extras: centos.mirror.colo-serv.net * updates: mirror.calgah.comResolving Dependencies--> Running transaction check---> Package tree.x86_64 0:1.6.0-10.el7 will be installed-->Finished Dependency Resolution
: In the above you have::* Lines with an IP address telling you which worker the output is for/from.:* Messages from the controller (e.g. "Executing task...", and "run: ...").:* Output from the worker ("out: ..."):* Output on the controller from your fab file ("worker1" which came from the "print()" call)Dependencies Resolved
======================================================================================================================== Package Arch Version Repository Size========================================================================================================================Installing:You should get used to the above tree x86_64 1. It's a lot of output but it's important to understand where every part is coming from, so you are able to debug problems when they happen6.0-10.el7 base 46 k
Transaction Summary== Part 2: Set up more administrative tasks======================================================================================================================Install 1 Package
Total download size:Let's pretend that we need collect the disk usage on several machines so that we can plan for storage maintenance46 kInstalled size: 87 kDownloading packages:tree-1.6.0-10.el7.x86_64.rpm | 46 kB 00:00:00Running transaction checkRunning transaction testTransaction test succeededRunning transaction Installing : tree-1.6.0-10.el7.x86_64 1/1 Verifying : tree-1.6.0-10. We'll set up a simple example of such a deployment hereel7.x86_64 1/1
=== Getting the disk usage on remote worker ===Installed: tree.x86_64 0:1.6.0-10.el7
:Add a getDiskUsage() function to your fabfile.py file:Complete! :<source lang="python"># to get the disk usage on remote workerdef getDiskUsage(): current_time = run('date') diskusage = run('df -H') header = 'Current Disk Usage at '+current_time print(header) print(diskusage)[student@centos7 ~]$
</source>
: Please note that when you run the "sudo" command the first time, it asks you for the user's password (i.e. user student's password). Let's now remote the "tree" package:<source lang='bash'>
[student@centos7 ~]$ yum remove tree -y
Loaded plugins: fastestmirror
You need to be root to perform this command.
[student@centos7 ~]$ sudo yum remove tree -y
Loaded plugins: fastestmirror
Resolving Dependencies
--> Running transaction check
---> Package tree.x86_64 0:1.6.0-10.el7 will be erased
--> Finished Dependency Resolution
:Note that each call to "run()" will run a command on the worker. In this function we get the date/time of the remote work, and then get the disk usage. The print() function print out both the values returned.Dependencies Resolved
======================================================================================================================== Package Arch Version Repository Size========================================================================================================================Removing:If you try to run it the same way as before: tree x86_64 1.6.0-10.el7 @base 87 k
<pre>$ fab --fabfileTransaction Summary=fabfile.py -H 192.168.122.169 getDiskUsage</pre>=======================================================================================================================Remove 1 Package
Installed size:You should get the following output87 kDownloading packages:<source lang="bash">[rchan@centos7 lab8]$ fab Running transaction checkRunning transaction testTransaction test succeededRunning transaction Erasing : tree--fabfile=fabfile1.6.py 0-H 192.16810.122el7.169 getDiskUsagex86_64 1/1[192.168.122.169] Executing task 'getDiskUsage'[192.168.122.169] run Verifying : date[192tree-1.1686.122.169] out: Sun Nov 0-10 13:17:16 EST 2019[192.168el7.122.169] out: x86_64 1/1
[192.168.122.169] runRemoved: df -H[192.168.122.169] out: Filesystem Size Used Avail Use% Mounted on[192.168.122.169] out: devtmpfs 947M 0 947M 0% /dev[192.168.122tree.169] out: tmpfs 964M x86_64 0 964M 0% /dev/shm[192.168.122.169] out: tmpfs 964M 91.7M 954M 2% /run[192.168.1226.169] out: tmpfs 964M 0 964M 0% /sys/fs/cgroup[192.168.122.169] out: /dev/mapper/centos-root 7.7G 5.6G 2.1G 73% /[192.168.122.169] out: /dev/vda1 1.1G 298M 766M 29% /boot[192.168.12210.169] out: tmpfs 193M 17k 193M 1% /run/user/42[192.168.122.169] out: tmpfs 193M 0 193M 0% /run/user/1000[192.168.122.169] out: el7
Current Disk Usage at Sun Nov 10 13:17:16 EST 2019Complete!Filesystem Size Used Avail Use% Mounted on[student@centos7 ~]$devtmpfs 947M 0 947M 0% </devsource>tmpfs 964M 0 964M 0% /dev/shmtmpfs 964M 9: The above tests confirm that the student user is allowed to run the yum command to install and remove rpm package. Now let's logout from the VM and go back to matrix.7M 954M 2% /On matrix, try to runthe sudo command using SSH:<source lang='bash'>[student@centos7 ~]$ exittmpfs 964M 0 964M 0% /sys/fs/cgrouplogoutConnection to myvmlab.senecacollege.ca closed./dev/mapper/centos[raymond.chan@mtrx-node05pd lab8]$ ssh -root 7p 7211 student@myvmlab.7G 5senecacollege.6G 2.1G 73% /ca "sudo yum install tree -y"sudo: no tty present and no askpass program specified/dev/vda1 1[raymond.1G 298M 766M 29% /bootchan@mtrx-node05pd lab8]$tmpfs 193M 17k 193M 1% </run/user/42source>tmpfs 193M 0 193M 0% /run/user/1000: The above error indicated that you need a tty for the SSH session to prompt you for the sudo password. Please look up the ssh man page to find out the option which turn on a tty for the SSH session.
Done.=== Run the privileged yum command on remote machine using ad-hoc fab command ===: Let's try the corresponding ad-hoc fab command on your VM:<source lang='bash'>Disconnecting from 192fab --host=myvmlab.168.122.169senecacollege.ca --port=7200 --user=student -- 'sudo yum install tree -y'</source>: Type in your user student's password when prompted for "sudo password", the yum install command to install the tree rpm package should be executed successfully.If the tree rpm package is already installed, you can remove it with the following ad-hoc fab command: <source lang='bash'>fab --host=myvmlab. donesenecacollege.ca --port=7200 --user=student -- 'sudo yum remove tree -y'
</source>
: Try remove the "tree" rpm package with the appropriate ad-hoc fab command.
= INVESTIGATION 3:YouRunning the fab command in script mode =: From investigation 2, we can see that running '''fab'''ll find that yum prompts in ad-hoc mode is quick, straight forward, and easy. However, the rich output generated can not be easily captured and processed. If you have a need to answer questionscapture and process the output generated by the commands executed on the remote machines, which you donthe solution is to run the '''fab'''t want to do command in an automated environmentscript mode. And also yum prints too much output, which also isn: The first step in running the '''fab'''t helpful command in an automated environmentscript mode is to create a fabric script file. We: Let'll fix it by adding two switches s start with a simple fabric script file to yumdemonstrate some basic concepts that use the API from the Fabric python library.: "-y" On matrix, cd to your lab8 directory and "create a simple fabric script file named '''fabfile.py''' (this is the default filename used by the fab command when you invoke it without the '-d1"f' optino):
== PART 1:Notice also that all Non-privileged task example =====Create non-privileged tasks: Getting the hostname of remote machines===: Add the four commands can be run as many times as you want, following contents to the result will be the samedefault fabric script called "fabfile. This is not always so easypy" in your lab8 directory:<source lang="python">from fabric.api import *
# set the name of the user login to the remote host
env.user = 'student'
# Define the task to get the hostname of remote machines:
def getHostname():
name = run("hostname")
print("The host name is:",name)
</source>
=== Update all : To check for syntax error in the fabric script, run the following command in the lab8 directory where it contains the rpm packages on remote worker ===:Let's pretend that we need to update software packages installed on several machines due to security patchesfabric script named "fabfile. Let's name the task as 'performSoftwareUpdate()'py":<source lang="pythonbash"># to perform software update on remote workerdef performSoftwareUpdate(): status = run('yum update fab -y') print(status)l
</source>
: Do you should get a syntax check with the "fab -l" commandlist of tasks defined in your fabfile.: When you try to run it the same way as before, you encounter some issue as shown belowpy:<source lang="bash">[rchan@centos7 lab8]$ fab --fabfile=fabfile.py -H 192.168.122.169 performSoftwareUpdate[192.168.122.169] Executing task 'performSoftwareUpdate'[192.168.122.169] run: yum update -yl[192.168.122.169] out: Loaded plugins: fastestmirror, langpacks[192.168.122.169] out: You need to be root to perform this command.[192.168.122.169] outAvailable commands:
 Fatal error: run() received nonzero return code 1 while executing! Requested: yum update -yExecuted: /bin/bash -l -c "yum update -y" Aborting.Disconnecting from 192.168.122.169... done. getHostname
</source>
: As you already know, you need superuser privilege in order to To perform software update the task of getHostname on a Linux system. There are two ways your VM (replace with the actual port # for connecting to do it your VM), run the fab command on Fabric. The first one is simple. Edit you fabfile.py and change the env.user line as shown belowmatrix:<source lang="pythonbash">[raymond.chan@mtrx-node05pd lab8]$ fab --hosts=myvmlab.senecacollege.ca --port=7200 getHostname[myvmlab.senecacollege.ca] Executing task 'getHostname'[myvmlab.senecacollege.ca] run: hostname[myvmlab.senecacollege.ca] out: centos7[myvmlab.senecacollege.ca] out:
env.user = 'root'The host name is: centos7
Done.
Disconnecting from myvmlab.senecacollege.ca:7200... done.
[raymond.chan@mtrx-node05pd lab8]$
</source>
: Save Notice that there is no need to specify the fabfile.py with user name at the change and run '''fab''' command line since we defined it againin the fabric script file (env.user = 'student').: If you see the password prompt again, make sure Also notice that you we can ssh capture the host name returned from your controller as the "hostname" command and print it out together with an descriptive text in a regular user to your worker vm as root without passwordline.
== Part 2: Set up In the firewall ==above executed '''fab''' command, the fab program imports the fabric script named "fabfile.py" and execute the getHostname function on the VM connect at port 7200 on myvmlab.senecacollege.ca. Note that the port number for your first will likely be of a different value.
: Recall that in our OPS courses we've been using iptables instead of firewalldIf you did all the setup right and you got a password prompt when execute the above command, which is installed by default in CentOS. Letread the prompt carefully and see who's make sure password it was prompting you for. If it is not for the user student, verify that our workers you have that set up as well. In the same '''following line in your fabfile.py''' and you've been using all along, add a new function like thiscan ssh to your VM as the user student without password:
: <source lang="python"># Will uninstall firewalld and replace it with iptablesdef setupFirewall(): run("yum -y -d1 remove firewalld") run("yum -y -d1 install iptables-services") run("systemctl enable iptables") run("systemctl start iptables")env.user = 'student'</source>
: That should by now look prett obvious. On In the above output from the worker you're going to uninstall firewalld''fab''' command, install iptablesyou have::* Lines with the FQDN of the remote machine you are working on.:* Messages from the controller workstation (e.g. "Executing task...", and make sure that "run: ...").:* Output from the iptables service is runningremote machine ("out: ..."):* Output generated on the controller workstation from your fab file (the print statement)
: Execute You should get used to the function for worker1 and double-check that above messages from the '''fab''' command. It's a lot of output but it worked's important to understand where every part is coming from, so you are able to debug problems when they happen.
==PART 2: Privileged Tasks Examples = Allow access ====Creat privileged tasks: install and remove rpm package on remote machines===: Add the following two new functions to Apache through the firewall ==end of the fabric script "fabfile.py" in your lab8 directory:<source lang='bash'>
def installPackage(pkg='dummy'): The default setup of iptables also doesn cmd = 'yum install ' + pkg + 't allow access to our web server. We-y'll need to add some more to our function to allow it. This would probably make more sense in setupWebServer status = sudo(cmd) but for now let's put it into setupFirewall print(status):
def removePackage(pkg): if pkg == '': cmd = 'yum remove dummy -y' else: <source lang cmd ="python">'yum remove ' + pkg + ' -y' runstatus = sudo("iptables -I INPUT -p tcp --dport 80 -j ACCEPT"cmd) runprint("iptables-save > /etc/sysconfig/iptables"status)
</source>
: Note that both functions take one function argument in different ways. However, if no function argument is passed when calling the function, both will default to a string value of "dummy". Both functions call the sudo() from the fabric.api to execute the command contained in the "cmd" object on the remote machine via sudo.
: To check for any syntax error in your updated fabric script, run the following command in the same directory as the fabfile.py:<source lang='bash'>
fab -l
</source>
: You should get a list of tasks defined similar to the following:<source lang='bash'>
[raymond.chan@mtrx-node05pd lab8]$ fab -l
Available commands:
getHostname installPackage removePackage[raymond.chan@mtrx-node05pd lab8]$</source>: Easy enoughIf you only need to connect to the same remote machine, but thereyou can specify the host and port number in the fabfile.py to save some typing when executing the fab command. Add the following two lines after the env.user line in your fabfile.py:<source lang='bash'>env.port = '7200' # <-- please replace with the actual value of your VM's port numberenv.hosts =['myvmlab.senecacollege.ca'] </source>: You can also store the user's on problem - if we run password in this more than once, we're going file so that it will respond to end up with duplicate iptables rules the "sudo password" prompt for port 80 sudo(check with iptables ) call. It is not safe to do so as you can configure the sudo module on the remote machine not to ask for sudo password.: Now you can run the fab command without the "--host" and "--L)port" option.: Run the following two fab commands, note the results and compare their difference:<source lang='bash'>fab installPackage
fab installPackage: In order to avoid that - we have to first check whether tree</source>: Run the following two fab commands, note the rule exists before we add it. We can do that like thisresults and compare their difference:<source lang='bash'>fab removePackage
fab removePackage: <source lang="bash">iptables -C INPUT -p tcp --dport 80 -j ACCEPT"tree</source>
== Part 2: Create remote task for updating rpm packages ==: Unfortunately that command answers Add a new function called "yes" or updatePackage"to your fabfile.py according to the following requirements::* Accept optional function argument as the rpm package name:* If no" by succeeding or failing depending on whether that rule exists. In Fabric function argument was given when a command fails - called, default to all the packages installed: The output of the entire fab file execution stopsupdatePackage when executed, assuming that it's an unrecoverable error. We need to prevent that with another with statementshould produce similar output as shown below:1. Update a single package: <source lang="python"'bash'> with settings(warn_only=True): firewallAlreadySetUp = run("iptables -C INPUT -p tcp --dport 80 -j ACCEPT") if firewallAlreadySetUp.return_code == 1fab updatePackage: ... move your iptables rules setup here ...tree
</source>
: Sample output:<source lang='bash'>
[raymond.chan@mtrx-node05pd lab8]$ fab updatePackage:tree
[myvmlab.senecacollege.ca] Executing task 'updatePackage'
[myvmlab.senecacollege.ca] sudo: yum update tree -y
[myvmlab.senecacollege.ca] out: sudo password:
[myvmlab.senecacollege.ca] out: Loaded plugins: fastestmirror
[myvmlab.senecacollege.ca] out: Loading mirror speeds from cached hostfile
[myvmlab.senecacollege.ca] out: * base: less.cogeco.net
[myvmlab.senecacollege.ca] out: * extras: centos.mirror.ca.planethoster.net
[myvmlab.senecacollege.ca] out: * updates: less.cogeco.net
[myvmlab.senecacollege.ca] out: No packages marked for update
[myvmlab.senecacollege.ca] out:
Loaded plugins: Test your new setupFirewall function on worker1, and make sure it opens access to Apache but does not create duplicate rules every time it's runfastestmirrorLoading mirror speeds from cached hostfile * base: less.cogeco.net * extras: centos.mirror.ca.planethoster.net * updates: less.cogeco.netNo packages marked for update
Done.Disconnecting from myvmlab.senecacollege.ca:7200... done.[raymond.chan@mtrx-node05pd lab8]$</source>:2. Update all installed package:<source lang= INVESTIGATION 3'bash'>fab updatePackage: Multiplying your work </source>: The following output had been trimmed, only showing the first few lines:<source lang='bash'>[myvmlab.senecacollege.ca] Executing task 'updatePackage'[myvmlab.senecacollege.ca] sudo: yum update -y[myvmlab.senecacollege.ca] out: sudo password:[myvmlab.senecacollege.ca] out: Loaded plugins: fastestmirror[myvmlab.senecacollege.ca] out: Loading mirror speeds from cached hostfile[myvmlab.senecacollege.ca] out: * base: less.cogeco.net[myvmlab.senecacollege.ca] out: * extras: centos.mirror.ca.planethoster.net[myvmlab.senecacollege.ca] out: * updates: less.cogeco.net...
Verifying : After completing all the previous parts of the lab systemd- you should have a working fabfile219-73.el7_8.5.py with two working functionsx86_64 53/54 Verifying : setupFirewall() and setupWebServer()systemd-libs-219-73.el7_8.5.x86_64 54/54
'''** Optional **'''You were asked to test them on worker1Removed: kernel. Now let's run these two functions on all your workers at the same timex86_64 0:3. The command is almost the same, except for the list of IP addresses:10.0-862.el7
<source lang="bash">fab --fabfile=fabfileInstalled: kernel.py -H 192x86_64 0:3.16810.56.11,192.168.56.12,192.168.560-1127.13,192.168.56.14,192.168.561.15 setupWebServer</source>el7
Updated: Again bind- your IP addresses will be different but the command will be the sameexport-libs.x86_64 32:9.11.4-16.P2.el7_8.6 binutils.x86_64 0:2.27-43.base.el7_8.1 ca-certificates.noarch 0:2020.2.41-70.0.el7_8 device-mapper.x86_64 7:1.02.164-7.el7_8.2 device-mapper-event.x86_64 7:1.02.164-7.el7_8.2 device-mapper-event-libs.x86_64 7:1.02.164-7.el7_8.2 device-mapper-libs.x86_64 7:1.02.164-7.el7_8.2 kernel-tools.x86_64 0:3.10.0-1127.13.1.el7 kernel-tools-libs.x86_64 0:3.10.0-1127.13.1.el7 lvm2.x86_64 7:2.02.186-7.el7_8.2 lvm2-libs.x86_64 7:2.02.186-7.el7_8.2 microcode_ctl.x86_64 2:2.1-61.10.el7_8 net-snmp.x86_64 1:5.7.2-48.el7_8.1 net-snmp-agent-libs.x86_64 1:5.7.2-48.el7_8.1 net-snmp-libs.x86_64 1:5.7.2-48.el7_8.1 net-snmp-utils.x86_64 1:5.7.2-48.el7_8.1 ntp.x86_64 0:4.2.6p5-29.el7.centos.2 ntpdate.x86_64 0:4.2.6p5-29.el7.centos.2 python-perf.x86_64 0:3.10.0-1127.13.1.el7 rsyslog.x86_64 0:8.24.0-52.el7_8.2 selinux-policy.noarch 0:3.13.1-266.el7_8.1 selinux-policy-targeted.noarch 0:3.13.1-266.el7_8.1 systemd.x86_64 0:219-73.el7_8.8 systemd-libs.x86_64 0:219-73.el7_8.8 systemd-sysv.x86_64 0:219-73.el7_8.8 yum-plugin-fastestmirror.noarch 0:1.1.31-54.el7_8
: You can also reconfigure the firewall on all the workers at the same time, using a command like this on your controller:Complete!
<source lang="bash">fab --fabfile=fabfileDone.py -H 192Disconnecting from myvmlab.168senecacollege.56ca:7200.11,192.168.56done.12,192[raymond.168.56.13,192.168.56.14,192.168.56.15 setupFirewallchan@mtrx-node05pd lab8]$</source> And imagine that you might have 10, 50, 100 servers to do this on - could you do it without the automation?
= Final Task - Apply fabfile.py to your VM on myvmlab Lab Exercise: Create a Fabric task called makeUser() =: Since your account on your vm on myvmlab is a regular user with Study the Fabric API run(), sudo privilege. You need (), and put() and utilize them to make create a new task called makeUser(): The makeUser() function should perform the following changes to your fabfile.py before applying it to your vm on myvmlab::* Change envcreate a new user called "ops435p" with home directory "/home/ops435p".user from 'root' :* add it to your account on your vm in myvmlabthe sudo group called "wheel".:* Change all ask your professor for a ssh public key and add it to the file named "authorized_keys" in the commands ~ops435p/.ssh directory. Make sure that need super user privilege from calling you set the proper permissions on both the run() function to instead calling directory ~ops435p/.ssh and the sudo() functionfile "~ops435p/.ssh/authorized_keys.: Test Add the makeUser() to your updated final version of fabfile.py until you get . :Run the same result as when you apply it to new task makeUser() on your own worker VM.:Verify and confirm that your new makeUser() task is working correctly.
= LAB 8 SIGN-OFF (SHOW INSTRUCTOR) =
:'''Have Ready to Show Your Instructor:'''* Complete all the parts of the lab and show upload the version of your fabfile.py which works on your vm on myvmlabto Blackboard by the due date.  = LAB REVIEW =
[[Category:OPS435-Python]][[Category:rchan]]
1,760
edits