Changes

Jump to: navigation, search

OPS435 Python3 Lab 8

13,947 bytes added, 19:00, 4 July 2020
no edit summary
<font color='red'><u>'''This lab is currently being reviewed. The final version will be ready by November 11, 2019'''</u></font>
= LAB OBJECTIVES =
:0. Review SSH setup and remote shell execution:1. Use Explore the '''Fabric Python library and its command line tool "fab".:2. Create Fabric scripts utilizing Fabric''' program s API to execute administrative define tasks on remote host via Python functions under for the '''Fabricfab''' frameworkcommand.:23. Create python functions using Fabric API Use the '''fab''' command to execute fabric script to perform Linux system administrative pre-defined tasks on controlled remote Linux systemsmachines.
== Overview ==
: Completing this lab will give you Fabric is a taste Python library and command-line tool for streamlining the use of what is involved in automating remote SSH for application deployment or system/network administration tasks. It has two major components::# a command-line interface program called "fab" that lets you execute arbitrary Python functions :# a set of Python APIs that you can use and call in your Python functions to make executing shell commands over SSH much easier. : We will look at and are going use the Fabric API and its '''Fabricfab''' package command to define and execute Python functions (or tasks), to automate interactions with remote Linux machines in this lab. Using Fabric you can automate monitoring, deploying software, and updating many systems at the same time repeatedly.
== REFERENCE ==
| style="border: 2px solid black;" | Category
| style="border: 2px solid black;" | Resource Link
 
|- style="background-color:white;border:none;"
| style="border: 2px solid black;" valign="top"|
:Official '''Fabric''' website
| style="border: 2px solid black;" valign="top"|
:[http://www.fabfile.org/]
|- style="background-color:white;border:none;"
:[https://www.digitalocean.com/community/tutorials/how-to-use-fabric-to-automate-administration-tasks-and-deployments]
|- style="background-color:white;border:none;"
| style="border: 2px solid black;" valign="top"|
:Official '''Fabric''' website
| style="border: 2px solid black;" valign="top"|
:[http://www.fabfile.org/]
|}
: <font color='blue'>Please note that the version of Fabric installed on matrix.senecacollege.ca for this lab is 1.14 and it supports only Python version 2.</font> The Fabric script files we are going to create in this lab have to meet Python version 2.x requirements. (e.g. print is a keyword, not a built-in function in Python 2.x):2. You should have learned some experience on the following topics in OPS235 and or OPS335. Please review them to prepare for some of the tasks activities in this lab::* Configure create and allow configure a regular user to run the sudo commandon a Linux system. :* The man page on configure and manage sudoprivilege for a regular user:* Configure sudoers using the configuration file: /etc/sudoers.visudo command:* Managing critical system log files: /var/log/messagesusing the yum command to install, /var/log/maillogremove, /var/log/secureand update rpm packages
:* Retrieve current firewall setting using the iptables -L -n -v command
= INVESTIGATION 1: Extra VM Setup The Fabric Environment =
: In order to experience The Fabric environment consists of the following components: :# Controller workstation - the machine that has the Fabric package installed and runs the "fab" command:## the Fabric Python Library - the fabric package (already installed on matrix):## the Fabric API - fabric.api:## the Fabriccommand - '''fab's features '': run Fabric script, name of the script is default to fabfile.py in a realistic way, wethe current working directory unless specified otherwise with the '-f're going to set up several virtual machines option.:## Fabric script: contains fabric environment object value and Python functions (You need at least one more VMor tasks). To begin with they are all going to have be executed by the same configuration'''fab''' command. Please make sure that each VM has direct network connect with other VMs you wish to control and configure:# Remote machine: the target machine on which one or more Fabric tasks will be executed.:## running the ssh server daemon:## use public key (or password based) authentication for ssh connection
== PART 1 - Set up Configure and test your controller workstation ==: In this lab you will use your login account on matrix.senecacollege.ca as your Fabric controller workstation.
: In this lab you will use your existing vm The Fabric package version 1.14.0 has already been installed on matrix.senecacollege.ca. You should have access to the '''centos7fab''' as a workstation command on matrix. Login to control other VMs which we'll call workersmatrix.senecacollege. Later in ca and run the lab, we will try following command to control and monitor your vm in myvmlab using confirm the version of the fabfile we are going to develop. fabric package: Install '''fabric''' using yum. Once it's installed you should have a '<source lang='bash'>fab''' command available. --version</source>: Type the following command to get the command line optionoptions of the fab command:<source lang='bash'>
fab --help
</source>
:You should get something similar to the following:<source lang="bash">
Usage: fab [options] <command>[:arg1,arg2=val2,host=foo,hosts='h1;h2',...] ...
number of concurrent processes to use in parallel mode
</source>
<font color='green'><b>Please note and study the following command-line options as they will be used in some of the activities in this lab::# -H, :# -f, :# -i, :# -l, and :# --port options. :# --user== PART 2 :# --initial-sudo-password- Create master Worker image ==prompt </b></font>
== PART 2: Create a new virtual Connect to VM in myvmlab.senecacollege.ca ==: You should have received an email from ITS containing the following information::* account name: (usually 'student'):* password: (let's assume it is 'P@ssw0rd' for the following instruction in this lab):* port number for SSH access via myvmlab.senecacollege.ca (e.g. 7200): This VM will be used as the remote Linux machinein our Fabric environment. Login to matrix and try the following SSH command to test the connectivity between matrix and your assignment VM:<pre> [raymond.chan@mtrx-node05pd lab8]$ ssh -p 7200 student@myvmlab.senecacollege.ca student@myvmlab.senecacollege.ca's password: Last login: Fri Jul 3 11:06:24 2020 from mtrx-node05pd.dcm.senecacollege.ca</pre>: Once you are on your VM, try the following commands: hostname, id, and df, and allocate record the results for it 1GB later comparison with the results of RAM other commands:<source lang='bash'>[student@centos7 ~]$ hostnamecentos7[student@centos7 ~]$ iduid=1002(student) gid=1002(student) groups=1002(student),10(wheel)[student@centos7 ~]$ dfFilesystem 1K-blocks Used Available Use% Mounted ondevtmpfs 878260 0 878260 0% /devtmpfs 889792 0 889792 0% /dev/shmtmpfs 889792 9492 880300 2% /runtmpfs 889792 0 889792 0% /sys/fs/cgroup/dev/mapper/centos-root 38680112 1745524 36934588 5% //dev/sda2 1038336 331228 707108 32% /boot/dev/sda1 204580 11296 193284 6% /boot/efi/dev/mapper/centos-home 18880512 33160 18847352 1% /hometmpfs 177960 0 177960 0% /run/user/1002</source>:Logout from your VM and 8GB of disk spaceget back to matrix.:The previous SSH command when executed successfully, created a login shell on the remote machine. Install If the previous SSH command is followed by a Basic Web Server configuration specific bash command, it will be executed on the remote host instead of CentOS creating a login shell. Consider the following:<source lang='bash'>[raymond.chan@mtrx-node05pd lab8]$ ssh -p 7200 student@myvmlab.senecacollege.ca 'hostname;id;df'student@myvmlab.senecacollege.ca's password:centos7uid=1002(student) gid=1002(student) groups=1002(student),10(wheel)Filesystem 1K-blocks Used Available Use% Mounted ondevtmpfs 878260 0 878260 0% /devtmpfs 889792 0 889792 0% /dev/shmtmpfs 889792 9492 880300 2% /runtmpfs 889792 0 889792 0% /sys/fs/cgroup/dev/mapper/centos-root 38680112 1745608 36934504 5% //dev/sda2 1038336 331228 707108 32% /boot/dev/sda1 204580 11296 193284 6% /boot/efi/dev/mapper/centos-home 18880512 33160 18847352 1% /hometmpfs 177960 0 177960 0% /run/user/1002</source>:The three shell commands: hostname, id, and df were executed sequentially. Compare the outputs above with the previous results when executing the corresponding commands in that VM using the same CentOS login shell.iso file :Please note that you used were asked to provide the user's password for your first machine in this courseevery SSH connection.
==PART 3: Make sure that:Set up SSH login with public key authentication ==:* The hostname of the system is '''worker1'''.:* It has a static IP address appropriate In order for your virtual network.:* Create a regular user using controller workstation to automate tasks execution on your Seneca email name as the user name: [seneca_id]. :* Add this new regular user VM, you need to the '''wheel''' group using the following command:<source lang="bash"> usermod -a -G wheel [seneca_id]</source>This will allow the user configure your VM to run the '''sudo''' commandSSH public key authentication instead of password authentication.:* After installation ensure that you can access '''worker1''' from your main vm using the static IP address youYou've assigned done this in both OPS235 and OPS335, and here is a summary on how to do it.between your account on matrix and your VM:
=== Set up : Create a new SSH key login ===pair (one private, and one public) under your account on matrix.senecacollege.ca. : In order for an automated system Once you have both keys, you can use the '''ssh-copy-id''' command to be able copy your public key to connect the student account on your VM, replace the port number with the correct value for your VM:<source lang='bash'>ssh-copy-id -i ~/.ssh/id_rsa.pub -p 7200 student@myvmlab.senecacollege.ca</source>: The above command should add the contents of your pub key to ~/.ssh/authorized_keys under your student account on your VM .: Verify and administer it confirm that your account on matrix can SSH to your VM as 'student' without prompting for a password:<source lang='bash'>[raymond.chan@mtrx- you will need node05pd lab8]$ ssh -p 7200 student@myvmlab.senecacollege.caLast login: Fri Jul 3 12:46:19 2020 from mtrx-node05pd.dcm.senecacollege.ca[student@centos7 ~]$ exitlogoutConnection to be able to connect to it using SSH keysmyvmlab.senecacollege. You've done this in both OPS235 and OPS335ca closed.
: Create a new SSH key on your main VM with your regular user[raymond.chan@mtrx-node05pd lab8]$ ssh -p 7200 student@myvmlab. Please do not use rootsenecacollege. Then set things up so that your regular user on your 'ca 'date;hostname;id'Fri Jul 3 12:55:22 EDT 2020centos7uid=1002(student) gid=1002(student) groups=1002(student),10(wheel)[raymond.chan@mtrx-node05pd lab8]$</source>: If you got similar result as above, you have successfully configure your controller workstation and your VM''' can SSH to the worker VM as (the same regular user) without putting in a passworduse public key authentication.
=INVESTIGATION 2 - Running the fab command in ad-hoc mode =: The fab command relies on SSH to make the connection to the remote machine before executing the intended commands. The fab command can run in ad-hoc mode:<source lang='bash'>fab [options] -- [shell commands]</source>: When running the fab command in ad-hoc mode, it is very similar to running the SSH with commands attached at the end. == PART 3 1: running non- Clone the Workers privileged shell commands on remote machines ==: We're only simulating In the real world where you'd have hundreds of VMs in one or more cloudsfollowing example, but you can just imagine that we use the VMs you're creating ''fab''' to execute the "date", "hostname", and "id" command remotely on our VM. Try the following ad-hoc fab commands and record their results for later use, replace the port number with the correct value for your computer are actually being created on an Amazon or Microsoft serverVM:<source lang='bash'>[raymond.chan@mtrx-node05pd lab8]$ fab --host=myvmlab.senecacollege.ca --port=7200 --user=student -- 'date;hostname;id'[myvmlab.senecacollege.ca] Executing task '<remainder>'[myvmlab.senecacollege.ca] run: date;hostname;id[myvmlab.senecacollege.ca] out: Fri Jul 3 13:05:39 EDT 2020[myvmlab.senecacollege.ca] out: centos7[myvmlab.senecacollege.ca] out: uid=1002(student) gid=1002(student) groups=1002(student),10(wheel)[myvmlab.senecacollege.ca] out:
: '''** Optional **''' Make four clones of the master worker image you've just created. Then make sure that each of them has a unique IP address. That's all you're required to change manually. All the other configuration on the workers (inlcuding the hostnames) will be set by Fabric. Normally you would have some kind of automation doing all this cloning and IP address assignment as well, but we don't have time for that this semester.
Done.Disconnecting from myvmlab.senecacollege.ca: Make snapshots of all 7200... done.[raymond.chan@mtrx-node05pd lab8]$</source>: Note that there is no password prompting if you complete part 3 successfully, otherwise, the SSH server daemon on your workers so VM will prompt you for a password. The output from the fab's ad-hoc mode is not much different from the SSH command with shell command attached at the end, however, please note that you the additional information on the output from the fab command can easily restore them to be very useful for record keeping purpose - what has been done and whether the original state after you modify themcommands had been carried out successfully or not.
= INVESTIGATION = PART 2: Fabric practice running privileged commands on remote machines ==: We will start say that running an ad-hoc fab command is very similar to the SSH command with some basicsshell commands attached at the end. Fabric runs python programs on the controller and Let's try both with privileged commands, like the workers. You create an "instructionyum" file on your controller, and execute it on the controller using the '''fab''' program. When you do that - you specify which workers you want your instructions to be executed oncommand.
=== Run the "yum" command on remote machine with SSH ===: The instructions are stored in a python fileBy default, your VM doesn't have the "tree" rpm package installed. Let's start You can verify this with a simple one named '''fabfile.py''' the following SSH command (remember to replace the default filename port number with the correct value for fabyour VM): == PART 1: Simplest example ==: <source lang='bash'>[raymond.chan@mtrx-node05pd lab8]$ ssh -p 7200 student@myvmlab.senecacollege.ca "pythonyum list tree">Loaded plugins: fastestmirrorLoading mirror speeds from fabriccached hostfile * base: centos.mirror.api import colo-serv.net *extras: centos.mirror.colo-serv.net * updates: centos.mirror.ca.planethoster.net# Will get Available Packagestree.x86_64 1.6.0-10.el7 base[raymond.chan@mtrx-node05pd lab8]$</source>: Please note that the hostname of this workertree package is "Available", but not yet installed.:def getHostname()Let't try to install the "tree" package with the shell command "yum install tree -y":<source lang='bash'> name = run([raymond.chan@mtrx-node05pd lab8]$ ssh -p student@myvmlab.senecacollege.ca "hostnameyum install tree -y") print(name)Loaded plugins: fastestmirrorYou need to be root to perform this command.
</source>
: Using the "yum" command to query rpm package doesn't need special privilege, however, it does when you try to install or remove rpm packages.
: Your "student" account on your VM was configured to allow you to run the "sudo" command to perform software management using the "yum" command. Let's login to your VM and try the following "sudo" command to install and then remove the "tree" rpm package:<source lang='bash'>
[raymond.chan@mtrx-node05pd lab8]$ ssh -p 7200 student@myvmlab.senecacollege.ca
Last login: Fri Jul 3 16:51:07 2020 from mtrx-node05pd.dcm.senecacollege.ca
[student@centos7 ~]$ sudo yum install tree -y
[sudo] password for student:
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: less.cogeco.net
* extras: centos.mirror.colo-serv.net
* updates: mirror.calgah.com
Resolving Dependencies
--> Running transaction check
---> Package tree.x86_64 0:1.6.0-10.el7 will be installed
--> Finished Dependency Resolution
:All this will do is get the hostname of the worker and print it (on the controller). We run it on the controller like this:Dependencies Resolved
: <source lang="bash">fab --fabfile=fabfile====================================================================================================================== Package Arch Version Repository Size========================================================================================================================Installing: tree x86_64 1.py 6.0-H 192.168.5610.11 getHostname</source>el7 base 46 k
: In the command above we're using the fab program to read the file fabfile.py and execute the getHostname function on the worker 192.168.56.11. Note that the IP address of your first worker will likely be different.Transaction Summary========================================================================================================================Install 1 Package
Total download size: If you did all the setup right and you try to execute the command above 46 kInstalled size: 87 kDownloading packages:tree-1.6.0- you will get a password prompt10. Read the prompt carefully and see who's password it prompts youel7. If it is not the same as your [seneca_id], try to fix it by adding the following line to before the first '''def''' line your fab filex86_64.rpm | 46 kB 00:00:00Running transaction checkRunning transaction testTransaction test succeededRunning transaction Installing : tree-1.6.0-10.el7.x86_64 1/1 Verifying : tree-1.6.0-10.el7.x86_64 1/1
Installed:<source lang="python">env tree.user = '[seneca_id]'</source>x86_64 0:1.6.0-10.el7
Complete![student@centos7 ~]$</source>:That should have workedPlease note that when you run the "sudo" command the first time, and it asks youfor the user's password (i.e. user student's password). Let's now remote the "tree" package:<source lang='bash'd get output like >[student@centos7 ~]$ yum remove tree -yLoaded plugins: fastestmirrorYou need to be root to perform thiscommand.[student@centos7 ~]$ sudo yum remove tree -yLoaded plugins: fastestmirrorResolving Dependencies--> Running transaction check---> Package tree.x86_64 0:1.6.0-10.el7 will be erased--> Finished Dependency Resolution
<pre>$ fab --fabfile=fabfile.py -H 192.168.56.11 getHostname[192.168.56.11] Executing task 'getHostname'[192.168.56.11] run: hostname[192.168.56.11] out: www[192.168.56.11] out: Dependencies Resolved
worker1======================================================================================================================== Package Arch Version Repository Size========================================================================================================================Removing: tree x86_64 1.6.0-10.el7 @base 87 k
Done.Transaction SummaryDisconnecting from 192.168.56.11... done.========================================================================================================================</pre>Remove 1 Package
Installed size: In the above you have87 kDownloading packages::* Lines with an IP address telling you which worker the output is for/from.Running transaction checkRunning transaction testTransaction test succeededRunning transaction Erasing :* Messages from the controller (etree-1.g6. "Executing task0-10.el7..", and "runx86_64 1/1 Verifying : tree-1.6.0-10.")el7.:* Output from the worker ("out: ..."):* Output on the controller from your fab file ("worker1" which came from the "print()" call)x86_64 1/1
Removed:You should get used to the above tree. It's a lot of output but it's important to understand where every part is coming from, so you are able to debug problems when they happenx86_64 0:1.6.0-10.el7
Complete![student@centos7 ~]$</source>: The above tests confirm that the student user is allowed to run the yum command to install and remove rpm package. Now let's logout from the VM and go back to matrix. On matrix, try to run the sudo command using SSH:<source lang== Part 2'bash'>[student@centos7 ~]$ exitlogoutConnection to myvmlab.senecacollege.ca closed.[raymond.chan@mtrx-node05pd lab8]$ ssh -p 7211 student@myvmlab.senecacollege.ca "sudo yum install tree -y"sudo: no tty present and no askpass program specified[raymond.chan@mtrx-node05pd lab8]$</source>: Set The above error indicated that you need a tty for the SSH session to prompt you for the sudo password. Please look up web server ==the ssh man page to find out the option which turn on a tty for the SSH session.
=== Run the privileged yum command on remote machine using ad-hoc fab command ===:Let's pretend that we needed to deploy a web server try the corresponding ad-hoc fab command on several machines. Weyour VM:<source lang='bash'll set up a simple example of such a deployment here.>fab --host=myvmlab.senecacollege.ca --port=7200 --user= Install Apache ===student -- 'sudo yum install tree -y'</source>:Add a setupWebServer() function Type in your user student's password when prompted for "sudo password", the yum install command to your python file: install the tree rpm package should be executed successfully. If the tree rpm package is already installed, you can remove it with the following ad-hoc fab command:<source lang="python"'bash'># Will set up a working web server with a prefab --host=myvmlab.senecacollege.ca --port=7200 --user=student -built websitedef setupWebServer(): run("hostnamectl set-hostname www") run("'sudo yum install httpd") run("systemctl enable httpd") run("systemctl start httpd")remove tree -y'
</source>
: Try remove the "tree" rpm package with the appropriate ad-hoc fab command.
= INVESTIGATION 3:Note Running the fab command in script mode =: From investigation 2, we can see that each call running '''fab''' in ad-hoc mode is quick, straight forward, and easy. However, the rich output generated can not be easily captured and processed. If you have a need to capture and process the output generated by the commands executed on the remote machines, the solution is to "run()" will run a the '''fab''' command on in script mode.: The first step in running the worker'''fab''' command in script mode is to create a fabric script file. In this function we set : Let's start with a simple fabric script file to demonstrate some basic concepts that use the hostname of API from the machine Fabric python library.: On matrix, cd to "www", install Apache, enable the Apache service, your lab8 directory and start that service now. Pretty create a simple commandsfabric script file named '''fabfile.py''' (this is the default filename used by the fab command when you invoke it without the '-f' optino):
== PART 1:If you try Non-privileged task example =====Create non-privileged tasks: Getting the hostname of remote machines===: Add the following contents to run it the same way as beforedefault fabric script called "fabfile.py" in your lab8 directory:<source lang="python">from fabric.api import *
<pre>$ fab --fabfile# set the name of the user login to the remote hostenv.user =fabfile.py -H 192.168.56.11 setupWebServer</pre>'student'
# Define the task to get the hostname of remote machines:You'll find that yum prompts you to answer questions, which you don't want to do in an automated environment. And also yum prints too much output, which also isn't helpful in an automated environment. We'll fix it by adding two switches to yumdef getHostname(): name = run("-yhostname" and ) print("-d1The host name is:":,name)</source>
:Notice also that all of To check for syntax error in the four commands can be fabric script, run as many times as you want, the result will be following command in the lab8 directory where it contains the samefabric script named "fabfile. This is not always so easypy":<source lang="bash">fab -l</source>: you should get a list of tasks defined in your fabfile.py:<source lang="bash">[rchan@centos7 lab8]$ fab -lAvailable commands:
getHostname</source>:At this point if you log in To perform the task of getHostname on your VM (replace with the actual port # for connecting to worker1 - you should see a new hostnameyour VM), and httpd installed and running (try with run the fab command on matrix:<source lang="systemctl statusbash")>[raymond.chan@mtrx-node05pd lab8]$ fab --hosts=myvmlab.senecacollege.ca --port=7200 getHostname[myvmlab.senecacollege.ca] Executing task 'getHostname'[myvmlab.senecacollege.ca] run: hostname[myvmlab.senecacollege.ca] out: centos7[myvmlab.senecacollege.ca] out:
=== Deploy a website ===The host name is: centos7
:Now that we have a web server running, we also want to put a website on itDone. The website can be of any complexity, but to keep this demonstration simple we'll have a single HTML file. You can pretend that it's as complex as you likeDisconnecting from myvmlab. Create an '''indexsenecacollege.html''' file like this: :<source lang="html"><h1>My fancy web server</h1></source> ca:And since we're pretending that it's a large website with many files and directories, we'll compress it into an archive named '''webcontents7200.tar.bz2''' using a tar command. You've done this since OPS235:Once you have your archive, make sure it's in the same directory as your fab file. Then add the following to your setupWebServer() function: :<source lang="python"> with cd("/var/www/html/"): put("webcontents.tar.bz2", ".") run("tar xvf webcontents.tar.bz2") run("rm webcontents.tar[raymond.bz2")chan@mtrx-node05pd lab8]$
</source>
: Notice that there is no need to specify the user name at the '''fab''' command line since we defined it in the fabric script file (env.user = 'student'). Also notice that we can capture the host name returned from the "hostname" command and print it out together with an descriptive text in a line.
:There is something weird in In the code above that you havenexecuted 't seen before but it's required for some uses of Fabric: the 'fab''with''' statementcommand, the fab program imports the fabric script named "fabfile.py" and execute the getHostname function on the VM connect at port 7200 on myvmlab.senecacollege.ca. Note that the port number for your first will likely be of a different value.
:The problem is that separate '''run''' commands each If you did all the setup right and you got a password prompt when execute in a brand new sessionthe above command, each with its own shellread the prompt carefully and see who's password it was prompting you for. They are If it is not like separate lines for the user student, verify that you have the following line in a single shell script even though they look like they should beyour fabfile.py and you can ssh to your VM as the user student without password:
:That means if you run a cd command and then a tar command separately - the tar command will not run in the directory where you think it will<source lang="python">env. In order to fix this you have to nest commands inside a user = 'student''with''' - it's like a '''run''' but with persistant results.</source>
:The code we added to In the function will cd to above output from the '''fab''' command, you have::* Lines with the FQDN of the default web site directory remote machine you are working on .:* Messages from the workercontroller workstation (e.g. "Executing task...", upload your web contents tarball and "run: ...").:* Output from your controller to that directory the remote machine ("out: ..."):* Output generated on the worker, extract it, and delete controller workstation from your fab file (the tarball.print statement)
:After You should get used to the above messages from the '''fab''' command. It's a lot of output but it's done - important to understand where every part is coming from, so you should have a working web server and simple website on your worker1are able to debug problems when they happen.
== PART 2:Except you won't be able Privileged Tasks Examples =====Creat privileged tasks: install and remove rpm package on remote machines===: Add the following two new functions to access it because the end of the firewallfabric script "fabfile. Wepy" in your lab8 directory:<source lang='ll deal with that in the next section.bash'>
def installPackage(pkg== Part 2'dummy'): Set up the firewall cmd ='yum install ' + pkg + ' -y' status =sudo(cmd) print(status)
def removePackage(pkg): Recall if pkg == '': cmd = 'yum remove dummy -y' else: cmd = 'yum remove ' + pkg + ' -y' status = sudo(cmd) print(status)</source>: Note that both functions take one function argument in our OPS courses we've been using iptables instead of firewallddifferent ways. However, which if no function argument is installed by passed when calling the function, both will default to a string value of "dummy". Both functions call the sudo() from the fabric.api to execute the command contained in CentOSthe "cmd" object on the remote machine via sudo. Let's make sure that our workers have that set up : To check for any syntax error in your updated fabric script, run the following command in the same directory as well. In the same '''fabfile.py:<source lang='bash'>fab -l</source>: You should get a list of tasks defined similar to the following:<source lang=' youbash've been using all along, add a new function like this>[raymond.chan@mtrx-node05pd lab8]$ fab -lAvailable commands:
getHostname installPackage removePackage[raymond.chan@mtrx-node05pd lab8]$</source>: If you only need to connect to the same remote machine, you can specify the host and port number in the fabfile.py to save some typing when executing the fab command. Add the following two lines after the env.user line in your fabfile.py: <source lang="python"'bash'>env.port = '7200' # Will uninstall firewalld and <-- please replace it with iptablesthe actual value of your VM's port numberdef setupFirewall(): run("yum -y -d1 remove firewalld") run("yum -y -d1 install iptables-services") run("systemctl enable iptables") run("systemctl start iptables")env.hosts =['myvmlab.senecacollege.ca']
</source>
: You can also store the user's password in this file so that it will respond to the "sudo password" prompt for sudo() call. It is not safe to do so as you can configure the sudo module on the remote machine not to ask for sudo password.
: Now you can run the fab command without the "--host" and "--port" option.
: Run the following two fab commands, note the results and compare their difference:<source lang='bash'>
fab installPackage
fab installPackage: That should by now look prett obvious. On tree</source>: Run the worker you're going to uninstall firewalld, install iptablesfollowing two fab commands, note the results and make sure that the iptables service is running.compare their difference:<source lang='bash'>fab removePackage
fab removePackage: Execute the function for worker1 and double-check that it worked.tree</source>
==Part 2: Create remote task for updating rpm packages = Allow access =: Add a new function called "updatePackage" to your fabfile.py according to Apache through the firewall =following requirements::* Accept optional function argument as the rpm package name:* If no function argument was given when called, default to all the packages installed: The output of the updatePackage when executed, should produce similar output as shown below::1. Update a single package:<source lang='bash'>fab updatePackage:tree</source>: Sample output:<source lang='bash'>[raymond.chan@mtrx-node05pd lab8]$ fab updatePackage:tree[myvmlab.senecacollege.ca] Executing task 'updatePackage'[myvmlab.senecacollege.ca] sudo: yum update tree -y[myvmlab.senecacollege.ca] out: sudo password:[myvmlab.senecacollege.ca] out: Loaded plugins: fastestmirror[myvmlab.senecacollege.ca] out: Loading mirror speeds from cached hostfile[myvmlab.senecacollege.ca] out: * base: less.cogeco.net[myvmlab.senecacollege.ca] out: * extras: centos.mirror.ca.planethoster.net[myvmlab.senecacollege.ca] out: * updates: less.cogeco.net[myvmlab.senecacollege.ca] out: No packages marked for update[myvmlab.senecacollege.ca] out:
Loaded plugins: The default setup of iptables also doesn't allow access to our web serverfastestmirrorLoading mirror speeds from cached hostfile * base: less.cogeco.net * extras: centos.mirror.ca. We'll need to add some more to our function to allow itplanethoster. This would probably make more sense in setupWebServer() but net * updates: less.cogeco.netNo packages marked for now let's put it into setupFirewall():update
Done.Disconnecting from myvmlab.senecacollege.ca: <source lang="python"> run("iptables -I INPUT -p tcp --dport 80 -j ACCEPT")7200... done. run("iptables[raymond.chan@mtrx-save > /etc/sysconfig/iptables")node05pd lab8]$
</source>
 : Easy enough, but there's on problem - if we run this more than once, we're going to end up with duplicate iptables rules for port 80 (check with iptables -L). : In order to avoid that - we have to first check whether the rule exists before we add it2. We can do that like this: Update all installed package: <source lang="'bash"'>iptables -C INPUT -p tcp --dport 80 -j ACCEPT"</source> : Unfortunately that command answers "yes" or "no" by succeeding or failing depending on whether that rule exists. In Fabric when a command fails - the entire fab file execution stops, assuming that it's an unrecoverable error. We need to prevent that with another with statementupdatePackage: <source lang="python"> with settings(warn_only=True): firewallAlreadySetUp = run("iptables -C INPUT -p tcp --dport 80 -j ACCEPT") if firewallAlreadySetUp.return_code == 1: ... move your iptables rules setup here ...
</source>
: The following output had been trimmed, only showing the first few lines:<source lang='bash'>
[myvmlab.senecacollege.ca] Executing task 'updatePackage'
[myvmlab.senecacollege.ca] sudo: yum update -y
[myvmlab.senecacollege.ca] out: sudo password:
[myvmlab.senecacollege.ca] out: Loaded plugins: fastestmirror
[myvmlab.senecacollege.ca] out: Loading mirror speeds from cached hostfile
[myvmlab.senecacollege.ca] out: * base: less.cogeco.net
[myvmlab.senecacollege.ca] out: * extras: centos.mirror.ca.planethoster.net
[myvmlab.senecacollege.ca] out: * updates: less.cogeco.net
...
Verifying : Test your new setupFirewall function on worker1, and make sure it opens access to Apache but does not create duplicate rules every time it's runsystemd-219-73.el7_8.5.x86_64 53/54 Verifying : systemd-libs-219-73.el7_8.5.x86_64 54/54
= INVESTIGATION Removed: kernel.x86_64 0:3: Multiplying your work =.10.0-862.el7
Installed: After completing all the previous parts of the lab kernel.x86_64 0:3.10.0- you should have a working fabfile1127.py with two working functions: setupFirewall() and setupWebServer()13.1.el7
'''** Optional **'''You were asked to test them on worker1Updated: bind-export-libs.x86_64 32:9.11.4-16.P2.el7_8.6 binutils.x86_64 0:2.27-43.base.el7_8.1 ca-certificates.noarch 0:2020.2.41-70.0.el7_8 device-mapper.x86_64 7:1.02.164-7.el7_8.2 device-mapper-event.x86_64 7:1.02.164-7.el7_8.2 device-mapper-event-libs.x86_64 7:1.02.164-7.el7_8.2 device-mapper-libs.x86_64 7:1.02.164-7.el7_8.2 kernel-tools.x86_64 0:3.10.0-1127.13.1.el7 kernel-tools-libs.x86_64 0:3.10.0-1127.13.1.el7 lvm2.x86_64 7:2.02.186-7.el7_8.2 lvm2-libs.x86_64 7:2.02.186-7.el7_8.2 microcode_ctl.x86_64 2:2.1-61.10.el7_8 net-snmp.x86_64 1:5.7.2-48.el7_8.1 net-snmp-agent-libs.x86_64 1:5.7.2-48.el7_8.1 net-snmp-libs.x86_64 1:5.7.2-48.el7_8.1 net-snmp-utils.x86_64 1:5.7.2-48.el7_8.1 ntp.x86_64 0:4.2.6p5-29.el7.centos.2 ntpdate.x86_64 0:4.2.6p5-29.el7.centos.2 python-perf.x86_64 0:3.10.0-1127.13.1.el7 rsyslog.x86_64 0:8.24.0-52.el7_8.2 selinux-policy.noarch 0:3.13.1-266.el7_8.1 selinux-policy-targeted.noarch 0:3.13.1-266.el7_8.1 systemd.x86_64 0:219-73. Now let's run these two functions on all your workers at the same timeel7_8. The command is almost the same, except for the list of IP addresses8 systemd-libs.x86_64 0:219-73.el7_8.8 systemd-sysv.x86_64 0:219-73.el7_8.8 yum-plugin-fastestmirror.noarch 0:1.1.31-54.el7_8
<source lang="bash">fab --fabfile=fabfile.py -H 192.168.56.11,192.168.56.12,192.168.56.13,192.168.56.14,192.168.56.15 setupWebServer</source>Complete!
: Again - your IP addresses will be different but the command will be the sameDone: You can also reconfigure the firewall on all the workers at the same time, using a command like this on your controller: <source lang="bash">fab --fabfile=fabfileDisconnecting from myvmlab.py -H 192senecacollege.168ca:7200.56.11,192.168done.56[raymond.12,192.168.56.13,192.168.56.14,192.168.56.15 setupFirewallchan@mtrx-node05pd lab8]$</source> And imagine that you might have 10, 50, 100 servers to do this on - could you do it without the automation?
= Final Task - Apply fabfile.py to your VM on myvmlab Lab Exercise: Create a Fabric task called makeUser() =: Since your account on your vm on myvmlab is a regular user with Study the Fabric API run(), sudo privilege. You need (), and put() and utilize them to make create a new task called makeUser(): The makeUser() function should perform the following changes to your fabfile.py before applying it to your vm on myvmlab::* Change envcreate a new user called "ops435p" with home directory "/home/ops435p".user from 'root' :* add it to your account on your vm in myvmlabthe sudo group called "wheel".:* Change all ask your professor for a ssh public key and add it to the file named "authorized_keys" in the commands ~ops435p/.ssh directory. Make sure that need super user privilege from calling you set the proper permissions on both the run() function to instead calling directory ~ops435p/.ssh and the sudo() functionfile "~ops435p/.ssh/authorized_keys.: Test Add the makeUser() to your updated final version of fabfile.py until you get . :Run the same result as when you apply it to new task makeUser() on your own worker VM.:Verify and confirm that your new makeUser() task is working correctly.
= LAB 8 SIGN-OFF (SHOW INSTRUCTOR) =
:'''Have Ready to Show Your Instructor:'''* Complete all the parts of the lab and show upload the version of your fabfile.py which works on your vm on myvmlabto Blackboard by the due date.  = LAB REVIEW =
[[Category:OPS435-Python]][[Category:rchan]]
1,760
edits

Navigation menu