Difference between revisions of "OPS435 Python Assignment 2"

From CDOT Wiki
Jump to: navigation, search
(→‎Bonus item 10% extra)
 
(38 intermediate revisions by 2 users not shown)
Line 1: Line 1:
−
[[Category:rchan]][[Category:OPS435-Python]]
+
[[Category:OPS435-Python]][[Category:rchan]]
−
You can pick one of the following three programming tasks as your assignment 2:
+
=Assignment 2 - Usage Report=
−
* [[OPS435 Python Assignment 2 A| dbda]] - a python program to report the (n) day before or the (n) day after a given date in the "YYYYMMDD" format
+
'''Weight:''' 15% of the overall grade
−
* [[OPS435 Python Assignment 2 B| weekly usage report chart]] - get the weekly usage data from assignment 1 for a selected user and use the openpyxl module to create a Excel spread sheet with bar chart.
+
 
−
* [[OPS435 Python Assignment 2 C| report partition information]] on a block device (both physical and virtual) by reading the partition information directly from the device.
+
'''Due Date:''' Ask your professor for exact date
 +
 
 +
'''Late Penalty:''' 20% per school day, and note that this assignment must be completed satisfactorily in order to pass the course even if you get zero mark for this assignment.
 +
 
 +
==Overview==
 +
Most system administrators would like to know the utilization of their systems by their users. On a Linux system, each user's login records are normally stored in the binary file /var/log/wtmp. The login records in this binary file can not be viewed or edited directly using normal Linux text commands like 'less', 'cat', etc. The 'last' command is often used to display the login records stored in this file in a human readable form. Please check the man page of the 'last' command for available options. The following is the contents of the file named "usage_data_file", which is a sample output of the 'last' command with the '-Fiw' flag on:
 +
<pre>
 +
$ last -Fiw
 +
rchan    pts/9        10.40.91.236    Tue Feb 13 16:53:42 2018 - Tue Feb 13 16:57:02 2018  (00:03)   
 +
cwsmith  pts/6        10.40.43.94      Tue Feb 13 16:51:47 2018 - Tue Feb 13 16:56:13 2018  (00:04)   
 +
mlee18  pts/6        10.40.43.94      Tue Feb 13 16:50:20 2018 - Tue Feb 13 16:51:27 2018  (00:01)   
 +
hfang    pts/4        24.114.50.50    Tue Feb 13 16:31:38 2018 - Tue Feb 13 17:48:39 2018  (01:17)   
 +
bigia    pts/8        24.114.50.50    Tue Feb 13 19:28:43 2018 - Tue Feb 13 20:28:31 2018  (00:59)   
 +
rchan    pts/2        10.40.91.236    Tue Feb 13 16:22:00 2018 - Tue Feb 13 16:45:00 2018  (00:23)   
 +
asmith  pts/2        10.43.115.162    Tue Feb 13 16:19:29 2018 - Tue Feb 13 16:22:00 2018  (00:02)   
 +
tsliu2  pts/4        10.40.105.130    Tue Feb 13 16:17:21 2018 - Tue Feb 13 16:30:10 2018  (00:12)   
 +
mshana  pts/13      10.40.91.247    Tue Feb 13 16:07:52 2018 - Tue Feb 13 16:45:52 2018  (00:38)   
 +
asmith  pts/11      10.40.105.130    Tue Feb 13 14:07:43 2018 - Tue Feb 13 16:07:43 2018  (02:00)
 +
</pre>
 +
It is always desirable to have a daily, weekly, monthly usage reports by user or by remote host based on the above information.
 +
 
 +
== Tasks for this assignment ==
 +
In this assignment, your should preform the following activities:
 +
# Work out an algorithm for producing daily, weekly, and monthly usage reports by user or by remote host based on the information obtained from the 'last' command or any given files which contain similar information as the 'last' command.
 +
# Once you believe you have a workable algorithm, you should then <b>design the structure of your python script</b> by identifying the appropriate python objects and functions to be used for each task in your algorithm and the main control logic. Make sure to identify all the input data, computation tasks, and outputs.
 +
# implement your computational solution using a single python script. You can use any built-in functions and functions from the allowed python modules to implement your solution.
 +
# review your working python code to see whether you can improve the interface of each function to facilitate better code re-use (this process is called <b>refactoring</b>).
 +
 
 +
== Allowed Python Modules for this assignment ==
 +
* the <b>os, sys</b> modules
 +
* the <b>argparse</b> module
 +
** [https://docs.python.org/3/howto/argparse.html Argparse Tutorial] - should read this first.
 +
** [https://docs.python.org/3/library/argparse.html Argparse API reference information page]
 +
* the <b>time</b> module
 +
** [https://www.tutorialspoint.com/python3/python_date_time.htm Time module Tutorial]
 +
** [https://docs.python.org/3/library/time.html Time module API reference page]
 +
 
 +
== Instructions ==
 +
 
 +
=== Program Name and valid command line arguments ===
 +
Name your Python3 script as <code>ur_[student_id].py</code>. Create a symbolic link to your script as ur.py (e.g. use the command ln -s ur_rchan.py ur.py to create the link) so that you can refer to your script as ur.py. Your script must accept one or more "file name" as its command line parameters and other optional parameters as shown below. Your python script should produce the following usage text when run with the --help option:
 +
<pre>
 +
[rchan@centos7 a1]$ python3 ./ur.py -h
 +
usage: ur_rchan.py [-h] [-l {user,host}] [-r RHOST] [-t {daily,weekly,monthly}]
 +
            [-u USER] [-v]
 +
            F [F ...]
 +
 
 +
Usage Report based on the last command
 +
 
 +
positional arguments:
 +
  F                    list of files to be processed
 +
 
 +
optional arguments:
 +
  -h, --help            show this help message and exit
 +
  -l {user,host}, --list {user,host}
 +
                        generate user name or remote host IP from the given
 +
                        files
 +
  -r RHOST, --rhost RHOST
 +
                        usage report for the given remote host IP
 +
  -t {daily,weekly,monthly}, --type {daily,weekly,monthly}
 +
                        type of report: daily, weekly, and monthly
 +
  -u USER, --user USER  usage report for the given user name
 +
  -v, --verbose        tune on output verbosity
 +
 
 +
Copyright 2019 - Raymond Chan
 +
</pre>
 +
Replace the last line with your own full name
 +
 
 +
 
 +
If there is one file name provided at the command line, read the login/logout records from the contents of the given file. The format of each line in the file should be the same as the output of 'last -Fiw'. Filter out incomplete login/logout record (check for the number of fields in each record).
 +
 
 +
If there is more than one file name provided, merge all the files together with the first one at the top and the last one at the bottom. Read and process the file contents in that order in your program.
 +
 
 +
=== Header ===
 +
 
 +
All your Python codes for this program must be placed in a <font color='red'><b><u>single source file</u></b></font>. Please include the following declaration by <b><u>you</u></b> as the <font color='blue'><b>script level docstring</b></font> in your Python source code file (replace [Student_id] with your Seneca email user name, and "Student Name" with your own name):
 +
 
 +
<source>OPS435 Assignment 2 - Winter 2019
 +
Program: ur_[Stduent_id].py
 +
Author: "Student Name"
 +
The python code in this file ur_[Student_id].py is original work written by
 +
"Student Name". No code in this file is copied from any other source
 +
including any person, textbook, or on-line resource except those provided
 +
by the course instructor. I have not shared this python file with anyone
 +
or anything except for submission for grading. 
 +
I understand that the Academic Honesty Policy will be enforced and violators
 +
will be reported and appropriate action will be taken.
 +
</source>
 +
 
 +
=== Sample outputs ===
 +
The following are the reports generated by the usage report script (ur.py) with the "usage_data_file" mentioned in the overview section. You can download the file [https://scs.senecac.on.ca/~raymond.chan/ops435/a2/usage_data_file here] for testing your ur.py script.
 +
==== User List ====
 +
The following is the user list extracted from the usage_data_file created by the command:
 +
<pre>
 +
[rchan@centos7 a2]$ ./ur.py -l user usage_data_file
 +
</pre>
 +
 
 +
<pre>
 +
User list for usage_data_file
 +
=============================
 +
asmith
 +
bigia
 +
cwsmith
 +
hfang
 +
mlee18
 +
mshana
 +
rchan
 +
tsliu2
 +
</pre>
 +
 
 +
==== Remote Host List ====
 +
The following is the remote host list extracted from the usage_file_file created by the command:
 +
<pre>
 +
[rchan@centos7 a2]$ ./ur.py -l host usage_data_file
 +
</pre>
 +
 
 +
<pre>
 +
Host list for usage_data_file
 +
=============================
 +
10.40.105.130
 +
10.40.43.94
 +
10.40.91.236
 +
10.40.91.247
 +
10.43.115.162
 +
24.114.50.50
 +
</pre>
 +
 
 +
==== Daily Usage Report by User ====
 +
The following is a Daily Usage Report created for user rchan by the following command:
 +
<pre>
 +
[rchan@centos7 a2]$ ./ur.py -u rchan -t daily usage_data_file
 +
</pre>
 +
 
 +
<pre>
 +
Daily Usage Report for rchan
 +
============================
 +
Date          Usage in Seconds
 +
2018 02 13        1580
 +
Total            1580
 +
</pre>
 +
 
 +
==== Daily Usage Report by Remote Host====
 +
The following is a Daily Usage Report created for the Remote Host 10.40.105.103 by the command:
 +
<pre>
 +
[rchan@centos7 a2]$ ./ur.py -r 10.40.105.130 -t daily usage_data_file
 +
</pre>
 +
 
 +
<pre>
 +
Daily Usage Report for 10.40.105.130
 +
====================================
 +
Date          Usage in Seconds
 +
2018 02 13        7969
 +
Total            7969
 +
</pre>
 +
 
 +
==== Weekly Usage Report by User ====
 +
The following is a Weekly Usage Report created for user rchan by the command:
 +
<pre>
 +
[rchan@centos7 a2]$ ./ur.py -u rchan -t weekly usage_data_file
 +
</pre>
 +
 
 +
<pre>
 +
Weekly Usage Report for rchan
 +
=============================
 +
Week #        Usage in Seconds
 +
2018 07          1580
 +
Total            1580
 +
</pre>
 +
 
 +
==== Weekly Usage Report by Remote Host ====
 +
The following is a Weekly Usage Report created for the remote host 10.40.105.130 by the command:
 +
<pre>
 +
[rchan@centos7 a2]$ ./ur.py -r 10.40.105.130 -t weekly usage_data_file
 +
</pre>
 +
 
 +
<pre>
 +
Weekly Usage Report for 10.40.105.130
 +
=====================================
 +
Week #        Usage in Seconds
 +
2018 07          7969
 +
Total            7969
 +
</pre>
 +
 
 +
==== Monthly Usage Report by User ====
 +
The following is a Monthly Usage Report created for user rchan by the command:
 +
<pre>
 +
[rchan@centos7 a2-2018fall]$ ./ur.py -u rchan -t monthly usage_data_file
 +
</pre>
 +
 
 +
<pre>
 +
Monthly Usage Report for rchan
 +
==============================
 +
Month        Usage in Seconds
 +
2018 02          1580
 +
Total            1580
 +
</pre>
 +
 
 +
==== Monthly Usage Report by Remote Host ====
 +
The following is a Monthly Usage Report created for remote host 10.40.105.130 by the command:
 +
<pre>
 +
[rchan@centos7 a2-2018fall]$ ./ur.py -r 10.40.105.130 -t monthly usage_data_file
 +
</pre>
 +
 
 +
<pre>
 +
Monthly Usage Report for 10.40.105.130
 +
======================================
 +
Month        Usage in Seconds
 +
2018 02          7969
 +
Total            7969
 +
</pre>
 +
 
 +
== Suggested Script Development and Testing ==
 +
The following python functions (to be created) are useful in handling the following sub-tasks:
 +
* reads login records from files and filters out unwanted records
 +
* convert login records into proper python object type so that it can be processed using as much built-in functions as possible
 +
* create function which generates daily usage reports by user and/or by remote host
 +
* create function which generates weekly usage reports by user and/or by remote host
 +
* create function which generates monthly usage reports by user and/or by remote host
 +
To  help you with this assignment, [[OPS435 A1 Usage Report Template|here is a template]] you can use as a starting point in designing your own Python Usage Report script.
 +
 
 +
For each function, identify what type of objects should be passed to the function, and what type of objects should be returned to the caller.
 +
 
 +
Once you have finished coding a function, you should start a Python3 interactive shell, import your functions and manually call the function you want to test under interactive mode and verify its correctness.
 +
 
 +
Once you have all the individual function tested and that each is working properly, perform the final test with test data provided by your professor and verify that your script produces the correct results before submitting your python program on Blackboard.
 +
 
 +
== Sample login/logout records file and sample test run results==
 +
* [https://scs.senecac.on.ca/~raymond.chan/ops435/a2/ Sample test data files and sample test run results]
 +
 
 +
== Rubric ==
 +
 
 +
{| class="wikitable" border="1"
 +
! Task !!  Maximum mark !! Actual mark
 +
|-
 +
| Algorithm and program design||15 ||
 +
|-
 +
| Program Authorship Declaration ||5 ||
 +
|-
 +
| Program usage || 5 ||
 +
|-
 +
| Program Options || 15 ||
 +
|-
 +
| Generate user name list || 10 ||
 +
|-
 +
| Generate remote host IP list|| 10 ||
 +
|-
 +
| Daily Usage Report by User || 10 ||
 +
|-
 +
| Daily Usage Report by Remote Host || 10 ||
 +
|-
 +
| Weekly Usage Report by User || 5 ||
 +
|-
 +
| Weekly Usage Report by Remote Host || 5 ||
 +
|-
 +
| Monthly Usage Report by User || 5 ||
 +
|-
 +
| Monthly Usage Report by Remote Host || 5 ||
 +
|-
 +
 
 +
| '''Total''' || 100 ||
 +
 
 +
|}
 +
== Bonus item 10% extra ==
 +
* Create a function to capture that output of the 'last -Fiw' on the system your script is being executed and return the contents in the type of object as what you get from a regular login/logout record file.
 +
* Call this function and use the objects it returned when the file name input at the command line is 'last'.
 +
 
 +
== Submission ==
 +
After fully testing your program, submit your python script ur_[student_id].py and your algorithm and program design report ur_[student_id].pdf in pdf format on Blackboard by April 5th, 2019.

Latest revision as of 14:59, 14 March 2019

Assignment 2 - Usage Report

Weight: 15% of the overall grade

Due Date: Ask your professor for exact date

Late Penalty: 20% per school day, and note that this assignment must be completed satisfactorily in order to pass the course even if you get zero mark for this assignment.

Overview

Most system administrators would like to know the utilization of their systems by their users. On a Linux system, each user's login records are normally stored in the binary file /var/log/wtmp. The login records in this binary file can not be viewed or edited directly using normal Linux text commands like 'less', 'cat', etc. The 'last' command is often used to display the login records stored in this file in a human readable form. Please check the man page of the 'last' command for available options. The following is the contents of the file named "usage_data_file", which is a sample output of the 'last' command with the '-Fiw' flag on: 

$ last -Fiw
rchan    pts/9        10.40.91.236     Tue Feb 13 16:53:42 2018 - Tue Feb 13 16:57:02 2018  (00:03)    
cwsmith  pts/6        10.40.43.94      Tue Feb 13 16:51:47 2018 - Tue Feb 13 16:56:13 2018  (00:04)    
mlee18   pts/6        10.40.43.94      Tue Feb 13 16:50:20 2018 - Tue Feb 13 16:51:27 2018  (00:01)    
hfang    pts/4        24.114.50.50     Tue Feb 13 16:31:38 2018 - Tue Feb 13 17:48:39 2018  (01:17)    
bigia    pts/8        24.114.50.50     Tue Feb 13 19:28:43 2018 - Tue Feb 13 20:28:31 2018  (00:59)    
rchan    pts/2        10.40.91.236     Tue Feb 13 16:22:00 2018 - Tue Feb 13 16:45:00 2018  (00:23)    
asmith   pts/2        10.43.115.162    Tue Feb 13 16:19:29 2018 - Tue Feb 13 16:22:00 2018  (00:02)    
tsliu2   pts/4        10.40.105.130    Tue Feb 13 16:17:21 2018 - Tue Feb 13 16:30:10 2018  (00:12)    
mshana   pts/13       10.40.91.247     Tue Feb 13 16:07:52 2018 - Tue Feb 13 16:45:52 2018  (00:38)    
asmith   pts/11       10.40.105.130    Tue Feb 13 14:07:43 2018 - Tue Feb 13 16:07:43 2018  (02:00)

It is always desirable to have a daily, weekly, monthly usage reports by user or by remote host based on the above information.

Tasks for this assignment

In this assignment, your should preform the following activities:

  1. Work out an algorithm for producing daily, weekly, and monthly usage reports by user or by remote host based on the information obtained from the 'last' command or any given files which contain similar information as the 'last' command.
  2. Once you believe you have a workable algorithm, you should then design the structure of your python script by identifying the appropriate python objects and functions to be used for each task in your algorithm and the main control logic. Make sure to identify all the input data, computation tasks, and outputs.
  3. implement your computational solution using a single python script. You can use any built-in functions and functions from the allowed python modules to implement your solution.
  4. review your working python code to see whether you can improve the interface of each function to facilitate better code re-use (this process is called refactoring).

Allowed Python Modules for this assignment

Instructions

Program Name and valid command line arguments

Name your Python3 script as ur_[student_id].py. Create a symbolic link to your script as ur.py (e.g. use the command ln -s ur_rchan.py ur.py to create the link) so that you can refer to your script as ur.py. Your script must accept one or more "file name" as its command line parameters and other optional parameters as shown below. Your python script should produce the following usage text when run with the --help option: 

[rchan@centos7 a1]$ python3 ./ur.py -h
usage: ur_rchan.py [-h] [-l {user,host}] [-r RHOST] [-t {daily,weekly,monthly}]
             [-u USER] [-v]
             F [F ...]

Usage Report based on the last command

positional arguments:
  F                     list of files to be processed

optional arguments:
  -h, --help            show this help message and exit
  -l {user,host}, --list {user,host}
                        generate user name or remote host IP from the given
                        files
  -r RHOST, --rhost RHOST
                        usage report for the given remote host IP
  -t {daily,weekly,monthly}, --type {daily,weekly,monthly}
                        type of report: daily, weekly, and monthly
  -u USER, --user USER  usage report for the given user name
  -v, --verbose         tune on output verbosity

Copyright 2019 - Raymond Chan

Replace the last line with your own full name


If there is one file name provided at the command line, read the login/logout records from the contents of the given file. The format of each line in the file should be the same as the output of 'last -Fiw'. Filter out incomplete login/logout record (check for the number of fields in each record).

If there is more than one file name provided, merge all the files together with the first one at the top and the last one at the bottom. Read and process the file contents in that order in your program.

Header

All your Python codes for this program must be placed in a single source file. Please include the following declaration by you as the script level docstring in your Python source code file (replace [Student_id] with your Seneca email user name, and "Student Name" with your own name): 

OPS435 Assignment 2 - Winter 2019
Program: ur_[Stduent_id].py
Author: "Student Name"
The python code in this file ur_[Student_id].py is original work written by
"Student Name". No code in this file is copied from any other source 
including any person, textbook, or on-line resource except those provided
by the course instructor. I have not shared this python file with anyone
or anything except for submission for grading.  
I understand that the Academic Honesty Policy will be enforced and violators 
will be reported and appropriate action will be taken.

Sample outputs

The following are the reports generated by the usage report script (ur.py) with the "usage_data_file" mentioned in the overview section. You can download the file here for testing your ur.py script.

User List

The following is the user list extracted from the usage_data_file created by the command: 

[rchan@centos7 a2]$ ./ur.py -l user usage_data_file

User list for usage_data_file
=============================
asmith
bigia
cwsmith
hfang
mlee18
mshana
rchan
tsliu2

Remote Host List

The following is the remote host list extracted from the usage_file_file created by the command: 

[rchan@centos7 a2]$ ./ur.py -l host usage_data_file

Host list for usage_data_file
=============================
10.40.105.130
10.40.43.94
10.40.91.236
10.40.91.247
10.43.115.162
24.114.50.50

Daily Usage Report by User

The following is a Daily Usage Report created for user rchan by the following command: 

[rchan@centos7 a2]$ ./ur.py -u rchan -t daily usage_data_file

Daily Usage Report for rchan
============================
Date          Usage in Seconds
2018 02 13        1580
Total             1580

Daily Usage Report by Remote Host

The following is a Daily Usage Report created for the Remote Host 10.40.105.103 by the command: 

[rchan@centos7 a2]$ ./ur.py -r 10.40.105.130 -t daily usage_data_file

Daily Usage Report for 10.40.105.130
====================================
Date          Usage in Seconds
2018 02 13        7969
Total             7969

Weekly Usage Report by User

The following is a Weekly Usage Report created for user rchan by the command: 

[rchan@centos7 a2]$ ./ur.py -u rchan -t weekly usage_data_file

Weekly Usage Report for rchan
=============================
Week #        Usage in Seconds
2018 07           1580
Total             1580

Weekly Usage Report by Remote Host

The following is a Weekly Usage Report created for the remote host 10.40.105.130 by the command: 

[rchan@centos7 a2]$ ./ur.py -r 10.40.105.130 -t weekly usage_data_file

Weekly Usage Report for 10.40.105.130
=====================================
Week #        Usage in Seconds
2018 07           7969
Total             7969

Monthly Usage Report by User

The following is a Monthly Usage Report created for user rchan by the command: 

[rchan@centos7 a2-2018fall]$ ./ur.py -u rchan -t monthly usage_data_file

Monthly Usage Report for rchan
==============================
Month         Usage in Seconds
2018 02           1580
Total             1580

Monthly Usage Report by Remote Host

The following is a Monthly Usage Report created for remote host 10.40.105.130 by the command: 

[rchan@centos7 a2-2018fall]$ ./ur.py -r 10.40.105.130 -t monthly usage_data_file

Monthly Usage Report for 10.40.105.130
======================================
Month         Usage in Seconds
2018 02           7969
Total             7969

Suggested Script Development and Testing

The following python functions (to be created) are useful in handling the following sub-tasks:

  • reads login records from files and filters out unwanted records
  • convert login records into proper python object type so that it can be processed using as much built-in functions as possible
  • create function which generates daily usage reports by user and/or by remote host
  • create function which generates weekly usage reports by user and/or by remote host
  • create function which generates monthly usage reports by user and/or by remote host

To help you with this assignment, here is a template you can use as a starting point in designing your own Python Usage Report script.

For each function, identify what type of objects should be passed to the function, and what type of objects should be returned to the caller.

Once you have finished coding a function, you should start a Python3 interactive shell, import your functions and manually call the function you want to test under interactive mode and verify its correctness.

Once you have all the individual function tested and that each is working properly, perform the final test with test data provided by your professor and verify that your script produces the correct results before submitting your python program on Blackboard.

Sample login/logout records file and sample test run results

Rubric

Task Maximum mark Actual mark
Algorithm and program design 15
Program Authorship Declaration 5
Program usage 5
Program Options 15
Generate user name list 10
Generate remote host IP list 10
Daily Usage Report by User 10
Daily Usage Report by Remote Host 10
Weekly Usage Report by User 5
Weekly Usage Report by Remote Host 5
Monthly Usage Report by User 5
Monthly Usage Report by Remote Host 5
Total 100

Bonus item 10% extra

  • Create a function to capture that output of the 'last -Fiw' on the system your script is being executed and return the contents in the type of object as what you get from a regular login/logout record file.
  • Call this function and use the objects it returned when the file name input at the command line is 'last'.

Submission

After fully testing your program, submit your python script ur_[student_id].py and your algorithm and program design report ur_[student_id].pdf in pdf format on Blackboard by April 5th, 2019.

Retrieved from "https://wiki.cdot.senecapolytechnic.ca/w/index.php?title=OPS435_Python_Assignment_2&oldid=138360"