Changes

'''Weight:''' 1015% of the overall grade

* Complete the algorithm document requirements for this assignment script the first milestone and push to GitHub by November 27April 4, 2020 and submit on Blackboard 2021 by 11:59 PM,* Complete the your Python script and push to Github GitHub by December 11April 18, 2020 2021 at 11:59 PM, and* Copy your Python script into a Word document and submit to Blackboard by December 11April 18, 2020 2021 at 11:59 PM.

In this assignment, your should preform the following activities:# Complete you will create a detail algorithm for producing monthly script that can generate usage reports by user or by remote host based on the information stored in any given files generated off of output from the 'last' commandor from a file in a similar format. # Once you have complete You will use usage_data_file for testing but the detail algorithm, you script should then <b>design the structure of your python script</b> by identifying the appropriate python objects, functions and modules to also be used for each task in your algorithm tested on some other Linux machines and the main control logicon Matrix. Make sure to identify the followings:## input data, ## computation tasksDepending on the options selected, and ## outputs.# implement your computational solution using a single python script. You can use any built-in functions and functions from the python modules should list in the "Allowed Python Modules" section below to implement your solutionusers or remote IP addresses, either overall or limited by a specific date. # Test and review your working python code to see whether you can improve the interface of each function to facilitate better code re-use (this process is called <b>refactoring<It should also generate daily usage reports for specific users/b>)remote hosts, or weekly usage reports as well.

* The <b>timedatetime</b> module

** === Argparse ===Argparse is a much, much better way of dealing with command line arguments. It not only handles positional arguments, but options as well and will generate usage messages.It's <i>very highly recommended</i> that you spend at least a few minutes reading through the [https://docs.python.org/3/howto/argparse.html Argparse Tutorial] . === Datetime ===Since Python is a <i>batteries</i> included language, it's important to get accustomed with using some of the modules in the standard library. Since we are dealing with dates and times, you are required to work with the datetime module. The full docs can be found [https://docs.python.org/3/library/datetime.html here]. Datetime objects can be initialized from strings that match a particular format. One example is provided for you in the codebase: [https://docs.python.org/3/library/datetime.html#datetime.datetime.strptime the strptime] function. Once you have created datetime objects, you can do useful things with them: * <code> d1 > d2 </code> will return True is d1 is <b>later</b> than d2.* <code> d2 - should read d1 </code> will return a <i>timedelta</i> object, which is an amount of time between d2 and d1. More interesting methods are in the [https://docs.python.org/3/library/datetime.html#datetime.datetime.fold table] in the docs. === Timedelta ===When performing some operations with datetime objects, we may see timedelta objects being created. In math, <b>delta</b> refers to a difference. So a timedelta object is essentially an object that represents a duration. You can do useful things with timedelta objects:* <code> delta1 + delta2 </code> will add up two durations. For example, if delta1 is two hours and delta2 is three hours, then this firstoperation will return five hours.** <code> str(delta1) </code> will represent the timedelta in a friendly format: H:MM:SS if the duration is less than 24 hours.  More interesting methods are in the [https://docs.python.org/3/library/argparsedatetime.html Argparse API reference information page#datetime.datetime.fold table]in the docs.

Accept the Assignment #2 via the link on Blackboard, and clone the Github repository on a Linux machine of your choosing. Rename Your code should only be located in the file "a2_templateassignment2.py" to "a2_<your myseneca username>.py, just as we did in Assignment 1. You may also want to create a symbolic link using <code>ln -s a2_<myseneca_id>.py a2.py</code> to save time.

Name your Python3 script as <code>a2_[student_id].py</code>. Your script must accept one or more "file name" as its command line parameters and other optional parameters as shown below. Your python script should produce the following usage text when run with the --help option:

[eric@centos7 a1]$ python3 ./a2assignment2.py -husage: new_templateassignment2.py [-h] [-l {user,host}] [-r RHOST] [-t {daily,monthlyweekly}] [-u USERd DATE] [-su USER] [-v] F [F ...files]

F list of files file to be processed, if blank, will call last

-t {daily,monthlyweekly}, --type time {daily,monthlyweekly} type of report: daily day or monthlyweek -d DATE, --date DATE specify date for report

Copyright 2020 2021 - Eric Brauer 

You will that there is an 'args' object in a2_templateassignment2.py. Once the <code>parse_command_args()</code> function is called, it will return an args object. The command line arguments will be stored as attributes of that object. <b>Do not use sys.argv to parse arguments.</b> If there is only one file name provided at the command line, read the login/logout records from the contents of the given file. If the file name is "online", get the record on the system your script is being execute using the Linux command "last -iwF". The format of each line in the file should be the same as the output of 'last -Fiw'. Filter out incomplete login/logout record (hints: check for the number of fields in each record).

If there is more than one a file name providedat the command line, merge all read the login/logout records from the contents of the files together with given file. If there is not file name, get the first one at record on the top and system your script is being execute using the Linux command "last one at -iwF". The format of each line in the file should be the same as the bottomoutput of 'last -Fiw'. Read and process Filter out incomplete login/logout record (hints: check for the file contents number of fields in that order in your programeach record).

<source>OPS435 Assignment 2 - Fall 2020Winter 2021

<li><code>git add *assignment2.py</code>

<li> Copy a2_template.py into a2_<myseneca_id>.py. Replace with your Myseneca username.<li> Run the script itself. Investigate argparse. **<b>In the main block, print(args).** </b> Experiment with the various options.

<li> Use the check script to check your work: <code>./checkA2.py -f -v TestHelp</code>. It should succeed.<li> Investigate the `<code>parse_for_user()` </code> function, with the <code>usage_data_file</code>. **In main, call `parse_for_user()` with `output` as the argument. Investigate what's returned.**<li> `<code>parse_for_user()` </code> should take the list of lines from the file, and instead return a list of usernames. **<b>In main, print the title header and the output.** <b>Commit the change.</b><li> **<b>Once you have `output` --> `parse_for_user()` --> correct output being printed, use if conditions to print only when `-l user` is in the command line arguments.** </b><li> Test using <code>./checkA2.py -f -v TestList</code>. You should see some tests succeeding, but some failing. Use the check script to start implementing the functions needed for <b>-l host</b>.<li> <b>Continue committing these changes as your proceed.</b><li> Implement the same things as parse_for_user but for `parse_for_hosts`. Output Your script should now be sortedpassing the TestList tests. <li> Compare your output with Now implement the output below.-d <lidate> Write the `parse_for_daily()` function using the pseudocode givenoption. This should be taking the will filter your user list of lines from your file, and output a dictionary with start dates in DD/MM/YYYY format as based on the key and usage in seconds as date provided by the valueuser.<li> Use <code> {'01./01/1980': 1200, '02/01/1980': 2400, '03/01/1980': 2200} checkA2.py -f -v TestDate</code>to check your work. <lib> Once your `parse_for_daily()` function works, call it with You have completed the argparse options, and display the contents.first milestone!</b><li> Write (or modify) a function The next stage will be to do implement the same for remote hostsdaily/weekly reports.Use <code>TestDaily<li/code> Implement the outputting of the duration in HH:MM:SS instead of seconds. It's recommended you write a function to take in seconds and return a string. Call this when the `-s` option is absent. Make sure this is working with remote hosts as well. You should now have x of y tests passing.<licode>TestWeekly</code> Finally, implement with the `--monthly` option. Create a new function and get it working. start with seconds, then duration and make sure it works with remote as wellcheck script.

The second column should be 15 characters long and be right-aligned.  <pre>Daily Usage Report for rchan < same number of characters============================ <Date Usage < right justified13/02/2018 0:26:00 <15/02/2018 0:33:00Total 0:59:20llllllllllrrrrrrrrrrrrrrr < left column is 10 chars, right column is 15^left just.</pre>

The following are the reports generated by the usage report script (ur.py) with the "usage_data_file" mentioned in the overview section. You can download the file [https://scs.senecac.on.ca/~raymond.chan/ops435/a2/usage_data_file here] to test your ur.py script.

[eric@centos7 a2]$ ./a2assignment2.py -l user usage_data_file

[eric@centos7 a2]$ ./a2assignment2.py -l host usage_data_file

==== Daily Usage Report by User The Verbose Option ====The Either of the following are Daily Usage Reports created for user rchan. The output two tests can be displayed either in secondsmodified with the <code>--verbose</code> option. You shouldn't have to do anything to get this working:

[eric@centos7 a2]$ ./a2assignment2.py -u rchan l host -t daily v usage_data_file --seconds

Daily Usage Report Files to be processed: usage_data_fileType of args for files <class 'str'>Host list for rchanusage_data_file=============================Date Usage10.40.105.13013/02/2018 158010.40.91.23615/02/2018 198010.40.91.247Total 356010.43.115.162

...or by omitting the ==== List For Specific Day ====Specifying a <code>--secondsdate</code> option, in HH:YYYY-MM:SS -DD formatshould list all users or hosts that were logged in at some point during that date, even if their start time or end time is different. For example, user <code>cwsmith</code> logged in on Feb 14 and logged off on Feb 15, but they show up when the following command is run:

[eric@centos centos7 a2]$ ./a2assignment2.py -u rchan l user -d 2018-02-t daily 14 usage_data_file

Daily Usage Report User list for rchanusage_data_file=============================Date Usage13/02/2018 00:26:0015/02/2018 00:33:00Total 00:59:20cwsmith

==== Daily Usage Report by Remote Host====The following is a Daily Usage Report created This should work for the Remote Host 10.40.105.103 by the commandhosts as well:

[eric@centos7 a2]$ ./a2assignment2.py -r 10.40.105.130 l host -d 2018-02-t daily 14 usage_data_file -s

Daily Usage Report Host list for 10.40.105.130usage_data_file====================================Date Usage14/02/2018 1093113/02/2018 7969Total 1890010.40.105.130

Just as you did with <code>--If the user</code>types in an invalid date, your the script should also display halt and print the time in HH:MM:SS by omitting the <code>--seconds</code> option. ==== Monthly Usage Report by User ====The following is a Monthly Usage Report created for user rchan by the commanderror message:

[eric@centos7 a2]$ ./a2assignment2.py -u rchan l host -t monthly d 2018-02-xx usage_data_file -s

Monthly Usage Report for rchan==============================Date Usage02/2018 3560Total 3560not recognized. Use YYYY-MM-DD format.

[eric@centos7 centos a2]$ ./a2assignment2.py -u cwsmith rchan -t monthly daily usage_data_file

Monthly Daily Usage Report for cwsmithrchan================================

13/02/2018 03 0:0226:11000315/02/2018 00 0:3833:00Total 03 0:4059:1120

Be also sure to test with the <code>--verbose</code> ==== Monthly Daily Usage Report by Remote Host ====The following is a Monthly Daily Usage Report created for the remote host Remote Host 10.40.105.130 103 by the command:

[eric@centos7 a2]$ ./a2assignment2.py -r 10.40.105.130 -t monthly daily usage_data_file

Monthly Daily Usage Report for 10.40.105.130======================================

14/02/2018 3:02:1113/02/2018 05 2:1512:0049Total 05 5:15:00

As discussed before, this command should also accept the <code>--seconds</code> option. ==== List Users With Verbose Weekly Usage Report by User ====Calling any of The following is a Weekly Usage Report created for user cwsmith by the previous commands with the <code>--verbose</code> option should cause the script to output more informationcommand:

[eric@centos7 a2]$ ./a2assignment2.py -l user u cwsmith -t weekly usage_data_file -v

Files to be processed: ['usage_data_file']Type of args Weekly Usage Report for files <class 'list'>User list for usage_data_filecwsmith===============================asmithDate Usagecwsmith2018 06 3:02:11rchan2018 10 0:38:00tsliu2Total 3:40:11

[eric@centos7 a2]$ ./a2assignment2.py -r 10.40.105.130 -t monthly weekly usage_data_file -v

Files to be processed: ['usage_data_file']Type of args for files <class 'list'>usage report for remote host: 10.40.105.130usage report type: monthlyMonthly Weekly Usage Report for 10.40.105.130======================================

02/2018 0506 0:1502:0031Total 05 0:1502:0031

Running the script with "online" <B>no filename</b> as a file argument should call a subprocess.Popen object and run the command <code>last -Fiw</code>.

[eric@mtrx-node06pd ~]$ ./a2assignment2.py -l user online

[eric@mtrx-node06pd ~]$ ./a2assignment2.py -u adas20 -t daily online

=== Detail Algorithm Document ===Follow the standard computation procedure: input - process - ouput when creating the algorithm document Please note that there will no unit test for this assignment.==== input ====* get data (command line arguments/options) from the user using the functions provided by the argparse module* according to the arguments/options given at the command line, take appropriate processing action. ==== processing ====* based on the file(s) specified, read the contents of each file and use appropriate objects to store but it* based on the command line arguments/options, process the data accordingly, which includes** data preprocessing (split is still a multi-day record into single day record)** record processing (preform required computation)==== output ====* output the required report based on the processed data==== identify and select appropriate python objects and functions ====The following python functions (to be created, you may have more) are useful in handling the following sub-tasks:* reads login records from files and filters out unwanted records* convert login records into proper python object type so that it can be processed using as much built-in functions as possible * create functions which generate daily usage reports by user and/or by remote host* create functions which generate monthly usage reports by user and/or by remote hostrequirement.

To help you with this assignment=== First Milestone ===By April 4, you should use have your <code>TestHelp</code>, <code>TestList</code> and <code>TestDate</code> tests all passing. Make sure that the a2_template.py code is in the your GitHub repository as . I will use a starting point in designing your own Python Usage Report scriptpull request comment to give feedback, suggest changes or get you unstuck.

| Algorithm Submission First Milestone || 10 ||

| Daily/Monthly Weekly Functions || 10 ||

| Output Date Functions || 5 ||

| Other Output/other Functions || 5 ||

* Stage 1: Submit your algorithm document file to Blackboard Complete the first milestone on GitHub by November 27April 4, 20202021.* Stage 2: Use commits to push your python script for this assignment to Github.com. The final state of your repository will be looked at on December 11April 18, 2020 2021 at 11:59 PM.* Stage 3: Copy your python script into a Word document and submit to Blackboard by December 11April 18, 2020 2021 at 11:59 PM.