Open main menu

CDOT Wiki β

Changes

OPS445/A2-F23-NAANDD

16,943 bytes added, 11:57, 29 October 2023
initial commit for F23
<span id="assignment-2"></span>
= Assignment 2 =

<span id="overview"></span>
== Overview ==

Create a script that presents information about process memory usage in a user friendly way.

Processes are a way of abstracting the different threads on a Linux system. Whenever you run a program from the terminal, or launch an application from the GUI, you start one or more processes. Processes require memory and CPU time until they are closed or finish. You have already learned about investigating processes using commands such as <code>ps</code> or <code>top</code>. Your task will be to create a basic program similar to these.

<span id="program-function"></span>
=== Program Function ===

When your script runs without arguments, it will show how much is memory is being used out of the total available:

<code>./assignment2.py</code>

<pre>Memory [############ | 62%] 9408404/15221204</pre>
When your scripts is passed the name of a program, it will show how much memory each process is running:

<code>./assignment2.py firefox</code>

<pre>248203 [ | 0%] 73872/15221204
248165 [ | 0%] 73544/15221204
248163 [ | 0%] 74020/15221204
248154 [ | 0%] 50444/15221204
247738 [ | 1%] 103872/15221204
247715 [ | 1%] 124872/15221204
247422 [ | 0%] 54212/15221204
247301 [# | 3%] 388440/15221204
firefox [# | 6%] 943276/15221204</pre>
Using the <code>-H</code> option will have your program output memory totals in '''human readable format''' (GiB instead of billions of bytes).

Using the <code>-l</code> option with a number will adjust the length of the '''bar graph'''.

Please see the '''Sample Output''' section for more details.

<span id="how-it-works"></span>
=== How It Works ===

<span id="total-memory-use"></span>
==== Total Memory Use ====

Information about processes can be found in the <code>/proc</code> directory. Use <code>ls</code> to investigate this location now. Pay particular attention to the directories that are named after process IDs (pids) and files such as <code>meminfo</code>.

When we calculate memory usage on a Linux machine, we have to take many things into account: unused memory is considered ''wasted'', and so available memory is often used to ''cache'' data that is no longer is use, but may need to be accessed in the future. Generally any memory that isn’t '''free, or being used for buffering or caching''' is considered '''in use'''.

Memory that is considered ''available'' is that which can be used to start new applications without swapping.

'''For simplicity’s sake, for this assignment “Memory in use” = Total Memory - Available Memory.'''

Both of these memory amounts can be found inside the <code>/proc/meminfo</code> file.

<span id="tracking-processes-for-a-running-program"></span>
==== Tracking Processes For A Running Program ====

For this assignment, we will ask you to get the Process ID numbers (pids) of an application using the <code>pidof</code> command. As you will discover, simple CLI programs like <code>grep</code> might only use a single process, whereas a more complex GUI program like Firefox will use several.

<span id="memory-use-per-process"></span>
==== Memory Use Per Process ====

Memory usage for a single process can be very complex. When an application starts, a process will map a block of memory for possible usage. This is '''Virtual Set Size (Vss)'''. This number will not actually tell you how much ''physical memory'' is actually being used by the process.

'''Resident Memory (Rss)''' is closer to the actual amount of physical memory used by a process. However, a block of memory might be shared across several processes, the Resident memory amount inaccurate.

When you use a tool such as '''Process Monitor''' or '''top''', generally the memory amount being reported is '''Rss'''. So for simplicity’s sake, '''for this assignment the memory used by a process should be the resident total'''.

The total amount of Rss memory can be found in <code>/proc/pid</code>, where pid is the process id number. In this directory you will find a file called <code>smaps</code>. Your program will need to read this file, find each line that begins with '''Rss''', and sum the number to get total Rss memory used.

For more information on how memory and /proc in Linux refer to these manual pages:

* <code>man 5 proc</code>
* <code>man free</code>
* [https://www.golinuxcloud.com/check-memory-usage-per-process-linux/ This page] is also an excellent explanation of memory.

'''TL;DR''': The process we are using for this assignment is not accurate, but it will get us close enough!

<span id="restrictions"></span>
== Restrictions ==

* You may '''only use the following modules:'''
** argparse
** os
** sys
* No other modules are allowed

<span id="instructions"></span>

== Instructions ==

The assignment will be broken into two milestones and one final submission. Please update your repository for each milestone and complete the final submission in order to earn all marks.

* Milestone 1 will be due on '''March 31'''.
* Milestone 2 will be due on '''April 14'''.
* The Final submission will be due on '''April 21'''.


-----

<span id="getting-started"></span>
=== Getting Started ===

# Accept the assignment on Blackboard.
# Once you accepted the assignment, you will get access to a starting repo.
# Add your code to the ''existing'' <code>assignment2.py</code> file.
# Commit after '''each significant change''' to the code.
# '''You can never have too many commits.''' GitHub is your proof of work and your backup if things go wrong.

<span id="first-milestone"></span>
=== First Milestone ===

Complete the following functions. The function definitions are already provided in the <code>assignment2.py</code> file:

* percent_to_graph()
* get_sys_mem()
* get_avail_mem()

<span id="percent-to-graph-function"></span>
==== Percent to Graph Function ====

This function will convert a float between '''0.0 - 1.0''' into a string of hash symbols and spaces: '''‘####’'''.

This will require you to use a ''scaling formula''.

[[File:Scaling-formula.png|thumb|none|alt=equation|equation]]

<span id="get-total-and-available-system-memory"></span>

==== Get Total and Available System Memory ====

<code>get_sys_mem()</code> and <code>get_avail_mem()</code> will both use <code>open()</code> to open the <code>/proc/meminfo</code> file, and return the relevant memory amounts as '''integers'''.

Once you have completed these functions, you have everything you need to '''print the total memory used graph''' as you have seen at the top of this page!

<span id="checking-your-work"></span>
==== Checking Your Work ====

A check script is provided to you in the repository. Run the following checks to verify your work:

<code>python3 ./checkA2.py -f -v TestPercent</code>

<code>python3 ./checkA2.py -f -v TestMemFuncs</code>

<span id="feedback"></span>

==== Feedback ====

I will provide you feedback on '''GitHub'''. Check the '''Issues''' tab, and make any changes that are required before the next deadline. You can '''close the issue''' to indicate that you have seen the comment.

<span id="second-milestone"></span>
=== Second Milestone ===

Complete the following functions:

* parse_command_args()
* pids_of_prog()
* rss_mem_of_pid()

<span id="command-arguments-using-argparse"></span>
==== Command Arguments Using Argparse ====

You will be using a module in the standard library called '''Argparse'''. This will help handle more complex sets of options and arguments than simply using sys.argv. Refer to the [https://docs.python.org/3/howto/argparse.html#id1 argparse documentation] to complete the parse_command_args function. At minimum, your assignment should handle the following options and arguments:

* <code>-h</code> will print a usage message. This will automatically be created by argparse itself, you will not need to implement this. However, refer carefully to the sample output and ensure that your help message matches the required output.
* <code>-H</code> will print file sizes in Human readable format. For example, 1024 bytes will be printed as 1KiB, 1024 kilobytes will be printed as 1MiB, and so on.
* <code>-l &lt;number&gt;</code> will set the maximum length of the bar graph. The default should be 20 character. This option will require an option argument that is an integer.
* Your script will also check for one optional positional argument which contains the name of a running program for scanning.

A function has been provided for you, most of the work is already complete. However, you will have to add one more option to it in order to get full marks.

Your assignment should be able to produce the following:

<code>./assignment2.py -h</code>

<pre>usage: assignment2.py [-h] [-H] [-l LENGTH] [program]

Memory Visualiser -- See Memory Usage Report with bar charts

positional arguments:
program if a program is specified, show memory use of all
associated processes. Show only total use if not.

options:
-h, --help show this help message and exit
-H, --human-readable Prints sizes in human readable format
-l LENGTH, --length LENGTH
Specify the length of the graph. Default is 20.

Copyright 2023</pre>
<span id="checking-your-work-1"></span>
==== Checking Your Work ====

Run the following checks to verify your work:

<code>python3 ./checkA2.py -f -v TestParseArgs</code>

<code>python3 ./checkA2.py -f -v TestPidList</code>

<code>python3 ./checkA2.py -f -v TestPidMem</code>

<span id="feedback-1"></span>

==== Feedback ====

I will provide you feedback on '''GitHub'''. Check the '''Issues''' tab, and make any changes that are required before the next deadline. You can '''close the issue''' to indicate that you have seen the comment.

<span id="final-submission"></span>
=== Final Submission ===

For the final step, you will need to tie everything together:

* Use the <code>args</code> object to check the options and arguments inputted by the user.
* Display the output in formatted columns. You will probably want to use f-strings, and set a specific width for each variable so that everything lines up.
* Make sure the <code>-H</code> and <code>-l</code> options change your output.

<span id="add-human-readable-output"></span>
==== Add Human Readable Output ====

The <code>meminfo</code> file has been storing memory amounts in ''kibibytes'' (kiB). One kibibytes equals 1024 bytes and is the correct way of presenting a number of bytes in a human readable format.

When the user uses the option <code>-H</code>, your script should present the amount of memory using mebi (MiB), gibi (GiB) or tebibytes (TiB).

This function is provided for you, but you should '''comment this code to explain how it works'''.

<span id="final-checks"></span>
==== Final Checks ====

Run the check script without arguments to verify all parts of your assignment. Output will be checked manually.

<code>python3 ./checkA2.py -f -v</code>

<span id="submitting-your-code-for-review"></span>
==== Submitting Your Code For Review ====

# Push your code to GitHub before the deadline.
# In addition, '''submit your code to Blackboard'''. A link will be provided.

<span id="other-considerations"></span>
== Other Considerations ==

A significant amount of your mark will be based on the things that ''aren’t'' your code. Please review the following guidelines to maximise your grade.

<span id="comments-and-documentation"></span>
=== Comments And Documentation ===

'''You need to be commenting your code'''. The following documentation is required:

* '''in-line comments''': Any line of code that is doing something non-obvious should be commented. Explain '''why''' you are doing something, rather than '''what''' you are doing.
* '''function docstrings''': After your <code>def</code> line, you enter a docstring inside ““. Any function that doesn’t already come with a docstring should have one.
* '''top-level docstring''': You will have noticed that the top of your <code>assignment1.py</code> file already has this docstring. Complete the '''Academic Honesty declaration''' and complete the docstring.

<span id="pep"></span>
=== PEP ===

The [https://peps.python.org/pep-0008/ PEP-8 Style Guide] is an official Python document that describes best practices for formatting your code. '''You should follow this guide as much as possible'''. You may find that [https://code.visualstudio.com/docs/python/linting using a linter] to check style to be useful.

<span id="functions-and-variables"></span>
=== Functions and Variables ===

* In addition to the required functions, you may create as many functions as you need.
* Functions should be in lower case, and spaces should be represented with an underscore. For example: <code>function_name</code>.
* Any data used inside of a function should be passed in as a parameter. Avoid global variables.
* Variables should have a sensible name. Avoid naming things <code>x</code>.
* Variables should be in lower case, and spaces represented by an underscore. For example: <code>start_date</code>.

<span id="git-commits"></span>
=== Git Commits ===

A workplace will have its own policy about how often to commit, and how to document commits. For us, '''git commits are your proof of work'''. Assignments that lack commits are subject to '''Academic Integrity review'''.

* A good practice is to create a commit for '''every significant change''' to your code. '''At the very least, commit after completing each of the required functions.'''
* An acceptable commit message needs to short, but should also describe the change. For example: <code>git commit -m &quot;completed the leap_year function&quot;</code>.

<span id="rubric"></span>
== Rubric ==

{|
! Element
!align="right"| Marks
|-
| '''Milestone 1:'''
|align="right"|
|-
| TestPercent Checks
|align="right"| 4
|-
| TestMemFuncs Checks
|align="right"| 6
|-
| '''Milestone 2:'''
|align="right"|
|-
| TestParseArgs Checks
|align="right"| 2
|-
| TestPidList Checks
|align="right"| 4
|-
| TestPidMem Checks
|align="right"| 4
|-
| '''Final Submission:'''
|align="right"|
|-
| Final Checks
|align="right"| 10
|-
| Output Presentation
|align="right"| 5
|-
| Comments and Documentation
|align="right"| 5
|-
| github use
|align="right"| 5
|-
| functions and style
|align="right"| 2
|-
| error checking
|align="right"| 3
|}

<span id="sample-output"></span>
== Sample Output ==

<code>./assignment2.py</code>

<pre>Memory [############# | 65%] 9890952/15221204</pre>
<code>./assignment2.py firefox</code>

<pre>295117 [ | 1%] 76748/15221204
295071 [ | 1%] 76692/15221204
295067 [ | 1%] 76500/15221204
295065 [ | 0%] 51912/15221204
294659 [ | 1%] 119720/15221204
294629 [ | 2%] 228976/15221204
294606 [ | 0%] 62112/15221204
294524 [ | 2%] 379096/15221204
firefox [# | 7%] 1071756/15221204</pre>
<code>./assignment2.py -H firefox</code>

<pre>295117 [ | 1%] 74.95 MiB/14.52 GiB
295071 [ | 1%] 74.89 MiB/14.52 GiB
295067 [ | 1%] 74.89 MiB/14.52 GiB
295065 [ | 0%] 50.70 MiB/14.52 GiB
294659 [ | 1%] 109.53 MiB/14.52 GiB
294629 [ | 1%] 145.89 MiB/14.52 GiB
294606 [ | 0%] 60.66 MiB/14.52 GiB
294524 [ | 2%] 327.28 MiB/14.52 GiB
firefox [# | 6%] 918.79 MiB/14.52 GiB</pre>
<code>./assignment2.py -l 40 nvim</code>

<pre>293104 [ | 0%] 22636/15221204
nvim [ | 0%] 22636/15221204</pre>
<code>./assignment2.py -l 50</code>

<pre>Memory [################################## | 68%] 10282860/15221204</pre>
<code>./assignment2.py oopsie</code>

<pre>oopsie not found.</pre>
<code>./assignment2.py -h</code>

<pre>usage: assignment2.py [-h] [-H] [-l LENGTH] [program]

Memory Visualiser -- See Memory Usage Report with bar charts

positional arguments:
program if a program is specified, show memory use of all
associated processes. Show only total use if not.

options:
-h, --help show this help message and exit
-H, --human-readable Prints sizes in human readable format
-l LENGTH, --length LENGTH
Specify the length of the graph. Default is 20.

Copyright 2023</pre>