DPS909/OSD600 Fall 2017 Lab 8

Introduction: Using Continuous Integration (CI) in Open Source Projects

In this lab, we will build on the work you began in Lab 6/7. Our goal will be to setup a complete CI solution for our project using GitHub and TravisCI. We will also spend some time working with build systems, linters, and testing frameworks.

Step 1: Connect GitHub and TravisCI

We'll follow the steps in https://docs.travis-ci.com/user/getting-started/#To-get-started-with-Travis-CI

First, we will use your GitHub repo from Lab 6/7. Using your GitHub Login, sign-in to TravisCI.

Next, go to your TravisCI Profile page, sync your repositories (if not already visible), and find your GitHub Repo from Lab 6/7. Enable integration between TravisCI and GitHub by flipping the toggle switch to "on".

Step 2: Create a .travis.yml file in your repo

You need to tell TravisCI how to configure, build, and test your project. You do this via a special file that lives in the root of your project repo, named .travis.yml (the name is important, note the initial period). The format of the file is documented here. Here are some language specific docs:

• JavaScript with node.js
• Python
• Go
• C++
• Rust
• C#
• etc. See docs for other languages.

After you create your .travis.yml file, add, commit, and push it to your GitHub repository. You should be able to see the .travis.yml file on GitHub.

Step 3: Automate Your Build

Now you need to create automated "scripts" for your project, which TravisCI can run on every checkin. The method for accomplishing this will differ depending on your chosen programming language and environment. For example:

• JavaScript and node.js, you will use npm scripts in your package.json file.
• Python, you will use a custom script, for example pyscript.
• Go, you will use a Makefile or Go build file.
• C++, you will use a Makefile.
• Rust, you will use a Cargo.toml file.
• C#, you will use xbuild and a solution file.

You can start small and add more automation as you go. Initially, you should focus on compiling your code and/or running a Linter like eslint, Go's lint, Rust's clippy, etc. Linters allow us to find style issues, detect errors, and generally to standardize our code when many people are working on it in parallel.

You should be able to run your build locally (i.e., without TravisCI). If you can't run it, TravisCI won't be able to run it. Here are some things to consider:

• Do you specify all your dependencies? Have you used external modules or packages that need to be specified in some configuration file? TravisCI will need to know what they are so it can install them
• Do you have the proper script(s) added to the right file(s) to get your code to build, your linters to run? If not, figure out what your language/environment expects and do that. Remember to look at other people's projects on GitHub to see how they do things: everything TravisCI needs will be included in the repo (i.e., it will be "code" you can also read).
• Can you get your build/linting to pass/fail locally? What if you make an error on purpose? And after you fix it, do you see your build scripts change their result?

One you have everything configured locally, add, commit, and push your changes to GitHub. Visit your TravisCI page to confirm that your build is working.

Step 4: Make Your Code Testable

Go back to the code you wrote in the last lab, see if it needs to be made more testable. Testable code means:

• Code you can include in other code. You should write modules, packages, libraries vs. an executable or web server. You want a set of functions you can call from your test code.
• Code that takes input and returns output. You don't want your code to print things, display messages, or otherwise interact with the user. The "user" of your library will be another developer using code, not an application.

For example, this is wrong:

function add(a, b) {
  var result = a + b;
  console.log("The sum is " + result); 
}

Instead you would do:

function add(a, b) {
  return a + b;
}

This second version of the code is easy to test, since you can pass values for a and b and expect to get back known results.

Fix your code so that you can easily test it.

Step 5: Write Tests

Now that your code is testable, it's time to write some tests. First, pick a testing framework/tool:

• JavaScript/node.js: https://mochajs.org/ or https://facebook.github.io/jest/
• Python: Unittest or pytest or the like
• Go: built-in testing framework
• Rust: built-in testing framework
• etc.

Here are some sample tests you might write, see if you can come up with more:

• filename without path. Given a "/home/kim/mydata.txt" expect to get back "mydata.txt"
• file size in bytes. Given a path like "/home/kim/mydata.txt" of size 129 bytes, expect to get back 129. You'll need to create a file in your project (e.g., in a tests/ dir) that you can use.
• sha1 digest for a file. Given a file which contains the text "The quick brown fox jumps over the lazy dog" expect to get back 2fd4e1c67a2d28fced849ee1bb76e7391b93eb12
• MD5 digest for a file. Given a file which contains the text "The quick brown fox jumps over the lazy dog" expect to get back 9e107d9d372bb6826bd81d3542a419d6

You should keep each test small, focused on only one scenario (e.g., a given input). Write many small tests to test your functions vs. one giant program that tries to do everything. Other things to consider:

• What should happen if the input data is wrong, missing, or refers to incorrect file info? You should write tests that prove your code fails in known ways (e.g., throws an exception, returns an error code, etc)
• Any data you need in your tests (e.g., special files you want to test against) should be placed in your repo just like any other code.
• You don't need to write tests for things that your code doesn't do. For example, if you use an external library, or built-in function, to calculate hashes, you don't need to write a hundred tests for all cases of hashing. Your code is just calling that code, so test to make sure that it behaves and fails in the proper ways.

Step 6: Running Your Tests, Locally and on TravisCI

Now that you have your tests written, you need to modify your build scripts to automatically run them, and tell TravisCI how to do this via a single command. As outlined above in Step 2, this will differ depending on your programming language and environment. For example, in a node.js JavaScript project, you specify the script to run when npm test is called via your package.json file. You should be able to run your entire test suite (all your tests) from the command line with a single command. If you can do that, TravisCI should be able to as well.

Step 7: Triggering Builds on TravisCI

TravisCI will run your build/test cycle every time you commit new code on master or open a Pull Request. You can try triggering a build by committing/pushing a new change to your master branch. This should (eventually) cause TravisCI to re-build your code. You can check on your project's builds at https://travis-ci.org/{github-user-name}/{github-repo-name}/builds. Make sure you can get TravisCI to re-build when you update your repo, and also make sure you can get your build to complete and be Green (passed) vs. Red (failed).

Submission

Please add your info to the table below. Include a blog post about the process above, what you learned, what you found interesting/new, what you got stuck on or found hard, and any other ideas you might have had while doing this. Did you know about TravisCI before doing this? What did you think of it?

#	Name	GitHub Repo (URL)	TravisCI Project Builds (URLs)	Blog Post (URL)
1	Michael Pierre	https://github.com/MPierre9/FileInfo	https://travis-ci.org/MPierre9/FileInfo/builds/306046300 https://travis-ci.org/MPierre9/FileInfo/builds/306045572	https://michaelpierreblog.wordpress.com/2017/11/22/building-automated-tests-and-implementing-travis-ci/
2	Svitlana Galianova	https://github.com/svitlana-galianova/FileInfo	https://travis-ci.org/svitlana-galianova/FileInfo/builds/306140438 (Pass) https://travis-ci.org/svitlana-galianova/FileInfo/builds/306139255 (Fail)	http://svitlanagalianova.blogspot.ca/2017/11/review-is-travisci-my-happy-place.html
3	Dan Epstein	https://github.com/Securter/FileInfoFinder	https://travis-ci.org/Securter/FileInfoFinder/builds/306553740 (Pass) https://travis-ci.org/Securter/FileInfoFinder/builds/305962935 (Fail)	http://www.danepstein.ca/travis-ci-automated-testing-with-ruby/
4	Jay Yin	https://github.com/jayyyin/python-filelib-thingy	https://travis-ci.org/jayyyin/python-filelib-thingy/branches	http://jyopensource.blogspot.ca/2017/11/making-automated-testing-for-python-lib.html
5	Phil Henning	https://github.com/PhillypHenning/Bradley-site	https://travis-ci.org/PhillypHenning/Bradley-site/branches	https://wordpress.com/post/bluesockphil.wordpress.com/1065
6	Marco Beltempo	https://github.com/marcobeltempo/fileside	https://travis-ci.org/marcobeltempo/fileside/branches	https://www.marcobeltempo.com/open-source/continuous-integration/
7	Anthony LoMagno	https://github.com/ajlomagno/File-Info-Tool	https://travis-ci.org/ajlomagno/File-Info-Tool/builds/306523013?utm_source=github_status&utm_medium=notification	https://anthonylomagno.wordpress.com/2017/12/16/software-testing-the-right-way/
8	Leonel Jara	https://github.com/lejara/easy-python-file-utils	https://travis-ci.org/lejara/easy-python-file-utils/builds	https://lejara.wordpress.com/2017/11/29/learning-travis-ci-pytest-and-more-python/
9	Joao Rodrigues	https://github.com/jmrodriguesgoncalves/filepy	https://travis-ci.org/jmrodriguesgoncalves/filepy	https://jmrodriguesgoncalves.blogspot.ca/2017/11/lab-8.html
10	Sean Prashad	https://github.com/SeanPrashad/Fiffy	https://travis-ci.org/SeanPrashad/Fiffy/builds	https://medium.com/seanprashad/continuous-learning-91e7510fd60a
11	Steven De Filippis	https://github.com/St3v3n-D/file_info_lib	https://travis-ci.org/St3v3n-D/file_info_lib/builds	https://dps909.defilippis.ca/index.php/2017/11/28/finding-bugs-and-future-proofing-against-newer-ones-with-travis-ci/
12	Earle White	https://github.com/5earle/File_Info	https://travis-ci.org/5earle/File_Info	https://ewhite7blog.wordpress.com/2017/11/29/testing-python-with-travis-ci/
13	Joshua Longhi	https://github.com/Jlonghi/go-file-utils	https://travis-ci.org/Jlonghi/go-file-utils/builds/307339272 https://travis-ci.org/Jlonghi/go-file-utils/builds/307337315	https://jlonghiblog.wordpress.com/2017/11/29/travisci/
14	Avedis Zeitounilian	https://github.com/Avedis777/OpenSourceLabs	https://travis-ci.org/Avedis777/OpenSourceLabs	http://avedis777.blogspot.ca/2017/11/automated-testing.html
15	Fateh Sandhu	https://github.com/Fatehsandhu/DPS-OpenSource	https://travis-ci.org/Fatehsandhu/DPS-OpenSource/builds	https://firefoxmacblog.wordpress.com/2017/12/01/travis-tests/
16	Jiel Selmani	https://github.com/jselmani/Extractor	https://travis-ci.org/jselmani/Extractor/builds	https://www.jielselmani.me/blog/2017/12/1/learning-travis-and-how-you-can-too
17	Mat Babol	https://github.com/mmBabol/file-info	https://travis-ci.org/mmBabol/file-info	http://mmbabol.blogspot.ca/2017/12/setting-up-complete-ci-solution-for.html
18	Mithilan Sivanesan	https://github.com/Mithilan16/GoLangFileUtils	https://travis-ci.org/Mithilan16/GoLangFileUtils	https://mithilanblog.wordpress.com/2017/12/07/travici-automated-testing-and-building-of-code/
19	Eric Schvartzman	https://github.com/ericschv/node-extract-file-info	https://travis-ci.org/ericschv/node-extract-file-info
20	Hans van den Pol	https://github.com/HansvandenPol/FileFunctions	https://travis-ci.org/HansvandenPol/FileFunctions/builds/312799439	https://opensourcetoronto.wordpress.com/2017/12/11/learning-travis-ci/
21	Marvin Sanchez	https://github.com/msanchez5/DestinyFileIO	https://travis-ci.org/msanchez5/DestinyFileIO	https://marvinrsanchez.wordpress.com/learning-travis-ci
22	Nicholas Krause	https://github.com/xerofoify/Python-File-Utils	https://travis-ci.org/xerofoify/Python-File-Utils/builds/313562447	nicholas95com.wordpress.com/2017/12/08/travis-ci-tests/
23	Haoyu Yang	https://github.com/feihaozi77/BuildingJSLibrary	https://travis-ci.org/feihaozi77/BuildingJSLibrary	http://haoyu1337.blogspot.ca/2018/01/working-with-travis-ci.html
24	Hadi Saeed	http://github.com/hadialshakhori/python	https://travis-ci.org/HadiAlshakhori/Python/builds/318851876	https://techbreaksblog.wordpress.com/2017/12/25/how-to-use-continuous-integration-with-python-github-and-travis-ci/
25	Azusa Shimazaki	https://github.com/azusaaz/WhatsTheFile	https://travis-ci.org/azusaaz/WhatsTheFile	http://assmith2017.blogspot.ca/2017/12/using-continuous-integration-ci-in-open.html
26	Parthkumar Patel	https://github.com/ppatel221/xtractor	https://travis-ci.org/ppatel221/xtractor	https://ppatel221.wordpress.com/2017/12/28/testing-with-travis-ci/
27	Teddy Prempeh	https://github.com/teddypee/File_Info	https://travis-ci.org/teddypee/File_Info	https://wordpress.com/post/teddyprempeh.wordpress.com/156
28	Gaurav Verma	https://github.com/GauravV-02/JS-Extractor	https://travis-ci.org/GauravV-02/JS-Extractor/builds/324850405	https://gblogs2017.wordpress.com/2018/01/04/continuous-integration/
29	Eric Suarez	https://github.com/ericsuhn/jsfile-utils	https://travis-ci.org/ericsuhn/jsfile-utils	https://esoscode.wordpress.com/2018/01/04/learning-about-travis-ci/
30	Igor Naperkovskiy	https://github.com/naperkovskiy/FileInfoExtractor	https://travis-ci.org/naperkovskiy/FileInfoExtractor/builds/328920382?utm_source=email&utm_medium=notification	https://naperkovskiyblog.wordpress.com/2018/01/15/fileinfoextractor-app/
31
32
33
34
35
36
37
38
39
40