667
edits
Changes
Created page with "==Introduction: Using Continuous Integration (CI) in Open Source Projects== In this lab, we will build on the work you began in Lab 6/7...."
==Introduction: Using Continuous Integration (CI) in Open Source Projects==
In this lab, we will build on the work you began in [[DPS909/OSD600 Fall 2017 Lab 6/7|Lab 6/7]]. Our goal will be to setup a complete CI solution for our project using GitHub and TravisCI. We will also spend some time working with build systems, linters, and testing frameworks.
==Step 1: Connect GitHub and TravisCI==
We'll follow the steps in https://docs.travis-ci.com/user/getting-started/#To-get-started-with-Travis-CI
First, we will use your GitHub repo from [[DPS909/OSD600 Fall 2017 Lab 6/7|Lab 6/7]]. Using your GitHub Login, [https://travis-ci.org/auth sign-in to TravisCI].
Next, go to your [https://travis-ci.org/profile TravisCI Profile page], sync your repositories (if not already visible), and find your GitHub Repo from [[DPS909/OSD600 Fall 2017 Lab 6/7|Lab 6/7]]. Enable integration between TravisCI and GitHub by flipping the toggle switch to "on".
==Step 2: Create a .travis.yml file in your repo==
You need to tell TravisCI how to configure, build, and test your project. You do this via a special file that lives in the root of your project repo, named <code>.travis.yml</code> (the name is important, note the initial period). The [https://docs.travis-ci.com/user/customizing-the-build format of the file is documented here]. Here are some language specific docs:
* [https://docs.travis-ci.com/user/languages/javascript-with-nodejs/ JavaScript with node.js]
* [https://docs.travis-ci.com/user/languages/python/ Python]
* [https://docs.travis-ci.com/user/languages/go/ Go]
* [https://docs.travis-ci.com/user/languages/cpp/ C++]
* [https://docs.travis-ci.com/user/languages/rust/ Rust]
* [https://docs.travis-ci.com/user/languages/csharp/ C#]
* etc. See [https://docs.travis-ci.com/user/customizing-the-build/#Specifying-Runtime-Versions docs for other languages].
After you create your <code>.travis.yml</code> file, add, commit, and push it to your GitHub repository. You should be able to see the <code>.travis.yml</code> file on GitHub.
==Step 3: Automate Your Build==
Now you need to create automated "scripts" for your project, which TravisCI can run on every checkin. The method for accomplishing this will differ depending on your chosen programming language and environment. For example:
* JavaScript and node.js, you will use [https://docs.npmjs.com/misc/scripts npm scripts] in your [https://docs.npmjs.com/files/package.json package.json] file.
* Python, you will use [https://docs.travis-ci.com/user/languages/python/#Default-Build-Script a custom script, for example pyscript].
* Go, you will use a [https://docs.travis-ci.com/user/languages/go/#Default-Build-Script Makefile or Go build file].
* C++, you will use a [https://docs.travis-ci.com/user/languages/cpp/#Default-Build-Script Makefile].
* Rust, you will use a [https://docs.travis-ci.com/user/languages/rust/#Default-Build-Script Cargo.toml file].
* C#, you will use [https://docs.travis-ci.com/user/languages/csharp/#Script xbuild and a solution file].
You can start small and add more automation as you go. Initially, you should focus on compiling your code and/or running a Linter like eslint, Go's lint, Rust's clippy, etc.
You should be able to run your build locally (i.e., without TravisCI). I you can't run it, TravisCI won't be able to run it. Here are some things to consider:
* Do you specify all your dependencies? Have you used external modules or packages that need to be specified in some configuration file? TravisCI will need to know what they are so it can install them
* Do you have the proper script(s) added to the right file(s) to get your code to build, your linters to run? If not, figure out what your language/environment expects and do that. Remember to look at other people's projects on GitHub to see how they do things: everything TravisCI needs will be included in the repo (i.e., it will be "code" you can also read).
* Can you get your build/linting to pass/fail locally? What if you make an error on purpose? And after you fix it, do you see your build scripts change their result?
One you have everything configured locally, add, commit, and push your changes to GitHub. Visit your TravisCI page to confirm that your build is working.
==Step 4: Make Your Code Testable==
Go back to the code you wrote in the last lab, see if it needs to be made more testable. Testable code means:
* Code you can include in other code. You should write modules, packages, libraries vs. an executable or web server. You want a set of functions you can call from your test code.
* Code that takes input and returns output. You don't want your code to print things, display messages, or otherwise interact with the user. The "user" of your library will be another developer using code, not an application.
For example, this is wrong:
<pre>
function add(a, b) {
var result = a + b;
console.log("The sum is " + result);
}
</pre>
Instead you would do:
<pre>
function add(a, b) {
return a + b;
}
</pre>
This second version of the code is easy to test, since you can pass values for <code>a</code> and <code>b</code> and expect to get back known results.
Fix your code so that you can easily test it.
==Step 5: Write Tests==
Now that your code is testable, it's time to write some tests. First, pick a testing framework/tool:
* JavaScript/node.js: https://mochajs.org/ or https://facebook.github.io/jest/
* Python: [http://docs.python-guide.org/en/latest/writing/tests/ Unittest or pytest or the like]
* Go: [https://golang.org/pkg/testing/ built-in testing framework]
* Rust: [https://doc.rust-lang.org/book/second-edition/ch11-01-writing-tests.html built-in testing framework]
* etc.
Here are some sample tests you might write, see if you can come up with more:
* filename without path. Given a "/home/kim/mydata.txt" expect to get back "mydata.txt"
* file size in bytes. Given a path like "/home/kim/mydata.txt" of size 129 bytes, expect to get back 129. You'll need to create a file in your project (e.g., in a tests/ dir) that you can use.
* sha1 digest for a file. Given a file which contains the text "The quick brown fox jumps over the lazy dog" expect to get back 2fd4e1c67a2d28fced849ee1bb76e7391b93eb12
* MD5 digest for a file. Given a file which contains the text "The quick brown fox jumps over the lazy dog" expect to get back 9e107d9d372bb6826bd81d3542a419d6
You should keep each test small, focused on only one scenario (e.g., a given input). Write many small tests to test your functions vs. one giant program that tries to do everything. Other things to consider:
* What should happen if the input data is wrong, missing, or refers to incorrect file info? You should write tests that prove your code fails in known ways (e.g., throws an exception, returns an error code, etc)
* Any data you need in your tests (e.g., special files you want to test against) should be placed in your repo just like any other code.
* You don't need to write tests for things that your code doesn't do. For example, if you use an external library, or built-in function, to calculate hashes, you don't need to write a hundred tests for all cases of hashing. Your code is just calling that code, so test to make sure that it behaves and fails in the proper ways.
==Step 6: Running Your Tests, Locally and on TravisCI==
Now that you have your tests written, you need to modify your build scripts to automatically run them, and tell TravisCI how to do this via a single command. As outlined above in Step 2, this will differ depending on your programming language and environment. For example, in a node.js JavaScript project, you specify the script to run when <code>npm test</code> is called via your <code>package.json</code> file. You should be able to run your entire test suite (all your tests) from the command line with a single command. If you can do that, TravisCI should be able to as well.
==Step 7: Triggering Builds on TravisCI==
TravisCI will run your build/test cycle every time you commit new code on <code>master</code> or open a Pull Request. You can try triggering a build by committing/pushing a new change to your <code>master</code> branch. This should (eventually) cause TravisCI to re-build your code. You can check on your project's builds at https://travis-ci.org/{github-user-name}/{github-repo-name}/builds. Make sure you can get TravisCI to re-build when you update your repo, and also make sure you can get your build to complete and be Green (passed) vs. Red (failed).
==Submission==
Please add your info to the table below. Include a blog post about the process above, what you learned, what you found interesting/new, what you got stuck on or found hard, and any other ideas you might have had while doing this. Did you know about TravisCI before doing this? What did you think of it?
{| class="wikitable"
! style="font-weight: bold;" | #
! style="font-weight: bold;" | Name
! style="font-weight: bold;" | GitHub Repo (URL)
! style="font-weight: bold;" | TravisCI Project Builds (URLs)
! style="font-weight: bold;" | Blog Post (URL)
|-
| 1
|
|
|
|
|-
| 2
|
|
|
|
|-
| 3
|
|
|
|
|-
| 4
|
|
|
|
|-
| 5
|
|
|
|
|-
| 6
|
|
|
|
|-
| 7
|
|
|
|
|-
| 8
|
|
|
|
|-
| 9
|
|
|
|
|-
| 10
|
|
|
|
|-
| 11
|
|
|
|
|-
| 12
|
|
|
|
|-
| 13
|
|
|
|
|-
| 14
|
|
|
|
|-
| 15
|
|
|
|
|-
| 16
|
|
|
|
|-
| 17
|
|
|
|
|-
| 18
|
|
|
|
|-
| 19
|
|
|
|
|-
| 20
|
|
|
|
|-
| 21
|
|
|
|
|-
| 22
|
|
|
|
|-
| 23
|
|
|
|
|-
| 24
|
|
|
|
|-
| 25
|
|
|
|
|-
| 26
|
|
|
|
|-
| 27
|
|
|
|
|-
| 28
|
|
|
|
|-
| 29
|
|
|
|
|-
| 30
|
|
|
|
|-
| 31
|
|
|
|
|-
| 32
|
|
|
|
|-
| 33
|
|
|
|
|-
| 34
|
|
|
|
|-
| 35
|
|
|
|
|-
| 36
|
|
|
|
|-
| 37
|
|
|
|
|-
| 38
|
|
|
|
|-
| 39
|
|
|
|
|-
| 40
|
|
|
|
|}
In this lab, we will build on the work you began in [[DPS909/OSD600 Fall 2017 Lab 6/7|Lab 6/7]]. Our goal will be to setup a complete CI solution for our project using GitHub and TravisCI. We will also spend some time working with build systems, linters, and testing frameworks.
==Step 1: Connect GitHub and TravisCI==
We'll follow the steps in https://docs.travis-ci.com/user/getting-started/#To-get-started-with-Travis-CI
First, we will use your GitHub repo from [[DPS909/OSD600 Fall 2017 Lab 6/7|Lab 6/7]]. Using your GitHub Login, [https://travis-ci.org/auth sign-in to TravisCI].
Next, go to your [https://travis-ci.org/profile TravisCI Profile page], sync your repositories (if not already visible), and find your GitHub Repo from [[DPS909/OSD600 Fall 2017 Lab 6/7|Lab 6/7]]. Enable integration between TravisCI and GitHub by flipping the toggle switch to "on".
==Step 2: Create a .travis.yml file in your repo==
You need to tell TravisCI how to configure, build, and test your project. You do this via a special file that lives in the root of your project repo, named <code>.travis.yml</code> (the name is important, note the initial period). The [https://docs.travis-ci.com/user/customizing-the-build format of the file is documented here]. Here are some language specific docs:
* [https://docs.travis-ci.com/user/languages/javascript-with-nodejs/ JavaScript with node.js]
* [https://docs.travis-ci.com/user/languages/python/ Python]
* [https://docs.travis-ci.com/user/languages/go/ Go]
* [https://docs.travis-ci.com/user/languages/cpp/ C++]
* [https://docs.travis-ci.com/user/languages/rust/ Rust]
* [https://docs.travis-ci.com/user/languages/csharp/ C#]
* etc. See [https://docs.travis-ci.com/user/customizing-the-build/#Specifying-Runtime-Versions docs for other languages].
After you create your <code>.travis.yml</code> file, add, commit, and push it to your GitHub repository. You should be able to see the <code>.travis.yml</code> file on GitHub.
==Step 3: Automate Your Build==
Now you need to create automated "scripts" for your project, which TravisCI can run on every checkin. The method for accomplishing this will differ depending on your chosen programming language and environment. For example:
* JavaScript and node.js, you will use [https://docs.npmjs.com/misc/scripts npm scripts] in your [https://docs.npmjs.com/files/package.json package.json] file.
* Python, you will use [https://docs.travis-ci.com/user/languages/python/#Default-Build-Script a custom script, for example pyscript].
* Go, you will use a [https://docs.travis-ci.com/user/languages/go/#Default-Build-Script Makefile or Go build file].
* C++, you will use a [https://docs.travis-ci.com/user/languages/cpp/#Default-Build-Script Makefile].
* Rust, you will use a [https://docs.travis-ci.com/user/languages/rust/#Default-Build-Script Cargo.toml file].
* C#, you will use [https://docs.travis-ci.com/user/languages/csharp/#Script xbuild and a solution file].
You can start small and add more automation as you go. Initially, you should focus on compiling your code and/or running a Linter like eslint, Go's lint, Rust's clippy, etc.
You should be able to run your build locally (i.e., without TravisCI). I you can't run it, TravisCI won't be able to run it. Here are some things to consider:
* Do you specify all your dependencies? Have you used external modules or packages that need to be specified in some configuration file? TravisCI will need to know what they are so it can install them
* Do you have the proper script(s) added to the right file(s) to get your code to build, your linters to run? If not, figure out what your language/environment expects and do that. Remember to look at other people's projects on GitHub to see how they do things: everything TravisCI needs will be included in the repo (i.e., it will be "code" you can also read).
* Can you get your build/linting to pass/fail locally? What if you make an error on purpose? And after you fix it, do you see your build scripts change their result?
One you have everything configured locally, add, commit, and push your changes to GitHub. Visit your TravisCI page to confirm that your build is working.
==Step 4: Make Your Code Testable==
Go back to the code you wrote in the last lab, see if it needs to be made more testable. Testable code means:
* Code you can include in other code. You should write modules, packages, libraries vs. an executable or web server. You want a set of functions you can call from your test code.
* Code that takes input and returns output. You don't want your code to print things, display messages, or otherwise interact with the user. The "user" of your library will be another developer using code, not an application.
For example, this is wrong:
<pre>
function add(a, b) {
var result = a + b;
console.log("The sum is " + result);
}
</pre>
Instead you would do:
<pre>
function add(a, b) {
return a + b;
}
</pre>
This second version of the code is easy to test, since you can pass values for <code>a</code> and <code>b</code> and expect to get back known results.
Fix your code so that you can easily test it.
==Step 5: Write Tests==
Now that your code is testable, it's time to write some tests. First, pick a testing framework/tool:
* JavaScript/node.js: https://mochajs.org/ or https://facebook.github.io/jest/
* Python: [http://docs.python-guide.org/en/latest/writing/tests/ Unittest or pytest or the like]
* Go: [https://golang.org/pkg/testing/ built-in testing framework]
* Rust: [https://doc.rust-lang.org/book/second-edition/ch11-01-writing-tests.html built-in testing framework]
* etc.
Here are some sample tests you might write, see if you can come up with more:
* filename without path. Given a "/home/kim/mydata.txt" expect to get back "mydata.txt"
* file size in bytes. Given a path like "/home/kim/mydata.txt" of size 129 bytes, expect to get back 129. You'll need to create a file in your project (e.g., in a tests/ dir) that you can use.
* sha1 digest for a file. Given a file which contains the text "The quick brown fox jumps over the lazy dog" expect to get back 2fd4e1c67a2d28fced849ee1bb76e7391b93eb12
* MD5 digest for a file. Given a file which contains the text "The quick brown fox jumps over the lazy dog" expect to get back 9e107d9d372bb6826bd81d3542a419d6
You should keep each test small, focused on only one scenario (e.g., a given input). Write many small tests to test your functions vs. one giant program that tries to do everything. Other things to consider:
* What should happen if the input data is wrong, missing, or refers to incorrect file info? You should write tests that prove your code fails in known ways (e.g., throws an exception, returns an error code, etc)
* Any data you need in your tests (e.g., special files you want to test against) should be placed in your repo just like any other code.
* You don't need to write tests for things that your code doesn't do. For example, if you use an external library, or built-in function, to calculate hashes, you don't need to write a hundred tests for all cases of hashing. Your code is just calling that code, so test to make sure that it behaves and fails in the proper ways.
==Step 6: Running Your Tests, Locally and on TravisCI==
Now that you have your tests written, you need to modify your build scripts to automatically run them, and tell TravisCI how to do this via a single command. As outlined above in Step 2, this will differ depending on your programming language and environment. For example, in a node.js JavaScript project, you specify the script to run when <code>npm test</code> is called via your <code>package.json</code> file. You should be able to run your entire test suite (all your tests) from the command line with a single command. If you can do that, TravisCI should be able to as well.
==Step 7: Triggering Builds on TravisCI==
TravisCI will run your build/test cycle every time you commit new code on <code>master</code> or open a Pull Request. You can try triggering a build by committing/pushing a new change to your <code>master</code> branch. This should (eventually) cause TravisCI to re-build your code. You can check on your project's builds at https://travis-ci.org/{github-user-name}/{github-repo-name}/builds. Make sure you can get TravisCI to re-build when you update your repo, and also make sure you can get your build to complete and be Green (passed) vs. Red (failed).
==Submission==
Please add your info to the table below. Include a blog post about the process above, what you learned, what you found interesting/new, what you got stuck on or found hard, and any other ideas you might have had while doing this. Did you know about TravisCI before doing this? What did you think of it?
{| class="wikitable"
! style="font-weight: bold;" | #
! style="font-weight: bold;" | Name
! style="font-weight: bold;" | GitHub Repo (URL)
! style="font-weight: bold;" | TravisCI Project Builds (URLs)
! style="font-weight: bold;" | Blog Post (URL)
|-
| 1
|
|
|
|
|-
| 2
|
|
|
|
|-
| 3
|
|
|
|
|-
| 4
|
|
|
|
|-
| 5
|
|
|
|
|-
| 6
|
|
|
|
|-
| 7
|
|
|
|
|-
| 8
|
|
|
|
|-
| 9
|
|
|
|
|-
| 10
|
|
|
|
|-
| 11
|
|
|
|
|-
| 12
|
|
|
|
|-
| 13
|
|
|
|
|-
| 14
|
|
|
|
|-
| 15
|
|
|
|
|-
| 16
|
|
|
|
|-
| 17
|
|
|
|
|-
| 18
|
|
|
|
|-
| 19
|
|
|
|
|-
| 20
|
|
|
|
|-
| 21
|
|
|
|
|-
| 22
|
|
|
|
|-
| 23
|
|
|
|
|-
| 24
|
|
|
|
|-
| 25
|
|
|
|
|-
| 26
|
|
|
|
|-
| 27
|
|
|
|
|-
| 28
|
|
|
|
|-
| 29
|
|
|
|
|-
| 30
|
|
|
|
|-
| 31
|
|
|
|
|-
| 32
|
|
|
|
|-
| 33
|
|
|
|
|-
| 34
|
|
|
|
|-
| 35
|
|
|
|
|-
| 36
|
|
|
|
|-
| 37
|
|
|
|
|-
| 38
|
|
|
|
|-
| 39
|
|
|
|
|-
| 40
|
|
|
|
|}