Difference between revisions of "DPS909 & OSD600 Winter 2019"
(→Week 10) |
(→Week 12) |
||
(2 intermediate revisions by the same user not shown) | |||
Line 347: | Line 347: | ||
* Work on your Release 0.3 | * Work on your Release 0.3 | ||
+ | |||
+ | == Week 11 == | ||
+ | |||
+ | * Complete your 0.3 and begin/continue 0.4 | ||
+ | |||
+ | * Continuous Integration (CI) | ||
+ | ** Lots of free-for-open-source CI services: [https://travis-ci.com/ TravisCI], [https://azure.microsoft.com/en-ca/services/devops/pipelines/ Azure Pipelines], [https://circleci.com/ CircleCI], [https://www.appveyor.com/ AppVeyor], [https://jenkins.io/ Jenkins] | ||
+ | |||
+ | * Automating build, test, and deploy steps with Travis CI | ||
+ | ** https://docs.travis-ci.com/ | ||
+ | ** https://docs.travis-ci.com/user/for-beginners/ | ||
+ | |||
+ | * Travis uses YAML for configuration | ||
+ | ** YAML Ain't Markup Language (YAML) | ||
+ | ** https://en.wikipedia.org/wiki/YAML | ||
+ | ** Usually uses the .yml extension | ||
+ | ** https://learnxinyminutes.com/docs/yaml/ | ||
+ | |||
+ | * Stages of a Travis CI Build | ||
+ | ** [https://docs.travis-ci.com/user/job-lifecycle/#the-job-lifecycle Stages of Build Lifecycle] | ||
+ | ** Adding OS-level dependencies https://docs.travis-ci.com/user/installing-dependencies/ | ||
+ | |||
+ | * Running Tests per Commit/PR | ||
+ | ** https://docs.travis-ci.com/user/languages/javascript-with-nodejs/ | ||
+ | ** Assumes you want to `npm install` and run `npm test` | ||
+ | ** Therefore, you need a working install and test process via `package.json` | ||
+ | |||
+ | * Deployment options: | ||
+ | ** GitHub and gh-pages | ||
+ | *** https://pages.github.com/ | ||
+ | *** https://help.github.com/categories/github-pages-basics/ | ||
+ | *** Enable HTTPS https://help.github.com/articles/securing-your-github-pages-site-with-https/ | ||
+ | *** Custom 404 page https://help.github.com/articles/creating-a-custom-404-page-for-your-github-pages-site/ | ||
+ | *** https://docs.travis-ci.com/user/deployment/pages/ | ||
+ | *** https://docs.travis-ci.com/user/status-images/ | ||
+ | ** [https://docs.travis-ci.com/user/deployment/cargo/ Deploy Rust crates to Cargo] | ||
+ | ** [https://docs.travis-ci.com/user/deployment/releases/ Deploy binary releases to GitHub Releases] | ||
+ | ** [https://docs.travis-ci.com/user/deployment/heroku/ Deploy web apps to Heroku] | ||
+ | ** [https://docs.travis-ci.com/user/deployment/npm/ Deploy node modules to npm] | ||
+ | ** [https://docs.travis-ci.com/user/deployment/pypi/ Deploy python packages to PyPi] | ||
+ | |||
+ | * Some example .travis.yml files in OSS projects | ||
+ | ** https://github.com/filerjs/filer/blob/master/.travis.yml | ||
+ | ** https://github.com/devtools-html/debugger.html/blob/master/.travis.yml | ||
+ | ** https://github.com/datalocale/dataviz-finances-gironde/blob/master/.travis.yml | ||
+ | |||
+ | * [https://github.com/humphd/learn-travis Walkthrough: setting up Travis for a node.js app] | ||
+ | |||
+ | == Week 12 == | ||
+ | |||
+ | * Discussion of open source maintenance, maintainers, and our expectations of "free" software | ||
+ | |||
+ | * BDFL | ||
+ | ** Linux: https://www.linuxjournal.com/content/25-years-later-interview-linus-torvalds | ||
+ | ** Python: https://lwn.net/Articles/759654/, https://www.theregister.co.uk/2018/07/13/python_creator_guido_van_rossum_quits/ | ||
+ | ** Node modules: https://www.npmjs.com/package/request, https://github.com/request/request/issues/3142 | ||
+ | |||
+ | ** event-stream hack | ||
+ | *** https://twitter.com/garybernhardt/status/1067111872225136640 | ||
+ | *** https://github.com/dominictarr/event-stream/issues/116 | ||
+ | *** https://schneid.io/blog/event-stream-vulnerability-explained/ Hack explained | ||
+ | *** https://blog.npmjs.org/post/180565383195/details-about-the-event-stream-incident | ||
+ | *** https://blog.tidelift.com/event-stream-100-million-downloads-unmaintained-hacked.-now-can-we-pay-the-maintainers | ||
+ | |||
+ | ** Emerging models | ||
+ | ** https://opencollective.com/, e.g., https://opencollective.com/babel | ||
+ | ** https://tidelift.com/ | ||
+ | ** https://www.patreon.com/, e.g., https://www.patreon.com/henryzhu, https://www.patreon.com/sindresorhus | ||
+ | ** LTS for Node Modules - https://medium.com/@nodejs/call-to-action-accelerating-node-js-growth-e4862bee2919 | ||
+ | *** https://github.com/nodejs/package-maintenance |
Latest revision as of 08:18, 3 April 2019
Contents
Week 1
- Releases
- 4 releases, some with multiple bugs/PRs required
- Chance to work on real code, real projects
- Big learning curve, lots of time required
- Amazing chance to gain experience, network, build your skills and resume
- Discussion/Readings
- Copyright (Copyright in Canada video)
- https://twitter.com/stan_sdcollins/status/1079395470731030528
- IANAL
- Who created it, "owns" it.
- Set of exclusive rights granted to the work's creator
- "The right to copy," to produce or reproduce a work or substantial portion thereof
- Copyright is automatic when a work is created, you don't have to register it.
- Copyright in Canada
- Copyright Guide
- In a software project, there can be many copyright holders (e.g., many contributors), or all contributors may assign their copyright to the project (e.g., CLA, which we'll cover later)
- Copyright (Copyright in Canada video)
- What is Open Source?
- First open technologies and projects we'll be using:
Week 2
- Release 0.1 Overview
- Due Wed Jan 30th
- node fs module vs. filer
- Licenses
- Rights, privileges, responsibilities, etc. applicable to someone other than the work's creator
- "Terms and Conditions"
- These must be granted by a copyright holder
- No License
- What can you do with code you find that has no license?
- what can I, can't I do?
- Public Domain
- SQLite, which is now used by literally everybody, see http://www.sqlite.org/famous.html
- Unlicense
- BSD License
- Family of Licenses, including 2-Clause BSD, 3-Clause BSD (aka New BDS), 4-Clause BSD
- "Why you should use a BSD style license for your Open Source Project"
- BSD Licenses code is usually compatible with other open/closed code, when you want to mix them.
- Example software projects licensed under the BSD License:
- Summary:
- You need to retain the license and copyright notice
- You can use it commercially or non-commercially (privately)
- You can distribute it freely
- You can modify it freely
- Open Source and Code Reading
- truncate
-
echo "data" > file
-
> file
-
-
fs.truncate()
- A great blog doing something similar: Deep Dive on the node fs module and fs.access() by Safia Abdalla
- truncate
Week 3
- Readings/Resources
- Filing and Fixing a bug: a cookbook approach
- set up git and GitHub
- https://help.github.com/ has lots of great articles to help you. You can also view video guides or read the printed guides
- setup your username in git
- setup your email address in git
- specify which editor git should use, for example you can use vscode
- setup line endings (CRLF vs. LF) in git, extra notes for Windows users
- setup ssh keys for GitHub
- In GitHub, create a fork of the repo you want to work on
- On your computer, clone your forked repo
- On your computer, add a remote named "upstream" for the original repo (vs. your fork)
- On GitHub, find or create an Issue for the change you want to make
- On your computer, create and checkout a branch for your work, e.g., issue-1234 for Issue #1234
- On your computer, make code changes, test them, add, and commit on your branch. Repeat as necessary.
- On your computer, push your changes (commits) to your fork (origin)
- On GitHub, create a Pull Request for your changes to get sent to the upstream repo
- On your computer, fix any problems pointed out by your reviewer(s), add the file(s), commit, and push again to update your pull request
- set up git and GitHub
- Real world example, fixing a bug in Filer
- https://github.com/filerjs/filer/issues/628 - Add tests for fs.writeFile to increase coverage
Week 4
- Learning Licenses: MIT
- MIT License
- The MIT License, Line by Line
- One of the most widely used licenses in Open Source
- Like the BSD License, nothing about patents (created before software was patentable in the US)
- Example software projects licensed under the BSD License:
- More Git
- Understanding how git works
- SHAs
- commits, trees, blobs
- branches
- Working Directory vs. Staging Area vs. HEAD
- Git Walkthrough Part I
- Git Walkthrough Part II
- Some basic git commands you should make sure you know how to use:
git clone
- clone an existing repository (i.e., one you've forked on GitHub)git status
- check what's happening with your repo, working directory, branch infogit add
- add a file, files, or folder(s) of file(s)git commit
- commit changes in the staging areagit log
- look back at existing commitsgit diff
- look at the difference between what's in the working directory and staging area, or between two commitsgit rm
- remove a filegit mv
- move or rename a filegit reset
- update the staging area, and perhaps working directory, with files from another commit (e.g., HEAD)git checkout
- switch to a branch or commit, or create, or get files from a branch/commit
- Understanding how git works
Week 5
- Finish Lab 3, Example: 3 projects I found recently
- Also, see Linuxbrew for Windows 10
- Release 0.2
- Lab 4 - first PR and Status update blog post (Friday Feb 15)
- Merging with git
- Where
git branch
splits histories apart,git merge
brings them back together - Understanding DIFFs and Patch files
-
git diff
,git show
,git log -p
, etc. to show DIFFs - Pull Requests also have links to get the raw .diff and .patch
- How to read a DIFF file
-
- Types of Merges: Fast Forward, Recursive Merges are the most common
-
--ff-only
to force a fast-forward (only the branch pointer is moved, no new commit is created) - 3-way merges: two branch commits with a common ancestor (new commit is created with multiple parents)
- Can have any number of parents though: one of the larges is a 66 commit octopus merge in the Linux kernel
-
- How to merge
- start with a clean working directory
-
commit
your work if you can; or -
stash
(git stash list
,git stash show
,git stash pop
)
-
- checkout the branch you want to merge into
-
git merge branch_to_merge_into_this_branch
- start with a clean working directory
- Various flags and commands to know:
-
git merge --squash
-
git merge --abort
-
git merge --continue
-
git branch -d
-
- Merge Conflicts
- Conflict markers
<<<<<<<<<
,=============
,>>>>>>>>>>>>
- Conflict markers
- Doing big merges in git
- Where
Week 6
- 0.2 Updates
- Interesting projects you've found?
- https://codetribute.mozilla.org/
- Update your Info on the 0.2 wiki with status blog post, PR(s)
-
git rebase branch
- Replay commits on a new base branch/commit
- Process goes like this:
- git finds a common ancestor commit of the branch you're on, and the one you're rebasing onto
- git calculates DIFFs for each, saves them to disk
- git checks out the commit you want to branch onto, and begins to replay those diffs one by one
- if there is a merge conflict, the rebase pauses so you can fix things
- use
git rebase --continue
orgit rebase --abort
to move forward after such a pause - use
git rebase --skip
to ignore the current commit and keep going
- Never rebase commits that are shared publicly in another repo. Only do it on commits you own locally (e.g., a topic branch you are working on)
- Don't use rebase to get rid of commits in a public branch, use
git revert commit-sha
instead to apply an inverse commit - If you rebase a branch you've pushed (e.g., for a pull request), when you push, use
git push origin branch-name -f
(f means force and will overwrite) -
git rebase -i
for interactive rebase- shows a script of all commits in reverse order (order they will be replayed). You can hand edit this to remove, re-order, or combine commits
- You can squash on the same branch by rebasing on
HEAD~n
where n is how many commits back from HEAD to go
-
git cherry-pick SHA
to add a commit to the current branch
Week 7
- Continue working on 0.2
- Discussion of any issues/questions you have
- Update blog post #2 due on Friday
- 2 Weeks left, you should have ~2 PRs done by Friday to stay on track
- Rebase and Merge Demo
- Upstream: https://github.com/copy/v86 (24 commits ahead)
- Forked: https://github.com/humphd/v86/tree/filer-9p-lastknowngood (33 commits ahead)
- Last common commit: https://github.com/copy/v86/commit/f6ae3ea58aee15f2a9be71f2847738367ef31747
- Open Source Case Study: Visual Studio Code
- https://code.visualstudio.com/
- https://github.com/Microsoft/vscode
- https://en.wikipedia.org/wiki/Visual_Studio_Code
- Technologies
- Electron
- Monaco Editor
- TypeScript
- xterm.js
- node.js, express, and hundreds of JavaScript modules
- Fixing Bugs in VSCode
- UI Bugs:
- Crash Bugs:
- Localization Bug: https://github.com/Microsoft/vscode/issues/49211
Week 8
- Finish and Submit 0.2
- Bug Fix Case Study and Walkthrough
- GitHub Desktop: MIT licensed, Electron app written in TypeScript and React for working with git/GitHub
- GitHub Desktop Repository
- Issue 6390
- Walkthrough of Fixing Bug 6390
Week 9
- Releases 0.3 and 0.4
- Lab 8 due Fri, Mar 15
- Release 0.3 due Fri, Mar 29
- Release 0.4 due Fri, Apr 12
- Open Source Case Study: Redis
- Redis (REmote DIctionary Server)
- https://github.com/antirez/redis - ~175K lines of code
- Cross-platform, high performance, in-memory, key/value, data structure database server. Written in mostly in C, as well as Tcl and Lua, with front-ends in just about every language and platform.
- BSD 3-Clause
- Started in 2009 by Salvatore Sanfilippo (antirez)
- Since 2015, development has been sponsored by Redis Labs (see https://en.wikipedia.org/wiki/Redis_Labs)
- Redis is among the most popular NoSQL databases in the world, and the most popular key/value store. It is used by everyone:
- GitHub
- StackOverflow
- Snapchat
- Shopify
- AirBnB
- Uber
- Tumblr
- Slack
- Medium
- Imgur
- Kickstarter
- Common Use Cases:
- User Session Cache (e.g., reduce DB lookups for user info, shopping cart data)
- Full Page Cache (e.g., by URL or route)
- Queues (e.g., Message Queue, Worker Queue)
- Counting (e.g., metrics, analytics)
- Pub/Sub (e.g., chat systems, notifications)
- Redis Tutorial and Walkthrough: https://try.redis.io/
- Writing Code Comments based on 32e0d237 commit
- His YouTube channel has more video discussions of the code in Redis
Week 10
- Starting a new Open Source Project: browser-mime
- The problem: https://github.com/humphd/nohost/issues/6
- mime-db
- mime-types
- Discussion
- talk to the community
- https://developer.mozilla.org/en-US/docs/Web/API/PerformanceNavigationTiming
- research data
- CORS and Data URIs (we need a small server for same-origin analysis)
- Puppeteer for automation
- How to modify mime-types code to use our new db? Git Submodule vs Git Subtree
-
git submodule add https://github.com/jshttp/mime-types.git
to create andgit submodule update --remote
to update -
git subtree add --prefix mime-types https://github.com/jshttp/mime-types.git master --squash
to create andgit subtree pull --prefix mime-types https://github.com/jshttp/mime-types.git master --squash
-
- Using linting and pretty-printing tools in projects
- Work on your Release 0.3
Week 11
- Complete your 0.3 and begin/continue 0.4
- Continuous Integration (CI)
- Lots of free-for-open-source CI services: TravisCI, Azure Pipelines, CircleCI, AppVeyor, Jenkins
- Automating build, test, and deploy steps with Travis CI
- Travis uses YAML for configuration
- YAML Ain't Markup Language (YAML)
- https://en.wikipedia.org/wiki/YAML
- Usually uses the .yml extension
- https://learnxinyminutes.com/docs/yaml/
- Stages of a Travis CI Build
- Stages of Build Lifecycle
- Adding OS-level dependencies https://docs.travis-ci.com/user/installing-dependencies/
- Running Tests per Commit/PR
- https://docs.travis-ci.com/user/languages/javascript-with-nodejs/
- Assumes you want to `npm install` and run `npm test`
- Therefore, you need a working install and test process via `package.json`
- Deployment options:
- GitHub and gh-pages
- https://pages.github.com/
- https://help.github.com/categories/github-pages-basics/
- Enable HTTPS https://help.github.com/articles/securing-your-github-pages-site-with-https/
- Custom 404 page https://help.github.com/articles/creating-a-custom-404-page-for-your-github-pages-site/
- https://docs.travis-ci.com/user/deployment/pages/
- https://docs.travis-ci.com/user/status-images/
- Deploy Rust crates to Cargo
- Deploy binary releases to GitHub Releases
- Deploy web apps to Heroku
- Deploy node modules to npm
- Deploy python packages to PyPi
- GitHub and gh-pages
- Some example .travis.yml files in OSS projects
Week 12
- Discussion of open source maintenance, maintainers, and our expectations of "free" software
- BDFL
- Linux: https://www.linuxjournal.com/content/25-years-later-interview-linus-torvalds
- Python: https://lwn.net/Articles/759654/, https://www.theregister.co.uk/2018/07/13/python_creator_guido_van_rossum_quits/
- Node modules: https://www.npmjs.com/package/request, https://github.com/request/request/issues/3142
- event-stream hack
- https://twitter.com/garybernhardt/status/1067111872225136640
- https://github.com/dominictarr/event-stream/issues/116
- https://schneid.io/blog/event-stream-vulnerability-explained/ Hack explained
- https://blog.npmjs.org/post/180565383195/details-about-the-event-stream-incident
- https://blog.tidelift.com/event-stream-100-million-downloads-unmaintained-hacked.-now-can-we-pay-the-maintainers
- event-stream hack
- Emerging models
- https://opencollective.com/, e.g., https://opencollective.com/babel
- https://tidelift.com/
- https://www.patreon.com/, e.g., https://www.patreon.com/henryzhu, https://www.patreon.com/sindresorhus
- LTS for Node Modules - https://medium.com/@nodejs/call-to-action-accelerating-node-js-growth-e4862bee2919