Difference between revisions of "DPS909 & OSD600 Fall 2018"
(→Week 12) |
(→Week 12) |
||
(6 intermediate revisions by 2 users not shown) | |||
Line 391: | Line 391: | ||
== Week 12 == | == Week 12 == | ||
+ | |||
+ | * News | ||
+ | ** event-stream hack | ||
+ | *** https://twitter.com/garybernhardt/status/1067111872225136640 | ||
+ | *** https://github.com/dominictarr/event-stream/issues/116 | ||
+ | *** https://schneid.io/blog/event-stream-vulnerability-explained/ Hack explained | ||
+ | *** https://blog.npmjs.org/post/180565383195/details-about-the-event-stream-incident | ||
+ | *** https://blog.tidelift.com/event-stream-100-million-downloads-unmaintained-hacked.-now-can-we-pay-the-maintainers | ||
+ | *** https://tidelift.com/ | ||
+ | ** LTS for Node Modules - https://medium.com/@nodejs/call-to-action-accelerating-node-js-growth-e4862bee2919 | ||
+ | *** https://github.com/nodejs/package-maintenance | ||
+ | *** https://www.npmjs.com/package/readable-stream | ||
+ | *** https://www.npmjs.com/package/mqtt - "MQTT.js is another good example which is used by AWS, Microsoft, and IBM" | ||
+ | ** https://arstechnica.com/tech-policy/2018/11/how-i-changed-the-law-with-a-github-pull-request/ | ||
+ | ** https://humphd.github.io/pretty-effective/ | ||
* [[OSD & DPS909 Fall 2018 Release 0.4]] | * [[OSD & DPS909 Fall 2018 Release 0.4]] |
Latest revision as of 11:59, 30 November 2018
Contents
Week 1
- Releases
- 4 releases, some with multiple bugs/PRs required
- Chance to work on real code, real projects
- Big learning curve, lots of time required
- Amazing chance to gain experience, network, build your skills and resume
- Discussion/Readings
- Copyright (Copyright in Canada video)
- IANAL
- Who created it, "owns" it.
- Set of exclusive rights granted to the work's creator
- "The right to copy," to produce or reproduce a work or substantial portion thereof
- Copyright is automatic when a work is created, you don't have to register it.
- Copyright in Canada
- Copyright Guide
- In a software project, there can be many copyright holders (e.g., many contributors), or all contributors may assign their copyright to the project (e.g., CLA, which we'll cover later)
- What is Open Source?
- Microsoft buys GitHub for 7.5 Billion - "We're all in on open source"
- Julia Evans, "How to teach yourself hard things"
- Copyright (Copyright in Canada video)
- First open technologies and projects we'll be using:
Week 2
- Release 0.1 Overview
- Due Friday Sept 28th
- node.js
- node fs module vs. filer
- synchronous vs. asynchronous functions in JavaScript
- node's callback function pattern vs. Promises
- Licenses
- Rights, privileges, responsibilities, etc. applicable to someone other than the work's creator
- "Terms and Conditions"
- These must be granted by a copyright holder
- No License
- What can you do with code you find that has no license?
- what can I, can't I do?
- Public Domain
- SQLite, which is now used by literally everybody, see http://www.sqlite.org/famous.html
- Unlicense
- BSD License
- Family of Licenses, including 2-Clause BSD, 3-Clause BSD (aka New BDS), 4-Clause BSD
- "Why you should use a BSD style license for your Open Source Project"
- BSD Licenses code is usually compatible with other open/closed code, when you want to mix them.
- Example software projects licensed under the BSD License:
- Summary:
- You need to retain the license and copyright notice
- You can use it commercially or non-commercially (privately)
- You can distribute it freely
- You can modify it freely
- Open Source and Code Reading
Week 3
- Readings/Resources
- Filing and Fixing a bug: a cookbook approach
- set up git and GitHub
- https://help.github.com/ has lots of great articles to help you. You can also view video guides or read the printed guides
- setup your username in git
- setup your email address in git
- specify which editor git should use, for example you can use vscode
- setup line endings (CRLF vs. LF) in git
- setup ssh keys for GitHub
- In GitHub, create a fork of the repo you want to work on
- On your computer, clone your forked repo
- On your computer, add a remote named "upstream" for the original repo (vs. your fork)
- On GitHub, find or create an Issue for the change you want to make
- On your computer, create and checkout a branch for your work, e.g., issue-1234 for Issue #1234
- On your computer, make code changes, test them, add, and commit on your branch. Repeat as necessary.
- On your computer, push your changes (commits) to your fork (origin)
- On GitHub, create a Pull Request for your changes to get sent to the upstream repo
- On your computer, fix any problems pointed out by your reviewer(s), add the file(s), commit, and push again to update your pull request
- set up git and GitHub
- Real world examples:
- Filing, Fixing a bug in Filer
- Adding a new Feature, Tests, and Docs to Filer - support node's new recursive fs.mkdir in Filer
Week 4
- Learning Licenses: MIT
- MIT License
- The MIT License, Line by Line
- One of the most widely used licenses in Open Source
- Like the BSD License, nothing about patents (created before software was patentable in the US)
- Example software projects licensed under the BSD License:
- More Git
- Git Walkthrough Part I
- Git Walkthrough Part II
- Some basic git commands you should make sure you know how to use:
git clone
- clone an existing repository (i.e., one you've forked on GitHub)git status
- check what's happening with your repo, working directory, branch infogit add
- add a file, files, or folder(s) of file(s)git commit
- commit changes in the staging areagit log
- look back at existing commitsgit diff
- look at the difference between what's in the working directory and staging area, or between two commitsgit rm
- remove a filegit mv
- move or rename a filegit reset
- update the staging area, and perhaps working directory, with files from another commit (e.g., HEAD)git checkout
- switch to a branch or commit, or create, or get files from a branch/commit
Week 5
- Release 0.2: Hacktoberfest
- For Tuesday October 9th (Lab 3)
- First (of 5) PR completed with a Blog Post
- Submit via https://github.com/humphd/hacktoberfest-at-seneca-2018/wiki/Student-Submissions
- Add projects that you think are good for other students to https://github.com/humphd/hacktoberfest-at-seneca-2018/wiki/Interesting-Open-Source-Projects
- For Tuesday October 9th (Lab 3)
- Merging with git
- Where
git branch
splits histories apart,git merge
brings them back together - Understanding DIFFs and Patch files
-
git diff
,git show
,git log -p
, etc. to show DIFFs - Pull Requests also have links to get the raw .diff and .patch
- How to read a DIFF file
-
- Types of Merges: Fast Forward, Recursive Merges are the most common
-
--ff-only
to force a fast-forward (only the branch pointer is moved, no new commit is created) - 3-way merges: two branch commits with a common ancestor (new commit is created with multiple parents)
- Can have any number of parents though: one of the larges is a 66 commit octopus merge in the Linux kernel
-
- How to merge
- start with a clean working directory
-
commit
your work if you can; or -
stash
(git stash list
,git stash show
,git stash pop
)
-
- checkout the branch you want to merge into
-
git merge branch_to_merge_into_this_branch
- start with a clean working directory
- Various flags and commands to know:
-
git merge --squash
-
git merge --abort
-
git merge --continue
-
git branch -d
-
- Merge Conflicts
- Conflict markers
<<<<<<<<<
,=============
,>>>>>>>>>>>>
- Conflict markers
- Doing big merges in git
- Where
Week 6
- Hacktoberfest Updates
- Add any interesting projects you find/work on to this list on the wiki
- Update your Info on the submissions wiki page by Monday:
- Add your Name beside your GitHub username
- New Pull Request
- New Blog Post
- Any Issues you're working on
- Week 1 Summary
-
git rebase branch
- Replay commits on a new base branch/commit
- Process goes like this:
- git finds a common ancestor commit of the branch you're on, and the one you're rebasing onto
- git calculates DIFFs for each, saves them to disk
- git checks out the commit you want to branch onto, and begins to replay those diffs one by one
- if there is a merge conflict, the rebase pauses so you can fix things
- use
git rebase --continue
orgit rebase --abort
to move forward after such a pause - use
git rebase --skip
to ignore the current commit and keep going
- Never rebase commits that are shared publicly in another repo. Only do it on commits you own locally (e.g., a topic branch you are working on)
- Don't use rebase to get rid of commits in a public branch, use
git revert commit-sha
instead to apply an inverse commit - If you rebase a branch you've pushed (e.g., for a pull request), when you push, use
git push origin branch-name -f
(f means force and will overwrite) -
git rebase -i
for interactive rebase- shows a script of all commits in reverse order (order they will be replayed). You can hand edit this to remove, re-order, or combine commits
- You can squash on the same branch by rebasing on
HEAD~n
where n is how many commits back from HEAD to go
- PRs needing a Rebase
Week 7
- Hacktoberfest Week 2
- Update your Info on the submissions wiki page by Monday:
- Add your Name beside your GitHub username
- New Pull Request
- New Blog Post
- Any Issues you're working on
- Update your Info on the submissions wiki page by Monday:
- Rebase Review and Demo
- Open Source Case Study: Visual Studio Code
- https://code.visualstudio.com/
- https://github.com/Microsoft/vscode
- https://en.wikipedia.org/wiki/Visual_Studio_Code
- Technologies
- Electron
- Monaco Editor
- TypeScript
- xterm.js
- node.js, express, and hundreds of JavaScript modules
- Fixing Bugs in VSCode
- UI Bugs:
- Crash Bugs:
- Localization Bug: https://github.com/Microsoft/vscode/issues/49211
Week 8
- Hacktoberfest 0.2 Due this Week (Wed Oct 31)
- Finish all 5 Pull Requests + Blog Posts
- Write a 6th and final Blog Post about all of your contributions. Include links to the Bugs you fixed and Pull Requests. Talk about what you learned, your growth through the experience, what went well, what you would do differently next time, and your reflections on Hacktoberfest in general.
- Make sure all PRs and Blog Posts (including 6th conclusion post) are up at https://github.com/humphd/hacktoberfest-at-seneca-2018/wiki/Student-Submissions
- 0.3 and 0.4 Releases, Labs 5-10
- Complete 3 larger PRs in external open source projects + 3 PRs in internal OSD/DPS open source run projects
- For 0.3, follow a 2 + 1 pattern: either 2 external and 1 internal, or 2 internal and 1 external.
- For 0.4, follow a 1 + 2 pattern: do the opposite of what you did in 0.3
- Consider working on a project you began working with during Hacktoberfest, though you aren't limited to this.
- PRs for 0.3 and 0.4 are about increasing quality vs quantity. You need to work on larger fixes/features than many of you did for Hacktoberfest. If you want to work on small bugs, you'll need to combine them together.
- Each Monday, a blog post discussing your work from the previous week is due. Link to bugs you are working on, PRs or branches in progress, talk about what you learned, what you're still thinking about, and any plans you have for upcoming weeks.
- You do not need to complete a PR every week for 0.3 and 0.4; though you will need to make progress each week in order to blog, and to stay on track.
- Open Source Case Study: Redis
- Redis (REmote DIctionary Server)
- https://github.com/antirez/redis - ~175K lines of code
- Cross-platform, high performance, in-memory, key/value, data structure database server. Written in mostly in C, as well as Tcl and Lua, with front-ends in just about every language and platform.
- BSD 3-Clause
- Started in 2009 by Salvatore Sanfilippo (antirez)
- Since 2015, development has been sponsored by Redis Labs (see https://en.wikipedia.org/wiki/Redis_Labs)
- Redis is among the most popular NoSQL databases in the world, and the most popular key/value store. It is used by everyone:
- GitHub
- StackOverflow
- Snapchat
- Shopify
- AirBnB
- Uber
- Tumblr
- Slack
- Medium
- Imgur
- Kickstarter
- Common Use Cases:
- User Session Cache (e.g., reduce DB lookups for user info, shopping cart data)
- Full Page Cache (e.g., by URL or route)
- Queues (e.g., Message Queue, Worker Queue)
- Counting (e.g., metrics, analytics)
- Pub/Sub (e.g., chat systems, notifications)
- Redis Tutorial and Walkthrough: https://try.redis.io/
Week 9
- OSD & DPS909 Fall 2018 Release 0.3
- OSD/DPS Fall 2018 Open Source Project Ideas
- Make sure you have chosen your projects (both new and existing) and have either filed bugs, or found things to work on. Your blog (due today) should discuss and explain what you plan to do over the coming 3 weeks.
- Spend the time in class talking with people about the projects, filing bugs, doing research, and figuring out your own ways to contribute.
Week 10
- Case Study: Prettier
- Using linting and pretty-printing tools in projects
- Complete your OSD & DPS909 Fall 2018 Release 0.3
Week 11
- 0.4 Project Options:
- Help fix eslint issues in Firefox (talk to Dave if you want to get involved so we can co-ordinate with Mozilla)
- Help fix flake8 issues in Pandas (talk to
@Alexander Ponomaroff
on Slack if you want to help)
- Continuous Integration
- Lots of free-for-open-source services: TravisCI, Azure Pipelines, CircleCI, AppVeyor, Jenkins
- See also Abstruse CI, Appcenter, Assertible, Badwolf, Bitrise, BrowserStack, Buildkite, ChronoCI, Codacy, Codefresh, Codeship, ConcourseCI, ContinuousPHP, Drone, Ebert, GoCD, HoundCI, Hydra, Probo.CI, Semaphore, Shippable, TeamCity, VSTS, Wercker
- Automating build, test, and deploy steps with Travis CI
- Travis uses YAML for configuration
- YAML Ain't Markup Language (YAML)
- https://en.wikipedia.org/wiki/YAML
- Usually uses the .yml extension
- https://learnxinyminutes.com/docs/yaml/
- Stages of a Travis CI Build
- Stages of Build Lifecycle
- Adding OS-level dependencies https://docs.travis-ci.com/user/installing-dependencies/
- Running Tests per Commit/PR
- https://docs.travis-ci.com/user/languages/javascript-with-nodejs/
- Assumes you want to `npm install` and run `npm test`
- Therefore, you need a working install and test process via `package.json`
- Deployment options:
- GitHub and gh-pages
- https://pages.github.com/
- https://help.github.com/categories/github-pages-basics/
- Enable HTTPS https://help.github.com/articles/securing-your-github-pages-site-with-https/
- Custom 404 page https://help.github.com/articles/creating-a-custom-404-page-for-your-github-pages-site/
- https://docs.travis-ci.com/user/deployment/pages/
- https://docs.travis-ci.com/user/status-images/
- Deploy Rust crates to Cargo
- Deploy binary releases to GitHub Releases
- Deploy web apps to Heroku
- Deploy node modules to npm
- Deploy python packages to PyPi
- GitHub and gh-pages
- Some example .travis.yml files in OSS projects
Week 12
- News
- event-stream hack
- https://twitter.com/garybernhardt/status/1067111872225136640
- https://github.com/dominictarr/event-stream/issues/116
- https://schneid.io/blog/event-stream-vulnerability-explained/ Hack explained
- https://blog.npmjs.org/post/180565383195/details-about-the-event-stream-incident
- https://blog.tidelift.com/event-stream-100-million-downloads-unmaintained-hacked.-now-can-we-pay-the-maintainers
- https://tidelift.com/
- LTS for Node Modules - https://medium.com/@nodejs/call-to-action-accelerating-node-js-growth-e4862bee2919
- https://github.com/nodejs/package-maintenance
- https://www.npmjs.com/package/readable-stream
- https://www.npmjs.com/package/mqtt - "MQTT.js is another good example which is used by AWS, Microsoft, and IBM"
- https://arstechnica.com/tech-policy/2018/11/how-i-changed-the-law-with-a-github-pull-request/
- https://humphd.github.io/pretty-effective/
- event-stream hack
- Reading Code to Fix Your Own
-
git blame
- GitHub File History
- GitHub Issues, Pull Requests
-
- Examples
- Firefox ESLint - "have any of you dealt with your no-undef errors yet? - I am wondering where this 'SpecialPowers' comes from and if it can be ignored" - https://dxr.mozilla.org/mozilla-central/source/testing/mochitest/nested_setup.js#2
-
fs.access()
- Apple's CVS source, Linux file.c - "How to structure a Python CLI?" - Tensorflow retrain.py, Python Fire, AWS cli
- CreativeCollab, other apps using boardgame.io: fields-of-arle, agricola
- mySeneca Extension, learning from Refined Twitter