Difference between revisions of "DPS909 & OSD600 Fall 2019"

From CDOT Wiki
Jump to: navigation, search
(→‎Week 9)
 
(7 intermediate revisions by the same user not shown)
Line 361: Line 361:
 
** Redis Tutorial and Walkthrough: https://try.redis.io/
 
** Redis Tutorial and Walkthrough: https://try.redis.io/
 
* [http://antirez.com/news/124 Writing Code Comments] based on [https://github.com/antirez/redis/tree/32e0d2376fe91e76be04bb62825af5d95737b13e 32e0d237 commit]
 
* [http://antirez.com/news/124 Writing Code Comments] based on [https://github.com/antirez/redis/tree/32e0d2376fe91e76be04bb62825af5d95737b13e 32e0d237 commit]
 +
** "[W]riting comments is of paramount importance in order to produce good code, that is maintainable in the long run and understandable by others and by the authors during modifications and debugging activities."
 +
** "So comments can be, for me, a tool for lowering the cognitive load of the reader."
 
* His [https://www.youtube.com/channel/UCDDG9vOcmgwlslJJpCWjqOg YouTube channel] has more video discussions of the code in Redis
 
* His [https://www.youtube.com/channel/UCDDG9vOcmgwlslJJpCWjqOg YouTube channel] has more video discussions of the code in Redis
  
Line 380: Line 382:
 
*** https://github.com/LearningLocker/learninglocker
 
*** https://github.com/LearningLocker/learninglocker
 
*** https://github.com/ParabolInc/action
 
*** https://github.com/ParabolInc/action
 +
** [https://github.com/Seneca-CDOT/telescope/issues/112 https://github.com/Seneca-CDOT/telescope/issues/112] has notes on some research into these projects.  Please add more as you find it.
 +
 +
== Week 10 ==
 +
 +
* Managing change on a fast moving project with lots of people
 +
** Documentation doesn't work, project "rules" don't work
 +
** Communication works (Watching Issues, Slack, in person discussions) and
 +
** You have to enforce standards across a project using tools and automation
 +
** Continuous Integration (CI) - [https://github.com/Seneca-CDOT/telescope/issues/43 we have Travis CI enabled]
 +
** Static Analysis Tools
 +
*** [https://github.com/Seneca-CDOT/telescope/pull/38 we have eslint for JavaScript using the Airbnb JavaScript Style Guide]
 +
*** [https://github.com/Seneca-CDOT/telescope/pull/125 we have stylelint for CSS]
 +
** Automated Tests with [https://jestjs.io/docs/en/getting-started Jest]
 +
** Next we will [https://github.com/Seneca-CDOT/telescope/issues/96 add Prettier for standard code formatting].  Get your 0.3 PRs in before the weekend, or get ready for merge conflicts when this lands.
 +
 +
* [https://humphd.github.io/pretty-effective/ Case Study - Prettier]
 +
** Web site: https://prettier.io/
 +
** Twitter: https://twitter.com/PrettierCode
 +
** GitHub: https://github.com/prettier/prettier
 +
** [https://www.youtube.com/watch?v=hkfBvpEfWdA  James Long introducing Prettier (video)]
 +
** [https://www.youtube.com/watch?v=3p6XR2VeHRw Visualization of Prettier Development (video)]
 +
** [https://prettier.io/docs/en/install.html Installing Prettier]
 +
 +
* Reviews
 +
** PRs must have a title that explains the fix, links to an Issue #
 +
** PRs must have a full description.  If this is UI code, show a screenshot, or explain the fix, talk about how to review it, how to test it, what's not done, what's going to happen in further PRs, etc.
 +
** All PRs must pass CI: eslint, stylelint, unit tests
 +
** Don't let people land code with unrelated commits (e.g., merges with master)
 +
** Ask people to update their master with upstream, and rebase
 +
** Don't let people land code with changes to unrelated lines/files (e.g., package.json, whitespace changes)
 +
** Make sure one PR doesn't undo the work of another (e.g., bad merge, erasing existing code)
 +
** Ask yourself how we'll test every piece of code we take.  We need to be able to trust it going forward.
 +
** PRs for us to try reviewing together:
 +
*** [https://github.com/Seneca-CDOT/telescope/pull/102 #102 - Added Extracting of URLs from blog feed function]
 +
*** [https://github.com/Seneca-CDOT/telescope/pull/123 #123 - Fix #74: Validate blog URLs.]
 +
*** [https://github.com/Seneca-CDOT/telescope/pull/136 #136 - Added feed list parser]
 +
*** [https://github.com/Seneca-CDOT/telescope/pull/137 #137 - added pagination to feed for #40]
 +
 +
* Research on Bull/Redis code
 +
** https://github.com/Seneca-CDOT/telescope/issues/112 has lots of ideas for Issues you could file and fix
 +
 +
== Week 11 ==
 +
 +
* 0.3 Recap
 +
** Make sure your PRs, Issues, and Blog posts are [https://wiki.cdot.senecacollege.ca/wiki/OSD_%26_DPS909_Fall_2019_-_Release_0.3 posted on the 0.3 page].  I'll mark 0.3 and 0.4 together, in case you're still cleaning up work from the past week (you have time).
 +
** Over 100 Pull Requests, 67% are already merged
 +
** 476 commits to 325 files totaling more than 20K lines of code edited/added
 +
** Top Repos: Telescope! followed by Microsoft, Mozilla, Facebook, Uber, Zeit, Elastic, GatsbyJS, .NET, and 100 others
 +
 +
* Telescope Updates
 +
** [https://gource.io/ Gource visualization of the repo]
 +
** Tour of the Code
 +
** TODO - Plan for 0.4
 +
** How to get involved
 +
*** triage issues
 +
*** reviews
 +
*** help debug things others are stuck on in PRs
 +
*** help people with git issues
 +
*** fixing bugs
 +
*** [https://github.com/Seneca-CDOT/telescope/issues/251 project management]
 +
 +
* Discussion of git Workflows
 +
** Value of CI - [https://travis-ci.org/Seneca-CDOT/telescope Travis CI builds] and [https://circleci.com/gh/Seneca-CDOT/telescope Circle CI builds]
 +
*** [https://treeherder.mozilla.org/#/jobs?repo=mozilla-central CI for Firefox]
 +
** Rebasing vs Merging
 +
** Linear history is better than a tangled web of merges
 +
*** gitk on Telescope vs.
 +
*** gitk on Prettier, node.js, WebKit, Gutenberg
 +
** Many, Small PRS are better than a few big ones
 +
*** easier to review
 +
*** easier to merge
 +
** Break your work up, leverage the community of developers we have
 +
** Dealing with package.json and package-lock.json
 +
 +
* Case Study
 +
** [https://github.com/humphd/desktop/tree/good-first-experience-issue-6390#walkthrough-fixing-a-bug-in-github-desktop Walkthrough: using history to debug with git bisect]
 +
 +
== Week 11 ==
 +
 +
* 0.4 Status Check
 +
** [https://github.com/Seneca-CDOT/telescope/pulls Pull Requests]
 +
** [https://github.com/Seneca-CDOT/telescope/issues Issues]
 +
** Countdown: 2 weeks to a working program.  What risks do we see?  How to address them?
 +
 +
* Case Study: [https://google.github.io/eng-practices/ Code Review at Google]
 +
** [https://google.github.io/eng-practices/review/developer/cl-descriptions.html Creating Reviewable Code]
 +
** [https://google.github.io/eng-practices/review/developer/small-cls.html Preferring Small Changes]
 +
** [https://google.github.io/eng-practices/review/reviewer/standard.html The Standard Code Review]
 +
** [https://google.github.io/eng-practices/review/reviewer/looking-for.html How to Review Code]
 +
** [https://google.github.io/eng-practices/review/developer/handling-comments.html How to Handle Review Comments]
 +
 +
* Case Study: [https://engineering.shopify.com/blogs/engineering/successfully-merging-work-1000-developers Shopify developers using git, CI, merging]
 +
** Over 1000 developers working on the code
 +
** Shopify changes 40 times a day
 +
** 400 commits a day to master
 +
** Master must always be green (passing CI)
 +
** Master must stay close to production
 +
** Emergency merges must be fast
 +
** /shipit GitHub integration
 +
** Merge queue to manage merges to master

Latest revision as of 11:07, 24 November 2019

Week 1

  • Course introduction
  • Labs
    • Weekly labs, typically done in class
    • Labs are due on the Friday of the week they are assigned by midnight
    • Marked using Pass/Fail scheme
    • All labs must be completed to pass the course
    • Lab 1 is available now
  • Releases
    • 4 releases, some with multiple bugs/PRs required, including participating in Hacktoberfest 2019
    • Due Dates: Sept 20, Oct 31, Nov 20, Dec 6
    • Chance to work on real code, real projects
    • Big learning curve, lots of time required
    • Amazing chance to gain experience, network, build your skills and resume
    • Work with new and emerging technologies, gain exposure to tech outside the classroom
  • Discussion/Readings
    • Copyright (Copyright in Canada video)
      • IANAL
      • Who created it, "owns" it.
      • Set of exclusive rights granted to the work's creator
      • "The right to copy," to produce or reproduce a work or substantial portion thereof
      • Copyright is automatic when a work is created, you don't have to register it.
      • Copyright in Canada
      • Copyright Guide
      • In a software project, there can be many copyright holders (e.g., many contributors), or all contributors may assign their copyright to the project (e.g., CLA, which we'll cover later)

Week 2

  • Licenses
    • Rights, privileges, responsibilities, etc. applicable to someone other than the work's creator
    • "Terms and Conditions"
    • These must be granted by a copyright holder

Week 3

  • Introducing git and GitHub
    • Content Addressable Filesystem and Snapshots
    • Distributed: Local vs. Remote development
    • .git directory
    • Content Integrity, SHAs (Secure Hash Algorithm)
      • git init
      • echo 'test content' | git hash-object -w --stdin
      • ls .git/objects
      • git cat-file -p d670460b4b4aece5915caf5c68d12f560a9fe3e4
    • Blobs, Trees, and Commits
    • Branches, master
    • Working Directory, Staging Area, Repository
    • What do these commands really do?
      • git clone url-to-git-repo
      • git add file.txt
      • git status
      • git rm file.txt
      • git commit -m "Added file.txt"
    • Remotes, origin, upstream

Week 4

Week 5

  • Forking vs. Merging
    • Anyone can fork, not everyone can get work merged back in
    • My Repo: my house my rules
    • Some famous Forks
      • Firefox from Mozilla Suite
      • WebKit from KDE's KHTML
      • Blink from WebKit
      • Ubuntu from Debian
      • Sun's StarOffice became OpenOffice became LibreOffice
      • WordPress from Cafelog
      • MariaDB from MySQL
      • FireOS (Amazon for Kindle) from Android
      • io.js from node.js, which eventually became the official node.js
    • Example Fork
      • Semantic UI
      • Fomantic UI - "Fomantic was created to continue active development of Semantic-UI and has the intent to be merged back into the master repository once active development can restart."
  • Merging with git
    • Where git branch splits histories apart, git merge brings them back together
    • Understanding DIFFs and Patch files
    • Types of Merges: Fast Forward, Recursive Merges are the most common
      • --ff-only to force a fast-forward (only the branch pointer is moved, no new commit is created)
      • 3-way merges: two branch commits with a common ancestor (new commit is created with multiple parents)
      • Can have any number of parents though: one of the larges is a 66 commit octopus merge in the Linux kernel
    • How to merge
      • start with a clean working directory
        • commit your work if you can; or
        • stash (git stash list, git stash show, git stash pop)
      • checkout the branch you want to merge into
      • git merge branch_to_merge_into_this_branch
    • Various flags and commands to know:
      • git merge --squash
      • git merge --abort
      • git merge --continue
      • git branch -d
    • Merge Conflicts
      • Conflict markers <<<<<<<<<, =============, >>>>>>>>>>>>
    • Doing big merges in git
  • TODO
    • Merging 3 PRs example
    • Release 0.2
    • Lab 4 - Complete your first Hacktoberfest PR, and write your first blog post about your fix. See guidelines in the Release 0.2 page. Place both links in a new section under the Submission section.

Week 6

  • 0.2 Updates
    • How's it Going?
      • Things I read in your blog posts:
        • Setting up projects takes time, research (Docker, Gatsby, Makefile, Gems, npm builds etc)
        • Trouble running projects
        • Watch Windows line ending issues
        • Need to fix things the way the project wants vs. how you want
        • Include links in your blog posts when you talk about things
    • Interesting projects you've found?
    • Stats for Week 1
      • 44 PRs: +3,358/-941 lines of code across 178 files in 37 repositories
      • 19 have already been merged
      • 21 people didn't get Lab 1 done (no PR/blog)
  • git commit --amend
    • Add to, correct, or otherwise alter your previous commit and/or comment message
    • Use --no-edit to leave the commit message alone
  • git rebase branch
    • Replay commits on a new base branch/commit
    • Process goes like this:
      • git finds a common ancestor commit of the branch you're on, and the one you're rebasing onto
      • git calculates DIFFs for each, saves them to disk
      • git checks out the commit you want to branch onto, and begins to replay those diffs one by one
      • if there is a merge conflict, the rebase pauses so you can fix things
      • use git rebase --continue or git rebase --abort to move forward after such a pause
      • use git rebase --skip to ignore the current commit and keep going
    • Never rebase commits that are shared publicly in another repo. Only do it on commits you own locally (e.g., a topic branch you are working on)
    • Don't use rebase to get rid of commits in a public branch, use git revert commit-sha instead to apply an inverse commit
    • If you rebase a branch you've pushed (e.g., for a pull request), when you push, use git push origin branch-name -f (f means force and will overwrite)
    • git rebase -i for interactive rebase
      • shows a script of all commits in reverse order (order they will be replayed). You can hand edit this to remove, re-order, or combine commits
    • You can squash on the same branch by rebasing on HEAD~n where n is how many commits back from HEAD to go
  • git cherry-pick SHA to add a commit to the current branch

Week 7

Week 8

  • Continue working on 0.2
  • Wrap-up and Discussion of Hacktoberfest 2019
  • Discussion of Labs for the rest of the term.
    • Every week you'll need to contribute in some way to our open source project. This could be a small PR, reviewing code, adding some docs, writing a test, working in issues, doing research and writing background material, etc.
    • Your lab blog post each week should discuss what you did

Week 9

  • Hacktoberfest 2019 Stats
    • 62 Students and a 95% completion rate (4 or more PRs)
    • 246 PRs, with 56% currently merged
    • 647 Commits to 881 files
    • 32K lines of code added, 14K lines removed
    • Languages
      • Most popular: JS/TS (124), Python (25), HTML/CSS (28), C/C++/C# (27), Java (14)
      • New/Interesting: Swift (4), PHP (3), Go (2), Rust (2), and 1 in each of OCaml, PowerShell, Ruby, Elixir, Kotlin, Shell
    • Notable Projects:
      • xterm.js, Polymer, Bitcoin, Angular, Ethereum, VSCode, Microsoft (Calculator, React Native Windows, STL), Jest, Facebook, WordPress, node.js, Nasa, Salesforce, Mozilla, Home Assistant, Google, Instacart,
  • Releases 0.3 - due Friday Nov 15
    • Be working on your external PR
    • Everyone needs a telescope Issue by the end of the week. We need to make this happen together
    • Friday's Lab will require you to document what you are doing for both.
  • Open Source Case Study: Redis
    • Redis (REmote DIctionary Server)
    • https://github.com/antirez/redis - ~175K lines of code
    • Cross-platform, high performance, in-memory, key/value, data structure database server. Written in mostly in C, as well as Tcl and Lua, with front-ends in just about every language and platform.
    • BSD 3-Clause
    • Started in 2009 by Salvatore Sanfilippo (antirez)
    • Since 2015, development has been sponsored by Redis Labs (see https://en.wikipedia.org/wiki/Redis_Labs)
    • Redis is among the most popular NoSQL databases in the world, and the most popular key/value store. It is used by everyone:
      • Twitter, Instagram, GitHub, StackOverflow, Pinterest, Snapchat, Shopify, Airbnb, Uber, Tumblr, Slack, Medium, Imgur, Kickstarter, etc.
    • Common Use Cases:
      • User Session Cache (e.g., reduce DB lookups for user info, shopping cart data)
      • Full Page Cache (e.g., by URL or route)
      • Queues (e.g., Message Queue, Worker Queue)
      • Counting (e.g., metrics, analytics)
      • Pub/Sub (e.g., chat systems, notifications)
    • Redis Tutorial and Walkthrough: https://try.redis.io/
  • Writing Code Comments based on 32e0d237 commit
    • "[W]riting comments is of paramount importance in order to produce good code, that is maintainable in the long run and understandable by others and by the authors during modifications and debugging activities."
    • "So comments can be, for me, a tool for lowering the cognitive load of the reader."
  • His YouTube channel has more video discussions of the code in Redis

Week 10

  • Reviews
    • PRs must have a title that explains the fix, links to an Issue #
    • PRs must have a full description. If this is UI code, show a screenshot, or explain the fix, talk about how to review it, how to test it, what's not done, what's going to happen in further PRs, etc.
    • All PRs must pass CI: eslint, stylelint, unit tests
    • Don't let people land code with unrelated commits (e.g., merges with master)
    • Ask people to update their master with upstream, and rebase
    • Don't let people land code with changes to unrelated lines/files (e.g., package.json, whitespace changes)
    • Make sure one PR doesn't undo the work of another (e.g., bad merge, erasing existing code)
    • Ask yourself how we'll test every piece of code we take. We need to be able to trust it going forward.
    • PRs for us to try reviewing together:

Week 11

  • 0.3 Recap
    • Make sure your PRs, Issues, and Blog posts are posted on the 0.3 page. I'll mark 0.3 and 0.4 together, in case you're still cleaning up work from the past week (you have time).
    • Over 100 Pull Requests, 67% are already merged
    • 476 commits to 325 files totaling more than 20K lines of code edited/added
    • Top Repos: Telescope! followed by Microsoft, Mozilla, Facebook, Uber, Zeit, Elastic, GatsbyJS, .NET, and 100 others
  • Discussion of git Workflows
    • Value of CI - Travis CI builds and Circle CI builds
    • Rebasing vs Merging
    • Linear history is better than a tangled web of merges
      • gitk on Telescope vs.
      • gitk on Prettier, node.js, WebKit, Gutenberg
    • Many, Small PRS are better than a few big ones
      • easier to review
      • easier to merge
    • Break your work up, leverage the community of developers we have
    • Dealing with package.json and package-lock.json

Week 11

  • 0.4 Status Check
    • Pull Requests
    • Issues
    • Countdown: 2 weeks to a working program. What risks do we see? How to address them?
  • Case Study: Shopify developers using git, CI, merging
    • Over 1000 developers working on the code
    • Shopify changes 40 times a day
    • 400 commits a day to master
    • Master must always be green (passing CI)
    • Master must stay close to production
    • Emergency merges must be fast
    • /shipit GitHub integration
    • Merge queue to manage merges to master
Retrieved from "https://wiki.cdot.senecapolytechnic.ca/w/index.php?title=DPS909_%26_OSD600_Fall_2019&oldid=142205"