[EPIC] Pushing features upstream - Jupyter Web App

[saffaalvi]

We will be pushing our newly added features and customizations upstream.

Jupyter Web App

Bring in features from StatCan/jupyter-apis to StatCan/kubeflow (master branch), specifically into components/crud-web-apps/jupyter since the jupyter-web-app component is to be deprecated soon (kubeflow#5553 deprecation status mentioned here)

• Move jupyter-apis (frontend) into StatCan/kubeflow jupyter-web-app Move jupyter-apis (frontend) into StatCan/kubeflow jupyter-web-app #44
  After merging jupyter-apis into jupyter-web-app, we have decided not to move forward with this method. More details in the issue.

• Figure out how to run local development of crud-web-apps/jupyter Test local development of crud-web-apps/jupyter #46

• Commits to start implementing into upstream (will be completed when merged upstream):

  • Support hiding registry or version in list of notebook images Implement hiding registry and tag in image dropdown upstream #51 - finished creating PR for this upstream
    - Jun 8: feat(jupyter): Support hiding image registry/version (17d85bd)
    - Jun 19: fix(jupyter): resolve build error (99722d6)
  • Jun 26: feat(jupyter): Update volume options + dropdown for existing volumes (adbb1a5)
    - This was already implemented upstream on July 24 (kubeflow/kubeflow@d30d769)
  • Moving fonts into the service Implement moving fonts into the service upstream #50 - finished creating PR for this upstream
    - Aug 26: chore(fonts): add fonts as assets (StatCan/jupyter-apis@2a1317b)
    - Sep 1: chore(fonts): add fonts to service (StatCan/jupyter-apis@e9c0fdb)
  • Support notebook volume management Implement notebook volume management upstream #48 - it seems like something is already being implemented for this upstream, this will be put on hold until they release what they have done so far so there's no overlap
    - Sept 17: fix: delete old notebooks (pvc) (StatCan/jupyter-apis@2095188)
    - Sep 23: feat: Add icons to main tables + refactor (StatCan/jupyter-apis@fdb507f)
    - Oct 2: fix: Add status to pvc table + refactor (StatCan/jupyter-apis@49fc2b3)
    - Oct 6: feat: Convey more info via volume status column (StatCan/jupyter-apis@4fabd14)
  • Add Official Languages Implement Official Languages with Angular ngx upstream #49 - finished creating PR for this upstream
    - Nov 4: feat: Add Official Languages with ngx-translate (StatCan/jupyter-apis@2e213ec)
    - Jan 21: feat: Add env variable to set language of the notebook server (StatCan/jupyter-apis@83af673)
  • Add env variable to set language of the notebook server Implement the Language Env Variable #55
  • Add localize Fr tranlations Add localize fr translations #66
  • Validation for mounted pvc Implement validation for mounted pvc upstream #53
    - Jan 4: feat: Add validation for mounted pvc (StatCan/jupyter-apis@f2614b6)
    - Jan 4: fix: show error when element is untouched (StatCan/jupyter-apis@22805a3)
  • Jan 7: fix: add check for runnning notebook server (StatCan/jupyter-apis@b8a3683)
    - This commit added a check to use ReadyReplicas to calculate notebook server status. I think upstream has already started using ReadyReplicas to process the status in crud-web-apps status.py
  • Fix npm vulnerabilities Implement our npm vulnerability fixes upstream #52
    • Aug 19 : fix(npm): fix vulnerabilities (squashed commit) (StatCan/jupyter-apis@d32875d)
    • Jan 25: fix npm vulnerabilities (StatCan/jupyter-apis@197905b)

• The major features that are likely the most important to get upstream are:

  • Support notebook volume management Implement notebook volume management upstream #48
  • Add Official Languages Implement Official Languages with Angular ngx upstream #49

[davidspek]

Regarding volume management, there is the new Volumes Web App that is being added. However, after the release I plan to work on a proposal that integrates https://github.com/filebrowser/filebrowser into this with a controller. I also have an idea how this could be integrated with the notebook images to overcome the ReadWriteOnce limitation of many storage classes.

[saffaalvi]

To get features upstream, I’ve been using the following process:

1. Move the issue for that feature (on the DAaaS board) from Backlog to In Progress
2. Clone the master branch from kubeflow/kubeflow locally
3. Make the changes for the feature/customization based on the commits that were made for that feature (documented in this EPIC)
   • We have documented how to get a stable (enough) local development to test out changes for crud-web-apps/jupyter in this issue: Test local development of crud-web-apps/jupyter #46. You will need to get this working so you can test and see if your changes are working as they should be.
4. Push local branch to our StatCan/kubeflow repository as a new branch
5. Ask someone from our team to review your branch and test out the changes before creating the PR upstream. Move issue from In Progress to Review
6. After reviewed, create a PR from your branch in the StatCan/kubeflow repo to be merged into kubeflow/kubeflow master branch
7. Update this EPIC to note that the PR for that feature/issue has been made upstream
   • Move the issue for that feature back into the Backlog once it has been created upstream with the triage/blocked label. The issue will be closed once the PR has been merged upstream.

*Note: you will have to sign a Contributor’s License Agreement (CLA) either on behalf of only yourself or your employer. You will need to go through StatCan’s internal process of which agreement you will be signing. Make sure the email address on your git commits have the same email address as the one used to sign the CLA.