Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make schema-derived JSON and YAML files downloadable via docs website #2302

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

eecavanna
Copy link
Collaborator

@eecavanna eecavanna commented Dec 8, 2024

In this branch, I made a proof of concept of making the schema-derived YAML and JSON files be accessible via the GitHub Pages-hosted documentation website instead of via the Git repository, itself. They would be generated by a GitHub Actions workflow based upon the source files in the Git repository, so they'd always be consistent with those source files (whereas, today, keeping them consistent is a manual process).

Here's a screenshot of a barebones web page that has links to each file. This is a proof of concept, not necessarily the most user friendly presentation of these files. We can iterate on this as we did when introducing a refscan-generated diagram into the documentation a few weeks ago. For example, we could display the actual contents of each file within an in-page code viewer widget, such as the one provided by CodeMirror.

image

When I click on any of the links, my web browser either displays its contents (if it's a JSON file) or prompts me to choose where I want it to be downloaded to (if it's a YAML file). This behavior is web browser-dependent, but we (the creators of the web page) can do things to tell the web browser what we'd prefer it do.

image

Each of the files can be downloaded via curl, as shown here:

image

@eecavanna eecavanna self-assigned this Dec 8, 2024
@eecavanna eecavanna linked an issue Dec 8, 2024 that may be closed by this pull request
Copy link

github-actions bot commented Dec 8, 2024

PR Preview Action v1.4.8
🚀 Deployed preview to https://microbiomedata.github.io/nmdc-schema/pr-preview/pr-2302/
on branch gh-pages at 2024-12-08 23:37 UTC

@eecavanna
Copy link
Collaborator Author

@turbomam and I discussed this PR today. Here's my public "notes" list, after having that discussion:

  1. TODO: @eecavanna to document (at least in this PR's description) how these changes would affect how the nmdc_data.py file accesses the derived schema files.
    1. When invoking those functions via the PyPI package, the derived files will be present since they're generated during the package build process (via make all).
    2. When doing local development in the schema repo, (if we omit the derived files from the repo) the derived files will only be present once the developer has derived them.
      1. Today, the derived files exist in the repo; but those copies are only in sync with the latest time someone built/committed them, and the schema source files are not necessarily in the same shape as they were when that took place.
      2. I volunteered to document the fact that, before someone can call those functions locally (i.e. not via a PyPI package), they'll have to do a few things; e.g. have poetry, make, and yq installed; run poetry install; and run make squeaky-clean all. I volunteered to write a quick start guide for local development.
  2. Note: Both of us think it would be OK to not make the two .py files be downloadable via this web page. We assume people only consume those via the PyPI package, not by pulling them from the Git repo.
  3. Note: Making the JSON/YAML files available on a web page opens up some opportunities for us to make them interactive/engaging in some ways.
    1. I brought this up briefly, before realizing it was a can of worms I didn't want to open at this time. After I brought it up, @turbomam put forth a couple examples of way we would make the schema be easier to consume: (a) offer more powerful searches than what web browser provide (e.g. regex searches) and (b) apply syntax highlighting. Anyway, I will shelve this pursuit for now (I consider it to be scope creep relative to the task at hand; which is: getting derived files out of the repo).
  4. TODO: @eecavanna to update the .gitignore file accordingly.
  5. TODO: @eecavanna to try to programmatically represent the dependence that make gendoc has upon the output of make all, without making gendoc be a heavyweight target.

Once I'm ready for @turbomam to take another look at this PR branch, I'll tell him that. It's still in a draft state.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Omit derived files from repo (generate them locally instead)
1 participant