Add Pydantic Data Model to Document Node Attributes Data Structure

[ytausch]

Currently, there is no documentation about the data model of the cf-graph that this bot implicitly uses.

This is the first step of improving this:

• add a Pydantic model for the node attributes
• add a CI check that validates the model against the current cf-graph state
• the above step lead to the discovery of invalid feedstocks and node attributes files, which are listed in test_validate.py.
• README updates
• add PR template encouraging contributors to update the Pydantic model

Documenting the data model also included discovering some undocumented features of the meta.yaml file. I submitted corresponding upstream docs updates (conda/conda-build#5214, conda/conda-build#5215, conda/conda-build#5217, conda/conda-build#5218)

[beckermr]

This PR is great. I do want to point out that I likely won't have the time or expertise in the short-term to maintain this enhancement by myself.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 71.44%. Comparing base (1db2f38) to head (d77d479).

❗ Current head d77d479 differs from pull request most recent head ddcdc37. Consider uploading reports for the commit ddcdc37 to get more accurate results

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #2239   +/-   ##
=======================================
  Coverage   71.44%   71.44%           
=======================================
  Files          99       99           
  Lines        9839     9839           
=======================================
  Hits         7029     7029           
  Misses       2810     2810           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[beckermr]

@jaimergp has a PR on smithy to add a pydantic model for the conda_forge.yml. This PR should probably pull that model directly from smithy instead of having its own.

[ytausch]

@jaimergp has a PR on smithy to add a pydantic model for the conda_forge.yml. This PR should probably pull that model directly from smithy instead of having its own.

I have noted this here (at the same time as your comment):

cf-scripts/conda_forge_tick/models/conda_forge_yml.py

Lines 18 to 22 in 3b5d0da

	TODO Note: There is currently an open PR that aims to add a Pydantic model for the `conda-forge.yml` file to
	conda-smithy. This PR is not yet merged, so the important parts of the model are defined here.
	In the future, cf-scripts should depend on conda-smithy to obtain the `conda-forge.yml` model from there.
	"""

[beckermr]

The smithy PR will be merged by the end of the week. We should move to using the stuff from smithy now.

[ytausch]

The smithy PR will be merged by the end of the week. We should move to using the stuff from smithy now.

I will not extend the current conda-forge.yml model definition from now since more fields are not used by the bot. We can either wait for the smithy PR to be merged and then change this here or migrate to using smithy as a dependency in a separate PR. I was thinking about the latter option.

In the meantime, I contributed my knowledge to the smithy PR (as comments).

[beckermr]

OK sounds good. I'll leave this PR open then and maybe @jaimergp can help me put in the changes needed to use the new smithy items. Thanks a bunch!

[ytausch]

Confirmation: mypy did not discover any new type issues with the changes of this PR.

[jaimergp]

Thanks! This is on my list for the week. I'll get back to you.

[jaimergp]

The schema is now in conda-smithy@main if you want to update the PR!

[ytausch]

The schema is now in conda-smithy@main if you want to update the PR!

Yes, I will look into it now :)

[ytausch]

The schema is now in conda-smithy@main if you want to update the PR!

blocked by conda-forge/conda-smithy#1858

[beckermr]

The latest smithy is now released. You need v3.32.0

[ytausch]

I want to clarify this PR's status. The following tasks are open, waiting to be resolved:

• Set extra=forbid for Pydantic Schema Fields conda-forge/conda-smithy#1865 (some node files fail to validate because of this)
• remove top-level platform keys conda-forge/libitk-feedstock#88 (invalid feedstock with easy fix)
• conda-forge.yml Pydantic Model: Single Values instead of Lists conda-forge/conda-smithy#1869 (some node files fail to validate because of this)

Those all deal with special cases of the conda-forge.yml validation and only affect the "known bad" feedstock list in the test. Besides that, I consider this PR complete and I will not add any changes related to the issues above.

/Cc @0xbe7a