CI test for input_data_list

[alperaltuntas]

introduces two CI tests

1. Check whether any input file names in MOM_input.yaml is missing in input_data_list.yaml.
2. Check whether any input files listed in input_data_list.yaml is missing in svn inputdata repository.

Limitation: This CI test only checks filenames and not absolute/relative paths.

Fixes: #194, #195

Current check_input_data_list.py output:

ValueError: Below parameter value(s) in MOM_input.yaml are suspected to be input file name(s), but are not present in input_data_list.yaml. If these are indeed input files, please add them to input_data_list.yaml. If not, please update this CI test module to exclude them from the check e.g., by adding them to the EXCEPTIONS list.

  MISOMIP_181108.nc
  dz_max_90th_quantile.nc
  ecosys_jan_IC_omip_latlon_1x1_180W_c230331.nc
  MOM_channels_global_025
  energy_new_tx2_3_conserve_230415_cdf5.nc
  MISOMIP_IC
  zstar_75layer_2.5m_248.4m-2024-03-29.nc
  MOM_channels_global_tx2_3v2_240501
  geothermal_davies2013_tx2_3_20240318_cdf5.nc
  ocean_topo_tx2_3v2_240501.nc
  hybrid_75layer_zstar2.50m-2020-11-23.nc

Current check_input_data_repo.py output (if input_data_list were completed).

ValueError: Below file names are listed in input_data_list.yaml but are missing in the svn inputdata repository. If these files are not needed, please remove them from input_data_list.yaml. If they are needed, please import them to the svn repository.

  ecosys_jan_IC_omip_latlon_1x1_180W_c230331.nc
  hybrid_75layer_zstar2.50m-2020-11-23.nc
  riv_nut.gnews_gnm.r05_to_tx2_3v2_nnsm_e250r250_230914.20240202.nc

[alperaltuntas]

This PR is now ready to be reviewed.

[alperaltuntas]

I've just pushed another commit that updates input_data_list to add all ten of the missing input files discovered by this new CI test. I've also rimported below files which were found to be missing in svn input data by the newly added CI test.

hybrid_75layer_zstar2.50m-2020-11-23.nc
riv_nut.gnews_gnm.r05_to_tx2_3v2_nnsm_e250r250_230914.20240202.nc

There is still one missing file in svn inputdata. I'll follow up with a seperate GitHub issue, but otherwise, this PR is ready for evaluation and merge.

[manishvenu]

Looks good!

[alperaltuntas]

@mnlevy1981, any comments/suggestions for this PR?

[mnlevy1981]

Would it make more sense to update MOM_input.yaml and ParamGen so that we can set some variables as input_data_check: True and that variable / filename would be added to the checklist?

It seems like we are duplicating a lot of information between MOM_input.yaml and input_data_list.yaml and, as we've seen, it's hard to keep that duplicate information in sync. Let's just keep it all in MOM_input.yaml and remove input_data_list.yaml altogether.

If that seems like too big of a task for now, I'm okay with bringing in these updates but I do think we eventually want to rely solely on MOM_input.yaml.

[mnlevy1981]

Also, I added ecosys_jan_IC_omip_latlon_1x1_180W_c230331.nc to the inputdata repository so the check_input_data_repo test passes

[alperaltuntas]

I like the idea of removing input_data_list.yaml, though it's probably best to do that in a subsequent PR. This test would still be needed (with alterations) so I'll merge this as is unless there are further comments/suggestions.