You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently various plotting-evaluation scripts and notebooks are scattered around the repository. Create a master notebook that calculates all the results from the trainings of the three training scenarios - dm_multiclass, jet_regression and binary_classification.
There is an evaluator written for all of these training scenarios under enreg/tools/ - decay_mode_evaluator.py, regression_evaluator.py and tagger_evaluator.py.
An example of their use can be found in notebooks/paper_performance_plotting.ipynb
The master notebook should be executable by papermill. An example of its usage can be found using a simple google search.
Ideally dm_evaluator would need also the capability to combine different evaluators in order to make direct comparisons. The direct comparison would be Fig.6 from our most recent paper. Fig.6 is produced currently with the notebooks/DM_CM.ipynb notebook.
As a next step, it would be preferred if also the metrics tracked during training could be plotted within this master script for each training scenario - currently the comparisons are done with notebooks/foundation_model_eval.ipynb and notebooks/losses.ipynb
The text was updated successfully, but these errors were encountered:
Currently various plotting-evaluation scripts and notebooks are scattered around the repository. Create a master notebook that calculates all the results from the trainings of the three training scenarios - dm_multiclass, jet_regression and binary_classification.
There is an evaluator written for all of these training scenarios under enreg/tools/ - decay_mode_evaluator.py, regression_evaluator.py and tagger_evaluator.py.
An example of their use can be found in notebooks/paper_performance_plotting.ipynb
The master notebook should be executable by papermill. An example of its usage can be found using a simple google search.
Ideally dm_evaluator would need also the capability to combine different evaluators in order to make direct comparisons. The direct comparison would be Fig.6 from our most recent paper. Fig.6 is produced currently with the notebooks/DM_CM.ipynb notebook.
As a next step, it would be preferred if also the metrics tracked during training could be plotted within this master script for each training scenario - currently the comparisons are done with notebooks/foundation_model_eval.ipynb and notebooks/losses.ipynb
The text was updated successfully, but these errors were encountered: