Feat 1078 modelschema #1109
Feat 1078 modelschema #1109
Conversation
| name: str = Field(..., title="Name") | ||
| developer_full_name: Optional[str] = Field(default=None, title="Name of developer") | ||
| developer_institution: Optional[Organization.ONE_OF] = Field(default=None, title="Institute where developed") | ||
| modality: Modality.ONE_OF = Field(..., title="Modality") |
There was a problem hiding this comment.
I wonder if we want a more generic version of modality as well, especially for models like an LLM that is not taking experimental data specifically as input (ie text, image, video etc). Not sure what an alternative name for that would be though.
There was a problem hiding this comment.
If this is meant to include all models, I agree with Tom that there won't always be training data. In 'task trained' RNNs, the model is trained on a supervised target output. I imagine some models might be 'pre-trained' in this setup and then trained on data at a later stage of training.
src/aind_data_schema/core/model.py
Outdated
| std: Decimal = Field(..., title="Standard deviation") | ||
|
|
||
|
|
||
| class PerformanceScore(AindModel): |
There was a problem hiding this comment.
do these only apply to supervised models?
| value: Any = Field(..., title="Metric value") | ||
|
|
||
|
|
||
| class ModelEvaluation(AindModel): |
There was a problem hiding this comment.
might be clearer to have a distinct ModelTraining as a subclass of ModelEvaluation. This would add a cross_validation_method (instead of validation_folds) along with a list of training_parameters and anything else we want to highlight (perhaps a specific field for data augmentation if present?).
There was a problem hiding this comment.
actually, now I'm wondering if these should both subclass DataProcess, with ModelEvaluation just adding the performance field? It's a bit off conceptually in the case of unsupervised methods, but I think we're already considering simulation as within the scope of DataProcess so maybe no different from that.
| name: str = Field(..., title="Name") | ||
| developer_full_name: Optional[str] = Field(default=None, title="Name of developer") | ||
| developer_institution: Optional[Organization.ONE_OF] = Field(default=None, title="Institute where developed") | ||
| modality: Modality.ONE_OF = Field(..., title="Modality") |
There was a problem hiding this comment.
I wonder if we want a more generic version of modality as well, especially for models like an LLM that is not taking experimental data specifically as input (ie text, image, video etc). Not sure what an alternative name for that would be though.
|
closing in favor of #1166 |
closes #1078