Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prune down what goes into the fit #2221

Open
scarlehoff opened this issue Nov 21, 2024 · 0 comments
Open

Prune down what goes into the fit #2221

scarlehoff opened this issue Nov 21, 2024 · 0 comments

Comments

@scarlehoff
Copy link
Member

See here #2217 (comment)

This is due to the fact that by the time we go into the fit we still have access to all information from loading the datasets (the fktables in terms of huge pandas dataframes, loaded with and without cuts, same for the data, etc).
We don't need any of that during the fit, so we should drop all that before getting to the hyperparametrizable function.

This doesn't really make a difference in a normal fit (probably it is using more memory than needed but nothing too serious) but when doing a hyperparameter scan the memory starts becoming a bottleneck.

Not something urgent, but it would be a nice addition.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant