Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for scipy.sparse matrices #4

Open
Defasium opened this issue Jan 13, 2023 · 1 comment
Open

Support for scipy.sparse matrices #4

Defasium opened this issue Jan 13, 2023 · 1 comment

Comments

@Defasium
Copy link

Hi, @olegranmo!
It would be very cool if methods fit, predict and transform could accept scipy.sparse matrices like most of the sklearn api models: LogisticRegression, MultinomialNB, RandomForest, etc...
For example in
https://github.com/cair/tmu/blob/main/examples/MNISTDemo.py#L18
Converting binarized X_train into scipy.sparse.csr_matrix can lower RAM consumption by a large factor.
This would be very convinient in case of relatively large datasets (with over 1 million examples). Or when there are a lot of features (like high-res images).

@olegranmo
Copy link
Member

Great point, @Defasium! Will add support for sparse matrixes at the first opportunity. Currently, the class TMAutoEncoder uses sparse input matrixes to deal with large text datasets.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants