You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is the project ready for production use? What is the minimum required hardware to run version 7B? (recommended CPU? how many CPU threads?)
Can the project handle 8GB of RAM? 32GB is excessive!
The text was updated successfully, but these errors were encountered:
Hey,
I think at the moment it is not production-ready. it loads the whole model into the memory and if you do not have enough resources it slows down the system ( and maybe hangs).
So you can use it for testing purposes but in my opinion, it is not ready for prod.
Is the project ready for production use? What is the minimum required hardware to run version 7B? (recommended CPU? how many CPU threads?)
Can the project handle 8GB of RAM? 32GB is excessive!
The text was updated successfully, but these errors were encountered: