Does using multiple processes for a python app use memory sharing? #1243
-
|
I know that for uwsgi, according to docs:
Does NGINX Unit do something similar? If I set |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 8 replies
-
|
In short yes. Lets look at a Unit process tree comprising unit and 5 python application processes... You'll notice we create a "prototype" process, we do that for each application (regardless of language), from there we fork the prescribed number of "application" processes. You can see that the application processes are children of the prototype process. Prototype processes were added in e207415 Applications processes may come and go but the prototype process will live for the duration of the application. |
Beta Was this translation helpful? Give feedback.
-
|
Is garbage collection frozen for preexisting objects (or otherwise handled) after the fork? I am asking since I know that forking in python without doing that risk triggering copy-on-write just because of garbage collection (garbage collection being "write"). |
Beta Was this translation helpful? Give feedback.
-
|
Not AFAIACT... |
Beta Was this translation helpful? Give feedback.
-
|
Not sure how often it is relevant. I originally learned about it from this video: https://www.youtube.com/watch?v=Hgw_RlCaIds |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the link to the video. That made me dig a little deeper. My answer here may change slightly what you take away from my original answer. Now gc.freeze is some Python API and obviously when we do the fork(2) we are still in C, but that is all moot anyway... The prototype processes only load the required Unit language module, e.g for python The prototype process doesn't load or run any application code, that only happens in the application processes. |
Beta Was this translation helpful? Give feedback.
-
|
Oh, ok so then I would say the answer to my original question is more of a "no" than a "yes" (even though what you are doing there is of course very good). For me, the issue was to do with applications which load machine learning models, and trying to run multiple copies of such an app without having multiple copies of the model in memory. I noticed that when was doing things like this with uwsgi, having multiple processes did not affect the total memory consumption significantly, but when I was using NGINX Unit, each copy of the app added 0.5 GB of memory consumption (which seems to be approximately how much memory is used by the loaded model). This means if I still want the same performance of my app, I will have to use much more memory. I see that gunicorn also has some support for this, using a "preload" option: https://www.joelsleppy.com/blog/gunicorn-application-preloading/ How would you feel about my making this a feature request? |
Beta Was this translation helpful? Give feedback.
-
|
It's an interesting problem/idea... At the moment I couldn't definitively say that it is something we would want to implement or even could implement. Remember we have more than just Python to worry about. Then there is the question of if we loaded an application that opened a bunch of database connections or somesuch, then when we forked the application processes each process would have the same connections and that's not gong to end well. Then there is the reload issue as mentioned which may be problematic for us. How are these models loaded? Are they contained in files that are say mmap(2)ed or are they generated at run-time? I can see ways of doing this, but of course it requires a bit more work on the application side... |
Beta Was this translation helpful? Give feedback.
-
|
I will say, to me, this way of doing stuff that uwsgi does (at least if I understand the docs correctly) is a bit strange, and they even call it "(ab)use" in their docs. So I fully understand if you consider it too hacky. I can't really make any recommendations myself. As for the models in my case, they are tar balls that are decompressed, then the tar balls contain some pickle files, which are depickled into python objects. The decompression and deserialization is entirely from python, using tarfile and pickle libraries. (I don't really think this pickling is a good format, personally.) Sadly I don't even know what "mmap(2)" is (even after trying to look it up) so I can't really answer that part of your question. |
Beta Was this translation helpful? Give feedback.
Thanks for the link to the video. That made me dig a little deeper.
My answer here may change slightly what you take away from my original answer.
Now gc.freeze is some Python API and obviously when we do the fork(2) we are still in C, but that is all moot anyway...
The prototype processes only load the required Unit language module, e.g for python
$(modulesdir}/python.unit.so, which in turn loads the required libraries for that language. e.g the python prototype process loads these (the PHP language module loads a lot more things...)