You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When an application detects that a file is not needed anymore it can call vine_remove_file and this deletes the file from the workers, and from the manager's file table. When not using temp files this seems like the correct behavior.
However, when using temp files, the application may decide to retrieve some of the files as checkpoints of the computation. Given space considerations, it may be desirable to delete this files from the workers but not entirely from the manager. Should a worker be lost with its temp files, the manager can send the saved files so that the computation does not have to start from scratch.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
When an application detects that a file is not needed anymore it can call
vine_remove_file
and this deletes the file from the workers, and from the manager's file table. When not using temp files this seems like the correct behavior.However, when using temp files, the application may decide to retrieve some of the files as checkpoints of the computation. Given space considerations, it may be desirable to delete this files from the workers but not entirely from the manager. Should a worker be lost with its temp files, the manager can send the saved files so that the computation does not have to start from scratch.
Beta Was this translation helpful? Give feedback.
All reactions