Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve common file perf #24

Open
agraubert opened this issue Feb 4, 2020 · 1 comment
Open

Improve common file perf #24

agraubert opened this issue Feb 4, 2020 · 1 comment
Labels
Internal Under-the-hood or Dev-ops enhancements

Comments

@agraubert
Copy link
Collaborator

Right now common files are processed in several steps, which is slow for big jobs. Would be significantly faster if the localizer pulls them out of the job spec as they are discovered.

However, right now, the current version indexes common files based on their input location, which allows for auto-detection of common files anywhere within the job spec
@agraubert agraubert added the Internal Under-the-hood or Dev-ops enhancements label Feb 4, 2020
@agraubert agraubert mentioned this issue Feb 4, 2020
Open
@agraubert
Copy link
Collaborator Author

With the upcoming array file changes, maybe move localization directives for common files upwards into the pick_common_inputs function and delete the keys from the input spec

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Internal Under-the-hood or Dev-ops enhancements
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@agraubert