Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Caching integration #72

Open
wants to merge 159 commits into
base: master
Choose a base branch
from
Open

Caching integration #72

wants to merge 159 commits into from

Commits on Oct 31, 2019

  1. added storage object to represent caches and associated readout funct… 

    …ionality
    @tfesenbecker
    tfesenbecker committed Oct 31, 2019
    Configuration menu
    Copy the full SHA
    9215d64 View commit details
    Browse the repository at this point in the history

  2. extended CLI to support storage files

    @tfesenbecker
    tfesenbecker committed Oct 31, 2019
    Configuration menu
    Copy the full SHA
    0a46da7 View commit details
    Browse the repository at this point in the history

  3. extended simulator to support storage files

    @tfesenbecker
    tfesenbecker committed Oct 31, 2019
    Configuration menu
    Copy the full SHA
    e727b69 View commit details
    Browse the repository at this point in the history

  4. added new drone attribute sitename connecting drones and storage elem… 

    …ents
    @tfesenbecker
    tfesenbecker committed Oct 31, 2019
    Configuration menu
    Copy the full SHA
    d64954d View commit details
    Browse the repository at this point in the history

  5. added file provider object connecting storage objects and jobs

    @tfesenbecker
    tfesenbecker committed Oct 31, 2019
    Configuration menu
    Copy the full SHA
    5597434 View commit details
    Browse the repository at this point in the history

  6. added different caching/cache cleaning/walltime recalculation algorithms

    @tfesenbecker
    tfesenbecker committed Oct 31, 2019
    Configuration menu
    Copy the full SHA
    bb1dcbe View commit details
    Browse the repository at this point in the history

  7. renamed storage readout

    @tfesenbecker
    tfesenbecker committed Oct 31, 2019
    Configuration menu
    Copy the full SHA
    8d6db96 View commit details
    Browse the repository at this point in the history

Commits on Nov 1, 2019

  1. fixed debug output

    @tfesenbecker
    tfesenbecker committed Nov 1, 2019
    Configuration menu
    Copy the full SHA
    fb150db View commit details
    Browse the repository at this point in the history

  2. renamed storage input reader

    @tfesenbecker
    tfesenbecker committed Nov 1, 2019
    Configuration menu
    Copy the full SHA
    69072ae View commit details
    Browse the repository at this point in the history

Commits on Nov 2, 2019

  1. updated Job class

    @tfesenbecker
    tfesenbecker committed Nov 2, 2019
    Configuration menu
    Copy the full SHA
    53ebec5 View commit details
    Browse the repository at this point in the history

  2. replaced function modifying walltime by function with property decorator

    @tfesenbecker
    tfesenbecker committed Nov 2, 2019
    Configuration menu
    Copy the full SHA
    f997223 View commit details
    Browse the repository at this point in the history

  3. Revert "replaced function modifying walltime by function with propert… 

    …y decorator"

This reverts commit f997223.
    @tfesenbecker
    tfesenbecker committed Nov 2, 2019
    Configuration menu
    Copy the full SHA
    2e2c06f View commit details
    Browse the repository at this point in the history

  4. replaced function modifying walltime by function with property decorator

    @tfesenbecker
    tfesenbecker committed Nov 2, 2019
    Configuration menu
    Copy the full SHA
    110b3e9 View commit details
    Browse the repository at this point in the history

  5. resolving PEP8 issues

    @tfesenbecker
    tfesenbecker committed Nov 2, 2019
    Configuration menu
    Copy the full SHA
    b032a0d View commit details
    Browse the repository at this point in the history

  6. Merge branch 'master' of https://github.com/MatterMiners/lapis into c… 

    …achingextension
    @tfesenbecker
    tfesenbecker committed Nov 2, 2019
    Configuration menu
    Copy the full SHA
    7753d0d View commit details
    Browse the repository at this point in the history

  7. fixed file provider bug (wrong inputfiles dictionary)

    @tfesenbecker
    tfesenbecker committed Nov 2, 2019
    Configuration menu
    Copy the full SHA
    5123034 View commit details
    Browse the repository at this point in the history

Commits on Nov 4, 2019

  1. Update lapis/cli/simulate.py 

    Co-Authored-By: Eileen Kuehn <[email protected]>
    @tfesenbecker @eileen-kuehn
    tfesenbecker and eileen-kuehn authored Nov 4, 2019
    Configuration menu
    Copy the full SHA
    1c2fe9f View commit details
    Browse the repository at this point in the history

  2. renamed function get_used_storage to _calculate_used_storage

    @tfesenbecker
    tfesenbecker committed Nov 4, 2019
    Configuration menu
    Copy the full SHA
    8739ce9 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'cachingextension' of https://github.com/tfesenbecker/lapis 

     into cachingextension
    @tfesenbecker
    tfesenbecker committed Nov 4, 2019
    Configuration menu
    Copy the full SHA
    0b5a922 View commit details
    Browse the repository at this point in the history

  4. attached fileprovider to drone instead of job and passed it via make_… 

    …drone instead of scheduler
    @tfesenbecker
    tfesenbecker committed Nov 4, 2019
    Configuration menu
    Copy the full SHA
    855242a View commit details
    Browse the repository at this point in the history

  5. reworked file coverage function to return a score

    @tfesenbecker
    tfesenbecker committed Nov 4, 2019
    Configuration menu
    Copy the full SHA
    bfadacb View commit details
    Browse the repository at this point in the history

  6. added proper __repr__ function

    @tfesenbecker
    tfesenbecker committed Nov 4, 2019
    Configuration menu
    Copy the full SHA
    3f30c58 View commit details
    Browse the repository at this point in the history

Commits on Nov 7, 2019

  1. added file classes

    @tfesenbecker
    tfesenbecker committed Nov 7, 2019
    Configuration menu
    Copy the full SHA
    2b214aa View commit details
    Browse the repository at this point in the history

  2. moved caching algorithm and associated cache cleanup to it's own class

    @tfesenbecker
    tfesenbecker committed Nov 7, 2019
    Configuration menu
    Copy the full SHA
    2bd91d7 View commit details
    Browse the repository at this point in the history

  3. Redesign of the storage class and associated changes

    @tfesenbecker
    tfesenbecker committed Nov 7, 2019
    Configuration menu
    Copy the full SHA
    29576eb View commit details
    Browse the repository at this point in the history

  4. put walltime getter and walltime recalculation back in seperate methods

    @tfesenbecker
    tfesenbecker committed Nov 7, 2019
    Configuration menu
    Copy the full SHA
    146fbe3 View commit details
    Browse the repository at this point in the history

  5. added parallel treatment of jobs input files in file provider

    @tfesenbecker
    tfesenbecker committed Nov 7, 2019
    Configuration menu
    Copy the full SHA
    7ef8dd9 View commit details
    Browse the repository at this point in the history

  6. fixed failed unit test that were caused by Drone without file provide… 

    …r, one unit test fails because job has no drone, ticket already exists
    @tfesenbecker
    tfesenbecker committed Nov 7, 2019
    Configuration menu
    Copy the full SHA
    b94ab82 View commit details
    Browse the repository at this point in the history

  7. Merge branch 'master' of https://github.com/MatterMiners/lapis into c… 

    …achingextension
    @tfesenbecker
    tfesenbecker committed Nov 7, 2019
    Configuration menu
    Copy the full SHA
    1e9e795 View commit details
    Browse the repository at this point in the history

Commits on Nov 8, 2019

  1. changed scoring to take filesizes into consideration

    @tfesenbecker
    tfesenbecker committed Nov 8, 2019
    Configuration menu
    Copy the full SHA
    191df2b View commit details
    Browse the repository at this point in the history

  2. Merge branch 'cachingextension' into feature/storageimprovement

    @tfesenbecker
    tfesenbecker authored Nov 8, 2019
    Configuration menu
    Copy the full SHA
    a635318 View commit details
    Browse the repository at this point in the history

  3. Merge pull request #1 from tfesenbecker/feature/storageimprovement 

    Feature/storageimprovement
    @tfesenbecker
    tfesenbecker authored Nov 8, 2019
    Configuration menu
    Copy the full SHA
    6f7ace1 View commit details
    Browse the repository at this point in the history

  4. fixed bug from merge

    @tfesenbecker
    tfesenbecker committed Nov 8, 2019
    Configuration menu
    Copy the full SHA
    75165ad View commit details
    Browse the repository at this point in the history

  5. Merge branch 'cachingextension' of https://github.com/tfesenbecker/lapis 

     into cachingextension
    @tfesenbecker
    tfesenbecker committed Nov 8, 2019
    Configuration menu
    Copy the full SHA
    d943ed6 View commit details
    Browse the repository at this point in the history

  6. Merge pull request #2 from tfesenbecker/feature/storageimprovement 

    fixed bug from merge
    @tfesenbecker
    tfesenbecker authored Nov 8, 2019
    Configuration menu
    Copy the full SHA
    9453632 View commit details
    Browse the repository at this point in the history

  7. removed debug output to fix unit test

    @tfesenbecker
    tfesenbecker committed Nov 8, 2019
    Configuration menu
    Copy the full SHA
    32faa38 View commit details
    Browse the repository at this point in the history

Commits on Nov 11, 2019

  1. First steps towards including everything concerning caching into moni… 

    …toring
    @tfesenbecker
    tfesenbecker committed Nov 11, 2019
    Configuration menu
    Copy the full SHA
    78a6f18 View commit details
    Browse the repository at this point in the history

  2. renamed method

    @tfesenbecker
    tfesenbecker committed Nov 11, 2019
    Configuration menu
    Copy the full SHA
    c7c2e03 View commit details
    Browse the repository at this point in the history

  3. split processing of job into file transfer and actual calculation

    @tfesenbecker
    tfesenbecker committed Nov 11, 2019
    Configuration menu
    Copy the full SHA
    73ada77 View commit details
    Browse the repository at this point in the history

  4. refactored storage and file provider objects in order to use Pipe

    @tfesenbecker
    tfesenbecker committed Nov 11, 2019
    Configuration menu
    Copy the full SHA
    02a79bb View commit details
    Browse the repository at this point in the history

Commits on Nov 12, 2019

  1. added monitoring for remote and storage connections

    @tfesenbecker
    tfesenbecker committed Nov 12, 2019
    Configuration menu
    Copy the full SHA
    63664e0 View commit details
    Browse the repository at this point in the history

  2. small fix in monitoring

    @tfesenbecker
    tfesenbecker committed Nov 12, 2019
    Configuration menu
    Copy the full SHA
    2df3841 View commit details
    Browse the repository at this point in the history

Commits on Nov 13, 2019

  1. Merge branch 'master' of https://github.com/MatterMiners/lapis into f… 

    …eature/includepipes
    @tfesenbecker
    tfesenbecker committed Nov 13, 2019
    Configuration menu
    Copy the full SHA
    a6a9783 View commit details
    Browse the repository at this point in the history

Commits on Nov 14, 2019

  1. Merge branch 'master' of https://github.com/MatterMiners/lapis into f… 

    …eature/includepipes
    @tfesenbecker
    tfesenbecker committed Nov 14, 2019
    Configuration menu
    Copy the full SHA
    6e5cdd7 View commit details
    Browse the repository at this point in the history

Commits on Nov 17, 2019

  1. adapted job walltime to new job processing in order to fix job event … 

    …monitoring
    @tfesenbecker
    tfesenbecker committed Nov 17, 2019
    Configuration menu
    Copy the full SHA
    cf5b3ab View commit details
    Browse the repository at this point in the history

  2. minor clean ups

    @tfesenbecker
    tfesenbecker committed Nov 17, 2019
    Configuration menu
    Copy the full SHA
    3768f5b View commit details
    Browse the repository at this point in the history

  3. added cache modelation via cachehitrate

    @tfesenbecker
    tfesenbecker committed Nov 17, 2019
    Configuration menu
    Copy the full SHA
    df17230 View commit details
    Browse the repository at this point in the history

  4. Update lapis/cachealgorithm.py 

    Co-Authored-By: Max Fischer <[email protected]>
    @tfesenbecker @maxfischer2781
    tfesenbecker and maxfischer2781 authored Nov 17, 2019
    Configuration menu
    Copy the full SHA
    128425f View commit details
    Browse the repository at this point in the history

  5. Update lapis/storage.py 

    Co-Authored-By: Max Fischer <[email protected]>
    @tfesenbecker @maxfischer2781
    tfesenbecker and maxfischer2781 authored Nov 17, 2019
    Configuration menu
    Copy the full SHA
    04223e4 View commit details
    Browse the repository at this point in the history

  6. Update lapis/storage.py 

    Co-Authored-By: Max Fischer <[email protected]>
    @tfesenbecker @maxfischer2781
    tfesenbecker and maxfischer2781 authored Nov 17, 2019
    Configuration menu
    Copy the full SHA
    59830d7 View commit details
    Browse the repository at this point in the history

  7. Update lapis/storage.py 

    Co-Authored-By: Max Fischer <[email protected]>
    @tfesenbecker @maxfischer2781
    tfesenbecker and maxfischer2781 authored Nov 17, 2019
    Configuration menu
    Copy the full SHA
    a7b3323 View commit details
    Browse the repository at this point in the history

  8. Update lapis/storage.py 

    Co-Authored-By: Max Fischer <[email protected]>
    @tfesenbecker @maxfischer2781
    tfesenbecker and maxfischer2781 authored Nov 17, 2019
    Configuration menu
    Copy the full SHA
    eff97c4 View commit details
    Browse the repository at this point in the history

  9. Update lapis/storage.py 

    Co-Authored-By: Max Fischer <[email protected]>
    @tfesenbecker @maxfischer2781
    tfesenbecker and maxfischer2781 authored Nov 17, 2019
    Configuration menu
    Copy the full SHA
    ffca7a3 View commit details
    Browse the repository at this point in the history

  10. Update lapis/storage.py 

    Co-Authored-By: Max Fischer <[email protected]>
    @tfesenbecker @maxfischer2781
    tfesenbecker and maxfischer2781 authored Nov 17, 2019
    Configuration menu
    Copy the full SHA
    97e3f83 View commit details
    Browse the repository at this point in the history

  11. resolved PEP8 issue

    @tfesenbecker
    tfesenbecker committed Nov 17, 2019
    Configuration menu
    Copy the full SHA
    a4ceec4 View commit details
    Browse the repository at this point in the history

Commits on Nov 19, 2019

  1. Merge pull request #61 from tfesenbecker/feature/includepipes 

    Feature/includepipes
    @tfesenbecker
    tfesenbecker authored Nov 19, 2019
    Configuration menu
    Copy the full SHA
    b2cb120 View commit details
    Browse the repository at this point in the history

  2. minor fix

    @tfesenbecker
    tfesenbecker committed Nov 19, 2019
    Configuration menu
    Copy the full SHA
    cdeeea6 View commit details
    Browse the repository at this point in the history

Commits on Nov 20, 2019

  1. moved definition of remote throughput to CLI input, storage object th… 

    …roughput to storage input file and caching via fixed cache hitrate to CLI input
    @tfesenbecker
    tfesenbecker committed Nov 20, 2019
    Configuration menu
    Copy the full SHA
    edfc8f1 View commit details
    Browse the repository at this point in the history

  2. Extended cache algorithm documentation

    @tfesenbecker
    tfesenbecker committed Nov 20, 2019
    Configuration menu
    Copy the full SHA
    25a2a23 View commit details
    Browse the repository at this point in the history

  3. implemented minor changes requested in PRs

    @tfesenbecker
    tfesenbecker committed Nov 20, 2019
    Configuration menu
    Copy the full SHA
    b374cf6 View commit details
    Browse the repository at this point in the history

  4. Update lapis/storage.py

    @tfesenbecker
    tfesenbecker committed Nov 20, 2019
    Configuration menu
    Copy the full SHA
    4487db6 View commit details
    Browse the repository at this point in the history

Commits on Nov 22, 2019

  1. Merge branch 'master' of https://github.com/MatterMiners/lapis into f… 

    …eature/caching
    @tfesenbecker
    tfesenbecker committed Nov 22, 2019
    Configuration menu
    Copy the full SHA
    521529f View commit details
    Browse the repository at this point in the history

Commits on Nov 26, 2019

  1. updated usim version requirement to 0.4.2

    @tfesenbecker
    tfesenbecker committed Nov 26, 2019
    Configuration menu
    Copy the full SHA
    5ebf272 View commit details
    Browse the repository at this point in the history

  2. completed renaming of file provider to connection

    @tfesenbecker
    tfesenbecker committed Nov 26, 2019
    Configuration menu
    Copy the full SHA
    1e814ea View commit details
    Browse the repository at this point in the history

Commits on Nov 27, 2019

  1. fixed job and simulator unit tests

    @tfesenbecker
    tfesenbecker committed Nov 27, 2019
    Configuration menu
    Copy the full SHA
    43361c9 View commit details
    Browse the repository at this point in the history

  2. fixed job and simulator unit tests

    @tfesenbecker
    tfesenbecker committed Nov 27, 2019
    Configuration menu
    Copy the full SHA
    d47789f View commit details
    Browse the repository at this point in the history

  3. replaced Storage.__repr__ to match the other classes

    @tfesenbecker
    tfesenbecker committed Nov 27, 2019
    Configuration menu
    Copy the full SHA
    a443618 View commit details
    Browse the repository at this point in the history

  4. added missing default values for unit test compatibility

    @tfesenbecker
    tfesenbecker committed Nov 27, 2019
    Configuration menu
    Copy the full SHA
    cca1859 View commit details
    Browse the repository at this point in the history

  5. extended monitoring

    @tfesenbecker
    tfesenbecker committed Nov 27, 2019
    Configuration menu
    Copy the full SHA
    36f5966 View commit details
    Browse the repository at this point in the history

  6. fixed PEP8 issue

    @tfesenbecker
    tfesenbecker committed Nov 27, 2019
    Configuration menu
    Copy the full SHA
    be5482f View commit details
    Browse the repository at this point in the history

  7. updated via_usim decorator

    @eileen-kuehn
    eileen-kuehn committed Nov 27, 2019
    Configuration menu
    Copy the full SHA
    4a358cf View commit details
    Browse the repository at this point in the history

  8. added statistics about jobs in DummyScheduler

    @eileen-kuehn
    eileen-kuehn committed Nov 27, 2019
    Configuration menu
    Copy the full SHA
    d0051c2 View commit details
    Browse the repository at this point in the history

  9. made unit tests succeed again

    @eileen-kuehn
    eileen-kuehn committed Nov 27, 2019
    Configuration menu
    Copy the full SHA
    4cae9be View commit details
    Browse the repository at this point in the history

  10. made enabling of monitoring explicit

    @eileen-kuehn
    eileen-kuehn committed Nov 27, 2019
    Configuration menu
    Copy the full SHA
    19ff945 View commit details
    Browse the repository at this point in the history

  11. blackened file

    @eileen-kuehn
    eileen-kuehn committed Nov 27, 2019
    Configuration menu
    Copy the full SHA
    719fc97 View commit details
    Browse the repository at this point in the history

  12. changed cli to also start without any caching information

    @eileen-kuehn
    eileen-kuehn committed Nov 27, 2019
    Configuration menu
    Copy the full SHA
    b87064d View commit details
    Browse the repository at this point in the history

  13. changed assignment of connections a bit

    @eileen-kuehn
    eileen-kuehn committed Nov 27, 2019
    Configuration menu
    Copy the full SHA
    c1ae198 View commit details
    Browse the repository at this point in the history

  14. removed creation of connection module from test as it is not required

    @eileen-kuehn
    eileen-kuehn committed Nov 27, 2019
    Configuration menu
    Copy the full SHA
    e57cd00 View commit details
    Browse the repository at this point in the history

  15. converted storage and file sizes to bytes

    @eileen-kuehn
    eileen-kuehn committed Nov 27, 2019
    Configuration menu
    Copy the full SHA
    6df1b27 View commit details
    Browse the repository at this point in the history

  16. Merge branch 'master' into feature/caching

    @eileen-kuehn
    eileen-kuehn committed Nov 27, 2019
    Configuration menu
    Copy the full SHA
    aced023 View commit details
    Browse the repository at this point in the history

  17. corrected access to numberofaccesses

    @eileen-kuehn
    eileen-kuehn committed Nov 27, 2019
    Configuration menu
    Copy the full SHA
    000be38 View commit details
    Browse the repository at this point in the history

  18. changed signature of StoredFile and adapted in IO operations

    @eileen-kuehn
    eileen-kuehn committed Nov 27, 2019
    Configuration menu
    Copy the full SHA
    08c6432 View commit details
    Browse the repository at this point in the history

  19. improved storage

    @eileen-kuehn
    eileen-kuehn committed Nov 27, 2019
    Configuration menu
    Copy the full SHA
    883b758 View commit details
    Browse the repository at this point in the history

  20. Merge branch 'master' into feature/caching

    @eileen-kuehn
    eileen-kuehn committed Nov 27, 2019
    Configuration menu
    Copy the full SHA
    df362a5 View commit details
    Browse the repository at this point in the history

  21. minimum required usim version set to 0.4.3

    @eileen-kuehn
    eileen-kuehn committed Nov 27, 2019
    Configuration menu
    Copy the full SHA
    d5673b4 View commit details
    Browse the repository at this point in the history

  22. renamed remove_from_storage and add_to_storage to remove and add

    @eileen-kuehn
    eileen-kuehn committed Nov 27, 2019
    Configuration menu
    Copy the full SHA
    65fd2be View commit details
    Browse the repository at this point in the history

  23. made free_space a property of storage

    @eileen-kuehn
    eileen-kuehn committed Nov 27, 2019
    Configuration menu
    Copy the full SHA
    325194d View commit details
    Browse the repository at this point in the history

  24. removed method find_file from storage

    @eileen-kuehn
    eileen-kuehn committed Nov 27, 2019
    Configuration menu
    Copy the full SHA
    22e4ea7 View commit details
    Browse the repository at this point in the history

  25. added todo

    @eileen-kuehn
    eileen-kuehn committed Nov 27, 2019
    Configuration menu
    Copy the full SHA
    7faf2ee View commit details
    Browse the repository at this point in the history

  26. ignored B006 for flake8

    @eileen-kuehn
    eileen-kuehn committed Nov 27, 2019
    Configuration menu
    Copy the full SHA
    e7a21e7 View commit details
    Browse the repository at this point in the history

  27. if file is available on storage, transfer now receives correct size

    @eileen-kuehn
    eileen-kuehn committed Nov 27, 2019
    Configuration menu
    Copy the full SHA
    a06dc62 View commit details
    Browse the repository at this point in the history

  28. fixed position of noqa

    @eileen-kuehn
    eileen-kuehn committed Nov 27, 2019
    Configuration menu
    Copy the full SHA
    3e3752b View commit details
    Browse the repository at this point in the history

  29. made determine_inputfile_source private to connection

    @eileen-kuehn
    eileen-kuehn committed Nov 27, 2019
    Configuration menu
    Copy the full SHA
    a570d50 View commit details
    Browse the repository at this point in the history

  30. renamed transfer_inputfiles to transfer_files

    @eileen-kuehn
    eileen-kuehn committed Nov 27, 2019
    Configuration menu
    Copy the full SHA
    4ce52df View commit details
    Browse the repository at this point in the history

  31. removed queue from file lookup in storage and improved determine inpu… 

    …tfile source
    @eileen-kuehn
    eileen-kuehn committed Nov 27, 2019
    Configuration menu
    Copy the full SHA
    6eb4615 View commit details
    Browse the repository at this point in the history

  32. improved stream file in connection

    @eileen-kuehn
    eileen-kuehn committed Nov 27, 2019
    Configuration menu
    Copy the full SHA
    7a7492f View commit details
    Browse the repository at this point in the history

  33. introduced HitrateStorage that transfers data based on a cache hitrate

    @eileen-kuehn
    eileen-kuehn committed Nov 27, 2019
    Configuration menu
    Copy the full SHA
    6a8801b View commit details
    Browse the repository at this point in the history

  34. fixed position of noqa

    @eileen-kuehn
    eileen-kuehn committed Nov 27, 2019
    Configuration menu
    Copy the full SHA
    43577ac View commit details
    Browse the repository at this point in the history

  35. removed cachehitrate from connection

    @eileen-kuehn
    eileen-kuehn committed Nov 27, 2019
    Configuration menu
    Copy the full SHA
    19bde1f View commit details
    Browse the repository at this point in the history

  36. connection now sets reference to remote_connection for storage

    @eileen-kuehn
    eileen-kuehn committed Nov 27, 2019
    Configuration menu
    Copy the full SHA
    0c2bd3d View commit details
    Browse the repository at this point in the history

  37. storage objects are now created based on specified cache hit rate

    @eileen-kuehn
    eileen-kuehn committed Nov 27, 2019
    Configuration menu
    Copy the full SHA
    122a351 View commit details
    Browse the repository at this point in the history

Commits on Nov 28, 2019

  1. adapted usage of caching for jobs

    @eileen-kuehn
    eileen-kuehn committed Nov 28, 2019
    Configuration menu
    Copy the full SHA
    23a7d7b View commit details
    Browse the repository at this point in the history

  2. introduced calculation efficiency to job

    @eileen-kuehn
    eileen-kuehn committed Nov 28, 2019
    Configuration menu
    Copy the full SHA
    e2e8f57 View commit details
    Browse the repository at this point in the history

  3. introduced calculation efficiency for jobs to cli

    @eileen-kuehn
    eileen-kuehn committed Nov 28, 2019
    Configuration menu
    Copy the full SHA
    b8e51fb View commit details
    Browse the repository at this point in the history

  4. added more type hints for job

    @eileen-kuehn
    eileen-kuehn committed Nov 28, 2019
    Configuration menu
    Copy the full SHA
    2056b41 View commit details
    Browse the repository at this point in the history

  5. removed initialisation of connection

    @eileen-kuehn
    eileen-kuehn committed Nov 28, 2019
    Configuration menu
    Copy the full SHA
    ddb5727 View commit details
    Browse the repository at this point in the history

  6. moved caching related monitoring to extra file

    @eileen-kuehn
    eileen-kuehn committed Nov 28, 2019
    Configuration menu
    Copy the full SHA
    f730a51 View commit details
    Browse the repository at this point in the history

  7. each simulation run now can be identified

    @eileen-kuehn
    eileen-kuehn committed Nov 28, 2019
    Configuration menu
    Copy the full SHA
    1bb6b13 View commit details
    Browse the repository at this point in the history

  8. added caching-specific monitoring information to documentation

    @eileen-kuehn
    eileen-kuehn committed Nov 28, 2019
    Configuration menu
    Copy the full SHA
    4c771a0 View commit details
    Browse the repository at this point in the history

  9. added type hints for simulator

    @eileen-kuehn
    eileen-kuehn committed Nov 28, 2019
    Configuration menu
    Copy the full SHA
    250ddae View commit details
    Browse the repository at this point in the history

  10. changed sizes for storage to bytes

    @eileen-kuehn
    eileen-kuehn committed Nov 28, 2019
    Configuration menu
    Copy the full SHA
    ddba83b View commit details
    Browse the repository at this point in the history

  11. fixed bug leading to full RAM

    @tfesenbecker
    tfesenbecker committed Nov 28, 2019
    Configuration menu
    Copy the full SHA
    f49c4ef View commit details
    Browse the repository at this point in the history

  12. Merge branch 'feature/caching' of https://github.com/MatterMiners/lapis… 

    … into feature/caching
    @tfesenbecker
    tfesenbecker committed Nov 28, 2019
    Configuration menu
    Copy the full SHA
    7d92a69 View commit details
    Browse the repository at this point in the history

  13. added RemoteStorage

    @eileen-kuehn
    eileen-kuehn committed Nov 28, 2019
    Configuration menu
    Copy the full SHA
    0b3b81e View commit details
    Browse the repository at this point in the history

  14. renamed storagesize to size and ensured correct units

    @eileen-kuehn
    eileen-kuehn committed Nov 28, 2019
    Configuration menu
    Copy the full SHA
    e2650c0 View commit details
    Browse the repository at this point in the history

  15. ensured that size is always int

    @eileen-kuehn
    eileen-kuehn committed Nov 28, 2019
    Configuration menu
    Copy the full SHA
    0adb6d0 View commit details
    Browse the repository at this point in the history

  16. renamed method again

    @eileen-kuehn
    eileen-kuehn committed Nov 28, 2019
    Configuration menu
    Copy the full SHA
    ac73c82 View commit details
    Browse the repository at this point in the history

  17. fixed semmle issue

    @eileen-kuehn
    eileen-kuehn committed Nov 28, 2019
    Configuration menu
    Copy the full SHA
    17b23da View commit details
    Browse the repository at this point in the history

  18. added type hints

    @eileen-kuehn
    eileen-kuehn committed Nov 28, 2019
    Configuration menu
    Copy the full SHA
    09ba7a4 View commit details
    Browse the repository at this point in the history

  19. removed cachealgorithm from storage and moved to connection

    @eileen-kuehn
    eileen-kuehn committed Nov 28, 2019
    Configuration menu
    Copy the full SHA
    eae5555 View commit details
    Browse the repository at this point in the history

Commits on Nov 29, 2019

  1. fixed bug leading to full RAM again

    @tfesenbecker
    tfesenbecker committed Nov 29, 2019
    Configuration menu
    Copy the full SHA
    2e6c1dd View commit details
    Browse the repository at this point in the history

  2. fixed bug leading to full RAM again

    @tfesenbecker
    tfesenbecker committed Nov 29, 2019
    Configuration menu
    Copy the full SHA
    0e3c573 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'feature/caching' of https://github.com/MatterMiners/lapis… 

    … into feature/caching
    @tfesenbecker
    tfesenbecker committed Nov 29, 2019
    Configuration menu
    Copy the full SHA
    27c1ea7 View commit details
    Browse the repository at this point in the history

  4. fix hit rate based caching functionality

    @tfesenbecker
    tfesenbecker committed Nov 29, 2019
    Configuration menu
    Copy the full SHA
    2a61f00 View commit details
    Browse the repository at this point in the history

  5. added first test for storage io

    @eileen-kuehn
    eileen-kuehn committed Nov 29, 2019
    Configuration menu
    Copy the full SHA
    4a95f3a View commit details
    Browse the repository at this point in the history

  6. adapted access to connection for RemoteStorage

    @eileen-kuehn
    eileen-kuehn committed Nov 29, 2019
    Configuration menu
    Copy the full SHA
    0e89302 View commit details
    Browse the repository at this point in the history

  7. added new test for storage input

    @eileen-kuehn
    eileen-kuehn committed Nov 29, 2019
    Configuration menu
    Copy the full SHA
    5fc8b37 View commit details
    Browse the repository at this point in the history

  8. fixed assignment of remote storage

    @eileen-kuehn
    eileen-kuehn committed Nov 29, 2019
    Configuration menu
    Copy the full SHA
    d3ff0da View commit details
    Browse the repository at this point in the history

  9. reverted change of transfer signature and added typehints

    @eileen-kuehn
    eileen-kuehn committed Nov 29, 2019
    Configuration menu
    Copy the full SHA
    cbc1507 View commit details
    Browse the repository at this point in the history

  10. introduced interface for storage

    @eileen-kuehn
    eileen-kuehn committed Nov 29, 2019
    Configuration menu
    Copy the full SHA
    d0124d1 View commit details
    Browse the repository at this point in the history

  11. added docstrings to storage interface

    @eileen-kuehn
    eileen-kuehn committed Nov 29, 2019
    Configuration menu
    Copy the full SHA
    c7af203 View commit details
    Browse the repository at this point in the history

  12. extended tests

    @eileen-kuehn
    eileen-kuehn committed Nov 29, 2019
    Configuration menu
    Copy the full SHA
    0019583 View commit details
    Browse the repository at this point in the history

  13. removed public update method from storage and made update private

    @eileen-kuehn
    eileen-kuehn committed Nov 29, 2019
    Configuration menu
    Copy the full SHA
    01efb93 View commit details
    Browse the repository at this point in the history

  14. added unit conversion for storageelement connection and remote connec… 

    …tion throughput_limit, filesize, usedsize
    @tfesenbecker
    tfesenbecker committed Nov 29, 2019
    Configuration menu
    Copy the full SHA
    a7177f7 View commit details
    Browse the repository at this point in the history

  15. reformated debug output

    @tfesenbecker
    tfesenbecker committed Nov 29, 2019
    Configuration menu
    Copy the full SHA
    d684dc8 View commit details
    Browse the repository at this point in the history

  16. added debug output and fixed wrong function call

    @tfesenbecker
    tfesenbecker committed Nov 29, 2019
    Configuration menu
    Copy the full SHA
    328e6dc View commit details
    Browse the repository at this point in the history

  17. added debug output and fixed wrong function call

    @tfesenbecker
    tfesenbecker committed Nov 29, 2019
    Configuration menu
    Copy the full SHA
    0e49c32 View commit details
    Browse the repository at this point in the history

  18. Merge branch 'feature/caching' of https://github.com/MatterMiners/lapis… 

    … into feature/caching
    @tfesenbecker
    tfesenbecker committed Nov 29, 2019
    Configuration menu
    Copy the full SHA
    98dbba1 View commit details
    Browse the repository at this point in the history

Commits on Nov 30, 2019

  1. Merge branch 'master' of https://github.com/MatterMiners/lapis into f… 

    …eature/caching
    @tfesenbecker
    tfesenbecker committed Nov 30, 2019
    Configuration menu
    Copy the full SHA
    93589ba View commit details
    Browse the repository at this point in the history

Commits on Dec 3, 2019

  1. reduced debug output, added default value for job_repr

    @tfesenbecker
    tfesenbecker committed Dec 3, 2019
    Configuration menu
    Copy the full SHA
    2351f51 View commit details
    Browse the repository at this point in the history

  2. Added unittests to test functionality of Connection class with Hitrat… 

    …eStorage storage elements
    @tfesenbecker
    tfesenbecker committed Dec 3, 2019
    Configuration menu
    Copy the full SHA
    9e3a446 View commit details
    Browse the repository at this point in the history

  3. extended type hints

    @tfesenbecker
    tfesenbecker committed Dec 3, 2019
    Configuration menu
    Copy the full SHA
    5cdcf35 View commit details
    Browse the repository at this point in the history

  4. fixed error in _usedstorage calculation

    @tfesenbecker
    tfesenbecker committed Dec 3, 2019
    Configuration menu
    Copy the full SHA
    2747bce View commit details
    Browse the repository at this point in the history

  5. storage.py now catches exception caused by not specified storage cont… 

    …ent config
    @tfesenbecker
    tfesenbecker committed Dec 3, 2019
    Configuration menu
    Copy the full SHA
    4f6132f View commit details
    Browse the repository at this point in the history

  6. fixed test_storage unit tests

    @tfesenbecker
    tfesenbecker committed Dec 3, 2019
    Configuration menu
    Copy the full SHA
    fa0e98f View commit details
    Browse the repository at this point in the history

  7. added new unit tests for hitrate based caching

    @tfesenbecker
    tfesenbecker committed Dec 3, 2019
    Configuration menu
    Copy the full SHA
    d01ba77 View commit details
    Browse the repository at this point in the history

  8. removed forgotten debug output

    @tfesenbecker
    tfesenbecker committed Dec 3, 2019
    Configuration menu
    Copy the full SHA
    5477344 View commit details
    Browse the repository at this point in the history

  9. storage_content_reader() handles empty files correctly now

    @tfesenbecker
    tfesenbecker committed Dec 3, 2019
    Configuration menu
    Copy the full SHA
    2dc77ff View commit details
    Browse the repository at this point in the history

  10. fixed line length

    @tfesenbecker
    tfesenbecker committed Dec 3, 2019
    Configuration menu
    Copy the full SHA
    c573ac5 View commit details
    Browse the repository at this point in the history

  11. fixed line length

    @tfesenbecker
    tfesenbecker committed Dec 3, 2019
    Configuration menu
    Copy the full SHA
    f6fec85 View commit details
    Browse the repository at this point in the history

  12. Monitor.run now properly closes the sampling aiter

    @maxfischer2781
    maxfischer2781 committed Dec 3, 2019
    Configuration menu
    Copy the full SHA
    2fb8afe View commit details
    Browse the repository at this point in the history

  13. made the linter happy

    @maxfischer2781
    maxfischer2781 committed Dec 3, 2019
    Configuration menu
    Copy the full SHA
    a5d13de View commit details
    Browse the repository at this point in the history

  14. added default value for calculation_efficiency

    @tfesenbecker
    tfesenbecker committed Dec 3, 2019
    2 Configuration menu
    Copy the full SHA
    d277354 View commit details
    Browse the repository at this point in the history

  15. remove calculation_efficiency default value

    @tfesenbecker
    tfesenbecker committed Dec 3, 2019
    Configuration menu
    Copy the full SHA
    a165249 View commit details
    Browse the repository at this point in the history

  16. corrected wrong attribute names

    @tfesenbecker
    tfesenbecker committed Dec 3, 2019
    Configuration menu
    Copy the full SHA
    691d523 View commit details
    Browse the repository at this point in the history

  17. fixed usage of transfer() interface and debug outputs

    @tfesenbecker
    tfesenbecker committed Dec 3, 2019
    Configuration menu
    Copy the full SHA
    b8ae864 View commit details
    Browse the repository at this point in the history

  18. fixed debug output leading to failing unit test

    @tfesenbecker
    tfesenbecker committed Dec 3, 2019
    Configuration menu
    Copy the full SHA
    73d6352 View commit details
    Browse the repository at this point in the history

Commits on Dec 4, 2019

  1. fixed bug in filesize unit conversion

    @tfesenbecker
    tfesenbecker committed Dec 4, 2019
    Configuration menu
    Copy the full SHA
    3c58797 View commit details
    Browse the repository at this point in the history

Commits on Jan 20, 2020

  1. Extended hitrate based caching to support different cache hitrates fo… 

    …r every file and cache
    @tfesenbecker
    tfesenbecker committed Jan 20, 2020
    Configuration menu
    Copy the full SHA
    616a7e1 View commit details
    Browse the repository at this point in the history