Move autoremove builds logic to a scheduled job #2468
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
it would be good to verify that this doesn't cause any performance degradations on a "production sized" database |
williamjallen
requested changes
Sep 30, 2024
zackgalbreath
force-pushed
the
remove_builds_revamp
branch
2 times, most recently
from
be7f3ef
to
173e373
Compare
zackgalbreath
force-pushed
the
remove_builds_revamp
branch
3 times, most recently
from
29a9574
to
d14b1f1
Compare
williamjallen
previously approved these changes
Oct 11, 2024
There was a problem hiding this comment.
Looks good to me. @zackgalbreath feel free to merge once you've taken a look at the changes I pushed.
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
|
Moving this to 3.7 as per our in-person discussion today |
zackgalbreath
force-pushed
the
remove_builds_revamp
branch
3 times, most recently
from
1e6422f
to
c0ed119
Compare
Separate the automatic removal of old builds from CDash's submission parsing logic. Instead, old builds will now be periodically cleaned up as a scheduled job. We use a new, lower priority queue to schedule these pruning jobs so they don't interfere with asychronous submission parsing. This commit moves the remove_builds function (and related helper functions) from common.php to DatabaseCleanupUtils. It also removes the "Cleanup database" from upgrade.php and moves this functionality to a new Artisan command instead: db:cleanup. We use this new db:cleanup command to remove shared records, such as testoutput. This allows us to remove a big chunk of custom logic from our removeBuilds funcion. While writing this commit, the following tables were already handled by db:cleanup: - buildfailuredetails - configure - configureerror - coveragefile - test2image The following tables represent potentially shared data that wasn't already handled by db:cleanup: - note - buildupdate - testoutput - updatefile - image - uploadfile The following tables were found to already have cascade-on-delete foreign keys, and thus their explicit DELETE logic was deemed safe to remove: - build2uploadfile - dynamicanalysisdefect - label2dynamicanalysis - label2buildfailure
zackgalbreath
force-pushed
the
remove_builds_revamp
branch
from
c0ed119
to
fc9df9e
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Decouple the cleanup of old builds from the submission parsing step