Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Staged Update #79

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Staged Update #79

wants to merge 1 commit into from

Conversation

angelini
Copy link
Contributor

@angelini angelini commented Mar 19, 2024

Adds the ability to stage an update and then commit it at a later point in time.

You call the existing Update endpoint with a new flag, this should be backwards compatible because the flag will default to false, and then instead of writing to objects the rows will be written to dl.staged_objects.

You can then follow up with a call to a new RPC endpoint Commit which will actually make that version the current version for your project.

The main, fairly conservative, decision that I took was to only allow you to commit a version if no updates have happened since you staged it.

This means if you:

  • Update to v2
  • Staged Updated to v3
  • Update to v3
  • Attempt to Commit the staged v3 => this will fail

I think this makes it much easier to reason about, but it could be problematic in practice where multiple systems are trying to update a project at once.

Remaining TODOs:

  • Add tests
  • Ensure that the GC can't delete objects that are currently staged
  • Add JS interfaces for staging and committing

@angelini angelini force-pushed the staged_update branch 8 times, most recently from 485ccfb to 1bd935c Compare March 20, 2024 13:04
@angelini angelini marked this pull request as ready for review March 20, 2024 13:04
@angelini angelini requested review from scott-rc, airhorns and MikePresman and removed request for airhorns March 20, 2024 13:04
@angelini
Copy link
Contributor Author

/cc @airhorns

if isStaged {
batch.Queue(`
INSERT INTO dl.staged_objects (project, start_version, stop_version, path, hash, mode, size, packed)
VALUES ($1, NULL, $2, $3, NULL, NULL, NULL, NULL)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is setting start_version as NULL a pattern used to distinguish the fact that its staged? I see this here and in DeleteObject

Copy link
Collaborator

@scott-rc scott-rc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

The main, fairly conservative, decision that I took was to only allow you to commit a version if no updates have happened since you staged it.

I like using this strategy at first -- let's see if we can get away with it.

@angelini
Copy link
Contributor Author

@scott-rc @MikePresman we've decided to pause this feature as Kyle is working on an alternative that would not require this.

You can avoid spending time reviewing this for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants