Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix the issue where compaction incorrectly drops a key when there is a snapshot with a sequence number of zero. #13155

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

tcwzxx
Copy link

@tcwzxx tcwzxx commented Nov 25, 2024

The compaction will incorrectly drop a key under the following conditions:

  1. Open an empty database.
  2. Use the IngestExternalFile API to ingest an SST file (the global sequence number will be 0).
  3. Create a snapshot (the snapshot sequence number will be 0).
  4. Trigger compaction; the key in the above SST file will be dropped.

The drop condition is found here:

} else if (last_snapshot == current_user_key_snapshot_ ||
(last_snapshot > 0 &&
last_snapshot < current_user_key_snapshot_)) {
// If the earliest snapshot is which this key is visible in

The condition does not explicitly check if a previous key exists.

Fix: Add a check of last_sequence != kMaxSequenceNumber to verify if there is a previous key

Copy link
Member

@cbi42 cbi42 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix. Can you add a release note for the bug fix under https://github.com/facebook/rocksdb/tree/main/unreleased_history?

@facebook-github-bot
Copy link
Contributor

@cbi42 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

@tcwzxx has updated the pull request. You must reimport the pull request before landing.

@tcwzxx
Copy link
Author

tcwzxx commented Nov 26, 2024

Thanks for the fix. Can you add a release note for the bug fix under https://github.com/facebook/rocksdb/tree/main/unreleased_history?

Thanks for the reivew. The release note has been added.

@facebook-github-bot
Copy link
Contributor

@cbi42 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@tcwzxx
Copy link
Author

tcwzxx commented Nov 28, 2024

@cbi42 Hi, could you help me merge this PR?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants