Add SstFileReader::MayMatch API#30
Draft
murphy-4o wants to merge 8 commits into
Draft
Conversation
murphy-4o
force-pushed
the
unique-key-rocksdb-maymatch
branch
from
0ba0b80 to
3faa153
Compare
murphy-4o
force-pushed
the
unique-key-rocksdb-maymatch
branch
from
3faa153 to
6374942
Compare
Exposes the SST bloom filter via a thin virtual that bypasses the full MultiGet scaffolding (KeyContext / GetContext / sort / LookupKey). Also adds skip_filters parameter to SstFileReader::MultiGet to avoid redundant bloom check on pre-filtered keys. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
murphy-4o
force-pushed
the
unique-key-rocksdb-maymatch
branch
from
6374942 to
e6240f2
Compare
Expose ReadOptions on SstFileReader::MayMatch so callers can request no-IO behavior through read_tier. The earlier commit message text about SstFileReader::MultiGet was stale; this branch only adds the bloom-only MayMatch path.
murphy-4o
changed the title
Make the format sanity check compare PR changes against ClickHouse/rocksdb master instead of the facebook/rocksdb upstream remote fetched by the workflow.
Clarify MayMatch fallback semantics in comments and skip filter setup entirely for empty batches.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
UNIQUE KEY uses RocksDB SST readers mostly as a negative filter: incoming keys are checked against existing SSTs, and most SST/key pairs do not match.
For that mostly-negative path,
SstFileReader::MultiGetis still too heavy because it is a point-lookup API. Even when the bloom filter avoids data-block reads,MultiGetstill prepares lookup/range state and result/status plumbing needed to return values or not-found statuses.Summary
This PR adds that narrow bloom-only fast path.
MayMatchbatches user keys through the table filter and returns booleans, so definitely-absent keys avoid the rest of the point-lookup machinery before callers fall back to normal lookup work for maybe-present keys.In a negative-probe microbenchmark,
MultiGetwas about181 ns/key, while theMayMatchwas about27 ns/keywith the filter resident.SstFileReader::MayMatchand routes it throughTableReader::MayMatch.ReadOptionson the publicSstFileReader::MayMatchoverload so callers can request no-IO behavior throughread_tier.Other changes:
origin/masterbecause the workflow fetchesfacebook/rocksdbasupstream; without the override,format-diff.shchecks unrelated ClickHouse fork deltas instead of this PR's diff.