mirror of
https://github.com/facebook/rocksdb.git
synced 2026-07-08 07:05:23 +08:00
Compare commits
6 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 10dfc6acb8 | |||
| d46d4b7f82 | |||
| a127d1533b | |||
| 928ebd5b8b | |||
| afcf52a996 | |||
| e688e7b235 |
+36
@@ -1,6 +1,42 @@
|
||||
# Rocksdb Change Log
|
||||
> NOTE: Entries for next release do not go here. Follow instructions in `unreleased_history/README.txt`
|
||||
|
||||
## 11.5.2 (06/25/2026)
|
||||
### Bug Fixes
|
||||
* Reverted PR14831 that made range_lock_manager aware of reverse-order CF
|
||||
|
||||
## 11.5.1 (06/24/2026)
|
||||
### Bug Fixes
|
||||
* Fixed a bug where closing a read-only DB instance could delete live SST files created by a concurrent read-write DB sharing the same directory.
|
||||
|
||||
## 11.5.0 (06/16/2026)
|
||||
### New Features
|
||||
* External table readers that open files through `ExternalTableOptions::fs` now update RocksDB SST/file-read statistics and file IO listener callbacks, making external file IO activity visible in existing read metrics.
|
||||
* Added `DB::PrepareFileIngestion()`, a two-phase form of `IngestExternalFile`/`IngestExternalFiles`. `PrepareFileIngestion()` performs all of the work that does not require the DB mutex (validating the arguments, reading each external file's metadata, and linking/copying the files into the DB) and returns an opaque `FileIngestionHandle`. `DB::CommitFileIngestionHandle()` (or `DB::CommitFileIngestionHandles()` for several handles at once) then makes the prepared files visible; multiple handles are committed atomically in a single MANIFEST write, or none are. `FileIngestionHandle::Abort()` (or simply destroying the handle) cancels a prepared ingestion and rolls it back. This gives flexibility to the application to Prepare a file separately from when its committed, and may shorten an applications critical path on `IngestExternalFile`.
|
||||
* Added two stats histograms reporting the per-call latency (in microseconds) of each successful `IngestExternalFile`/`IngestExternalFiles` call, split by phase: `rocksdb.ingest.external.file.prepare.micros` (argument validation and reading/validating/linking the external files, which does not block live writes) and `rocksdb.ingest.external.file.run.micros` (the ingestion performed under the DB mutex while live writes are blocked). Failed ingestions are not recorded.
|
||||
* Added `DBOptions::use_direct_io_for_compaction_reads` (default false). When enabled, compaction-input SST reads use `O_DIRECT` while user reads remain buffered, avoiding page-cache eviction of hot user-read data by sequential compaction scans. Pair with `use_direct_io_for_flush_and_compaction = true` on write-heavy workloads for direct I/O on both compaction inputs and outputs. Rejected at Open when combined with `allow_mmap_reads = true`. No-op when `use_direct_reads = true` is already set (since all reads are already direct).
|
||||
|
||||
### Public API Changes
|
||||
* `DB::GetPreparedFileInfoForExternalSstIngestion()` can now prepare metadata for a live DB-generated SST file so it can be passed to `IngestExternalFileArg::file_infos`. The input path must exactly match a live table file owned by the source DB, and the returned metadata handle must outlive the ingestion.
|
||||
* `ExternalSstFileInfo` gained a `prepared_file_info` member produced by `SstFileWriter::Finish`, and `IngestExternalFileArg` gained a `file_infos` field: a vector of borrowed `const PreparedFileInfo*` pointers, parallel to `external_files`. The owning `ExternalSstFileInfo::prepared_file_info` handle must outlive the ingestion. When set, `IngestExternalFiles()` reuses the supplied per-file metadata instead of re-opening and scanning each file to recompute it, avoiding that extra I/O.
|
||||
* Corrected the public C++ option spelling from `memtable_veirfy_per_key_checksum_on_seek` to `memtable_verify_per_key_checksum_on_seek`. RocksDB continues to accept the old misspelled OPTIONS-file key for compatibility, while newly serialized OPTIONS files use the corrected key.
|
||||
* Added experimental read-scoped block buffer provider API, configured through `ReadOptions::read_scoped_block_buffer_provider`, for supported block-based table iterator scans and MultiScan reads to use caller-provided read-scoped storage for final data-block contents. When configured, supported provider-backed scan data-block reads bypass the data-block cache. The provider is ignored when mmap reads are enabled. RocksDB may still use ordinary temporary scratch for serialized block bytes, such as when a block may be compressed. Get and MultiGet do not currently provide API guarantees for this provider.
|
||||
* Added `FileSystem::SyncFile()` and `Env::SyncFile()` public APIs for syncing or fsyncing a file by name without requiring callers to reopen it as writable.
|
||||
|
||||
### Behavior Changes
|
||||
* The default `compression` for column families is now `kLZ4Compression` rather than `kSnappyCompression`. This affects only column families that do not explicitly set `compression`, and only newly-written SST files. The change is fully compatible: existing data remains readable with no migration needed, as RocksDB selects the decompressor per block. LZ4 offers slightly better compression ratios and decompression CPU efficiency vs. Snappy and similar compression CPU, tested across various server CPUs. When support is not compiled in, the fallback path for default compression is LZ4 -> Snappy -> NoCompression.
|
||||
* Disable parallel compression (ignore `CompressionOptions::parallel_threads`) for fast built-in compressors: Snappy, LZ4 (accelerated, not LZ4HC), and accelerated levels of ZSTD (level < 0). These do not generally benefit from parallel compression. This behavior can be overridden with a custom Compressor from a custom CompressionManager.
|
||||
* `PosixFileSystem::OptimizeForCompactionTableRead` now delegates to the base `FileSystem::OptimizeForCompactionTableRead` implementation before applying its Linux-only compaction-readahead clamp (added for #12038). The returned `FileOptions::use_direct_reads` therefore reflects `DBOptions::use_direct_reads`, consistent with the base class and with other `FileSystem` implementations. Previously the override ignored the base implementation and keyed both the returned options and the Linux readahead clamp off the incoming `FileOptions` rather than `DBOptions`. At the in-tree call site the incoming `FileOptions::use_direct_reads` already matches `DBOptions::use_direct_reads`, so this is effectively a no-op for existing configurations; it removes a latent inconsistency where a caller passing `FileOptions` whose `use_direct_reads` disagreed with `DBOptions` would have had the global flag silently ignored for compaction-input reads on Linux. (Note: the `#12038` readahead clamp still keys off the global `use_direct_reads`; it is not skipped for the compaction-only `use_direct_io_for_compaction_reads` path, since that flag enables O_DIRECT after this hook runs.)
|
||||
* Brought more unity to `kLZ4Compression` and `kLZ4HCCompression` by giving each access to the other's compression levels. Negative (fast) values previously only available to LZ4 and positive (slow) values previously only available to LZ4HC are now available to both, so configuring a non-default `compression_opts.level` now selects the LZ4 compressor variant. This is a behavior change for previously tolerated but dubious configurations such as positive compression level with `kLZ4Compression`. `kLZ4Compression` and `kLZ4HCCompression` keep their effective default levels, equivalent to -1 and 9 respectively.
|
||||
* Removed a discontinuity in `kZSTD` compression levels at `compression_opts.level == 0` by mapping that setting to `level = -1` rather than to `level = 3` as the ZSTD library does internally. This improves the compression auto-tuning landscape.
|
||||
|
||||
### Bug Fixes
|
||||
* Fixed a bug where the range lock manager would crash with an assertion failure when using a reverse comparator column family.
|
||||
|
||||
### Performance Improvements
|
||||
* Reduced commit latency when ingesting many external files by allowing `IngestExternalFileOptions::file_opening_threads` to open table readers for committed ingested files using multiple threads.
|
||||
* Reduced commit latency for large external file ingestions into the last level by adding `IngestExternalFileOptions::prefetch_lmax_index_and_filter_blocks`, which can skip commit-time index and filter block prefetching for cache-backed table metadata.
|
||||
|
||||
## 11.4.0 (06/02/2026)
|
||||
### Public API Changes
|
||||
* Added `rocksdb_options_set_memtable_batch_lookup_optimization()` and `rocksdb_options_get_memtable_batch_lookup_optimization()` to the C API, exposing the existing `AdvancedColumnFamilyOptions::memtable_batch_lookup_optimization` field. This allows C API users (and downstream language bindings) to enable the skip-list memtable's batch-lookup optimization for `MultiGet`, which caches the search path between consecutive keys and reduces per-key cost from O(log N) to O(log d) where d is the distance between consecutive keys.
|
||||
|
||||
@@ -176,6 +176,7 @@ DBImpl::DBImpl(const DBOptions& options, const std::string& dbname,
|
||||
const bool seq_per_batch, const bool batch_per_txn,
|
||||
bool read_only)
|
||||
: dbname_(dbname),
|
||||
read_only_(read_only),
|
||||
own_info_log_(options.info_log == nullptr),
|
||||
initial_db_options_(SanitizeOptions(dbname, options, read_only,
|
||||
&init_logger_creation_s_)),
|
||||
@@ -672,7 +673,8 @@ void DBImpl::UnregisterBlobDirectWriteColumnFamily() {
|
||||
Status DBImpl::MaybeWriteWalMarkersToManifestOnClose() {
|
||||
mutex_.AssertHeld();
|
||||
if (!mutable_db_options_.optimize_manifest_for_recovery ||
|
||||
!opened_successfully_ || versions_ == nullptr || logs_.empty()) {
|
||||
!opened_successfully_ || read_only_ || versions_ == nullptr ||
|
||||
logs_.empty()) {
|
||||
return Status::OK();
|
||||
}
|
||||
|
||||
@@ -880,7 +882,7 @@ Status DBImpl::CloseHelper() {
|
||||
// manifest file), it is not able to identify live files correctly. As a
|
||||
// result, all "live" files can get deleted by accident. However, corrupted
|
||||
// manifest is recoverable by RepairDB().
|
||||
if (opened_successfully_) {
|
||||
if (opened_successfully_ && !read_only_) {
|
||||
JobContext job_context(next_job_id_.fetch_add(1));
|
||||
FindObsoleteFiles(&job_context, true);
|
||||
|
||||
|
||||
@@ -1397,6 +1397,7 @@ class DBImpl : public DB {
|
||||
|
||||
protected:
|
||||
const std::string dbname_;
|
||||
const bool read_only_;
|
||||
// TODO(peterd): unify with VersionSet::db_id_
|
||||
std::string db_id_;
|
||||
// db_session_id_ is an identifier that gets reset
|
||||
|
||||
@@ -3223,7 +3223,7 @@ void DBImpl::ResumeAllCompactions() {
|
||||
void DBImpl::MaybeScheduleFlushOrCompaction() {
|
||||
mutex_.AssertHeld();
|
||||
TEST_SYNC_POINT("DBImpl::MaybeScheduleFlushOrCompaction:Start");
|
||||
if (!opened_successfully_) {
|
||||
if (!opened_successfully_ || read_only_) {
|
||||
// Compaction may introduce data race to DB open
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -8089,6 +8089,41 @@ INSTANTIATE_TEST_CASE_P(OpenFilesAsync, OpenFilesAsyncTest,
|
||||
::testing::Values(-1, 10),
|
||||
::testing::Bool()));
|
||||
|
||||
TEST_F(DBTest, ReadOnlyCloseDoesNotDeleteWriterFiles) {
|
||||
Options options = CurrentOptions();
|
||||
options.create_if_missing = true;
|
||||
options.disable_auto_compactions = true;
|
||||
DestroyAndReopen(options);
|
||||
|
||||
ASSERT_OK(Put("before", "value"));
|
||||
ASSERT_OK(Flush());
|
||||
|
||||
std::unique_ptr<DB> read_only_db;
|
||||
ASSERT_OK(DB::OpenForReadOnly(options, dbname_, &read_only_db));
|
||||
|
||||
ASSERT_OK(Put("after", "value"));
|
||||
ASSERT_OK(Flush());
|
||||
|
||||
std::vector<LiveFileMetaData> live_files;
|
||||
db_->GetLiveFilesMetaData(&live_files);
|
||||
ASSERT_GE(live_files.size(), 2);
|
||||
|
||||
std::string newest_sst_path;
|
||||
uint64_t newest_file_number = 0;
|
||||
for (const auto& live_file : live_files) {
|
||||
if (live_file.file_number > newest_file_number) {
|
||||
newest_file_number = live_file.file_number;
|
||||
newest_sst_path = live_file.directory + "/" + live_file.relative_filename;
|
||||
}
|
||||
}
|
||||
ASSERT_FALSE(newest_sst_path.empty());
|
||||
ASSERT_OK(env_->FileExists(newest_sst_path));
|
||||
|
||||
read_only_db.reset();
|
||||
|
||||
ASSERT_OK(env_->FileExists(newest_sst_path));
|
||||
}
|
||||
|
||||
// Test mix of races with async file open, reads, compactions
|
||||
TEST_P(OpenFilesAsyncTest, ConcurrentFileAccess) {
|
||||
Options options = CurrentOptions();
|
||||
|
||||
@@ -13,7 +13,7 @@
|
||||
// minor or major version number planned for release.
|
||||
#define ROCKSDB_MAJOR 11
|
||||
#define ROCKSDB_MINOR 5
|
||||
#define ROCKSDB_PATCH 0
|
||||
#define ROCKSDB_PATCH 2
|
||||
|
||||
// Make it easy to do conditional compilation based on version checks, i.e.
|
||||
// #if ROCKSDB_VERSION_GE(4, 5, 6)
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
The default `compression` for column families is now `kLZ4Compression` rather than `kSnappyCompression`. This affects only column families that do not explicitly set `compression`, and only newly-written SST files. The change is fully compatible: existing data remains readable with no migration needed, as RocksDB selects the decompressor per block. LZ4 offers slightly better compression ratios and decompression CPU efficiency vs. Snappy and similar compression CPU, tested across various server CPUs. When support is not compiled in, the fallback path for default compression is LZ4 -> Snappy -> NoCompression.
|
||||
@@ -1 +0,0 @@
|
||||
* Disable parallel compression (ignore `CompressionOptions::parallel_threads`) for fast built-in compressors: Snappy, LZ4 (accelerated, not LZ4HC), and accelerated levels of ZSTD (level < 0). These do not generally benefit from parallel compression. This behavior can be overridden with a custom Compressor from a custom CompressionManager.
|
||||
-1
@@ -1 +0,0 @@
|
||||
`PosixFileSystem::OptimizeForCompactionTableRead` now delegates to the base `FileSystem::OptimizeForCompactionTableRead` implementation before applying its Linux-only compaction-readahead clamp (added for #12038). The returned `FileOptions::use_direct_reads` therefore reflects `DBOptions::use_direct_reads`, consistent with the base class and with other `FileSystem` implementations. Previously the override ignored the base implementation and keyed both the returned options and the Linux readahead clamp off the incoming `FileOptions` rather than `DBOptions`. At the in-tree call site the incoming `FileOptions::use_direct_reads` already matches `DBOptions::use_direct_reads`, so this is effectively a no-op for existing configurations; it removes a latent inconsistency where a caller passing `FileOptions` whose `use_direct_reads` disagreed with `DBOptions` would have had the global flag silently ignored for compaction-input reads on Linux. (Note: the `#12038` readahead clamp still keys off the global `use_direct_reads`; it is not skipped for the compaction-only `use_direct_io_for_compaction_reads` path, since that flag enables O_DIRECT after this hook runs.)
|
||||
@@ -1 +0,0 @@
|
||||
Brought more unity to `kLZ4Compression` and `kLZ4HCCompression` by giving each access to the other's compression levels. Negative (fast) values previously only available to LZ4 and positive (slow) values previously only available to LZ4HC are now available to both, so configuring a non-default `compression_opts.level` now selects the LZ4 compressor variant. This is a behavior change for previously tolerated but dubious configurations such as positive compression level with `kLZ4Compression`. `kLZ4Compression` and `kLZ4HCCompression` keep their effective default levels, equivalent to -1 and 9 respectively.
|
||||
@@ -1 +0,0 @@
|
||||
Removed a discontinuity in `kZSTD` compression levels at `compression_opts.level == 0` by mapping that setting to `level = -1` rather than to `level = 3` as the ZSTD library does internally. This improves the compression auto-tuning landscape.
|
||||
@@ -1 +0,0 @@
|
||||
Fixed a bug where the range lock manager would crash with an assertion failure when using a reverse comparator column family.
|
||||
@@ -1 +0,0 @@
|
||||
External table readers that open files through `ExternalTableOptions::fs` now update RocksDB SST/file-read statistics and file IO listener callbacks, making external file IO activity visible in existing read metrics.
|
||||
@@ -1 +0,0 @@
|
||||
Added `DB::PrepareFileIngestion()`, a two-phase form of `IngestExternalFile`/`IngestExternalFiles`. `PrepareFileIngestion()` performs all of the work that does not require the DB mutex (validating the arguments, reading each external file's metadata, and linking/copying the files into the DB) and returns an opaque `FileIngestionHandle`. `DB::CommitFileIngestionHandle()` (or `DB::CommitFileIngestionHandles()` for several handles at once) then makes the prepared files visible; multiple handles are committed atomically in a single MANIFEST write, or none are. `FileIngestionHandle::Abort()` (or simply destroying the handle) cancels a prepared ingestion and rolls it back. This gives flexibility to the application to Prepare a file separately from when its committed, and may shorten an applications critical path on `IngestExternalFile`.
|
||||
@@ -1 +0,0 @@
|
||||
Added two stats histograms reporting the per-call latency (in microseconds) of each successful `IngestExternalFile`/`IngestExternalFiles` call, split by phase: `rocksdb.ingest.external.file.prepare.micros` (argument validation and reading/validating/linking the external files, which does not block live writes) and `rocksdb.ingest.external.file.run.micros` (the ingestion performed under the DB mutex while live writes are blocked). Failed ingestions are not recorded.
|
||||
@@ -1 +0,0 @@
|
||||
Added `DBOptions::use_direct_io_for_compaction_reads` (default false). When enabled, compaction-input SST reads use `O_DIRECT` while user reads remain buffered, avoiding page-cache eviction of hot user-read data by sequential compaction scans. Pair with `use_direct_io_for_flush_and_compaction = true` on write-heavy workloads for direct I/O on both compaction inputs and outputs. Rejected at Open when combined with `allow_mmap_reads = true`. No-op when `use_direct_reads = true` is already set (since all reads are already direct).
|
||||
-1
@@ -1 +0,0 @@
|
||||
Reduced commit latency when ingesting many external files by allowing `IngestExternalFileOptions::file_opening_threads` to open table readers for committed ingested files using multiple threads.
|
||||
@@ -1 +0,0 @@
|
||||
Reduced commit latency for large external file ingestions into the last level by adding `IngestExternalFileOptions::prefetch_lmax_index_and_filter_blocks`, which can skip commit-time index and filter block prefetching for cache-backed table metadata.
|
||||
-1
@@ -1 +0,0 @@
|
||||
`DB::GetPreparedFileInfoForExternalSstIngestion()` can now prepare metadata for a live DB-generated SST file so it can be passed to `IngestExternalFileArg::file_infos`. The input path must exactly match a live table file owned by the source DB, and the returned metadata handle must outlive the ingestion.
|
||||
@@ -1 +0,0 @@
|
||||
`ExternalSstFileInfo` gained a `prepared_file_info` member produced by `SstFileWriter::Finish`, and `IngestExternalFileArg` gained a `file_infos` field: a vector of borrowed `const PreparedFileInfo*` pointers, parallel to `external_files`. The owning `ExternalSstFileInfo::prepared_file_info` handle must outlive the ingestion. When set, `IngestExternalFiles()` reuses the supplied per-file metadata instead of re-opening and scanning each file to recompute it, avoiding that extra I/O.
|
||||
@@ -1 +0,0 @@
|
||||
Corrected the public C++ option spelling from `memtable_veirfy_per_key_checksum_on_seek` to `memtable_verify_per_key_checksum_on_seek`. RocksDB continues to accept the old misspelled OPTIONS-file key for compatibility, while newly serialized OPTIONS files use the corrected key.
|
||||
@@ -1 +0,0 @@
|
||||
Added experimental read-scoped block buffer provider API, configured through `ReadOptions::read_scoped_block_buffer_provider`, for supported block-based table iterator scans and MultiScan reads to use caller-provided read-scoped storage for final data-block contents. When configured, supported provider-backed scan data-block reads bypass the data-block cache. The provider is ignored when mmap reads are enabled. RocksDB may still use ordinary temporary scratch for serialized block bytes, such as when a block may be compressed. Get and MultiGet do not currently provide API guarantees for this provider.
|
||||
@@ -1 +0,0 @@
|
||||
Added `FileSystem::SyncFile()` and `Env::SyncFile()` public APIs for syncing or fsyncing a file by name without requiring callers to reopen it as writable.
|
||||
@@ -102,9 +102,14 @@ TEST_F(RangeLockingTest, BasicRangeLocking) {
|
||||
delete txn1;
|
||||
}
|
||||
|
||||
// CompareDbtEndpoints must honor ReverseBytewiseComparator direction
|
||||
// in length-disparity case.
|
||||
TEST_F(RangeLockingTest, RangeLockWithReverseComparator) {
|
||||
// A point-equality range lock passes [infimum(K), supremum(K)]: the same user
|
||||
// key with the infimum endpoint as the left bound and the supremum endpoint as
|
||||
// the right bound. The infimum endpoint must sort before the supremum endpoint
|
||||
// even under a reverse comparator (the suffix marker is positional, not key
|
||||
// content), otherwise the locktree's left <= right invariant is violated and
|
||||
// try_acquire_lock aborts. Regression test for a crash on MyRocks equality
|
||||
// lookups on a reverse ('rev:') column family.
|
||||
TEST_F(RangeLockingTest, PointRangeLockWithReverseComparator) {
|
||||
delete db;
|
||||
db = nullptr;
|
||||
ASSERT_OK(DestroyDB(dbname, options));
|
||||
@@ -121,19 +126,12 @@ TEST_F(RangeLockingTest, RangeLockWithReverseComparator) {
|
||||
auto cf = db->DefaultColumnFamily();
|
||||
|
||||
Transaction* txn0 = db->BeginTransaction(write_options, txn_options);
|
||||
Transaction* txn1 = db->BeginTransaction(write_options, txn_options);
|
||||
|
||||
ASSERT_OK(txn0->GetRangeLock(cf, Endpoint("aa"), Endpoint("a")));
|
||||
|
||||
auto s = txn1->GetRangeLock(cf, Endpoint("ab"), Endpoint("a"));
|
||||
ASSERT_TRUE(s.IsTimedOut());
|
||||
|
||||
ASSERT_OK(txn1->GetRangeLock(cf, Endpoint("b"), Endpoint("b")));
|
||||
// [infimum("a"), supremum("a")] : same user key, infimum -> supremum.
|
||||
ASSERT_OK(txn0->GetRangeLock(cf, Endpoint("a", false), Endpoint("a", true)));
|
||||
|
||||
txn0->Rollback();
|
||||
txn1->Rollback();
|
||||
delete txn0;
|
||||
delete txn1;
|
||||
}
|
||||
|
||||
// CompareDbtEndpoints must use CompareWithoutTimestamp for range lock
|
||||
|
||||
@@ -39,18 +39,6 @@ RangeLockManagerHandle* NewRangeLockManager(
|
||||
static const char SUFFIX_INFIMUM = 0x0;
|
||||
static const char SUFFIX_SUPREMUM = 0x1;
|
||||
|
||||
struct RangeTreeCmpContext {
|
||||
const Comparator* cmp;
|
||||
bool is_reverse;
|
||||
};
|
||||
|
||||
namespace {
|
||||
[[nodiscard]] bool IsReverseComparator(const Comparator* cmp) {
|
||||
return cmp == ReverseBytewiseComparator() ||
|
||||
cmp == ReverseBytewiseComparatorWithU64Ts();
|
||||
}
|
||||
} // namespace
|
||||
|
||||
// Convert Endpoint into an internal format used for storing it in locktree
|
||||
// (DBT structure is used for passing endpoints to locktree and getting back)
|
||||
void serialize_endpoint(const Endpoint& endp, std::string* buf) {
|
||||
@@ -213,6 +201,20 @@ void RangeTreeLockManager::UnLock(PessimisticTransaction* txn,
|
||||
((RangeTreeLockTracker*)range_tracker)->ReleaseLocks(this, txn, all_keys);
|
||||
}
|
||||
|
||||
// Comparator for range-lock endpoints. This is intentionally
|
||||
// direction-AGNOSTIC: it orders the user-key bytes with the column family's own
|
||||
// comparator, then breaks ties on the endpoint-type suffix byte (infimum before
|
||||
// supremum), which is a positional marker independent of the CF's sort
|
||||
// direction.
|
||||
//
|
||||
// CALLER REQUIREMENT: it is the caller's responsibility to pass in a flipped
|
||||
// (swapped start/end, with negated infimum/supremum flags) range for RangeLock
|
||||
// to work on a REVERSE-ORDERED column family. The locktree requires every range
|
||||
// to satisfy left <= right, and this comparator does NOT compensate for reverse
|
||||
// ordering on the caller's behalf. MyRocks does this flip today in
|
||||
// ha_rocksdb::set_range_lock() (see its "RangeFlagsShouldBeFlippedForRevCF"
|
||||
// comment). Do not re-add reverse-awareness here (that double-corrects callers
|
||||
// like MyRocks that already flip).
|
||||
int RangeTreeLockManager::CompareDbtEndpoints(void* arg, const DBT* a_key,
|
||||
const DBT* b_key) {
|
||||
const char* a = (const char*)a_key->data;
|
||||
@@ -225,24 +227,26 @@ int RangeTreeLockManager::CompareDbtEndpoints(void* arg, const DBT* a_key,
|
||||
|
||||
// Compare the values. The first byte encodes the endpoint type, its value
|
||||
// is either SUFFIX_INFIMUM or SUFFIX_SUPREMUM.
|
||||
const auto* ctx = static_cast<const RangeTreeCmpContext*>(arg);
|
||||
const auto* cmp = ctx->cmp;
|
||||
const bool is_reverse = ctx->is_reverse;
|
||||
Comparator* cmp = (Comparator*)arg;
|
||||
int res = cmp->CompareWithoutTimestamp(
|
||||
Slice(a + 1, min_len - 1), /*a_has_ts=*/false, Slice(b + 1, min_len - 1),
|
||||
/*b_has_ts=*/false);
|
||||
if (!res) {
|
||||
// The user keys compared equal above, so order by the endpoint-type suffix
|
||||
// byte (a positional boundary marker: infimum before the key, supremum
|
||||
// after) -- direction-independent; see the function-level comment.
|
||||
if (b_len > min_len) {
|
||||
int r = a[0] == SUFFIX_INFIMUM ? -1 : 1;
|
||||
return is_reverse ? -r : r;
|
||||
// a's user key is a prefix of b's; a is shorter.
|
||||
return a[0] == SUFFIX_INFIMUM ? -1 : 1;
|
||||
} else if (a_len > min_len) {
|
||||
int r = b[0] == SUFFIX_INFIMUM ? 1 : -1;
|
||||
return is_reverse ? -r : r;
|
||||
// b's user key is a prefix of a's; b is shorter.
|
||||
return b[0] == SUFFIX_INFIMUM ? 1 : -1;
|
||||
} else {
|
||||
// Same user key: infimum sorts before supremum.
|
||||
if (a[0] < b[0]) {
|
||||
return is_reverse ? 1 : -1;
|
||||
return -1;
|
||||
} else if (a[0] > b[0]) {
|
||||
return is_reverse ? -1 : 1;
|
||||
return 1;
|
||||
} else {
|
||||
return 0;
|
||||
}
|
||||
@@ -365,11 +369,10 @@ RangeLockManagerHandle::Counters RangeTreeLockManager::GetStatus() {
|
||||
}
|
||||
|
||||
std::shared_ptr<toku::locktree> RangeTreeLockManager::MakeLockTreePtr(
|
||||
toku::locktree* lt, std::unique_ptr<RangeTreeCmpContext> ctx) {
|
||||
toku::locktree* lt) {
|
||||
toku::locktree_manager* ltm = <m_;
|
||||
return std::shared_ptr<toku::locktree>(
|
||||
lt,
|
||||
[ltm, ctx = std::move(ctx)](toku::locktree* p) { ltm->release_lt(p); });
|
||||
lt, [ltm](toku::locktree* p) { ltm->release_lt(p); });
|
||||
}
|
||||
|
||||
void RangeTreeLockManager::AddColumnFamily(const ColumnFamilyHandle* cfh) {
|
||||
@@ -378,18 +381,15 @@ void RangeTreeLockManager::AddColumnFamily(const ColumnFamilyHandle* cfh) {
|
||||
InstrumentedMutexLock l(<ree_map_mutex_);
|
||||
if (ltree_map_.find(column_family_id) == ltree_map_.end()) {
|
||||
DICTIONARY_ID dict_id = {.dictid = column_family_id};
|
||||
auto ctx = std::make_unique<RangeTreeCmpContext>(RangeTreeCmpContext{
|
||||
cfh->GetComparator(), IsReverseComparator(cfh->GetComparator())});
|
||||
toku::comparator cmp;
|
||||
cmp.create(CompareDbtEndpoints, ctx.get());
|
||||
cmp.create(CompareDbtEndpoints, (void*)cfh->GetComparator());
|
||||
toku::locktree* ltree =
|
||||
ltm_.get_lt(dict_id, cmp,
|
||||
/* on_create_extra*/ static_cast<void*>(this));
|
||||
// This is ok to because get_lt has copied the comparator:
|
||||
cmp.destroy();
|
||||
|
||||
ltree_map_.insert(
|
||||
{column_family_id, MakeLockTreePtr(ltree, std::move(ctx))});
|
||||
ltree_map_.insert({column_family_id, MakeLockTreePtr(ltree)});
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -20,8 +20,6 @@ namespace ROCKSDB_NAMESPACE {
|
||||
|
||||
typedef DeadlockInfoBufferTempl<RangeDeadlockPath> RangeDeadlockInfoBuffer;
|
||||
|
||||
struct RangeTreeCmpContext;
|
||||
|
||||
// A Range Lock Manager that uses PerconaFT's locktree library
|
||||
class RangeTreeLockManager : public RangeLockManagerBase,
|
||||
public RangeLockManagerHandle {
|
||||
@@ -118,8 +116,7 @@ class RangeTreeLockManager : public RangeLockManagerBase,
|
||||
|
||||
RangeDeadlockInfoBuffer dlock_buffer_;
|
||||
|
||||
std::shared_ptr<toku::locktree> MakeLockTreePtr(
|
||||
toku::locktree* lt, std::unique_ptr<RangeTreeCmpContext> ctx = nullptr);
|
||||
std::shared_ptr<toku::locktree> MakeLockTreePtr(toku::locktree* lt);
|
||||
static int CompareDbtEndpoints(void* arg, const DBT* a_key, const DBT* b_key);
|
||||
|
||||
// Callbacks
|
||||
|
||||
Reference in New Issue
Block a user