mirror of
https://github.com/facebook/rocksdb.git
synced 2026-07-07 14:47:40 +08:00
main
993 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Peter Dillinger
|
6cceef9497 |
Bug fixes: surface embedded-blob resolution errors (#14906)
Summary:
Experimental embedded ("same-file") blob SST support (https://github.com/facebook/rocksdb/issues/14851) overloads blob file number 0 as both kInvalidBlobFileNumber and the same-file sentinel kCurrentFileBlobIndexFileNumber. Same-file references must be resolved by EmbeddedBlobResolvingIterator before they reach any generic file-metadata or integrated-BlobDB path, all of which reject file number 0 as invalid.
When resolving a same-file blob hit an error (e.g. a db_stress-injected blob-region read fault), EmbeddedBlobResolvingIterator::value() fell back to returning the RAW, unresolved same-file value, surfacing the error only via status()/Valid(). Compaction consumed the raw value before consulting status, leaking an unresolved same-file reference into the output stream. Depending on the value type this produced two different crash-test failures, both from this one root cause:
* T277566778 -- wide-column entity variant. The raw same-file entity (kTypeWideColumnEntity) reached FileMetaData::UpdateBoundaries, whose blob-ref scan (correctly) rejected file number 0: Flush failed: Corruption: Invalid blob file number (also observed from a background compaction). The tripwire fired on the leak, so the real injected read error was masked behind a misleading corruption status.
* T277310719 -- whole-value BlobIndex variant, and more dangerous because it escapes that tripwire. ResolveKeyType() rewrites the key type kTypeBlobIndex -> kTypeValue with no I/O (so it always succeeds); on the masked error the emitted entry is therefore {kTypeValue, raw BlobIndex bytes}. UpdateBoundaries only scans kTypeBlobIndex/entity types, so it does NOT reject this record: the corruption is silently written to the compaction output and persists. A later point lookup reads those raw BlobIndex bytes back as the value, which db_stress detects as db_stress: expected_value.cc:102: Assertion `ExpectedValue::IsValueBaseValid(value_base)' failed from TestGet. In the captured crash the aborting db_stress process had injected no faults itself -- it read corruption persisted by an earlier faults-on run (blackbox reuses the DB directory), confirming the persistence route.
Fix: EmbeddedBlobResolvingIterator now resolves eagerly for callers that do not opt into unprepared values (allow_unprepared_value=false, e.g. compaction), via a new EagerEmbeddedBlobResolvingIterator (the caller is selected in BlockBasedTable::NewIterator). Eager callers resolve the value during positioning, so a resolution error (blob-region I/O or corruption) is observable through status()/Valid() BEFORE value() is consumed, and value() never exposes an unresolved same-file BlobIndex. Lazy callers (allow_unprepared_value=true, user iteration) keep value laziness but must honor PrepareValue()'s result. This keeps the "callers never see an unresolved same-file blob" invariant structural, even on the error path. Using a template for EmbeddedBlobResolvingIterator minimizes unnecessary overheads.
Also hardens and documents the integrity tripwires that caught the leak, so they are not "fixed" by weakening them -- doing so would mask real corruption and could persist unresolvable same-file references into ordinary (non-embedded) SSTs, turning a transient error into permanent data loss: FileMetaData::UpdateBoundaries (write/output path), Version::GetBlob and Version::MultiGetBlob (integrated-BlobDB read path), plus the contract note in blob_constants.h.
Adds a test-only sync point
"BlockBasedTable::MaybeResolveEmbeddedValue:InjectError" (compiled out under NDEBUG) to simulate a blob-region resolution fault deterministically.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14906
Test Plan:
New unit tests in db_blob_index_test, both of which fail without the fix and pass with it:
* DBBlobIndexTest.EmbeddedBlobResolveErrorDuringCompactionNotMasked -- entity variant. Without the fix CompactRange fails with the masked "Invalid blob file number"; with the fix it fails with the injected IOError.
* DBBlobIndexTest.EmbeddedBlobResolveErrorWholeValueDuringCompactionNotMasked -- whole-value variant. Without the fix CompactRange SUCCEEDS (silently masking the error and persisting {kTypeValue, raw BlobIndex}); with the fix it fails with the injected IOError.
Confirmed the tests exercise the real root cause by temporarily neutralizing the eager resolution (MaybeEagerlyMaterialize): both tests then fail in their respective pre-fix modes (entity -> "Invalid blob file number"; whole-value -> compaction succeeds), then pass again once restored.
End-to-end: the db_stress reproducer for T277566778 (blackbox crash test with embedded-blob ingestion and fault injection) reproduced the "Invalid blob file number" crash before the fix and no longer reproduces with it.
Reviewed By: anand1976
Differential Revision: D110361228
Pulled By: pdillinger
fbshipit-source-id: 1faaa9a9ead726f17b611ff150418fa59ed75fd4
|
||
|
Josh Kang
|
7785264e5a |
Make file opens multi-threaded for commit external sst file ingestion (#14853)
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/14853 `IngestExternalFileOptions` now exposes `file_opening_threads` so external file ingestion can open table readers for newly committed SSTs in parallel. The requested thread count is carried from each ingestion job into commit and then into manifest `LogAndApply`, where `VersionBuilder::LoadTableHandlers()` uses it while installing the ingested files. This is intended for workloads that ingest many external SSTs in one call, where serial table-reader opens can dominate commit latency. The ingestexternalfile benchmark now has a flag for this option, and `db_stress` now creates multiple data SSTs per ingest operation so stress runs cover multi-file ingestion, file-info ingestion, and the parallel file-opening path together. Benchmarks Ran `db_bench --benchmarks=ingestexternalfile --use_existing_db=0 --num=2200000 --compression_type=none --statistics --use_direct_reads=true --ingest_external_file_batch_size=200 --ingest_external_file_num_batches=1 --ingest_external_file_use_file_info=true --ingest_external_file_fill_cache=false` while varying `--ingest_external_file_file_opening_threads`. The generated SST files were about 256 MiB each. | `file_opening_threads` | `rocksdb.ingest.external.file.prepare.micros` | `rocksdb.ingest.external.file.run.micros` | File opens | | 1 | 16,150 us | 327,136 us | 200 | | 32 | 16,665 us | 38,349 us | 200 | At the same ~256 MiB SST size, 2,000 files is roughly 500 GiB of SST data. A straight 10x extrapolation of the 200-file ingest histograms puts prepare+run at about 3.43s with `file_opening_threads=1` and 0.55s with `file_opening_threads=32`; run alone is about 3.27s vs. 0.38s. Reviewed By: pdillinger Differential Revision: D108347952 fbshipit-source-id: 49efa48524df64cace010b881030f63f771cbb95 |
||
|
Steph Pontikes
|
e18d41e08c |
lazily intialize iterators (#14772)
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/14772 Updated the iterator creation scheme to happen lazily (on request) as oppsed to eagerly. this allows us to prune the iterator tree structure at the time of requesting iterator preparation as opposed to creation, and allows pruning to become an implementation detail. Version now skips non-overlapping SST levels and files before adding children to the iterator tree, returns direct table iterators when a level has a single matching file, and uses pruned LevelIterator instances when multiple files in one non-L0 level match. The overload no longer prepares iterators during creation; callers that need prepared multiscan execution still call Prepare explicitly after construction, and MultiScan does that itself. Benchmark: ran `db_bench` in opt mode for the base revision and this diff, with `fillseq,compact,levelstats,multiscanrandom`, `--num=1000000`, `--reads=10000000`, single thread, fixed seeds, `--multiscan_use_async_io=false`, and `--use_multiscan=true`. Both A and B had exactly one SST file and no memtable/L0 data (`L0: 0 files`, `L1: 1 file, 61 MB`). `multiscanrandom` creates `MultiScanArgs` and calls `NewMultiScan(...)`, which reaches the new `NewIterator(..., scan_opts)` pruning path in this diff. ``` seed base A pruning B delta 424242 21824.333 17693.333 -18.9% 424243 24042.014 19424.056 -19.2% 424244 22424.974 17636.910 -21.4% 424245 22404.213 18612.840 -16.9% ``` Average: base `22673.9 us/op`, pruning `18341.8 us/op`, about `19.1%` faster. Reviewed By: xingbowang Differential Revision: D104904298 fbshipit-source-id: a742106a1d5813fb795a39eeeb35f8cddc02e886 |
||
|
zaidoon
|
7affaee1c4 |
Add use_direct_io_for_compaction_reads option (#14743)
Summary: Adds a new `DBOption use_direct_io_for_compaction_reads` (default false). When on, compaction-input SST files are opened with `O_DIRECT` so the sequential read-once data from compaction doesn't pollute the OS page cache and evict the hot user-read working set. User reads keep going through the buffered fast path. This protects user-read tail latency on write-heavy workloads without forcing user reads onto the existing global `use_direct_reads` knob (which pays in throughput and P50 — see the bench below). The interesting bit is that just flipping the FileOptions returned by `FileSystem::OptimizeForCompactionTableRead` doesn't actually trigger `O_DIRECT` at the kernel level. The TableCache (and `FileMetaData::pinned_reader`) is already holding buffered handles opened at flush time or at `DB::Open` via `LoadTableHandlers`. When compaction asks for an iterator, it gets back the cached buffered handle and the kernel never sees the `O_DIRECT` flag. So this PR also adds a small bypass path: - `TableCache::FindTable` / `NewIterator` learn a `open_ephemeral_table_reader` mode. When set, the pinned-reader fast path and the shared cache are skipped, `GetTableReader` is called directly with the caller's FileOptions, and ownership of the freshly opened TableReader is handed back via a `unique_ptr`. The iterator takes ownership via `RegisterCleanup` and frees the reader on destruction. - `VersionSet::MakeInputIterator` and `LevelIterator` plumb the flag through both L0 and L1+ compaction-input paths. - `CompactionJob::ProcessKeyValueCompaction` turns the bypass on when `use_direct_io_for_compaction_reads` is set, the global `use_direct_reads` is off, and `OptimizeForCompactionTableRead` produced `use_direct_reads=true` in the compaction-read FileOptions. The option is opt-in: when off, nothing changes for existing users. When on, only the compaction-input opens take the bypass path; user reads keep hitting the TableCache and the buffered fast path normally. There's also a small db_bench helper in the same PR: a new `--bgwriter_num` flag that lets the writer thread in `readwhilewriting` (and the other "while writing" variants) spread its puts across `[0, bgwriter_num)` instead of `[0, num)`. Without this the readers and writer share a key range and you can't have both a hot read subset and meaningful compaction work — this lets you have both. ### Benchmark Setup: Ubuntu 24.04 (kernel 7.0.5, OrbStack Linux VM on Apple Silicon), 14 vCPUs, virtio-blk disk, btrfs. MGLRU disabled (`echo 0 > /sys/kernel/mm/lru_gen/enabled`) so the kernel uses the classic active/inactive LRU. 14 GB DB (3.5M keys × 4 KB values), no compression. Each measurement run is pinned to a 1 GB cgroup via `systemd-run --scope -p MemoryMax=1G -p MemorySwapMax=0`. Page cache is dropped between configs. db_bench is Release build. Workload: `readwhilewriting` for 120s. 4 reader threads doing random reads over a hot key subset, plus 1 writer thread spreading overwrites across the full 3.5M-key keyspace (via `--bgwriter_num=3500000`) throttled at 200 MB/s, so there's continuous compaction running while the readers go. The size of the hot reader subset relative to available page cache controls how visible the optimization is. The Cassandra blog ([Lightfoot 2026](https://lightfoot.dev/direct-i-o-for-cassandra-compaction-cutting-p99-read-latency-by-5x/)) documented the same thing: biggest wins when the hot set is big enough to actually compete for cache, smaller wins when the hot set trivially fits, neutral when the hot set is way bigger than cache. So I ran two hot-set sizes. #### Small hot set: ~30 MB (~3% of the 1 GB cgroup) — N=5 iterations, mean (CV) `--num=7500`. The hot set is small enough that the page cache holds it without much trouble even under compaction, so the wins here are real but on the modest side. | Config | Throughput (ops/s) | Read P50 (µs) | Read P99 (µs) | Read P99.9 (µs) | Read P99.99 (µs) | |---|---|---|---|---|---| | buffered (default) | 233,477 (8.2%) | 16.09 | 82.24 | 721.0 | 2,102.5 | | direct_compaction_writes_only (existing knob alone) | 287,405 (2.8%) — **+23.1%** | 13.00 (−19.2%) | **66.77 (−18.8%)** | 553.9 (−23.2%) | 1,787.6 (−15.0%) | | direct_compaction_read_only (new knob alone) | 250,669 (2.4%) — +7.4% | 14.16 (−12.0%) | 102.99 (+25.2%) | 689.8 (−4.3%) | 1,801.3 (−14.3%) | | direct_compaction_read_write (new + existing, recommended) | 277,920 (3.3%) — **+19.0%** | **12.99 (−19.3%)** | 84.23 (+2.4%) | 613.4 (−14.9%) | **1,738.2 (−17.3%)** | | use_direct_reads=true (existing global) + write-side | 249,014 (2.5%) — +6.7% | 15.95 (−0.9%) | 68.78 (−16.4%) | **450.8 (−37.5%)** | 1,814.5 (−13.7%) | CV is 2.4–3.3% on the optimized configs (8.2% on buffered), so the deltas are real. With a hot set this small, the existing `use_direct_io_for_flush_and_compaction` knob is already doing most of the work — the new flag's main extra contribution here is P99.99 (combined wins it by ~2 points vs writes-only-alone). Worth noting: the new flag *alone* (without the existing write-side flag) improves P99.99 but regresses P99 by 25% on this small-hot-set workload, because direct compaction reads lose kernel readahead and compaction-output writes are still hitting the page cache. That regression goes away once you combine with the existing write-side flag, or once the hot set is bigger (see next table). So if you're using just one knob, use the existing one. If you're using this PR's flag, pair it with `use_direct_io_for_flush_and_compaction=true`. #### Larger hot set: ~400 MB (~40% of cache) — N=5 iterations, mean (CV) `--num=100000`. This is the case the Cassandra blog calls out — hot set big enough to actually fight compaction for cache. Their analogous setup (1M hot partitions, ~33% hot/cache) reported 1.93× p99 improvement. Numbers here are the headline: | Config | Throughput (ops/s) | Read P50 (µs) | Read P99 (µs) | Read P99.9 (µs) | Read P99.99 (µs) | |---|---|---|---|---|---| | buffered (default) | 68,959 (7.7%) | 44.81 | 541.22 | 2,225.2 | 11,334.5 | | direct_compaction_writes_only (existing knob alone) | 73,973 (10.3%) — +7.3% | 42.22 (−5.8%) | 456.27 (−15.7%) | 2,016.9 (−9.4%) | 9,190.0 (−18.9%) | | direct_compaction_read_only (new knob alone) | 84,337 (2.3%) — +22.3% | 38.66 (−13.7%) | 386.97 (−28.5%) | 1,644.8 (−26.1%) | 4,837.9 (−57.3%, 2.34×) | | direct_compaction_read_write (new + existing, recommended) | **104,923 (8.4%) — +52.2%** | **34.26 (−23.5%)** | **290.97 (−46.2%)** | **1,143.4 (−48.6%)** | **3,080.3 (−72.8%, 3.68×)** | | use_direct_reads=true (existing global) + write-side | 71,598 (9.1%) — +3.8% | 51.33 (+14.5%) | 297.91 (−45.0%) | 1,663.6 (−25.2%) | 6,530.0 (−42.4%) | Combined config gets a 3.68× p99.99 win, 1.86× p99, p50 down 23%, throughput up 52%. Same shape as the Cassandra blog's 1.93× p99 result — the improvement just lands at deeper percentiles for us because RocksDB's baseline data path is roughly 40× faster than Cassandra's (their buffered p99 was 35 ms, ours is 0.54 ms), so the cache-miss tail is further out. A few things worth calling out from this table: - The new flag is doing real work on top of the existing write-side flag here, not just shifting things around. Combined throughput is +42% over `direct_compaction_writes_only` alone, and combined p99.99 is 3× better. The existing knob alone gives a fairly modest +7% throughput / -19% p99.99 in this case — there's a clear gap that the new flag fills. - The new flag *alone* (no existing write-side flag) is also a real improvement here: +22% throughput, p99.99 down 57%. The P99 regression we saw in the small-hot-set case is gone, because the cache-protection effect now dominates the lost-readahead cost. - `use_direct_reads=true` (the existing global flag) actually regresses P50 by 14.5% in this workload — taking user reads off the page cache hurts you when the hot data could have been cached. It also gets the worst throughput of any direct config. It's not an equivalent way to get these gains. ### `compaction_readahead_size` matters when this flag is on Direct I/O bypasses kernel readahead, so RocksDB's own `DBOptions::compaction_readahead_size` becomes the only prefetch the iterator has. The default of 2 MB is enough and real users will get it automatically. **But `db_bench`'s `--compaction_readahead_size` CLI default is 0**, which defeats prefetch and makes direct compaction look slower than it actually is. If you're reproducing the numbers above, pass `--compaction_readahead_size=2097152` (or larger). - Recommended production config is `use_direct_io_for_compaction_reads=true` + `use_direct_io_for_flush_and_compaction=true`. Strongest configuration at every percentile and throughput in both benches. - The new flag is the read-side counterpart to `use_direct_io_for_flush_and_compaction`, which handles compaction-write cache pollution. They address different sources of pollution and compose. The gap between "combined" and "writes-only-alone" is 17 percentage points on p99.99 in the small-hot-set bench and 54 points in the larger one, so the new flag is contributing real value, especially as the hot set grows. - The new flag alone is also a real improvement when the hot set is big enough to compete with cache (+22% throughput, 2.34× p99.99 in the larger-hot-set bench). On a very small hot set it improves p99.99 but regresses p99, so pairing with the existing write-side flag is safer. - The benefit is workload-dependent. Small hot sets get modest tail-latency wins. Hot sets sized to actually compete for cache get the big multi-percentile wins shown above. Hot sets bigger than cache (not benched here but covered in the Cassandra blog) see no change either way — every read misses regardless. ### Reproducing Any Linux host (or a Linux VM on macOS via OrbStack / Multipass / lima): ```bash sudo apt-get install -y build-essential clang cmake git pkg-config \ libgflags-dev libsnappy-dev zlib1g-dev libbz2-dev liblz4-dev libzstd-dev cmake -DCMAKE_BUILD_TYPE=Release -DPORTABLE=1 -DWITH_GFLAGS=1 -DWITH_TESTS=0 .. make -j db_bench echo 0 | sudo tee /sys/kernel/mm/lru_gen/enabled ``` Build the source DB once, unrestricted memory: ```bash ./db_bench --benchmarks=fillrandom,compact,waitforcompaction,stats \ --db=/path/to/source_db --num=3500000 --key_size=16 --value_size=4096 \ --write_buffer_size=16777216 --target_file_size_base=16777216 \ --max_background_jobs=4 --compression_type=none --cache_size=4194304 \ --max_bytes_for_level_base=67108864 --disable_wal=1 --sync=0 ``` For each config, copy `source_db -> scratch_db`, run `sync && echo 3 > /proc/sys/vm/drop_caches`, then: ```bash sudo systemd-run --scope -p MemoryMax=1G -p MemorySwapMax=0 \ ./db_bench --use_existing_db=1 \ --benchmarks=readwhilewriting,stats --db=/path/to/scratch_db \ --threads=5 --duration=120 --statistics=true --histogram=1 \ --num=7500 --bgwriter_num=3500000 \ --key_size=16 --value_size=4096 \ --write_buffer_size=16777216 --target_file_size_base=16777216 \ --max_background_jobs=4 --compression_type=none \ --cache_size=4194304 --open_files=200 \ --skip_stats_update_on_db_open=true \ --max_bytes_for_level_base=67108864 \ --benchmark_write_rate_limit=209715200 \ --compaction_readahead_size=2097152 \ --rate_limiter_bytes_per_sec=0 \ --use_direct_reads={true|false} \ --use_direct_io_for_compaction_reads={true|false} \ --use_direct_io_for_flush_and_compaction={true|false} ``` For the larger hot-set table, change `--num=7500` to `--num=100000`. The five configs in the tables: - `buffered`: all three flags false. - `direct_compaction_writes_only`: `use_direct_io_for_flush_and_compaction=true`, the other two false. This is what users have today without this PR. - `direct_compaction_read_only`: `use_direct_io_for_compaction_reads=true`, the other two false. - `direct_compaction_read_write`: `use_direct_io_for_compaction_reads=true`, `use_direct_io_for_flush_and_compaction=true`, `use_direct_reads=false`. **Recommended.** - `direct_all`: `use_direct_reads=true`, `use_direct_io_for_flush_and_compaction=true`, `use_direct_io_for_compaction_reads=false`. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14743 Reviewed By: pdillinger Differential Revision: D108017601 Pulled By: xingbowang fbshipit-source-id: 4039d490d7e77b476db7a477a2f3d24738db6336 |
||
|
Peter Dillinger
|
62f05627be |
Reduce manifest rotation for foreground metadata ops (#14797)
Summary: Async WAL precreation in https://github.com/facebook/rocksdb/pull/14738 / D105020559 was motivated by slow file creation time on remote storage. MANIFEST does not need the same precreation treatment as WAL because most MANIFEST writes come from background flush and compaction work, but user-facing metadata operations can still pay MANIFEST rotation file creation latency inline. File ingestion performance is a particular concern to some Meta users. Relax the effective MANIFEST rotation limit by 25% for MANIFEST write batches containing any foreground VersionEdit, while keeping background-only flush/compaction batches on the configured or auto-tuned limit. This covers column family manipulation, external file ingestion and import, and DeleteFilesInRange(s). SetOptions remains expected to avoid MANIFEST writes; the test keeps a regression guard for that behavior. The relaxation is intentionally bounded. It reduces the chance that foreground metadata operations create a new MANIFEST inline, while still allowing foreground operations to rotate once the current MANIFEST is beyond the relaxed threshold. Heavier blocking operations like manual Flush or CompactRange already trigger additional file creation and do not get this treatment here, though that could be reconsidered later. This should reduce a potential latency hazard of manifest file size auto-tuning: more frequent MANIFEST rotations. With this change, rotation latency is shifted toward background-only MANIFEST batches when possible. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14797 Test Plan: Expanded DBEtc3Test.AutoTuneManifestSize to cover the foreground threshold behavior and the original auto-tuning behavior in separate phases: - verifies foreground-only CreateColumnFamily writes get only bounded 25% headroom by asserting the first four large-CF additions do not rotate and the fifth does; - verifies auto-tuned background thresholds still prevent excessive rotation; - verifies foreground operations stay below the relaxed threshold for CreateColumnFamily, IngestExternalFile, CreateColumnFamilyWithImport, and DeleteFilesInRanges; - verifies SetOptions still does not write to MANIFEST; - verifies a following background flush still rotates at the normal threshold; - preserves the persisted compacted manifest size close/reopen coverage. Reviewed By: xingbowang Differential Revision: D106578771 Pulled By: pdillinger fbshipit-source-id: f8e274032cd9e7f50e95b685c949242f95351498 |
||
|
Xingbo Wang
|
d7afd3bb3b |
Sync recovery SST directory before reused MANIFEST append (#14780)
Summary: - When `reuse_manifest_on_open` reuses the current MANIFEST, `DB::Open` recovery can flush WAL data into a new L0 SST and append the corresponding `VersionEdit` to that already-current MANIFEST. - If open later fails and the process crashes, the MANIFEST edit can be durable while the recovered SST directory entry is not, leaving the DB pointing at a missing SST. - Fsync the recovered SST's data directory before adding the file to the recovery edit when appending to a reused MANIFEST. - Add a regression test that injects failure after MANIFEST sync, simulates crash cleanup of files created after the last directory sync, and verifies the recovered key remains readable. ## Task - T272584339 Pull Request resolved: https://github.com/facebook/rocksdb/pull/14780 Test Plan: CI Reviewed By: anand1976 Differential Revision: D106201774 Pulled By: xingbowang fbshipit-source-id: a44a7d1263d5bc1d82b995c90eef1a825eab4182 |
||
|
Peter Dillinger
|
87c554b492 |
Persist compacted manifest size for auto-tuning across DB::Open (#14725)
Summary: last_compacted_manifest_file_size_ drives TuneMaxManifestFileSize() to compute the manifest rotation threshold, but it started at 0 on every DB::Open and was only populated after the first manifest rotation. This is really only a problem with reuse_manifest_on_open, because no fresh manifest is created on open. Add a new forward-compatible (safe-to-ignore) MANIFEST tag kLastCompactedManifestFileSize that records the approximate compacted manifest size at the end of WriteCurrentStateToManifest. During recovery, the value is loaded and used to immediately tune the rotation threshold. The record includes a rough estimate of its own overhead (~15 bytes) and must be the last record written by WriteCurrentStateToManifest for accurate estimation. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14725 Test Plan: Extended AutoTuneManifestSize in db_etc3_test to close and reopen with reuse_manifest_on_open after establishing a known auto-tuning state. Verifies that the manifest file number is preserved (no spurious rotation) and that subsequent CF additions don't trigger rotation -- proving the persisted compacted size keeps the tuned threshold correct. Verified the test fails when the recovery loading is disabled. Relax a fragile Java test that was dependent on the exact size of the manifest file. SHORT_TEST=1 ./tools/check_format_compatible.sh Reviewed By: anand1976 Differential Revision: D104464522 Pulled By: pdillinger fbshipit-source-id: 4f5d22d2e149bd40a523ee11780e5e3344803c19 |
||
|
Xingbo Wang
|
4707775ae9 |
Fix GetContext status propagation and blob-backed wide-column merge operands (#14640)
Summary: - propagate lower-level read and merge failures through `GetContext` via `read_status`, so `Get` and `GetEntity` preserve the original error instead of synthesizing `Corruption` when blob-backed reads or merge resolution fail - teach `GetMergeOperands` to resolve blob-backed default columns from wide-column entities, covering both the direct base-value path and the merge-plus-base path - add regression coverage for blob-read IO errors during `Get`/`GetEntity` merge resolution and for `GetMergeOperands` on blob-backed wide-column entities - fix the `DBFlushTest.MemPurgeCorrectLogNumberAndSSTFileCreation` test race by waiting for flush callbacks and cleaning up sync points ## Testing - `make db_blob_basic_test -j14` - `/usr/bin/perl -e 'alarm shift; exec ARGV' 60 ./db_blob_basic_test --gtest_filter='DBBlobBasicTest/DBBlobBasicIOErrorTest.GetBlob_IOError/*:DBBlobBasicTest/DBBlobBasicIOErrorTest.GetEntityMergeWithBlobBaseIOError/*'` ## Task T265824017, T265415808 Pull Request resolved: https://github.com/facebook/rocksdb/pull/14640 Reviewed By: anand1976 Differential Revision: D101690700 Pulled By: xingbowang fbshipit-source-id: 2b6fc357b37a01efa72a2d54dcff55be8992f42a |
||
|
Peter Dillinger
|
795f3bd61f |
Fix check-sources.sh non-ASCII check and remove non-ASCII from sources (#14729)
Summary: The non-ASCII character check in check-sources.sh used git grep -P (Perl regex), which requires git compiled with PCRE support. On systems without it, the command fails with exit code 128, which is != 1 (no match), so the check always reported a violation -- effectively dead. Even in CI where git has PCRE2 support, the check was silently broken: git grep -P uses PCRE2 in UTF mode by default, which interprets [\x80-\xFF] as a Unicode codepoint range (U+0080 to U+00FF). Characters like em-dash (U+2014), arrows (U+2192), and math symbols (U+2248, etc.) fall outside that range and were not detected. Only Latin-1 Supplement characters (U+0080-U+00FF) would have been caught. Replace with LC_ALL=C git grep using bash $'[\x80-\xff]' literal byte range, which works with basic regex in the C locale, and replace all non-ASCII characters in non-excluded source files: - em-dash to -- - arrow to -> - math symbols to ASCII equivalents (~=, <=, >=) - box-drawing characters to ASCII art Also exclude .github/ from the check, as scripts there can use non-ascii without disrupting RocksDB builds on non-UTF-8 systems. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14729 Test Plan: manual / CI (make check-sources passes clean) Reviewed By: hx235 Differential Revision: D104692574 Pulled By: pdillinger fbshipit-source-id: 1d884c21056dcd83558b825a04b867f1c08e3f45 |
||
|
Josh Kang
|
e07ccc3528 |
block GetCreationTimeOfOldestFile on async file open completion (#14723)
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/14723 ### Context `GetCreationTimeOfOldestFile()` assumed `max_open_files` = -1 meant every live SST had a pinned table reader. That is not true with `open_files_async`: recovery intentionally skips loading table files, `DB::Open()` returns, and `BGWorkAsyncFileOpen()` pins readers later. Any caller — e.g. fb_rocksdb's daily report at `FbRocksDb.cpp:1025` — invoking the API in the window between `DB::Open` returning and the background opener completing trips a debug assert. Reported in https://fb.workplace.com/groups/rocksdb/permalink/31668956482726231/. Removing the assert alone is insufficient. For legacy DBs whose manifest does not carry `file_creation_time`, `FileMetaData::TryGetFileCreationTime()` falls back to the pinned reader; with no reader, it returns `kUnknownFileCreationTime` and the function silently returns 0 (the "info unavailable" sentinel). The caller cannot distinguish "no info" from "raced with async open." ### Changes - Remove the invalid debug assert in `Version::GetCreationTimeOfOldestFile`. - Add a private helper `DBImpl::WaitForAsyncFileOpen()` that blocks on `bg_cv_` while `bg_async_file_open_state_ == kScheduled`. The synchronization machinery (`bg_async_file_open_state_` + `bg_cv_`) already exists — the destructor wait loop in `db_impl.cc:674-687` uses the same pattern. The helper is a no-op when `open_files_async = false`, and bails on `shutting_down_` so `DB::Close()` is not blocked by an in-flight caller. - Call `WaitForAsyncFileOpen()` at the top of `DBImpl::GetCreationTimeOfOldestFile()` (inside the `max_open_files == -1` branch). - Document the blocking behavior in `include/rocksdb/db.h`. - Replace the regression test with one that uses a `DBImpl::WaitForAsyncFileOpen::BeforeWait` sync point: spawn a thread that calls `GetCreationTimeOfOldestFile`, deterministically confirm it blocks inside the wait, release async open, confirm the caller wakes with the real value. ### Potential Followups (not included here) - Apply the same wait to `GetLiveFilesMetaData` and `GetColumnFamilyMetaData` — both zero out `oldest_ancester_time` / `file_creation_time` in the SST metadata they return during the async-open window (`db/version_set.cc:7877-7878`, `2090-2091`, `2172-2173`). - Address compaction-picker effects: TTL/periodic file selection (`db/version_set.cc:4039`, `4092-4094`), bottommost over-marking (`db/version_set.cc:4700-4701`), FIFO TTL/temperature pickers (`db/compaction/compaction_picker_fifo.cc:105-107`, `167-170`, `401-411`), and tiered-compaction output time inheritance (`db/compaction/compaction.cc:981`, `1000`). - Harden `FbRocksDb.cpp:1025` to check `status.ok()` instead of `status.code() != kNotSupported`. Reviewed By: mszeszko-meta Differential Revision: D104285992 fbshipit-source-id: ea46375ea1b3ba77fe6b548071aee1101ac0da77 |
||
|
Anand Ananthabhotla
|
c734b7cc60 |
Add reuse_manifest_on_open DBOption (#14704)
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/14704 Add an immutable DBOption `reuse_manifest_on_open` (default false). When enabled, `DB::Open` can keep using the recovered MANIFEST for the first post-open metadata update instead of rebuilding a fresh MANIFEST, which can reduce warm-open latency for DBs whose MANIFEST is expensive to regenerate. Reuse is still best-effort. If RocksDB cannot safely resume appending to the recovered MANIFEST, it falls back to the existing fresh-MANIFEST path. The option is also disabled under `best_efforts_recovery`. This diff also teaches the reopened MANIFEST writer to adopt the existing file size before appending, documents the small-`max_manifest_file_size` caveat for the reused path, and keeps the full warm-reopen composition working with `optimize_manifest_for_recovery`. Reviewed By: hx235, pdillinger Differential Revision: D103568447 fbshipit-source-id: f4f5c35ea3ef0b80a0d52d94be40c6bd11505999 |
||
|
Anand Ananthabhotla
|
02a2b3501d |
Add optimize_manifest_for_recovery DBOption (#14702)
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/14702 Add a mutable DBOption `optimize_manifest_for_recovery` (default false) as a temporary rollout / kill switch for warm-reopen MANIFEST optimizations. In this diff, enabling the option lets recovery skip MANIFEST updates during `DB::Open` when the recovered state is already reflected on disk, which reduces metadata appends after a clean shutdown and can lower warm-reopen latency on storage where MANIFEST appends are expensive. If the option is disabled, RocksDB follows the existing recovery path unchanged. The optimization is disabled under `best_efforts_recovery`, where recovery intentionally rewrites metadata as part of salvage, and the option is mutable so later diffs in this stack can share the same rollout knob. Reviewed By: pdillinger, hx235 Differential Revision: D103568448 fbshipit-source-id: 9ec930343e434f1bee6130bcdbd7738dddd92b6d |
||
|
Anand Ananthabhotla
|
4e2fe35bbb |
Gate file_open_metadata consumption on fast_sst_open option (#14676)
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/14676 When fast_sst_open is disabled, RocksDB was still passing previously-persisted file_open_metadata from the MANIFEST to NewRandomAccessFile. This could cause failures when the metadata becomes stale (e.g. expired filesystem credentials). This change gates the consumption of file_open_metadata in TableCache::GetTableReader on the fast_sst_open option. When fast_sst_open is false, previously persisted metadata is ignored and not passed to the filesystem via FileOptions::file_metadata. The fast_sst_open flag is threaded from MutableDBOptions through VersionSet -> ColumnFamilySet -> ColumnFamilyData -> TableCache at construction time, ensuring the gate is active before any table readers are opened during recovery. Dynamic changes via SetDBOptions are also propagated to all existing TableCache instances. Reviewed By: mszeszko-meta, xingbowang Differential Revision: D102735581 fbshipit-source-id: 9a2c4dc0644a2f65c36b2468605df57779e127cd |
||
|
Danny Chen
|
2e7cf42cda |
Add MANIFEST_VALIDATION_FAILURE_COUNT statistic (#14657)
Summary: CONTEXT: The manifest validation on close feature (verify_manifest_content_on_close) detects corruption but does not increment any statistics counter, making it harder to monitor in production. WHAT: Add a new ticker MANIFEST_VALIDATION_FAILURE_COUNT that is incremented each time content validation detects manifest corruption during DB::Close(). The counter fires per corruption detection, so it can increment up to 2 times per close (once on initial check, once after rewrite attempt). Updated all existing manifest validation tests to verify the counter value. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14657 Test Plan: - All 7 manifest validation tests pass with new stat assertions - 5x repeat with COERCE_CONTEXT_SWITCH=1 shows no flakiness - Full version_set_test suite (212 tests) passes Reviewed By: anand1976 Differential Revision: D102404260 Pulled By: dannyhchen fbshipit-source-id: 21a0aa1ad8de12a935caf5642e41ccf2a47b46d9 |
||
|
Xingbo Wang
|
af4e32945b |
Blob direct write v1: write-path blob separation with partitioned files (reduced scope) (#14535)
Summary: This PR introduces **blob direct write v1**, a reduced-scope write-path optimization where large values (>= `min_blob_size`) are written directly to blob files during `Put()` and replaced in the memtable with compact `BlobIndex` references. This avoids holding full values in memory until flush time. ### Motivation In the existing BlobDB architecture, values are written to the WAL and memtable in their full form and separated into blob files only at flush time. This means: - Large values are held in memory twice (raw in memtable + blob file at flush) - Blob I/O is serialized through a single flush thread per column family Blob direct write addresses both: values leave the write path as small `BlobIndex` references, and multiple **partitions** (configurable via `blob_direct_write_partitions`) allow concurrent blob writes with independent locks. ### Design (v1 — single-writer, WAL-disabled, reduced scope) The v1 design intentionally keeps scope narrow for correctness and reviewability: - **Single writer thread assumption**: no concurrent writes to the same partition file. One logical writer serializes the batch. - **WAL-disabled**: direct-write blob files are only registered in MANIFEST at flush time. WAL replay cannot recover unregistered blob references, so WAL is disabled for this v1. - **Flush-on-write**: each `AddRecord` call flushes to the OS immediately. - **FIFO generation batching**: each memtable switch creates one generation batch. Direct-write files for that memtable are sealed and registered atomically when the batch is flushed to MANIFEST. - **Round-robin partitions**: blob writes are distributed across `blob_direct_write_partitions` files using an atomic counter. ### New components | Component | Description | |---|---| | `BlobFilePartitionManager` | Owns N partition files per CF. Manages open/seal/register lifecycle tied to memtable generations. | | `BlobWriteBatchTransformer` | A `WriteBatch::Handler` that rewrites qualifying `Put` values as `BlobIndex` entries before the batch enters the write group. | ### Write path integration 1. `DBImpl::WriteImpl` calls `BlobWriteBatchTransformer::TransformBatch` before entering the writer group (for default write path), or before joining the batch group (for pipelined/unordered write). 2. Values >= `min_blob_size` are written to a partition file; the key is stored with a `BlobIndex` in the transformed batch. A rollback guard marks blob bytes as initial garbage if the write fails. 3. On `SwitchMemtable`, `RotateCurrentGeneration` moves active partitions into the next immutable batch. 4. `FlushMemTableToOutputFile` / `AtomicFlushMemTablesToOutputFiles` call `PrepareFlushAdditions` to seal partition files and collect `BlobFileAddition` + `BlobFileGarbage` entries registered to MANIFEST alongside the flush. 5. Shutdown paths (`CancelAllBackgroundWork`, `WaitForCompact` with `close_db=true`) force-flush all CFs with active direct-write managers to ensure blob files are registered before close. ### Read path - **Get/MultiGet**: `MaybeResolveBlobForWritePath` resolves `BlobIndex` references found in memtable or immutable memtable via `BlobFilePartitionManager::ResolveBlobDirectWriteIndex`, which first checks manifest-visible state and falls back to direct blob-file reads via `BlobFileCache`. - **Iterator**: `DBIter::BlobReader` is extended with a `BlobFilePartitionManager*` to resolve direct-write blob indexes during iteration. The unified `ResolveBlobDirectWriteIndex` path handles both manifest-visible and not-yet-flushed files. ### New options | Option | Default | Description | |---|---|---| | `enable_blob_direct_write` | `false` | Enable write-path blob separation for this CF. Requires `enable_blob_files = true`. Not dynamically changeable. | | `blob_direct_write_partitions` | `1` | Number of parallel partition files per CF. Not dynamically changeable. | ### Feature incompatibilities (reduced v1 scope) The following features are *not supported* when `enable_blob_direct_write = true`, and are enforced both in `db_stress_tool` validation and `db_crashtest.py` sanitization: **Write model constraints:** - `threads` must be 1 (single writer assumption) - `allow_concurrent_memtable_write` = 0 - `enable_pipelined_write` = 0 (transformation done before batch group, but pipelined path supported with pre-transform) - `two_write_queues` = 0 - `unordered_write` = 0 (transformation done before batch group, but unordered path supported with pre-transform) **WAL and recovery:** - `disable_wal` = 1 (required — WAL replay of unregistered blob files is out of v1 scope) - `best_efforts_recovery` = 0 - `reopen` = 0 (no crash-restart with WAL replay) - All WAL-related stress features disabled: `manual_wal_flush_one_in`, `sync_wal_one_in`, `lock_wal_one_in`, `get_sorted_wal_files_one_in`, `get_current_wal_file_one_in`, `track_and_verify_wals`, `rate_limit_auto_wal_flush`, `recycle_log_file_num` **Blob GC and dynamic options:** - `use_blob_db` = 0 (stacked BlobDB not supported) - `allow_setting_blob_options_dynamically` = 0 - `enable_blob_garbage_collection` = 0 - `blob_compaction_readahead_size` = 0 - `blob_file_starting_level` = 0 **Unsupported value types and APIs:** - Merge (`use_merge`, `use_full_merge_v1`) — merge values pass through untransformed - Entity APIs (`use_put_entity_one_in`, `use_get_entity`, `use_multi_get_entity`, `use_attribute_group`) - `use_timed_put_one_in` - User-defined timestamps (`user_timestamp_size`, `persist_user_defined_timestamps`, `create_timestamped_snapshot_one_in`) - Transactions (`use_txn`, `use_optimistic_txn`, `test_multi_ops_txns`, `commit_bypass_memtable_one_in`) — though `WriteCommittedTxn::CommitInternal` falls back from bypass-memtable to normal path when BDW is active - `IngestWriteBatchWithIndex` returns `NotSupported` - `inplace_update_support` = 0 **Fault injection:** - All write/read/metadata fault injection disabled (`sync_fault_injection`, `write_fault_one_in`, `metadata_write_fault_one_in`, `read_fault_one_in`, `metadata_read_fault_one_in`, `open_*_fault_one_in`) **Infrastructure/snapshot APIs:** - `remote_compaction_worker_threads` = 0 - `test_secondary` = 0 - `backup_one_in` = 0 - `checkpoint_one_in` = 0 - `get_live_files_apis_one_in` = 0 - `ingest_external_file_one_in` = 0 - `ingest_wbwi_one_in` = 0 ### Tests - `db/blob/db_blob_basic_test.cc`: ~660 lines of new direct-write unit tests covering basic put/get, multi-partition, flush/compaction, recovery, and error injection. - `db/blob/blob_file_cache_test.cc`: ~96 lines of new tests for direct-write blob file cache behavior. - `db/write_batch_test.cc`: ~96 lines of tests for WriteBatch with blob index entries. - `utilities/transactions/transaction_test.cc`: verifies transaction commit path falls back correctly with direct write enabled. - `db_stress_tool/`: full stress test support with `--enable_blob_direct_write` and `--blob_direct_write_partitions` flags, integrated into `db_crashtest.py` with 10% random selection alongside regular blob params. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14535 Test Plan: ``` make -j128 db_blob_basic_test && ./db_blob_basic_test make -j128 blob_file_cache_test && ./blob_file_cache_test make -j128 write_batch_test && ./write_batch_test make -j128 transaction_test && ./transaction_test make -j128 check ``` Stress test: ``` python3 tools/db_crashtest.py blackbox --enable_blob_direct_write=1 \ --enable_blob_files=1 --blob_direct_write_partitions=4 \ --disable_wal=1 --threads=1 ``` Reviewed By: pdillinger Differential Revision: D98766843 Pulled By: xingbowang fbshipit-source-id: 1577653826913a59d05680a87bce5534ac5a5e69 |
||
|
Josh Kang
|
5db0603613 |
Read-triggered compactions (#14426)
Summary: Add read-triggered compaction, a new feature that reduces read amplification by compacting SST files that receive high read traffic. When an SST file's read frequency (`num_reads_sampled / file_size`) exceeds a configurable threshold, it is marked for compaction to a lower level. The feature introduces two new options: a CF option `read_triggered_compaction_threshold` (default 0, disabled) and a DB option `max_periodic_compaction_trigger_seconds` (default 43200s) that controls how often the background thread re-evaluates compaction scores on quiet databases. Both options are dynamically changeable. Lowering `max_periodic_compaction_trigger_seconds` does add some overhead, but generally is minimal, so running this every couple of minutes in a production environment seems fairly reasonable. ## Key changes - **New CF option `read_triggered_compaction_threshold`** (`advanced_options.h`): When positive, files with `reads_per_byte > threshold` are marked for compaction. Files at the last non-empty level are skipped (bottommost compaction handles those separately). Marked files are sorted by hotness (reads_per_byte descending). - **New DB option `max_periodic_compaction_trigger_seconds`** (`options.h`): Replaces the hardcoded 12-hour ceiling in `ComputeTriggerCompactionPeriod()`. Essential for read-triggered compaction on quiet DBs since there are no writes to trigger score re-evaluation. - **Leveled compaction picker** (`compaction_picker_level.cc`): Adds read-triggered as the lowest-priority compaction reason in `SetupInitialFiles()`, using the existing `PickFileToCompact` helper. - **Universal compaction picker** (`compaction_picker_universal.cc`): Adds `PickReadTriggeredCompaction` as lowest priority. Refactors shared "find output level + compute overlapping inputs + create Compaction" logic from both `PickDeleteTriggeredCompaction` and `PickReadTriggeredCompaction` into `BuildCompactionToNextLevel`, handling both single-level and multi-level universal cases. - **Periodic trigger integration** (`db_impl.cc`): `TriggerPeriodicCompaction` now also fires for CFs with `read_triggered_compaction_threshold > 0`, even without time-based compaction configured. - **Stress test & db_bench support**: Both `db_stress` and `db_bench` support the new options. `db_crashtest.py` randomly enables read-triggered compaction and sets a short periodic trigger interval when enabled. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14426 Test Plan: **Unit tests**: - `compaction_picker_test` — 7 new tests: `ReadTriggeredCompactionDisabled`, `ReadTriggeredCompactionBelowThreshold`, `ReadTriggeredCompactionAboveThreshold`, `NeedsCompactionReadTriggered`, `ReadTriggeredPicksFile`, `UniversalReadTriggeredCompaction`, `ReadTriggeredSkipsLastLevel`, `UniversalReadTriggeredNoPickWhenNotMarked` - `db_compaction_test` — `ReadTriggeredCompaction` integration test verifying end-to-end behavior with sync points - Stress test coverage **Stress test**: ``` make V=1 -j "CRASH_TEST_EXT_ARGS=--duration=600 --max_key=2500000 --max_compaction_trigger_wakeup_seconds=10 --read_triggered_compaction_threshold=0.0001 --interval=600" blackbox_crash_test ``` - confirmed read triggered compactions from LOGS **Benchmark** (`db_bench`): Setup: 5M keys (100B values, 16B keys), leveled compaction, 5 levels, 4MB target file size. DB fully compacted, then 2M overlapping keys written without compaction to create L0/L1 overlap (82 files, ~294MB). LSM shape change during readrandom with read-triggered compaction: ``` BEFORE: L0=9 files (15MB), L1=4 (16MB), L2=20 (69MB), L3=49 (194MB) — 82 files, 294MB AFTER: L3=66 files (223MB) ``` | Benchmark | Config | avg ops/s | % change | |-----------|--------|-----------|----------| | readrandom (8 threads, 5M reads) | baseline (threshold=0) | 1,086,965 | — | | readrandom (8 threads, 5M reads) | threshold=0.000001, trigger=5s | 1,453,697 | **+33.7%** | Reviewed By: xingbowang Differential Revision: D97838716 Pulled By: joshkang97 fbshipit-source-id: a21fcb270c7fadd4f78d98b9c821982f220dd3f0 |
||
|
Danny Chen
|
b6f498b2c9 |
Add verify_manifest_content_on_close option (#14451)
Summary: Add a new mutable DB option `verify_manifest_content_on_close` (default: false). When enabled, on DB close the MANIFEST file is read back and all records are validated (CRC checksums via log::Reader and logical content via VersionEdit::DecodeFrom). If corruption is detected, a fresh MANIFEST is written from in-memory state using the existing LogAndApply recovery path. This complements the existing size validation in VersionSet::Close() with content validation, reusing the same manifest reading pattern as VersionSet::Recover(). Implementation plan: ## Part 1: New DB Option — verify_manifest_content_on_close - A new mutable bool DB option (default: false) that can be dynamically toggled via SetDBOptions() at runtime, following the pattern of other mutable manifest options like max_manifest_file_size. - Propagation: SetDBOptions() -> DBImpl::mutable_db_options_ -> versions_->UpdatedMutableDbOptions() -> VersionSet::verify_manifest_content_on_close_ ## Part 2: Core Implementation — Content Validation in VersionSet::Close() - Inserted after existing size check, before closed_ = true - Opens manifest as SequentialFileReader, creates log::Reader with checksum=true - Loops ReadRecord with WALRecoveryMode::kAbsoluteConsistency, decodes each record as VersionEdit - On corruption: fires OnIOError listeners, logs error, calls LogAndApply with empty edit to trigger manifest rewrite from in-memory state - If manifest can't be opened for reading: logs warning, doesn't fail close ## Part 3: Unit Tests (in version_set_test.cc) - ManifestContentValidationOnClose_Clean: enable option, normal close, verify no manifest rotation - ManifestContentValidationOnClose_CorruptRecord: enable option, corrupt manifest via SyncPoint, verify rotation occurs and DB reopens cleanly - ManifestContentValidationOnClose_Disabled: default off, verify content validation does not run - ManifestContentValidationOnClose_SizeCheckFails: truncate manifest so size check fails first, verify recovery via size-check path ## What Happens If a Corruption is Detected If corruption was detected, four things happen: 1. **Notify listeners** — Fires `OnIOError` on all registered event listeners (from db_options_->listeners) so monitoring/alerting systems can observe the corruption event. Uses `FileOperationType::kVerify` to categorize it. 2. **Permit unchecked errors** — `PermitUncheckedError()` silences RocksDB's debug-mode assertion that every `IOStatus` must be inspected. These statuses are informational-only here; the real recovery is via `LogAndApply`. 3. **Log the error** — Writes a `ROCKS_LOG_ERROR` message with the filename for operational visibility (grep-able in production logs). 4. **Rewrite the manifest via `LogAndApply`** — This is the actual recovery. `LogAndApply` is called with an empty `VersionEdit` (no changes). Internally, `LogAndApply` detects that the current `descriptor_log_` is null (it was reset at line 5551, or by the previous `LogAndApply` in the size-check path) and creates a brand-new MANIFEST file. It serializes the entire current in-memory LSM state — all column families, all levels, all file metadata, sequence numbers, etc. — into this new file. It then atomically updates the `CURRENT` file pointer to reference the new MANIFEST. This works because the in-memory state was built from the original manifest during `DB::Open()` and has been kept fully up to date through all subsequent operations (flushes, compactions, etc.) during the DB's lifetime. The on-disk manifest is essentially a journal of changes; `LogAndApply` with an empty edit produces a fresh, compacted snapshot of that state. ## Flow Diagram of Manifest Content Validation VersionSet::Close() │ ├─ Close descriptor_log_ and check size │ └─ Size mismatch? → LogAndApply (rewrite manifest) │ ├─ Content validation (if s.ok() && option enabled) │ ├─ Open manifest for sequential reading │ │ └─ Can't open? → WARN log, continue │ │ │ ├─ For each record: │ │ ├─ ReadRecord (CRC32 check, kAbsoluteConsistency) │ │ └─ DecodeFrom (VersionEdit logical check) │ │ │ └─ Corruption detected? │ ├─ Notify OnIOError listeners │ ├─ LOG_ERROR │ └─ LogAndApply (rewrite manifest from in-memory state) │ └─ closed_ = true; return s; ## How This Relates to the Existing Size Check The existing size check (lines 5556-5582) and the new content validation are complementary: | Check | What it catches | How it checks | |----------------|-----------------------------------------|----------------------------| | Size check | Truncation, partial writes, extra bytes | Compare expected vs actual file size | | Content check | Bit-rot, silent corruption, bad records | CRC32 + VersionEdit decode | The size check catches gross corruption (file too short or too long). The content check catches subtle corruption where the file is the right size but individual bytes have been flipped (e.g., storage media bit-rot, buggy filesystem, incomplete block write). Both recovery paths use the same mechanism: `LogAndApply` with an empty `VersionEdit` to rewrite the manifest from in-memory state. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14451 Reviewed By: xingbowang Differential Revision: D96004906 Pulled By: dannyhchen fbshipit-source-id: 0b0ecdada3a74e97d2cadbba2091b8b577f1d684 |
||
|
Josh Kang
|
42eff8b632 |
Add new heauristic 'num_collapsible_entry_reads_sampled' (#14434)
Summary: Add per-file sampling of "collapsible" entry reads (single deletions, merges, and kNotFound results) that may later be used to help inform read-triggered compactions. This is a better metric than `num_reads_sampled` as it is more targeted towards reads that could be avoided via compaction. The existing behavior of `num_reads_sampled` is that reads only gets sampled on iterator creation for a file. It is problematic because next/prev() calls are not sampled, nor are additional seeks(). This PR moves sampling to per-seek/next granularity within `LevelIterator` and adds a new `num_collapsible_entry_reads_sampled` counter that tracks how often a file serves entries that could be eliminated by compaction. Note only L1+ files have iterator seeks/nexts/prevs sampled. Introducing this at L0 would require wrapping table reader iterators, introducing a performance cost. ## Key changes - **New counter `num_collapsible_entry_reads_sampled`** in `FileSampledStats` tracks sampled reads that encounter deletions, single deletions, merges, or kNotFound results in both Get and Iterator paths. - **Moved sampling from file-open to per-operation** in `LevelIterator`: sampling now happens in `SampleRead()` called from `Seek()`, `SeekForPrev()`, `SeekToFirst()`, `SeekToLast()`, `Next()`, `NextAndGetResult()`, and `Prev()`. The `should_sample` parameter was removed from `LevelIterator`'s constructor. - **Differentiated sampling rate for Next() vs Seek()**: `should_sample_file_read_next()` uses a 64x lower sampling rate (`kFileReadSampleRate * 64`) since Next() is cheaper than Seek() and called more frequently. - **Collapsible tracking in Get path**: `Version::Get()` now increments the collapsible counter when `GetContext::State()` is `kNotFound`, `kMerge`, or `kDeleted`. - **Collapsible tracking in MultiGet path**: `MultiGetFromSST` also increments the collapsible counter for the same states. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14434 Test Plan: - Added new DB tests for both num_reads_sampled and num_collapsible_entry_reads_sampled ### Benchmark results (readrandom, readseq) Setup: 1M keys, 16-byte keys, 100-byte values, no compression, fillrandom+compact | Benchmark | Params | ops/s (main) | ops/s (feature) | % change | |------------|--------------------|-------------|--------------------------|----------| | readrandom | seed=1, threads=1 | 387,194 | 389,449 | +0.6% | | readseq | seed=1, threads=1 | 5,598,371 | 5,572,975 | -0.5% | No meaningful performance regression observed — differences are within run-to-run noise. Reviewed By: xingbowang Differential Revision: D95613793 Pulled By: joshkang97 fbshipit-source-id: 9dd09c9b7527b148424bde5686f4157c7a9e1214 |
||
|
Josh Kang
|
f25fb41da6 |
Add option to validate sst files in the background on DB open (#14322)
Summary: Add `open_files_async` option for faster DB startup. When enabled, SST file opening and validation is deferred to a background thread after `DB::Open` returns, reducing startup latency for databases with many SST files. WAL recovery remains synchronous. To support this, `FindTable` is extended with a pinning mechanism that stores the cache handle directly on `FileMetaData` via a new `PinnedTableReader` class, and sets the table reader atomically so subsequent reads skip cache lookups. `FileDescriptor::table_reader` is replaced with `PinnedTableReader pinned_reader` which wraps a `std::atomic<TableReader*>` with acquire/release ordering to safely handle concurrent access between the background opener and read threads. Should validations fail, the background opener sets a `kAsyncFileOpen` background error. Future read requests will look up the table reader again via the cache, and if any validations fail there it will get propagated to the user (existing behavior when `max_open_files > 0`). This feature is most useful when `max_open_files=-1`, because otherwise file opening is already capped at 16 files and DB open should be fast. ## Restrictions - This feature also is incompatible with fifo compaction because fifo compaction requires reading table properties under DB mutex. When table reader is unpinned, this may cause a DB hang. - This feature is also incompatible with `skip_stats_update_on_db_open=false` because it will result in even longer DB open ## Key changes - New `open_files_async` DB option with C, Java, and `db_bench` bindings - `BGWorkAsyncFileOpen` background worker that opens all SST files post-`DB::Open`, with shutdown awareness via `shutting_down_` flag - New `PinnedTableReader` class in `version_edit.h` — thread-safe wrapper holding `std::atomic<TableReader*>` and `Cache::Handle*` with proper acquire/release ordering. Replaces the old `FileDescriptor::table_reader` raw pointer and `FileMetaData::table_reader_handle` - Extract `LoadTableHandlersHelper` into `db/version_util.cc` — shared between `VersionBuilder::LoadTableHandlers` (for version edits during recovery) and `BGWorkAsyncFileOpen` (for base storage post-open) - `FindTable` extended with `pin_table_handle` and `out_table_reader` params — when pinning is enabled, the table reader is stored on `FileMetaData` so Get/MultiGet/Iterator skip redundant cache lookups. `FindTable` now performs the pinned-reader fast-path check internally instead of requiring callers to check `fd.table_reader` beforehand - Note: pinning is explicit (not default) because some callers create temporary `FileMetaData`s that would need to properly clean up table handles - `CompactedDBImpl` updated to use `FindTable` + pinning instead of raw `fd.table_reader` access for Get/MultiGet - New `kAsyncFileOpen` background error reason in `listener.h` and `error_handler.cc` - Add a check in ~DBImpl to ensure async file open task has not been forgotten to be scheduled in (future) subclasses of DBImpl. Certain subclasses that never use it will need to explicitly mark it. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14322 Test Plan: - `OpenFilesAsyncTest` parameterized over `num_flushes` (1, 20), `ReadType` (Get, MultiGet, Iterator), `max_open_files` (-1, 10), and `read_only` (true, false) - **ConcurrentFileAccess**: concurrent reads and compactions race with async opener - **AfterRead**: reads happen before async opener, verifying lazy open and that the opener sees already-pinned readers - **BeforeRead**: async opener completes first, verifying reads use pre-loaded table readers - **Shutdown**: DB closes before async opener starts, verifying clean cancellation with 0 file opens - **Error**: corrupted SST files, verifying `kAsyncFileOpen` background error is set and reads return corruption - **DropColumnFamily**: CF dropped before async opener runs, verifying the opener gracefully skips dropped CFs - Added to crash test ### Benchmark To simulate a high-latency remote filesystem, I set up a virtual filesystem with dm-delay using 10ms reads, 0 ms writes. ``` # Generate a DB with many L0 files TEST_TMPDIR=/data/users/jkangs/dm-delay-test/mnt ./db_bench -benchmarks=fillseq -disable_auto_compactions=true -write_buffer_size=1000 -num=1000000 ``` ``` ./db_bench -use_existing_db=true -db=/data/users/jkangs/dm-delay-test/mnt/dbbench -benchmarks=readrandom -reads=1 -report_open_timing=true -open_files_async=true -use_direct_reads -file_opening_threads=1 -skip_stats_update_on_db_open OpenDb: 25.1419 milliseconds ``` ``` ./db_bench -use_existing_db=true -db=/data/users/jkangs/dm-delay-test/mnt/dbbench -benchmarks=readrandom -reads=1 -report_open_timing=true -open_files_async=false -use_direct_reads -file_opening_threads=1 -skip_stats_update_on_db_open OpenDb: 23109.4 milliseconds ``` ### No read regressions On main branch ``` ./db_bench -use_existing_db=true -db=/dev/shm/dbbench -benchmarks=readrandom -seed=1 -threads=8 -duration=30 readrandom : 4.827 micros/op 1657100 ops/sec 30.005 seconds 49720992 operations; 183.3 MB/s (6198999 of 6198999 found) ``` On this branch ``` ./db_bench -use_existing_db=true -db=/dev/shm/dbbench -benchmarks=readrandom -seed=1 -threads=8 -duration=30 readrandom : 4.863 micros/op 1644808 ops/sec 30.007 seconds 49354992 operations; 182.0 MB/s (6099999 of 6099999 found) ./db_bench -use_existing_db=true -db=/dev/shm/dbbench -benchmarks=readrandom -seed=1 -threads=8 -duration=30 -open_files_async=true readrandom : 4.803 micros/op 1665392 ops/sec 30.004 seconds 49968992 operations; 184.2 MB/s (6222999 of 6222999 found) ``` Reviewed By: pdillinger, xingbowang Differential Revision: D93538033 Pulled By: joshkang97 fbshipit-source-id: 32ac70c112cd733b7c1e1c1e2e7ce6422318a5ae |
||
|
Andrew Chang
|
09bda51c50 |
Propagate file_checksum through FileOptions on NewRandomAccessFile (#14321)
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/14321 Add file_checksum and file_checksum_func_name fields to FileOptions so that downstream FileSystem implementations can access per-file checksum metadata when SST files are opened. The fields are populated from FileMetaData at all call sites where SST files are opened via NewRandomAccessFile: TableCache::GetTableReader, Version::GetTableProperties, and CompactionJob::ReadTablePropertiesDirectly. Also fixes the fallback path in TableCache::GetTableReader to use the local fopts (with temperature and checksum) instead of the original file_options. Added a kNoFileChecksumFuncName which is distinct from kUnknownFileChecksumFuncName: - kUnknownFileChecksumFuncName ("Unknown"): We have FileMetaData for this file, and the metadata says no checksum was computed (no factory was configured when the file was written). This is a property of the file itself. - kNoFileChecksumFuncName ("Unavailable"): We don't even have FileMetaData — we're opening this file in a context where there's no checksum metadata to propagate at all (e.g., SstFileDumper, SstFileReader, checksum generation). It's a property of the call site, not the file. So the assertion file_checksum.empty() is correct for both, but for different reasons — one says "the file has no checksum," the other says "we have no idea about this file's checksum." Reviewed By: pdillinger Differential Revision: D92728944 fbshipit-source-id: 8fd34ea22ca87090b26d0a55c921f354f97f1ffc |
||
|
Xingbo Wang
|
b040ab83e1 |
Add a new picking algorithm in fifo compaction (#14326)
Summary: Add a new kv ratio based compaction picking algorithm in fifo compaction Pull Request resolved: https://github.com/facebook/rocksdb/pull/14326 Test Plan: Unit test Reviewed By: pdillinger Differential Revision: D93257941 Pulled By: xingbowang fbshipit-source-id: fd2d0e1356c7b54682a1197475a1bd26cb45c9d4 |
||
|
Peter Dillinger
|
7ecc12110c |
Fix format compatibility issues, extend test (#14323)
Summary: See https://github.com/facebook/rocksdb/issues/14240 which brought this to my attention. Here I've added range deletions and compactions to the format compatible test, and fixed or worked-around compatibility issues (likely longstanding). The first fix was in Version::MaybeInitializeFileMetaData for an assertion failure simply from adding range deletions from some 5.x version. The second fix is a broader work-around for older SST files with unreliable num_entries/num_range_deletions/num_deletions statistics in their table properties. We depend on them only for some paranoid checks for compaction, so in my assessment the best way to deal with those files is to exclude the paranoid checks when dealing with the files with unrelaible data. (Details in code comments.) The important part is that compacting old files is exceptionally rare, so we aren't really interefering with the paranoid checks doing thier job on an ongoing basis. This depends on https://github.com/facebook/rocksdb/issues/14315 (just landed) because there is a remaining undiagnosed problem with some very early releases, but I'm not fixing that because its support is being dropped. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14323 Test Plan: test extended (ran locally excluding some releases) Reviewed By: xingbowang Differential Revision: D93032653 Pulled By: pdillinger fbshipit-source-id: f90b32f30ba4764692e68d23705f42c778e0dc1d |
||
|
Xingbo Wang
|
ea5e649225 |
Fix an infinite compaction loop bug with udt (#14228)
Summary: Problem The TEST_WaitForCompact in TimestampCompatibleCompactionTest.UdtTombstoneCollapsingTest would sometimes run forever, indicating an infinite compaction loop. Issue https://github.com/facebook/rocksdb/issues/14223 Root Cause In ComputeBottommostFilesMarkedForCompaction(), files were marked for bottommost compaction based only on the condition largest_seqno < oldest_snapshot_seqnum. However, for User-Defined Timestamps (UDT) columns, compaction can only zero sequence numbers when the file's maximum timestamp is below full_history_ts_low. When timestamps were above this threshold: 1. File gets marked for compaction (seqno condition met) 2. Compaction runs but cannot zero seqno (timestamp condition not met) 3. Output file immediately gets re-marked for compaction 4. Infinite loop Solution Added timestamp range tracking to FileMetaData and updated the marking logic to check timestamps before marking files. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14228 Test Plan: Unit test Reviewed By: pdillinger Differential Revision: D90586045 Pulled By: xingbowang fbshipit-source-id: addfa4f988db8c87fb513a1bf58ee54623a6c210 |
||
|
Hui Xiao
|
d2fe0ee389 |
Fix use-after-free in BlockBasedTable after best-efforts recovery retry (#14155)
Summary: **Context/Summary:** Best-efforts recovery can cause a use-after-free bug after retrying for a failed recovery attempt. The issue occurs in VersionSet::Reset(): - First recovery attempt: Opens SST files, caching BlockBasedTable objects in table_cache_ https://github.com/facebook/rocksdb/blob/ac412b10955d5a1d3d99aff8edf94eae1e4a22d5/db/version_edit_handler.cc#L565 - Recovery fails: Calls Reset() which deletes the old ColumnFamilySet (and all CFDs) https://github.com/facebook/rocksdb/blob/ac412b10955d5a1d3d99aff8edf94eae1e4a22d5/db/version_set.cc#L6631 - Creates new CFDs: But reuses the same table_cache_ https://github.com/facebook/rocksdb/blob/ac412b10955d5a1d3d99aff8edf94eae1e4a22d5/db/version_set.cc#L5579 - Bug: Cached BlockBasedTable objects contain now-dangling reference to previous CFD's member such as rep_->internal_comparator or rep_->ioptions as below. References instead of object copies are used for memory efficiency ``` struct BlockBasedTable::Rep { Rep(const ImmutableOptions& _ioptions, .. const InternalKeyComparator& _internal_comparato...)) {} ~Rep() { status.PermitUncheckedError(); } const ImmutableOptions& ioptions; ... const InternalKeyComparator& internal_comparator; ``` - Crash: Accessing any of the above reference in cached tables during read or compaction after recovery finishes triggers use-after-free This PR calls table_cache_->EraseUnRefEntries() to clear tables containing the dangling reference in VersionSet::Reset() before creating the new ColumnFamilySet. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14155 Test Plan: - Add new unit test that fails before the fix under ASAN run and pass after ``` [ RUN ] DBBasicTest.BestEffortRecoveryFailureWithTableCacheUseAfterFree ================================================================= ==1976446==ERROR: AddressSanitizer: heap-use-after-free on address 0x61e00000a8c8 at pc 0x7f6b21beae57 bp 0x7ffd65bacec0 sp 0x7ffd65baceb8 READ of size 8 at 0x61e00000a8c8 thread T0 #0 0x7f6b21beae56 in rocksdb::UserComparatorWrapper::user_comparator() const util/user_comparator_wrapper.h:29 // rep_->ioptions https://github.com/facebook/rocksdb/issues/1 0x7f6b21beb02b in rocksdb::InternalKeyComparator::user_comparator() const db/dbformat.h:421 https://github.com/facebook/rocksdb/issues/2 0x7f6b229a7a50 in rocksdb::BinarySearchIndexReader::NewIterator(rocksdb::ReadOptions const&, bool, rocksdb::IndexBlockIter*, rocksdb::GetContext*, rocksdb::BlockCacheLookupContext*) table/block_based/binary_search_index_reader.cc:62 https://github.com/facebook/rocksdb/issues/3 0x7f6b22a9a649 in rocksdb::BlockBasedTable::NewIndexIterator(rocksdb::ReadOptions const&, bool, rocksdb::IndexBlockIter*, rocksdb::GetContext*, rocksdb::BlockCacheLookupContext*) const table/block_based/block_based_table_reader.cc:1683 https://github.com/facebook/rocksdb/issues/4 0x7f6b22aa39be in rocksdb::BlockBasedTable::Get(rocksdb::ReadOptions const&, rocksdb::Slice const&, rocksdb::GetContext*, rocksdb::SliceTransform const*, bool) table/block_based/block_based_table_reader.cc:2533 https://github.com/facebook/rocksdb/issues/5 0x7f6b2241201c in rocksdb::TableCache::Get(rocksdb::ReadOptions const&, rocksdb::InternalKeyComparator const&, rocksdb::FileMetaData const&, rocksdb::Slice const&, rocksdb::GetContext*, rocksdb::MutableCFOptions const&, rocksdb::HistogramImpl*, bool, int, unsigned long) db/table_cache.cc:492 0x61e00000a8c8 is located 72 bytes inside of 2784-byte region [0x61e00000a880,0x61e00000b360) freed by thread T0 here: #0 0x7f6b248d20d7 in operator delete(void*, unsigned long) /home/engshare/third-party2/gcc/11.x/src/gcc-11.x/libsanitizer/asan/asan_new_delete.cpp:172 https://github.com/facebook/rocksdb/issues/1 0x7f6b21ca8703 in rocksdb::ColumnFamilyData::UnrefAndTryDelete() db/column_family.cc:785 https://github.com/facebook/rocksdb/issues/2 0x7f6b21cb25ee in rocksdb::ColumnFamilySet::~ColumnFamilySet() db/column_family.cc:1771 https://github.com/facebook/rocksdb/issues/3 0x7f6b225683df in std::default_delete<rocksdb::ColumnFamilySet>::operator()(rocksdb::ColumnFamilySet*) const (/data/users/huixiao/rocksdb/librocksdb.so.10.10+0x1f683df) https://github.com/facebook/rocksdb/issues/4 0x7f6b22568ceb in std::__uniq_ptr_impl<rocksdb::ColumnFamilySet, std::default_delete<rocksdb::ColumnFamilySet> >::reset(rocksdb::ColumnFamilySet*) /mnt/gvfs/third-party2/libgcc/d1129753c8361ac8e9453c0f4291337a4507ebe6/11.x/platform010/5684a5a/include/c++/trunk/bits/unique_ptr.h:182 https://github.com/facebook/rocksdb/issues/5 0x7f6b22550c52 in std::unique_ptr<rocksdb::ColumnFamilySet, std::default_delete<rocksdb::ColumnFamilySet> >::reset(rocksdb::ColumnFamilySet*) /mnt/gvfs/third-party2/libgcc/d1129753c8361ac8e9453c0f4291337a4507ebe6/11.x/platform010/5684a5a/include/c++/trunk/bits/unique_ptr.h:456 https://github.com/facebook/rocksdb/issues/6 0x7f6b224fa09e in rocksdb::VersionSet::Reset() db/version_set.cc:5587 https://github.com/facebook/rocksdb/issues/7 0x7f6b2250752c in rocksdb::VersionSet::TryRecover(std::vector<rocksdb::ColumnFamilyDescriptor, std::allocator<rocksdb::ColumnFamilyDescriptor> > const&, bool, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, bool*) db/version_set.cc:6640 https://github.com/facebook/rocksdb/issues/8 0x7f6b220c5a88 in rocksdb::DBImpl::Recover(std::vector<rocksdb::ColumnFamilyDescriptor, std::allocator<rocksdb::ColumnFamilyDescriptor> > const&, bool, bool, bool, bool, unsigned long*, rocksdb::DBImpl::RecoveryContext*, bool*) db/db_impl/db_impl_open.cc:565 previously allocated by thread T0 here: #0 0x7f6b248d1257 in operator new(unsigned long) /home/engshare/third-party2/gcc/11.x/src/gcc-11.x/libsanitizer/asan/asan_new_delete.cpp:99 https://github.com/facebook/rocksdb/issues/1 0x7f6b21cb30e0 in rocksdb::ColumnFamilySet::CreateColumnFamily(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned int, rocksdb::Version*, rocksdb::ColumnFamilyOptions const&, bool) db/column_family.cc:1827 https://github.com/facebook/rocksdb/issues/2 0x7f6b22516a11 in rocksdb::VersionSet::CreateColumnFamily(rocksdb::ColumnFamilyOptions const&, rocksdb::ReadOptions const&, rocksdb::VersionEdit const*, bool) db/version_set.cc:7715 https://github.com/facebook/rocksdb/issues/3 0x7f6b22494910 in rocksdb::VersionEditHandler::CreateCfAndInit(rocksdb::ColumnFamilyOptions const&, rocksdb::VersionEdit const&) db/version_edit_handler.cc:494 https://github.com/facebook/rocksdb/issues/4 0x7f6b2249005f in rocksdb::VersionEditHandler::Initialize() db/version_edit_handler.cc:209 https://github.com/facebook/rocksdb/issues/5 0x7f6b2248cd13 in rocksdb::VersionEditHandlerBase::Iterate(rocksdb::log::Reader&, rocksdb::Status*) db/version_edit_handler.cc:32 https://github.com/facebook/rocksdb/issues/6 0x7f6b225081db in rocksdb::VersionSet::TryRecoverFromOneManifest(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<rocksdb::ColumnFamilyDescriptor, std::allocator<rocksdb::ColumnFamilyDescriptor> > const&, bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, bool*) db/version_set.cc:6679 https://github.com/facebook/rocksdb/issues/7 0x7f6b225074a1 in rocksdb::VersionSet::TryRecover(std::vector<rocksdb::ColumnFamilyDescriptor, std::allocator<rocksdb::ColumnFamilyDescriptor> > const&, bool, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, bool*) db/version_set.cc:6635 https://github.com/facebook/rocksdb/issues/8 0x7f6b220c5a88 in rocksdb::DBImpl::Recover(std::vector<rocksdb::ColumnFamilyDescriptor, std::allocator<rocksdb::ColumnFamilyDescriptor> > const&, bool, bool, bool, bool, unsigned long*, rocksdb::DBImpl::RecoveryContext*, bool*) db/db_impl/db_impl_open.cc:565 ``` Reviewed By: anand1976 Differential Revision: D87991593 Pulled By: hx235 fbshipit-source-id: 2379b297ff592cadf02659e355cdc8e170917cfc |
||
|
Jay Huh
|
2f583aed8f |
Move prepared_iter size assertion after cleanup (#14144)
Summary: Fixing crash test failure caused by `prepared_iters_.size() == 0` Pull Request resolved: https://github.com/facebook/rocksdb/pull/14144 Test Plan: ``` python3 -u tools/db_crashtest.py --stress_cmd=./db_stress --cleanup_cmd='' --simple blackbox ``` Reviewed By: krhancoc Differential Revision: D87656914 Pulled By: jaykorean fbshipit-source-id: 9ef7cf4ea5d34fe9dee6219b32323e91a2ea3e5f |
||
|
Ryan Hancock
|
b9951ded37 |
Introducing Prepare all iterators for LevelIterator (#14100)
Summary:
This diff introduces the async prepare of all iterators within a MultiScan. The current state has each iterator be prepared as its needed, and with this diff, we prepare all iterators during the prepare phase of the Level Iterator, this will allow more time for each IO to be dispatched and serviced, increasing the odds that a block is ready as the scan seeks to it.
Benchmark is prefilled using
```
KEYSIZE=64
VALUESIZE=512
NUMKEYS=5000000
SCAN_SIZE=100
DISTANCE=25000
NUM_SCANS=15
THREADS=1
./db_bench --db=$DB \
--benchmarks="fillseq" \
--write_buffer_size=5242880 \
--max_write_buffer_number=4 \
--target_file_size_base=5242880 \
--disable_wal=1 --key_size=$KEYSIZE \
--value_size=$VALUESIZE --num=$NUMKEYS --threads=32
}
```
And benchmark ran is
```
run() {
echo 1 | sudo tee /proc/sys/vm/drop_caches
./db_bench --db=$DB --use_existing_db=1 \
--benchmarks=multiscan \
--disable_auto_compactions=1 --seek_nexts=$SCAN_SIZE \
--multiscan-use-async-io=1 \
--multiscan-size=$NUM_SCANS --multiscan-stride=$DISTANCE \
--key_size=$KEYSIZE --value_size=$VALUESIZE \
--num=$NUMKEYS --threads=$THREADS --duration=60 --statistics
}
```
The benchmark uses large stride sides to ensure that two scans would touch separate files. We reduce the size of the block cache to increase likelyhood of reads (and simulate larger data sets)
**Branch:**
```
Integrated BlobDB: blob cache disabled
RocksDB: version 10.8.0
Date: Tue Nov 11 13:26:29 2025
CPU: 166 * AMD EPYC-Milan Processor
CPUCache: 512 KB
Keys: 64 bytes each (+ 0 bytes user-defined timestamp)
Values: 512 bytes each (256 bytes after compression)
Entries: 5000000
Prefix: 0 bytes
Keys per prefix: 0
RawSize: 2746.6 MB (estimated)
FileSize: 1525.9 MB (estimated)
Write rate: 0 bytes/second
Read rate: 0 ops/second
Compression: Snappy
Compression sampling rate: 0
Memtablerep: SkipListFactory
Perf Level: 1
------------------------------------------------
multiscan_stride = 25000
multiscan_size = 15
seek_nexts = 100
DB path: [/data/rocksdb/mydb]
multiscan : 837.941 micros/op 1193 ops/sec 60.001 seconds 71605 operations; (multscans:71605)
```
**Baseline:**
```
Set seed to 1762898809121995 because --seed was 0
Initializing RocksDB Options from the specified file
Initializing RocksDB Options from command-line flags
Integrated BlobDB: blob cache disabled
RocksDB: version 10.9.0
Date: Tue Nov 11 14:06:49 2025
CPU: 166 * AMD EPYC-Milan Processor
CPUCache: 512 KB
Keys: 64 bytes each (+ 0 bytes user-defined timestamp)
Values: 512 bytes each (256 bytes after compression)
Entries: 5000000
Prefix: 0 bytes
Keys per prefix: 0
RawSize: 2746.6 MB (estimated)
FileSize: 1525.9 MB (estimated)
Write rate: 0 bytes/second
Read rate: 0 ops/second
Compression: Snappy
Compression sampling rate: 0
Memtablerep: SkipListFactory
Perf Level: 1
------------------------------------------------
multiscan_stride = 25000
multiscan_size = 15
seek_nexts = 100
DB path: [/data/rocksdb/mydb]
multiscan : 1129.916 micros/op 885 ops/sec 60.001 seconds 53102 operations; (multscans:53102)
```
Repeated for confirmation.
This introduces a ~20% improvement in latency and op/s.
Note: Benchmarks are single threaded as, when increasing thread count, we start seeing large amounts of overhead being induced by block cache contention, finally resulting in both baseline and branch becoming equal.
Further on network attached storage with high latency, the level iterator, preparing all iterators so a 20% improvement even at high thread counts.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14100
Reviewed By: anand1976
Differential Revision: D86913584
Pulled By: krhancoc
fbshipit-source-id: da9d0c890e25e392a33389ce6b80f9bfb84d3f85
|
||
|
Viraj Thakur
|
2cf81e0a20 |
fix compiler warning for mutex->AssertHeld (#14115)
Summary: We are seeing Github actions failures due to a compiler error: https://github.com/facebook/rocksdb/actions/runs/19190877461/job/54865138898?fbclid=IwY2xjawN_Hc9leHRuA2FlbQIxMQBicmlkETFZeGlpZXZXMGlDTVhTYldwc3J0YwZhcHBfaWQBMAABHp6JoIoMBbZq-8Kgfc1honBdkAbHAZzW2ORiCM2Br2D9utxtMlq6IIqUUQnu_aem_SOU-DDsjDDMB3mTncKfLwQ&brid=VRqQ-asf2myW425wX1qqhg When UpdatedMutableDbOptions is called from the VersionSet constructor, manifest_file_size_ is 0, and mu is nullptr. This is expected and fine, and we never enter the block where AssertHeld is called. All other times UpdatedMutableDbOptions is called, the mutex must be held. This PR just checks that mu is not null, to satisfy the compiler. We could alternatively intentionally crash if there is concern over a silent failure if mu is passed as nullptr Pull Request resolved: https://github.com/facebook/rocksdb/pull/14115 Reviewed By: pdillinger Differential Revision: D86733318 Pulled By: virajthakur fbshipit-source-id: ce9ed6275c9495a3ea2a12f984dbceef7b441e24 |
||
|
Ranjan Banerjee
|
9fbb68be17 |
Api to get SST file with key ranges for a particular level and key range (startKey, EndKey)rocksdb [Internal version] (#14009)
Summary: There are instances where an application might be interested in knowing the distribution in SST files for a key range in a particular level. This implementation creates an overloaded GetColumnFamilyMetaData api where (startKey, EndKey) can be passed along with level information to filter the necessary sst files along with the keyranges for each sst file Pull Request resolved: https://github.com/facebook/rocksdb/pull/14009 Reviewed By: anand1976 Differential Revision: D83389707 fbshipit-source-id: 6df1dc1f9233efe9000b03cc1831b3c618cbcef3 |
||
|
Peter Dillinger
|
37176a4a44 |
Auto-tune manifest file size (#14076)
Summary:
Adds auto-tuning of manifest file size to avoid the need to scale `max_manifest_file_size` in proportion to things like number of SST files to properly balance (a) manifest file write amp and new file creation, vs. (b) manifest file space amp and replay time, including non-incremental space usage in backups. (Manifest file write amp comes from re-writing a "live" record when the manifest file is re-created, or "compacted"; space amp is usage beyond what would be used by a compacted manifest file.) In more detail,
* Add new option `max_manifest_space_amp_pct` with default value of 500, which defaults to 0.2 write amp and up to roughly 5.0 space amp, except `max_manifest_file_size` is treated as the "minimum" size before re-creating ("compacting") the manifest file.
* `max_manifest_file_size` in a way means the same thing, with the same default of 1GB, but in a way has taken on a new role. What is the same is that we do not re-create the manifest file before reaching this size (except for DB re-open), and so users are very unlikely to see a change in default behavior (auto-tuning only kicking in if auto-tuning would exceed 1GB for effective max size for the current manifest file). The new role is as a file size lower bound before auto-tuning kicks in, to minimize churn in files considered "negligibly small." We recommend a new setting of around 1MB or even smaller like 64KB, and expect something like this to become the default soon.
* These two options along with `manifest_preallocation_size` are now mutable with SetDBOptions. The effect is nearly immediate, affecting the next write to the current manifest file.
Also in this PR:
* Refactoring of VersionSet to allow it to get (more) settings from MutableDBOptions. This touches a number of files in not very interesting ways, but notably we have to be careful about thread-safe access to MutableDBOptions fields, and even fields within VersionSet. I have decided to save copies of relevant fields from MutableDBOptions to simplify testing, etc. by not saving a reference to MutableDBOptions but getting notified of updates.
* Updated some logging in VersionSet to provide some basic data about final and compacted manifest sizes (effects of auto-tuning), making sure to avoid I/O while holding DB mutex.
* Added db_etc3_test.cc which is intended as a successor to db_test and db_test2, but having "test.cc" in its name for easier exclusion of test files when using `git grep`. Intended follow-up: rename db_test2 to db_etc2_test
* Moved+updated `ManifestRollOver` test to the new file to be closer to other manifest file rollover testing.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14076
Test Plan:
As for correctness, new unit test AutoTuneManifestSize is pretty thorough. Some other unit tests updated appropriately. Manual tests in the performance section were also audited for expected behavior based on the new logging in the DB LOG. Example LOG data with -max_manifest_file_size=2048 -max_manifest_space_amp_pct=500:
```
2025/10/24-11:12:48.979472 2150678 [/version_set.cc:5927] Created manifest 5, compacted+appended from 52 to 116
2025/10/24-11:12:49.626441 2150682 [/version_set.cc:5927] Created manifest 24, compacted+appended from 2169 to 1801
2025/10/24-11:12:52.194592 2150682 [/version_set.cc:5927] Created manifest 91, compacted+appended from 10913 to 8707
2025/10/24-11:13:02.969944 2150682 [/version_set.cc:5927] Created manifest 362, compacted+appended from 52259 to 13321
2025/10/24-11:13:18.815120 2150681 [/version_set.cc:5927] Created manifest 765, compacted+appended from 80064 to 13304
2025/10/24-11:13:35.590905 2150681 [/version_set.cc:5927] Created manifest 1167, compacted+appended from 79863 to 13304
```
As you can see, it only took a few iterations of ramp-up to settle on the auto-tuned max manifest size for tracking ~122 live SST files, around 80KB and compacting down to about 13KB. (13KB * (500 + 100) / 100 = 78KB). With the default large setting for max_manifest_file_size, we end up with a 232KB manifest, which is more than 90% wasted space. (A long-running DB would be much worse.)
As for performance, we don't expect a difference, even with TransactionDB because actual writing of the manifest is done without holding the DB mutex. I was not able to see a performance regression using db_bench with FIFO compaction and >1000 ~10MB SST files, including settings of -max_manifest_file_size=2048 -max_manifest_space_amp_pct={500,10,0}. No "hiccups" visible with -histogram either.
I also tried seeding a 1 second delay in writing new manifest files (other than the first). This had no significant effect at -max_manifest_space_amp_pct=500 but at 100 started causing write stalls in my test. In many ways this is kind of a worst case scenario and out-of-proportion test, but gives me more confidence that a higher number like 500 is probably the best balance in general.
Reviewed By: xingbowang
Differential Revision: D85445178
Pulled By: pdillinger
fbshipit-source-id: 1e6e07e89c586762dd65c65bb7cb2b8b719513f9
|
||
|
Changyu Bi
|
12b85c8ce9 |
Fix timestamp handling in LevelIterator MultiScan seeks (#14085)
Summary: As titled, this fixes some internal crash test failures when UDT is enabled. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14085 Test Plan: monitor crash tests. Reviewed By: anand1976 Differential Revision: D85617949 Pulled By: cbi42 fbshipit-source-id: da6fb21c0ca5803ea24e8daf7de8558321babcf4 |
||
|
Xingbo Wang
|
dce33f9443 |
Follow up on MultiScan change in #14040 (#14055)
Summary: * Address feedback from https://github.com/facebook/rocksdb/issues/14040 * Add additional test for MultiScan * Fix a bug when del range and data are in same file for multi-scan * Rewrite the cases need to be handled in SeekMultiScan Pull Request resolved: https://github.com/facebook/rocksdb/pull/14055 Test Plan: Unit test Reviewed By: cbi42, anand1976 Differential Revision: D84851788 Pulled By: xingbowang fbshipit-source-id: 0f69632733afb99685f6341badbf239681010c38 |
||
|
Changyu Bi
|
144e9f1e42 |
Fix compaction picking with L0 standalone range deletion file (#14061)
Summary: When a standalone range deletion file is ingested in L0, currently it is compacted with any overlapping L0 files. This is not desirable when we ingest new data on top of the range deletion file. This PR fixes the compaction picking logic to only consider L0 files older than the standalone range deletion file. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14061 Test Plan: added a new unit test and updated an existing one. Reviewed By: xingbowang Differential Revision: D84930780 Pulled By: cbi42 fbshipit-source-id: 65f4403ccb40ba964b9e65b09e2f7f7efebe81df |
||
|
anand76
|
112ff5bb70 |
Allow empty MultiScan result in BlockBasedTableIterator Prepare (#14046)
Summary: Currently in BlockBasedTableIterator's Prepare(), the index lookup for a MultiScan range is expected to return atleast 1 data block (unless UDI is in use). This is because there's an implicit assumption that only ranges intersecting with the keys in the file will be prepared. This assumption, however, doesn't hold if there are range deletions and the smallest and/or largest keys in the file extend beyond the keys in the file. The LevelIterator prunes the MultiScan ranges based on the smallest/largest key, so its possible for a range to only overlap the range deletion portion of the file and not overlap any of the data blocks. Furthermore, the BlockBasedTableIterator is now much more forgiving of Seek to targets outside of prepared ranges after https://github.com/facebook/rocksdb/issues/14040 . Keeping the above in mind, this PR removes the check in BlockBasedTableIterator for non-empty index result. It adds assertions in LevelIterator to verify that ranges are being properly pruned. Another side effect is we can no longer rely solely on a scan range having 0 data blocks (i.e cur_scan_start_idx >= cur_scan_end_idx) to decide if the iterator is out of bound. We can only do so for all but the last range prepared range. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14046 Test Plan: 1. Add unit test in db_iterator_test 2. Run crash test Reviewed By: xingbowang Differential Revision: D84623871 Pulled By: anand1976 fbshipit-source-id: 2418e629f92b1c46c555ddea3761140f700819e4 |
||
|
Xingbo Wang
|
1585f2240c |
Move the MultiScan seek key check to upper layer (#14040)
Summary: The current seek key validation is too strict. This change relaxes it at block iterator level, and add additional check at DB iterator level. The new contract is that when MultiScan is used, after prepared is called, each following seek must seek the start key of the prepared scan range in order. Otherwise, the iterator is set with error status. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14040 Test Plan: Unit test Reviewed By: anand1976 Differential Revision: D84292297 Pulled By: xingbowang fbshipit-source-id: 7b31f727e67e7c0bfc53c2f9a6552e0c3d324869 |
||
|
Xingbo Wang
|
27625f4fc2 |
Fix range delete file caused MultiScan issue (#14028)
Summary: When there is an ingested SST file that only contains delete range operations, MultiScan may return error "Scan does not intersect with file". This is due to file selection during Prepare uses the file smallest and largest key without considering whether there is any key in the file. This is only a temporary fix. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14028 Test Plan: Unit test Reviewed By: anand1976 Differential Revision: D83986964 Pulled By: xingbowang fbshipit-source-id: e0961ca854e2062c2457be4324817ba073ae785d |
||
|
anand76
|
035242415f |
Fix incorrect MultiScan handling of range limit between files (#14011)
Summary: This PR fixes a bug in how MultiScan handled a scan range limit falling in the key range between files. The bug was in LevelIterator, where Prepare() relied on FindFile to determine the lower bound file for the range limit. FindFile returns the smallest file index with `range.limit < file.largest_key`. However, that doesn't guarantee that the range overlaps the file, as the `range.limit` could be smaller than `file.smallest_key`. This also fixes a bug in BlockBasedTableIterator of Valid() returning true even if status() returned error. This was exposed by the previous bug. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14011 Test Plan: Add unit tests in db_iterator_test and table_test Reviewed By: cbi42 Differential Revision: D83496439 Pulled By: anand1976 fbshipit-source-id: a9d2d138d69d0c816d9f4160a984b273d00d683f |
||
|
Changyu Bi
|
862438a7a1 |
Fix handling of out-of-range scan option (#13995)
Summary: currently BlockBasedTableIterator::Prepare() fails the iterator with non-ok status if an out-of-range scan option is detected. This is due to the interaction between LevelIterator and BlockBasedTableIterator, see added comment above BlockBasedTableIterator::Prepare(). This can fail stress test for L0 files since it doesn't use LevelIterator and scan options are not pruned. This PR fixes this by adding an internal option to MultiScanArgs that enables this check. Pull Request resolved: https://github.com/facebook/rocksdb/pull/13995 Test Plan: - new unit test - stress test that fails before this pr: `python3 -u ./tools/db_crashtest.py whitebox --iterpercent=60 --prefix_size=-1 --prefixpercent=0 --readpercent=0 --test_batches_snapshots=0 --use_multiscan=1 --read_fault_one_in=0 --kill_random_test=88888 --interval=60 --multiscan_use_async_io=0 --mmap_read=0 --level0_file_num_compaction_trigger=20` Reviewed By: anand1976 Differential Revision: D83166088 Pulled By: cbi42 fbshipit-source-id: 241a7d43c8c00d9a98eea0cabb03d2174d51aae5 |
||
|
anand76
|
afbbc90b06 |
Fail multi scan upon Prepare failure or bad scan options (#13974)
Summary: Return a failure status for multi scan if Prepare fails, or if the scan options are unsupported, instead of falling back on a regular scan. This PR also fixes a bug in LevelIterator that caused max_prefetch_size to be ignored. Pull Request resolved: https://github.com/facebook/rocksdb/pull/13974 Test Plan: Add new test in db_iterator_test and table_test Reviewed By: xingbowang Differential Revision: D82843944 Pulled By: anand1976 fbshipit-source-id: f12756c40ebd38d8d4e4425e97438b6e766a4663 |
||
|
Xingbo Wang
|
5a1ff2cb14 |
Force caller to pass comparator in MultiScanArgs (#13970)
Summary: Force caller of MultiScanArgs to pass comparator. Pass comparator from CF handle to MultiScanArgs in NewMultiScan. Expand MultiScanArgs unit test with different comparator. Pull Request resolved: https://github.com/facebook/rocksdb/pull/13970 Test Plan: unit test Reviewed By: cbi42 Differential Revision: D82739270 Pulled By: xingbowang fbshipit-source-id: e709f4a333ad547c0ba6d24d8fb2b22e50e8a12f |
||
|
Changyu Bi
|
2620c85638 |
Support async IO for MultiScan (#13932)
Summary: add option MultiScanArgs::use_async_io option and implementation for using ReadAsync() for multiscan. Read requests are submitted during Prepare() and polled during actual scanning. Pull Request resolved: https://github.com/facebook/rocksdb/pull/13932 Test Plan: - updated existing unit test to use async_io. - crash test: `python3 -u ./tools/db_crashtest.py whitebox --iterpercent=60 --prefix_size=-1 --prefixpercent=0 --readpercent=0 --test_batches_snapshots=0 --use_multiscan=1 --read_fault_one_in=0 --kill_random_test=88888 --interval=60 --multiscan_use_async_io=1 --mmap_read=0` Benchmark: - Default multiscan benchmark: ``` Set up: /db_bench --benchmarks="fillseq,compact" --disable_wal=1 --threads=1 --num_levels=1 --compaction_style=2 --fifo_compaction_max_table_files_size_mb=1000 --write_buffer_size=268435456 Without async IO: ./db_bench --db="/tmp/rocksdbtest-543376/dbbench" --use_existing_db=1 --benchmarks=multiscan --disable_auto_compactions=1 --seek_nexts=100 --threads=32 --duration=10 --statistics=1 --use_direct_reads=1 --multiscan_use_async_io=0 multiscan : 415.569 micros/op 75805 ops/sec 10.355 seconds 784968 operations; (multscans:24999) rocksdb.read.async.micros COUNT : 0 With asycn IO: ./db_bench --db="/tmp/rocksdbtest-543376/dbbench" --use_existing_db=1 --benchmarks=multiscan --disable_auto_compactions=1 --seek_nexts=100 --threads=32 --duration=10 --statistics=1 --use_direct_reads=1 --multiscan_use_async_io=1 multiscan : 413.236 micros/op 76044 ops/sec 10.375 seconds 788968 operations; (multscans:24999) rocksdb.read.async.micros COUNT : 3916499 Similar performance. ``` - Larger scan, more scans per multiscan, do not coalesce IO so that async IO can progress while scanning, and use one thread: ``` multiscan_stride = 1000 multiscan_size = 100 seek_nexts = 1000 ./db_bench --db="/tmp/rocksdbtest-543376/dbbench" --use_existing_db=1 --benchmarks=multiscan --disable_auto_compactions=1 --threads=1 --duration=10 --statistics=0 --use_direct_reads=1 --cache_size=2097152 --multiscan_size=100 --multiscan_stride=1000 --seek_nexts=1000 --seed=1 --multiscan_coalesce_threshold=0 --multiscan_use_async_io=0 Without async IO: multiscan : 20495.205 micros/op 48 ops/sec 10.002 seconds 488 operations; (multscans:488) With async IO: multiscan : 18337.883 micros/op 54 ops/sec 10.013 seconds 546 operations; (multscans:546) ~10% improvement in throughput ``` Reviewed By: xingbowang Differential Revision: D82077818 Pulled By: cbi42 fbshipit-source-id: 66e32cf4039183c4841827409286dfbaa6dfbcd8 |
||
|
Changyu Bi
|
acf9d4e445 |
Fix UDT handling in MultiScan (#13938)
Summary: we saw some crash test failure at https://github.com/facebook/rocksdb/blob/f46242cef631351a5c8f4a7b0fb0935ec7fa61c8/table/block_based/block_based_table_iterator.cc#L964-L965. This is likely due to timestamp not being considered properly in some places in MultiScan code paths. This PR fixes the issue. Pull Request resolved: https://github.com/facebook/rocksdb/pull/13938 Test Plan: crash test with timestamp and multiscan: `python3 -u ./tools/db_crashtest.py whitebox --enable_ts --iterpercent=60 --prefix_size=-1 --prefixpercent=0 --readpercent=0 --test_batches_snapshots=0 --use_multiscan=1 --read_fault_one_in=0 --kill_random_test=88888 --interval=60` Reviewed By: anand1976 Differential Revision: D82175263 Pulled By: cbi42 fbshipit-source-id: 5d40ede1aec15f8faeaa7fd041b939e68611ff73 |
||
|
Changyu Bi
|
618f660eab |
Configurable multiscan IO coalescing threshold (#13886)
Summary: Add a new filed `io_coalesce_threshold` to MultiScanArgs to make IO coalescing threshold configurable. Pull Request resolved: https://github.com/facebook/rocksdb/pull/13886 Test Plan: db_bench showing less IO requests with higher io_coalesce_threshold ``` Single L0 file, iterator uses BlockBasedTableIterator directly, skipping LevelIterator DB Set up: ./db_bench --benchmarks="fillseq,compact" --disable_wal=1 --threads=1 --num_levels=1 --compaction_style=2 --fifo_compaction_max_table_files_size_mb=1000 --write_buffer_size=268435456 ./db_bench --db="/tmp/rocksdbtest-543376/dbbench" --use_existing_db=1 --benchmarks=multiscan --disable_auto_compactions=1 --seek_nexts=100 --threads=32 --duration=10 --statistics=1 --use_direct_reads=1 .. --multiscan_coalesce_threshold=0 rocksdb.non.last.level.read.bytes COUNT : 54591304136 rocksdb.non.last.level.read.count COUNT : 7680204 multiscan : 397.197 micros/op 79401 ops/sec 10.377 seconds 823968 operations; (multscans:24999) --multiscan_coalesce_threshold=16384 rocksdb.non.last.level.read.bytes COUNT : 95960989272 rocksdb.non.last.level.read.count COUNT : 912008 multiscan : 389.099 micros/op 81064 ops/sec 10.312 seconds 835968 operations; (multscans:25999) --multiscan_coalesce_threshold=163840 rocksdb.non.last.level.read.bytes COUNT : 98805008718 rocksdb.non.last.level.read.count COUNT : 827893 multiscan : 392.831 micros/op 80357 ops/sec 10.353 seconds 831968 operations; (multscans:25999) DB with multiple files in a level, iterator will use LevelIterator ./db_bench --benchmarks="fillseq,compact" --disable_wal=1 --threads=1 --num_levels=6 --num=10000000 ./db_bench --db="/tmp/rocksdbtest-543376/dbbench" --use_existing_db=1 --benchmarks=multiscan --disable_auto_compactions=1 --seek_nexts=100 --threads=32 --duration=10 --statistics=1 --use_direct_reads=1 --num=10000000 --multiscan_coalesce_threshold=0 multiscan : 1161.734 micros/op 26995 ops/sec 10.667 seconds 287968 operations; (multscans:8999) rocksdb.non.last.level.read.bytes COUNT : 23917753523 rocksdb.non.last.level.read.count COUNT : 2868907 --multiscan_coalesce_threshold=16384 rocksdb.non.last.level.read.bytes COUNT : 35022281853 rocksdb.non.last.level.read.count COUNT : 287375 multiscan : 1195.336 micros/op 26265 ops/sec 10.850 seconds 284968 operations; (multscans:8999) ``` Reviewed By: anand1976 Differential Revision: D80381441 Pulled By: cbi42 fbshipit-source-id: 57cc67df4a808e27c3a48ddf3ef6907bec131ee9 |
||
|
Ryan Hancock
|
0b44282a9d |
Introduction of MultiScanOptions (#13837)
Summary: To better support future options, and changes, we need to convert the std::vector<ScanOptions> to something more malleable. This diff introduces the MultiScanOptions structure and pipes it through the various points in the code in the Prepare path. Pull Request resolved: https://github.com/facebook/rocksdb/pull/13837 Test Plan: Ensure all associated tests pass ``` make check all ``` Reviewed By: cbi42 Differential Revision: D79655229 Pulled By: krhancoc fbshipit-source-id: 3a90fb7420e9655021de85ed0158b866f8bfba05 |
||
|
Changyu Bi
|
3bd7d968e1 |
Introduce column family option cf_allow_ingest_behind (#13810)
Summary: this option has the same functionality as DBOptions::allow_ingest_behind but allows the feature at per CF level. `DBOptions::allow_ingest_behind` is deprecated after this PR and users should use `cf_allow_ingest_behind` instead. Pull Request resolved: https://github.com/facebook/rocksdb/pull/13810 Test Plan: updated some existing tests to use the new option. Reviewed By: xingbowang Differential Revision: D79191969 Pulled By: cbi42 fbshipit-source-id: 0da45f6be472ace6754ad15df93d45ac86313837 |
||
|
Ryan Hancock
|
351d212777 |
Ensure Property Bags are Pushed Down to BlockBasedIterator (#13795)
Summary: This diff fixes up a miss in which the property_bag was not pushed down to the BlockBasedIterator. Pull Request resolved: https://github.com/facebook/rocksdb/pull/13795 Reviewed By: anand1976 Differential Revision: D78762294 Pulled By: krhancoc fbshipit-source-id: 8970b0a87e35d07d5a0dd16f360ec96859f66550 |
||
|
Ryan Hancock
|
fe68fbcd7f |
Prepare() Scan Option Pruning for LevelIterator (#13780)
Summary: This diff introduces the ScanOption Pruning, previously the intent was to do prefetching for each sub-iterator of the level iterator, however since BlockBasedIterator does not prefetch asynchronously, this optimization does not make sense just yet. For now we will prune the ScanOptions to the overlapping ranges and make sure they are properly piped to the underlying layers (during Prepare, and Seek). Pull Request resolved: https://github.com/facebook/rocksdb/pull/13780 Reviewed By: cbi42 Differential Revision: D78436869 Pulled By: krhancoc fbshipit-source-id: 681fe7f7f88b04b5c2d60cb3a5de01e03f6f8431 |
||
|
Xingbo Wang
|
0be850a000 |
Avoid divide by 0 in ComputeCompactionScore for FIFO compaction (#13767)
Summary: When max_table_files_size was accidentally configured with 0 value, engine could crash on divide by 0 operation. Although RocksDB do configuration validation during bootstrap, it typically does not do this for runtime dynamic parameter validation. Therefore, there is a chance where max_table_files_size could be set to 0. This PR only focuses on fixing a code path where max_table_files_size ack as divisor. Pull Request resolved: https://github.com/facebook/rocksdb/pull/13767 Test Plan: Unit test. Reviewed By: cbi42 Differential Revision: D78420516 Pulled By: xingbowang fbshipit-source-id: 6fdcc85b28a2c6319066665262b981e513719703 |
||
|
Changyu Bi
|
f081d145cf |
Backport internal changes (#13752)
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/13752 ... to github repo. This include changes from D77323287, D77473923 and the release note change in patch release: D77611483. Reviewed By: archang19 Differential Revision: D77670619 fbshipit-source-id: 37d877f3317c71de190128fa4da6b18f6dfcf3c5 |
||
|
anand76
|
24e2b05e61 |
Multi scan API (#13473)
Summary: A multi scan API for users to pass a set of scan ranges and have the table readers determine the optimal strategy for performing the scans. This might include coalescing of IOs across scans, for example. The requested scans should be in increasing key order. The scan start keys and other info is passed to NewMultiScanIterator, which in turn uses the newly added Prepare() interface in Iterator to update the iterator. The Prepare() takes a vector of ScanOptions, which contain the start keys and optional upper bounds, as well as user defined parameters in the property_bag taht are passed through as is to external table readers. The initial implementation plumbs this through to the ExternalTableReader. This PR also fixes an issue of premature destruction of the external table iterator after the first scan of the multi-scan. The `LevelIterator` treats an invalid iterator as a potential end of file and destroys the table iterator in order to move to the next file. To prevent that, this PR defines the `NextAndGetResult` interface that the external table iterator must implement. The result returned by `NextAndGetResult` differentiates between iterator invalidation due to out of bound vs end of file. Eventually, I envision the `MultiScanIterator` to be built on top of a producer-consumer queue like container, with RocksDB (producer) enqueueing keys and values into the container and the application (consumer) dequeueing them. Unlike a traditional producer consumer queue, there is no concurrency here. The results will be buffered in the container, and when the buffer is empty a new batch will be read from the child iterators. This will allow the virtual function call overhead to be amortized over many entries. TODO (in future PRs): 1. Update the internal implementation of Prepare to trim the ScanOptions range based on the intersection with the table key range, taking into consideration unbounded scans and opaque user defined bounds. 2. Long term, take advantage of Prepare in BlockBasedTableIterator, atleast for the upper bound case. Pull Request resolved: https://github.com/facebook/rocksdb/pull/13473 Reviewed By: pdillinger Differential Revision: D71447559 Pulled By: anand1976 fbshipit-source-id: 31668abb0c529aa1ac1738ae46c36cbddf9148f1 |
||
|
Yu Zhang
|
934cf2d40d |
Implement the DB::GetPropertiesOfTablesForLevels API (#13469)
Summary: As titled. This API returns the table properties of files per level. It can be handy for use cases that needed file's leveling info while retrieving TableProperties. We will use this API to later aggregate per level data write time info. Pull Request resolved: https://github.com/facebook/rocksdb/pull/13469 Test Plan: Added unit tests Reviewed By: pdillinger Differential Revision: D71353096 Pulled By: jowlyzhang fbshipit-source-id: dc1fbb2c97e4365fc8d7241f9a59c65fbf4fb766 |