Summary:
Add read-scoped block buffers for scan reads. Introduce an experimental read-scoped block buffer provider API, configured through ReadOptions::read_scoped_block_buffer_provider, so supported block-based table iterator scans and MultiScan data-block reads can use caller-provided read-scoped storage for final data-block contents.
When configured, supported provider-backed scan data-block reads bypass the data-block cache while preserving normal index/filter block-cache behavior. Known-uncompressed reads can attach provider cleanup to provider-backed read buffers without copying. Compressed reads decompress directly into provider-backed output, while maybe-compressed reads that turn out to be uncompressed copy once into provider-backed final contents. mmap reads ignore the provider.
Extend AlignedBuffer and RandomAccessFileReader direct-I/O paths to support external aligned allocations, then use that support for read-scoped iterator, async I/O, and MultiRead scratch buffers. Centralize read-scoped I/O policy, keep coalesced async reads safe when blocks are released before completion, and validate provider lease contracts.
Add focused coverage for read-scoped ownership, compressed and uncompressed blocks, direct I/O, data-block cache bypass behavior, invalid provider leases, async release handling, and stress-test provider invariants. Add public API release notes for read-scoped block buffers.
Bonus change: Fixed a flaky test in ReserveThread
## Testing
- CI
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14806
Reviewed By: anand1976
Differential Revision: D106999951
Pulled By: xingbowang
fbshipit-source-id: b1f23d4bab6318b6373ba2ca99a5c4d6a842dc5a
Summary:
- Propagate validated, sorted MultiScan range state from `DBIter::Prepare()` through `MultiScanArgs`.
- Mark block-based table IO jobs as already sorted when the public MultiScan ranges have been validated.
- Keep `IODispatcher::SubmitJob()` as the normalization boundary for unsorted callers, while allowing sorted callers to skip the defensive block-handle sort.
- Update private dispatcher coalescing helpers to consume sorted block indices and add debug assertions for that precondition.
## Testing
CI, new unit test
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14783
Reviewed By: anand1976
Differential Revision: D106301516
Pulled By: xingbowang
fbshipit-source-id: 99b7ffcaecbf27cb79f15feb4af8680ff1e422d9
Summary:
The non-ASCII character check in check-sources.sh used git grep -P (Perl regex), which requires git compiled with PCRE support. On systems without it, the command fails with exit code 128, which is != 1 (no match), so the check always reported a violation -- effectively dead.
Even in CI where git has PCRE2 support, the check was silently broken: git grep -P uses PCRE2 in UTF mode by default, which interprets [\x80-\xFF] as a Unicode codepoint range (U+0080 to U+00FF). Characters like em-dash (U+2014), arrows (U+2192), and math symbols (U+2248, etc.) fall outside that range and were not detected. Only Latin-1 Supplement characters (U+0080-U+00FF) would have been caught.
Replace with LC_ALL=C git grep using bash $'[\x80-\xff]' literal byte range, which works with basic regex in the C locale, and replace all non-ASCII characters in non-excluded source files:
- em-dash to --
- arrow to ->
- math symbols to ASCII equivalents (~=, <=, >=)
- box-drawing characters to ASCII art
Also exclude .github/ from the check, as scripts there can use non-ascii without disrupting RocksDB builds on non-UTF-8 systems.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14729
Test Plan: manual / CI (make check-sources passes clean)
Reviewed By: hx235
Differential Revision: D104692574
Pulled By: pdillinger
fbshipit-source-id: 1d884c21056dcd83558b825a04b867f1c08e3f45
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14569
ReadSet::ReadIndex() moves block values out of pinned_blocks_ via std::move,
but never releases the associated prefetch memory accounting. This causes
ReleaseBlock() and the destructor to skip ReleaseMemory() since they check
pinned_blocks_.GetValue() which returns null after the move. Over time, the
memory budget is exhausted and no further prefetches can be dispatched when
max_prefetch_memory_bytes is set. The bug was introduced in https://github.com/facebook/rocksdb/pull/14401.
The fix releases memory accounting in ReadIndex() when moving values out
(both for Case 1: block already available, and Case 2: after async IO
polling), and zeros block_sizes_ to prevent double-release.
Also adds multiscan_max_prefetch_memory_bytes option to db_stress/crashtest
for stress testing this code path.
Reviewed By: hx235
Differential Revision: D99488961
fbshipit-source-id: 5ddd1f50e2f6ebb357f86e013d781a790e7e558a
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14401
Unify the MultiScan and regular iterator codepaths in BlockBasedTableIterator by introducing a MultiScanIndexIterator that implements InternalIteratorBase<IndexValue>. During Prepare(), the original index iterator is swapped out for a MultiScanIndexIterator that wraps the prefetched block handles and scan range metadata. This allows SeekImpl() and FindBlockForward() to use the same code flow for both regular and MultiScan operations, eliminating the need for separate MultiScan-specific methods (SeekMultiScan, FindBlockForwardInMultiScan, MultiScanSeekTargetFromBlock, MultiScanUnexpectedSeekTarget, MultiScanLoadDataBlock, MarkPreparedRangeExhausted).
Key changes:
- New MultiScanIndexIterator class that manages scan range tracking, block handle iteration, forward-only seek enforcement, and wasted block counting
- InitDataBlock() loads blocks from ReadSet when MultiScan is active
- FindBlockForward() detects scan range boundaries via IsScanRangeExhausted() after index_iter_->Next()
- Disabled reseek optimization for MultiScan so MultiScanIndexIterator::Seek() is always called to update scan range tracking state
- Removed MultiScanState struct and all MultiScan-specific methods from BlockBasedTableIterator
- No changes to CheckDataBlockWithinUpperBound or CheckOutOfBound — they work as-is through iterate_upper_bound
- multi_scan_status_ intentionally not checked in Valid() hot path to avoid performance regression; when status is non-OK, block_iter_points_to_real_block_ is already false
- Fixed pre-existing bug in ReadSet::SyncRead() that used the base decompressor without compression dictionary, causing ZSTD data corruption when blocks with dictionary compression needed synchronous fallback reads
Reviewed By: xingbowang
Differential Revision: D93300655
fbshipit-source-id: 231059208e0cc512bc2ec43ff7055fcb2a2dc72d
Summary:
This diff introduces the IODispatcher into the BlockBasedTableIterator. This replaces much of the prefetch logic with the logic found in IODispatcher.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14255
Test Plan:
I ran the following benchmark, %change is within noise tolerance. There shouldn't be any large performance improvement with this change, more so there should also not be any performance degradation.
MultiScan Benchmark: Current Branch vs Main
Configuration:
- Threads: 4
- Ranges per scan: 10
- Stride: 5000
- Seek nexts: 100
- Cache: Cold (dropped before each run)
- Runs: 3
Results:
│ Mode │ Main (ops/sec) │ Current (ops/sec) │ Change │
│ Sync │ 8,901 │ 9,032 │ +1.5% │
│ Async │ 11,297 │ 11,947 │ +5.8% │
I further run db_stress test
```
make -j32 -f crash_test.mk J=32 blackbox_crash_test
```
Against my local machine for 60 minutes, on local flash, with async-io for multiscans always on.
Reviewed By: anand1976
Differential Revision: D91705195
Pulled By: krhancoc
fbshipit-source-id: acf2f944e8b715e99384c8cee79f8d241eadf5b8
Summary:
This diff introduces the IO Dispatcher, which will be used to simplify the code path for MultiScan, while further providing a centralized place to enact policy on how MultiScan is done (i.e., limit memory usage and pinned buffers for example). Right now this diff only encapsulates the functionality done during the Prepare of MultiScan.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14135
Reviewed By: anand1976
Differential Revision: D87837261
Pulled By: krhancoc
fbshipit-source-id: 2698910ade02bc3d182413ae07ce69fe7abb7ec5
Xingbo Wang