Compare commits

...

204 Commits

Author SHA1 Message Date
anand76 7d274331a6 Force push 2026-04-23 13:33:29 -07:00
anand76 30b150475b Force push 2026-04-23 13:28:31 -07:00
anand76 7eec357e30 Fix check_format_compatible.sh and revert folly hash 2026-04-21 16:04:13 -07:00
anand1976 0dab64fd3e Update version to 11.3.0 for 11.2 release 2026-04-20 11:54:12 -07:00
Xingbo Wang 6ff96384dd db_iter: eagerly resolve blob-backed iterator columns (#14632)
Summary:
Restore the iterator contract for blob-backed wide-column entities by eagerly materializing all blob-backed columns before a `DBIter` entry is exposed as valid. Also capture similar error inside compaction filter V4, so that the error is captured a surfaced up like older compaction filter V3.

## Problem

Lazy wide-column blob resolution let `DBIter` report `Valid() == true` before all columns were actually prepared. Callers could read `value()` successfully, then hit a blob-read failure in `columns()`, which flipped the iterator to invalid after the entry had already been observed as valid.

## Solution

- Restore back to old behavior.
- Resolve and materialize all blob-backed wide columns during iterator positioning.
- Keep `columns()` as a pure accessor once `Valid()` is true.
- Add coverage for eager blob resolution on iterator scans.
- Add coverage for blob resolution failures invalidating the iterator before the entry is exposed.
- Capture the same error in compaction filter and set db to bg error status if the error is not retriable.

## Next step
If we want lazy iterator resolution in the future, we will need an API that can surface lazy resolution failures explicitly instead of hiding I/O and status transitions inside value() or columns().

## Testing

- `make -j14 db_wide_blob_direct_write_test`
- `timeout 60s ./db_wide_blob_direct_write_test --gtest_filter='*DirectWriteIteratorValueScanEagerlyResolvesBlobColumns:*DirectWriteIteratorBlobResolutionErrorInvalidatesEntry'`

## Task
T265294130

# Bonus fix

  ## Non-compaction blob filtering fixes

  ### 1. Flush-time wide-entity lazy blob resolution now works correctly

  Blob direct-write can leave wide-entity blob references in memtables before flush. Flush already runs through `CompactionIterator`, but outside a real compaction it did not have blob read support. As a result, `FilterV4` lazy resolution on flush could fail immediately with
  `NotSupported("Blob fetcher not available")` instead of either:
  - resolving the blob successfully, or
  - surfacing the real blob read error and failing the flush

  This stack fixes that by plumbing blob read support into non-compaction table-file creation. `BuildTable()` now gives `CompactionIterator` enough context to construct a `BlobFetcher` for flush/recovery table building, including write-path fallback for direct-write blob files
  that are not yet manifest-visible.

  With that in place:
  - flush-time `FilterV4` can lazily resolve blob-backed wide columns
  - if resolution fails, the failure is latched and flush fails after `FilterV4()` returns
  - `bg_error` is set instead of silently preserving the entry

  ### 2. Plain blob fallback outside compaction now uses the resolved user value

  For plain blob-backed values, `FilterBlobByKey()` is allowed to return `kUndetermined`, which is supposed to fall back to the normal value-based filter path. That fallback worked in compaction, but the non-compaction `BuildTable()` path still treated plain blob indexes as
  unsupported/corrupt because it could not read the blob.

  This stack fixes that behavior for flush/recovery table-file creation:
  - when `FilterBlobByKey()` returns `kUndetermined`, RocksDB now eagerly resolves the plain blob value
  - `FilterV2`/`FilterV3`/`FilterV4` then see the actual user value as `ValueType::kValue`, not the `BlobIndex` encoding
  - legacy filters therefore behave the same way outside compaction as they already do inside compaction

  Wide-column entities still use the `FilterV4` lazy resolver path. The plain-blob fix is specifically about restoring the documented fallback behavior for blob-backed `kValue` records outside compaction.

  ## Why this matters

  Without these last two commits, flush-time filtering still had two inconsistent behaviors:
  - wide-entity lazy resolution could not actually read blob-backed columns in the direct-write case
  - plain blob-backed values could not fall back to normal value-based filtering outside compaction

  So even after fixing iterator validity and compaction error propagation, non-compaction table-file creation still had remaining contract holes. These commits make flush/recovery behave consistently with compaction.

  ## Coverage added

  The tests added in these commits cover:
  - flush-time plain-blob `FilterV3` fallback using the resolved user value
  - flush-time plain-blob `FilterV4` fallback using the resolved user value
  - flush-time direct-write wide-entity lazy resolver success
  - flush-time direct-write wide-entity lazy resolver failure on missing blob file, including flush failure and `bg_error` latching

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14632

Reviewed By: anand1976

Differential Revision: D101425233

Pulled By: xingbowang

fbshipit-source-id: 695b2df5189033d30309b35849815e31fd965664
2026-04-18 00:38:50 -07:00
Anand Ananthabhotla a4139ef9e9 IODispatcher: fall back to synchronous coalesced reads when async IO is unavailable (#14633)
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/14633

Reviewed By: xingbowang

Differential Revision: D101299989

fbshipit-source-id: ce30f91da23031b57b2f440f3cdc56f67cc673a0
2026-04-17 17:57:35 -07:00
Andrew Chang 3ef8b1b743 Guard auto-readahead against exhausted index iter (#14631)
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14631

Observed crash from ZippyDB iterate scans:

```
onFatalError: unexpected error : Invariant SignalFatal:
fbcode/zippydb/server/Main.cpp:handleSigToFatalCommon:281:FATAL false
failed: Signal 11 (SIGSEGV) at address 0x0 ... method: MultiIterate ...
RocksDB Activity: iterate
```

Key stack frames:
  rocksdb::BlockBasedTableIterator::IsNextBlockOutOfReadaheadBound()
    fbcode/rocksdb/src/table/block_based/block_based_table_iterator.h:471
  rocksdb::BlockBasedTableIterator::BlockCacheLookupForReadAheadSize()
    fbcode/rocksdb/src/table/block_based/block_based_table_iterator.cc:878
  rocksdb::FinalizeAsyncRead()
  rocksdb::FilePrefetchBuffer::PollIfNeeded()
  rocksdb::BlockFetcher::ReadBlockContents()
  rocksdb::BlockBasedTableIterator::InitDataBlock()
  rocksdb::DBIter::Next()

Reviewed By: xingbowang

Differential Revision: D101217429

fbshipit-source-id: e7657909a0f07526d070b214cb783f581b8e98ca
2026-04-17 12:36:29 -07:00
anand76 4bc0bff0e4 Mark IODispatcher as experimental (#14630)
Summary:
As titled

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14630

Reviewed By: xingbowang

Differential Revision: D101383049

Pulled By: anand1976

fbshipit-source-id: 19fb1c27bee465e691412a30681f9a21c70672f7
2026-04-17 12:12:13 -07:00
xingbowang 49ec219955 cmake: fix folly nightly build regressions (#14609)
Summary:
Fix nightly build regressions introduced by recent CI/toolchain changes.

This PR includes two parts:
- CMake/link fixes for the Folly and Folly Lite nightly jobs
- a targeted nightly workflow fix for the clang-21 ASAN/UBSAN + Folly job

Changes:
- link gflags explicitly for USE_FOLLY CMake tool and benchmark targets
- resolve USE_FOLLY_LITE glog via its installed library path instead of bare -lglog
- in the clang-21 nightly job, build Folly/getdeps with gcc/g++ instead of inheriting clang-21 for third-party dependency builds
- disable ccache only for that standalone Folly build step so getdeps/CMake does not inject the broken ccache compiler launcher for ASM

Root cause:
- recent CI/container changes exposed missing explicit link dependencies in the CMake Folly paths
- the clang-21 nightly job exported CC/CXX at job scope, so Folly getdeps inherited clang-21 into libiberty/binutils build logic that expects GCC-style driver behavior such as -print-multi-os-directory
- the same standalone Folly build path also misbehaved when ccache was auto-detected and used as a compiler launcher for assembler-with-cpp inputs

Verification:
- make format-auto
- git diff --check
- local CMake configure sanity check for default config
- upstream nightly reruns used to confirm failure signatures before and after the first-round fixes

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14609

Reviewed By: anand1976

Differential Revision: D100695693

Pulled By: xingbowang

fbshipit-source-id: 703546ad3ddc518ab80c936442709d65fe2d22af
2026-04-17 11:22:35 -07:00
Josh Kang 790294f3ad Disallow table_filters entirely when range tombstone conversion is enabled (#14618)
Summary:
The prior fix (https://github.com/facebook/rocksdb/issues/14586) disabled read-path range tombstone synthesis at the `DBIter` level when `table_filter` was set. However, synthesized tombstones persist in the memtable beyond the lifetime of the iterator that created them. A prior unfiltered iterator can synthesize a range tombstone that then silently affects a subsequent filtered iterator's results — the filtered scan sees the tombstone but not the SSTs it was derived from, allowing hidden SST state to corrupt the filtered view. See failed crash test in T264151327.

This PR takes the stronger approach of rejecting iterator creation outright (`InvalidArgument`) when `ReadOptions::table_filter` is used on a column family with `min_tombstones_for_range_conversion > 0`. This eliminates the entire class of interaction bugs between the two features rather than trying to suppress conversion in individual code paths.

## Example

- `min_tombstones_for_range_conversion = 2`
- L1 SST_keep: `Put(a), Put(b), Put(c)`   (`c` is the live boundary)
- L0 SST_dels: `Delete(a), Delete(b)`     (2 contiguous tombstones, newer)

### Step 1 — Iterator A (no filter)
A walks `Del(a) Del(b)` (2 contig) then `c` (live).
Synthesizes range tombstone `[a, c)` into the active memtable.

### Step 2 — Iterator B (`table_filter` skips SST_dels)
B's filter excludes SST_dels, so `a, b` from SST_keep should appear live.
But the memtable now holds range tombstone `[a, c)` from step 1, which
applies to B regardless of `table_filter`. B sees only `c` —
**a, b are silently hidden**.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14618

Test Plan:
- `TableFilterNotAllowed` test validates the rejection path end-to-end: unfiltered scan synthesizes the tombstone, filtered iterator is rejected with `InvalidArgument`, and the full-DB view remains correct via `VerifyIteration`.
- Crash test sanitization ensures `db_stress` won't hit the incompatible configuration.
- Run crash test sanitization unit test: `python3 tools/db_crashtest_test.py`
- Run iterator tests: `make db_iterator_test && ./db_iterator_test --gtest_filter="*TableFilterNotAllowed*"`

Reviewed By: xingbowang

Differential Revision: D100873156

Pulled By: joshkang97

fbshipit-source-id: dd0d123d94b1bfda2a4a5040681fcc7cf14f760e
2026-04-17 01:33:16 -07:00
Josh Kang 9886d52fc4 Disable range tomb conversion when using legacy prefix iterator (#14625)
Summary:
There is an assumption in the code that within a given prefix all keys can be seen after a seek to that prefix. The problem occurs when there is a Next() followed by a Prev() back into the same prefix. The merging iterator actually uses SeekForPrev during Prev for non-child iterators, and as a result changes the prefix being used. Now that we are back to our original prefix, the view is incomplete.

The fix is to simply disable range tombstone conversion in this legacy prefix mode.

### Example

Use a 1-byte prefix extractor, so the prefix is just the first character.

Think of the merged iterator as combining two children:

- Child A: `b8`
- Child B: `b7, c1`

Global sorted order is:

```text
b7, b8, c1
```

So:

- predecessor of `c1` should be `b8`
- predecessor of `b8` should be `b7`

## Sequence

Start in legacy prefix mode:

- `total_order_seek = false`
- `prefix_same_as_start = false`

Now do:

1. `Seek("b8")`
2. `Next()`
3. `Prev()`

## What should happen

```text
Seek("b8") -> b8
Next()     -> c1
Prev()     -> b8
```

Because in the global order, `b8` is immediately before `c1`.

## What the buggy iterator did

```text
Seek("b8") -> b8
Next()     -> c1
Prev()     -> b7
```

So it skipped `b8`.

## Why it happens

When `Prev()` switches direction, the merging iterator tells all non-current
 children to `SeekForPrev(current_key)`.

At that moment:

- current key is `c1`
- so the other children get `SeekForPrev("c1")`

But in legacy prefix mode, those child seeks are allowed to use prefix
filtering. So a child that only contains `b*` keys can effectively say:

```text
"I was asked for prefix c; I do not have prefix c."
```

That child drops out instead of returning its real global predecessor `b8`.

Then the current child simply moves back from `c1` to `b7`, and the merged
result becomes `b7`.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14625

Test Plan: Update unit tests to disable feature.

Reviewed By: xingbowang

Differential Revision: D101259748

Pulled By: joshkang97

fbshipit-source-id: 23e19c470f931406f96a34a7baebca89e91c7e39
2026-04-16 18:20:59 -07:00
xingbowang 36e50a61fc db_stress: exclude info logs from metadata read faults (#14616)
Summary:
- fix a false positive in `db_stress` PrefixScan fault accounting caused by info-log `FileExists()` probes going through metadata-read fault injection
- exclude `kInfoLogFile` from metadata-read injection and extend `FaultInjectionTestFS` filename parsing so LOG/LOG.old.* and `db_log_dir` info-log paths are recognized consistently
- add regression coverage showing info-log `FileExists()` bypasses metadata-read injection while manifest lookups still inject

## Testing
- make format-auto
- make -j48 fault_injection_fs_test db_stress
- timeout 60 ./fault_injection_fs_test
- timeout 60 ./fault_injection_fs_test --gtest_filter='*MetadataReadFaultExcludesInfoLogFiles*' --gtest_repeat=5

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14616

Reviewed By: hx235

Differential Revision: D100838168

Pulled By: xingbowang

fbshipit-source-id: 9ef996908b393aea2a8221fcdda67ed609643dbe
2026-04-16 18:20:44 -07:00
xingbowang 7b06a9ae7e Track blob garbage during flush of direct-write blob references (#14623)
Summary:
- meter blob garbage during `BuildTable()` when flush input can contain direct-write blob references, so overwrite elision and flush-time compaction filters register garbage in the manifest
- plumb `BlobFileGarbage` through flush and recovery manifest edits even when the flush produces no SST output
- add wide-column direct-write tests covering TTL-filtered flushes with both mixed live/expired records and all-expired input

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14623

Reviewed By: anand1976

Differential Revision: D101212841

Pulled By: xingbowang

fbshipit-source-id: e5803ddaf17b5dfd92312e42191b76e311257f3b
2026-04-16 16:45:13 -07:00
xingbowang df5695cf24 Fix workflow YAML parsing and add validation (#14624)
Summary:
This fixes the Claude review workflow YAML parse error and adds a CI validation step for GitHub Actions workflow YAML via `make check-
  workflow-yaml`.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14624

Reviewed By: archang19

Differential Revision: D101225058

Pulled By: xingbowang

fbshipit-source-id: 0f371dcbd70ece2c25220c96e43305d9665554b1
2026-04-16 16:06:32 -07:00
Xingbo Wang 820c4a77c1 dump disk usage on no space stress test failure (#14619)
Summary:
Detect out-of-space failures from combined db_stress stdout/stderr in the Python wrapper so both OpenAndCompact stdout failures and verification stderr failures trigger the same diagnostics.

When a match is found, print filesystem usage for /dev/shm and the db roots, then summarize per-directory and per-extension usage to make it clear which files consumed space. Add unit coverage for the failure matcher and suffix accounting.

This help triage any regression in additional file usage in stress test.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14619

Reviewed By: archang19

Differential Revision: D100984310

Pulled By: xingbowang

fbshipit-source-id: 23765ff405f0e64382e601b0da173ab6b37dba6d
2026-04-15 09:52:43 -07:00
Laurynas Biveinis 94b6566505 Fix CompareDbtEndpoints crash with timestamp-aware comparators (#14601) (#14611)
Summary:
`RangeTreeLockManager::CompareDbtEndpoints()` called `Comparator::Compare()` on range lock endpoint keys that never contain user-defined timestamps. With a timestamp-aware comparator (`timestamp_size() > 0`), this caused assertion failures in debug builds and silent endpoint misordering in release builds.

Replace `Compare()` with the 4-argument `CompareWithoutTimestamp()` using `a_has_ts=false, b_has_ts=false`, which is the correct contract for serialized range lock endpoints (format: `[1-byte suffix][key bytes]`, no timestamp).

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14611

Test Plan:
- New test `RangeLockWithTimestampComparator` reopens with `BytewiseComparatorWithU64Ts` and exercises range lock acquisition, conflict detection, and non-overlapping success with short keys.
- Verified RED (assertion failure) before fix, GREEN after.
- Full `range_locking_test` suite passes (16/16).
- Stress tested with `COERCE_CONTEXT_SWITCH=1 --gtest_repeat=5`.

Reviewed By: xingbowang

Differential Revision: D100775201

Pulled By: laurynas-biveinis

fbshipit-source-id: 58e3846e62e9b3cc5bdc69557458b245f90b3967
2026-04-15 09:38:59 -07:00
Anand Ananthabhotla 1cc216a45b Fast file open: retrieve, persist, and pass file open metadata (#14596)
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14596

When DBOptions::fast_sst_open is enabled, RocksDB retrieves opaque file system
metadata for SST files after flush, compaction, and external file ingestion via
FSRandomAccessFile::GetFileOpenMetadata(). This metadata is persisted in the
MANIFEST using a new forward-compatible NewFileCustomTag (kFileOpenMetadata = 17),
and passed back to the file system via FileOptions::file_metadata on subsequent
file opens. This accelerates DB open time on remote storage systems by allowing
the file system to skip expensive metadata RPCs.

The feature is gated by DBOptions::fast_sst_open (default false). Everything
works seamlessly regardless of the option setting, file metadata support, or
presence/absence of the metadata in the MANIFEST. The MANIFEST change is backward
compatible - older RocksDB versions safely ignore the new tag.

Reviewed By: xingbowang

Differential Revision: D100220973

fbshipit-source-id: f52de9dd853a50653b3297ab4a37a868fe41cc04
2026-04-14 19:07:29 -07:00
Josh Kang 978bf67426 Disable min_tombstones_for_range_conversion in crash tests (#14617)
Summary:
Temporarily disable the feature until all fixes land and crash tests are stabilized.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14617

Reviewed By: xingbowang

Differential Revision: D100853151

Pulled By: joshkang97

fbshipit-source-id: f2c9018f0b55891f6c0623902c3b961bc7f623aa
2026-04-14 13:52:59 -07:00
Xingbo Wang 6b8ca89f45 Ignore post-SIGTERM io_uring stderr in stress test (#14613)
Summary:
Blackbox crash tests intentionally terminate `db_stress` with `SIGTERM` on timeout. When `io_uring` is enabled, that shutdown can emit the expected
`PosixRandomAccessFile::MultiRead: io_uring_submit_and_wait returned terminal error: -9.`
message on `stderr`, which currently makes the timeout path look like a real failure.

This change filters only that known post-`SIGTERM` stderr after a timeout when the process output confirms the `SIGTERM` handler ran. Any other stderr is still surfaced and fails the test, while the ignored lines are appended to `stdout` so the signal remains visible in logs.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14613

Test Plan:
- Added unit tests covering fully ignored post-`SIGTERM` stderr
- Added unit tests covering mixed stderr where unrelated lines must still fail
- Added unit tests covering guard conditions when the run did not time out or did not print the `SIGTERM` marker
- Not run (not requested)

Reviewed By: archang19

Differential Revision: D100806204

Pulled By: xingbowang

fbshipit-source-id: 01248a371afae91bb43df55f88400baf155373f5
2026-04-14 10:02:10 -07:00
Xingbo Wang 50852b5c8d db_stress: document expected-state trace/replay contract (#14612)
Summary:
- add a `docs/components/` landing page and a stress-test docs index
- document the `db_stress` expected-state trace/replay lifecycle, file invariants, and prefix-recovery contract in `expected_state_trace.md`
- align `db_stress` comments with the restore semantics: missing trace entries are fatal, while extra tail trace entries are tolerated
- keep the new docs tree trackable and point repo instructions at the new component-docs entrypoint

## Testing
- Not run (documentation and comment updates only)

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14612

Reviewed By: joshkang97

Differential Revision: D100797173

Pulled By: xingbowang

fbshipit-source-id: 25be8c6239b9fdd84580818efe7520c371f9a46b
2026-04-14 09:58:20 -07:00
Peter Dillinger 9f474a1034 Enforce < 4GB sizes for keys, values, and some BlockBuilder inputs (#14461)
Summary:
BlockBuilder was somewhat inconsistent in its treatment of Slices whose size exceeds 4GB, which in a random corruption case could lead to (for example) serializing only the bottom 32 bits of a value size (uncorrupted) but appending the full multi-GB (corrupted) size. Because this is inner loop code, we don't want to pay CPU for extra conversions, data movements, or Status plumbing (already ruled out by Xingbo).

In this change we make BlockBuilder more internally consistent and lift the 32-bit size requirement to callers (of a few functions specifically, for now). To ensure that's satisfied, I've added additional checks near the perimeter of RocksDB to ensure keys and values do not exceed 4GB, plus an extra random corruption or backstop check in BlockBasedTableBuilder. In detail,

BlockBuilder (block_builder.cc/h):
* Add API comments documenting the < 4GB assumption on Add/AddWithLastKey
* Add debug assertions verifying input slice sizes < 4GB
* Simplify AddWithLastKeyImpl to use uint32_t locals, reducing static_cast
* Use only bottom 32 bits when appending derivative slices for consistency
* Update MaybeStripTimestampFromKey to also truncate to 32-bit size
* Add FIXME comments where buffer_/values_buffer_ sizes are truncated

BlockBasedTableBuilder (block_based_table_builder.cc):
* Add value size check (> uint32_t max) as a safety net against random corruption, since we have seen such corruptions in production

WriteBatch (write_batch.cc):
* Tighten existing key size checks to account for kNumInternalBytes (8-byte internal key suffix), using new kMaxWriteBatchKeySize constant
* Add missing size checks to Delete, SingleDelete, and DeleteRange (both Slice and SliceParts variants)

SstFileWriter (sst_file_writer.cc):
* Add key and value size checks in AddImpl and DeleteRangeImpl, which bypass WriteBatch and go directly to the table builder

MergeHelper (merge_helper.cc):
* Add merge result size check (> uint32_t max) in both overloads of TimedFullMergeImpl, returning Corruption if exceeded
* Add PartialMergeMulti result size check in MergeUntil

CompactionIterator (compaction_iterator.cc):
* Add size check on compaction filter output values (kChangeValue and kChangeWideColumnEntity), returning Corruption if > 4GB

meta_blocks.cc:
* Use Slice with uint32_t-truncated sizes in PropertyBlockBuilder::Finish to handle potentially oversized user property collector output

Needed follow-up:
* Check for blocks that are mis-encoded due to overflowing 32 bits for restart point offsets (and similar). See FIXME comments in block_builder.cc for why this is tricky.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14461

Test Plan:
New extreme-size unit tests, along with some testing infrastructure improvements:

Added test::HasBigMem() (in test_util/testharness.h) which returns true when the system has ≥128GB RAM (via sysconf(_SC_PHYS_PAGES) on Linux/macOS) or when ROCKSDB_BIGMEM_TESTS is set. All extreme-size tests use HasBigMem() to skip gracefully on smaller machines rather than being permanently disabled.

Where possible, tests use MemMapping::AllocateLazyZeroed() to supply large keys/values as Slices backed by anonymous mmap (cleaner with new MemMapping::AsSlice()). On Linux, read-only access to these pages maps to the shared kernel zero page, so the source data consumes no physical RAM — only the destination copy (e.g., WriteBatch::rep_) materializes, cutting peak memory roughly in half vs. std::string.

Tests (all enabled, skip via HasBigMem()):

./write_batch_test --gtest_filter='*LargeKeyValueSizeLimit*'
./external_sst_file_basic_test --gtest_filter='*LargeSizeSstFileWriter*'
./merge_test --gtest_filter='*LargeMergeResultRejected*'
./merge_helper_test --gtest_filter='*LargePartialMergeResultRejected*'
./db_compaction_test --gtest_filter='*CompactionFilterLargeValueRejected*'

All 5 pass (verified on a 128GB+ machine). On smaller machines, all 5 bypass cleanly with "insufficient memory for reliable continuous testing".

Reviewed By: xingbowang

Differential Revision: D96521899

Pulled By: pdillinger

fbshipit-source-id: 70f4b5e6a23ab074d60e653fbb7ddc5edbe162ab
2026-04-10 17:04:47 -07:00
Josh Kang f3a20854e0 Revert trace changes (#14600)
Summary:
Reverts the two commits https://github.com/facebook/rocksdb/pull/14578 https://github.com/facebook/rocksdb/pull/14575 that changed tracer behavior. The correct behavior is that trace should be written to before WAL.

Although a trace file may not be fully consumed, it is recycles after each crash, so there is no need to worry about applying an uncomited write. The original crash test was a false positive for a different error.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14600

Reviewed By: xingbowang

Differential Revision: D100397989

Pulled By: joshkang97

fbshipit-source-id: 3da6fb80f682ac6f9529c1a76eaf169e38cf2477
2026-04-10 16:04:37 -07:00
Xingbo Wang c47e35395b db: disable read-path tombstone conversion for partial UDT reads (#14595)
Summary:
T263957043 reproduces in whitebox crash testing when DBIter converts a run of point deletes into a range tombstone while reading with an older user-defined timestamp. The scan can observe an older delete for the boundary keys but miss newer live versions of the same keys and interior keys, so the synthesized tombstone is based on partial visibility and later hides valid max-timestamp reads.

## Problem

Read-path range tombstone conversion (`min_tombstones_for_range_conversion`) assumes the scan observes all interior live keys between the start and end of a contiguous delete run. With user-defined timestamps, a read at an older timestamp can see deletes but miss newer Puts for the same keys. The synthesized range tombstone then incorrectly covers those newer versions, causing data loss on subsequent max-timestamp reads.

## Fix

Gate read-path range conversion on full timestamp visibility. The optimization is now only enabled when:
1. There is no `table_filter` (existing guard — SSTs hidden by filter can break the contiguity assumption)
2. There is no `iter_start_ts` (time-travel scans see a subset of versions)
3. Either no read timestamp is set, or the read timestamp is the max timestamp (all `0xff` bytes)

This is done via a new helper `HasFullTimestampVisibility()` checked at `DBIter` construction time.

## Test Changes

- Updated all existing UDT test variants in `ReadPathRangeTombstoneTest` to read at max timestamp so the optimization remains covered where it is valid.
- Added `UDTOlderTimestampDisablesInsertion`: a regression test that writes at ts=1, deletes at ts=2, writes again at ts=3, then reads at ts=2. Verifies no range tombstone is synthesized and a subsequent max-timestamp read still sees all live values.

## Validation

- `make -j192 db_iterator_test`
- `./db_iterator_test --gtest_filter='*ReadPathRangeTombstoneTest*'`
- `./db_iterator_test --gtest_filter='*UDTOlderTimestampDisablesInsertion*' --gtest_repeat=5`
- Reran the original whitebox crash-test seed from T263957043 against the patched tree; crash-recovery verification passed

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14595

Reviewed By: joshkang97

Differential Revision: D100235999

Pulled By: xingbowang

fbshipit-source-id: 7a287f3c7fdc47428fda0ab89eedd5d43f663985
2026-04-10 10:47:35 -07:00
Xingbo Wang a9ac4df630 env: avoid unused info in TSAN mapping helper (#14597)
Summary:
TsanAnnotateMappedMemory() always builds a TsanMappedMemoryInfo payload
for TEST_SYNC_POINT_CALLBACK(), but in builds where sync points compile to a
no-op the local variable becomes unused and Clang fails with
-Werror,-Wunused-variable.

Move the explicit (void)info cast so it applies regardless of whether
__SANITIZE_THREAD__ is enabled. This keeps the helper warning-free in:
- non-TSAN builds
- builds where TEST_SYNC_POINT_CALLBACK expands to nothing
- TSAN builds that still want the SyncPoint payload for tests

This was missed by CI because the warning only appears in the
COMPILE_WITH_TSAN=1 + NDEBUG/release configuration. Our CI matrix covers
release builds without TSAN and TSAN builds in debug mode, but not the
TSAN+NDEBUG combination.

No behavior change is intended. The helper still:
- exposes the mapping metadata to SyncPoint-based tests
- calls AnnotateNewMemory() in TSAN builds
- remains a no-op with respect to runtime behavior in non-TSAN builds

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14597

Differential Revision: D100333855

Pulled By: xingbowang

fbshipit-source-id: 3ff0b0c8a51b4959b7a36984e40ac31e49da82eb
2026-04-10 08:43:03 -07:00
Xingbo Wang 4254c705a2 env: reset TSAN state for recycled mappings (#14594)
Summary:
TSAN was reporting false data-race warnings after the kernel reused a virtual address for a new mapping while TSAN still associated that address range with the old mapping.

This showed up in two RocksDB paths:
- `io_uring` setup on helper threads vs later `io_uring` `MultiRead` access
- `io_uring` setup vs file mmap reads when `use_mmap_reads` is enabled

## Changes

Introduce `TsanAnnotateMappedMemory()`, which calls `AnnotateNewMemory()` as soon as a fresh mapping exists so TSAN drops any stale shadow state for the recycled address range.

Apply the helper to:
- io_uring SQ/CQ/SQE mappings created by `CreateIOUring()`
- `PosixFileSystem` mmap-read mappings
- `PosixFileSystem` raw memory-mapped file buffers
- `PosixMmapFile` writable mmap region growth

Also add deterministic regression tests that force virtual-address reuse with `MAP_FIXED` for both the io_uring and mmap-read cases. The tests pass mapping metadata over a pipe so teardown/remap ordering is deterministic without introducing TSAN-visible synchronization that would mask the problem.

## Testing

- `COMPILE_WITH_TSAN=1 make -j192 env_test`
- `env_test --gtest_filter='EnvPosixTest.SupportedOpsNoAsyncIOOnIOUringInitFailure:EnvPosixTest.IOUringAddressReuseNoTsanFalsePositive:EnvPosixTest.MmapReadAddressReuseNoTsanFalsePositive'`

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14594

Reviewed By: archang19

Differential Revision: D100235746

Pulled By: xingbowang

fbshipit-source-id: 8ef8d85c08a7646540b9c11c3741978898a5af3d
2026-04-09 17:19:11 -07:00
Davis Rollman bad2d5b0af Migrate fbcode coro references from folly/experimental to folly/coro (5/6 - fbcode a-m)
Summary: Migrate coro references in remaining fbcode services (a-m).

Reviewed By: iahs

Differential Revision: D98841546

fbshipit-source-id: 83bb5a2ef6a29a1ff464efa0e4e0f76b48662ab0
2026-04-09 11:11:05 -07:00
xingbowang 1fe1b1bd46 Reduce claude review failure CI noise (#14591)
Summary:
### Summary

  Fix flaky Claude Code Review workflow failures caused by stale workflow_run SHAs.

  Problem:

  - The auto-review workflow is triggered from pr-jobs via workflow_run.
  - In some cases, by the time the review job starts, the PR branch head has already advanced.
  - Then Get PR info cannot find an open PR whose head SHA matches the older workflow_run.head_sha, so it fails with:
      - Could not find PR for SHA ... after retries
  - This creates unnecessary CI noise even though a newer review job will be triggered for the updated commit.

  Solution:

  - Treat “PR not found for this head SHA” as a stale run, not an error.
  - In Get PR info, replace workflow failure with:
      - a warning
      - skip=true
      - skip_reason=stale_sha:<sha>
  - Gate all later auto-review steps on steps.pr_info.outputs.skip != 'true'
  - Add a small logging step to report the skip reason

  Result:

  - stale auto-review runs exit successfully instead of failing
  - CI becomes less noisy
  - the newest commit still gets reviewed by the later workflow run

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14591

Reviewed By: mszeszko-meta

Differential Revision: D100190375

Pulled By: xingbowang

fbshipit-source-id: fa4a0479a10e953e52b3d99c2b483dae01d3e7fb
2026-04-09 11:01:12 -07:00
Peter Dillinger f3f7feba92 Document Flush(), Sync(), and Fsync() contracts; remove dead enum (#14590)
Summary:
Flush(), Sync(), and Fsync() on WritableFile and FSWritableFile were essentially undocumented. Added comprehensive API contracts covering durability guarantees (process crash vs. power failure), data readability after Flush(), the implication relationship (Sync implies Flush), and thread-safety (referencing IsSyncThreadSafe()). The contracts are written to be implementation-agnostic, avoiding assumptions about OS-level mechanisms, so they apply equally to local, remote, and virtual/wrapper filesystem implementations. Several such implementations have been audited for adherence.

Note that WritableFileWriter currently calls FSWritableFile::Flush() UNNECESSARILY for cases like writing SST files. This should be optimized away in follow-up assuming this interpretation of the contract is agreed upon.

Also removed the dead FileSystem::WriteLifeTimeHint enum, which was never referenced anywhere. The actively used enum is Env::WriteLifeTimeHint.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14590

Test Plan: Documentation-only change to public headers, plus removal of an unused enum with no references. No functional changes. Verified no compilation errors with existing tests.

Reviewed By: xingbowang

Differential Revision: D100183288

Pulled By: pdillinger

fbshipit-source-id: 7f1982c41e09fe39896dbc6cc328316be559ec4a
2026-04-09 10:45:50 -07:00
Peter Dillinger b7395b3fa4 Allow Compressor to override parallel_threads for SST building (#14580)
Summary:
Add Compressor::GetRecommendedParallelThreads() virtual method so that the compression subsystem can influence the parallel compression thread count used when building SST files. The base Compressor returns 0 (no opinion), while built-in compressors (via CompressorBase) return the parallel_threads value from their CompressionOptions. This gives CompressionManager a clean mechanism to override parallel_threads by customizing the CompressionOptions passed to GetCompressor() in its GetCompressorForSST() implementation.

The table builder now reads the thread count from the compressor after creation, rather than only from CompressionOptions directly. Hard structural constraints (partition_filters without decoupled mode, user_defined_index_factory) still force single-threaded compression regardless of the compressor's recommendation.

Also adds a sync point in MaybeStartParallelCompression for test observability, and compression_test coverage for the new functionality.

Bonus: extends CompressionManagerCustomCompression test to cover re-opening a DB at format_version=7 with a non-default compression manager that uses the built-in CompatibilityName ("BuiltinV2"). Verifies that data is readable after re-opening without the original manager, and that neither GetId() nor CompatibilityName() resolves to the custom manager via CreateFromString.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14580

Test Plan:
New unit tests in compression_test:
- GetRecommendedParallelThreads: verifies built-in compressors return the parallel_threads from their CompressionOptions for all supported compression types.
- CompressionManagerOverridesParallelThreads: end-to-end test with a custom CompressionManager that overrides parallel_threads from 1 to 4 in GetCompressorForSST, verified via sync point that parallel compression actually activates with the overridden thread count, plus data readback verification.

Existing parallel compression tests (DBCompressionTestMaybeParallel) continue to pass.

Reviewed By: xingbowang

Differential Revision: D99896343

Pulled By: pdillinger

fbshipit-source-id: 6b3a30856a78641714d33ec7ba2099f33533d3af
2026-04-08 21:22:52 -07:00
zaidoon 3b38fde293 Support UDI as primary index (#14547)
Summary:
Add `use_udi_as_primary_index` option to `BlockBasedTableOptions`. When
enabled, the UDI becomes the primary index — all reads (including
internal operations like compaction and VerifyChecksum) automatically
route through the UDI without needing `ReadOptions::table_index_factory`.

Both the standard binary search index and the UDI are always fully
built. The standard index serves as a safety fallback (e.g., for
backup/restore or rollback to a non-UDI configuration). A future
refactor will extract the index abstraction to allow skipping the
standard index build when the UDI is primary (see discussion below).

## Write path

- `UserDefinedIndexBuilderWrapper` always forwards `AddIndexEntry` and
  `OnKeyAdded` to both the internal standard builder and the UDI builder
- New `udi_is_primary_index` table property marks primary-mode SSTs
- Validates incompatible options at `DB::Open` and builder creation:
  partitioned index, partitioned filters, missing
  `user_defined_index_factory`

## Read path

- `UserDefinedIndexReaderWrapper` defaults to UDI when `udi_is_primary_`,
  even when `ReadOptions::table_index_factory` is null — this handles
  the 15+ internal call sites that don't set `table_index_factory`
- `use_udi_as_primary_index` automatically enforces `fail_if_no_udi_on_open`
  to prevent silent data loss if SSTs are opened without UDI support

## Rollback

Since the standard index is always fully populated, rollback from
primary mode is straightforward: set `use_udi_as_primary_index=false`.
No compaction required — SSTs written in primary mode are immediately
readable through the standard index.

## Public API

- `BlockBasedTableOptions::use_udi_as_primary_index` (default: false)
- `UserDefinedIndexBuilder::EstimatedSize()` — pure virtual, O(1) via
  running counter in the trie implementation

## Bug fixes (issues https://github.com/facebook/rocksdb/issues/14560, https://github.com/facebook/rocksdb/issues/14561, https://github.com/facebook/rocksdb/issues/14562)

Fixed trie index correctness bugs that caused crash test failures:

- **Always-on seqno encoding**: `must_use_separator_with_seq_` is now
  unconditionally true. Non-boundary separators store tag=0 (sentinel),
  same-user-key boundaries store the real tag, and the last block stores
  its real last-key tag. This fixes the `NonBoundaryTag` bug where
  non-boundary separators with `kMaxSequenceNumber` caused the post-seek
  correction to incorrectly advance past the correct block.
- **Standard index always built in primary mode**: An empty (stub)
  standard index block caused behavioral divergence in
  `BlockBasedTableIterator` under concurrent flush/compaction, leading to
  `test_batches_snapshots` prefix scan inconsistencies.

## Stress test

- `use_udi_as_primary_index` flag randomized by `db_crashtest.py`
- Both primary and secondary UDI modes exercised in crash tests
- Trie index probability halved (~6%) per reviewer request
- Re-enables trie crash tests (disabled by https://github.com/facebook/rocksdb/issues/14559)

## Tests

- Parameterized `TrieIndexDBTest` on UDI mode (secondary vs primary)
- New factory-level tests: non-boundary separator seek, Prev within
  overflow, SeekToLast with overflow, empty trie, overflow exhaustion,
  all-scans-exhausted
- New DB-level tests: multi-CF coalescing iterator, GetEntity with
  explicit snapshot, reverse iteration across same-user-key blocks,
  non-boundary separator seek correctness, rollback from primary

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14547

Reviewed By: anand1976

Differential Revision: D99494181

Pulled By: xingbowang

fbshipit-source-id: ca52a0d5c0e523770c80e1fe2b9b5d50406b67bc
2026-04-08 21:05:03 -07:00
Xingbo Wang 3070f73e97 Wide-column blob separation: lazy resolution through read, compaction, and write paths (#14386)
Summary:
Wide-column blob separation: lazy resolution through read, compaction, and write paths

Extend blob direct write to support wide-column entities (PutEntity), and add
lazy blob resolution for wide-column values across all read and compaction paths.

**Write path -- PutEntity blob separation:**
- BlobWriteBatchTransformer::PutEntityCF now extracts large column values
  (>= min_blob_size) to blob files and serializes V2 entities with BlobIndex
  references, matching the existing Put behavior.
- Add MaybePreprocessWideColumns() static helper to share blob extraction
  logic between the WriteBatch transformer and the new PutEntity fast path.
- Add PutEntityFastPath() in DBImpl that preprocesses columns (sort, blob
  extract, serialize) before calling WriteImpl, skipping the redundant
  WriteBatch transformation pass. Trace batch preserves the original columns.

**Read path -- blob resolution for Get/MultiGet/Iterator:**
- GetContext::SaveValue resolves V2 entity blob columns eagerly: for
  value (Get), resolves the default column's blob reference; for columns
  (GetEntity), resolves all blob columns and re-serializes as V1.
- DBIter::SetValueAndColumnsFromEntity detects V2 entities, deserializes
  with DeserializeV2, and eagerly resolves all blob columns via a new
  ReadPathBlobResolver. Resolved values are cached in the resolver and
  wide_columns_ Slices point into the cache, avoiding copies.
- Add ReadPathBlobResolver (new file) -- on-demand blob fetcher for the
  read path with per-column caching, used by both DBIter and GetContext.
- BlobFetcher gains allow_write_path_fallback to read from in-flight
  direct-write blob files not yet visible through Version (pre-flush reads).
- Memtable lookups for Get(key) on V2 entities with a blob default column
  now return the blob index with is_blob_index=true, triggering the
  existing BDW resolution in MaybeResolveWritePathValue.
- MaybeResolveWritePathValue (renamed from MaybeResolveDirectWriteBlobIndex)
  now also resolves V2 entity blob columns for GetEntity/MultiGetEntity,
  re-serializing as V1 after resolution.

**Compaction path -- filter, GC, and extraction:**
- CompactionIterator::InvokeFilterIfNeeded handles V2 entities: FilterV3
  gets eagerly-resolved column values for backward compatibility; FilterV4
  gets a CompactionBlobResolver for lazy on-demand resolution.
- Add CompactionFilter::FilterV4 with WideColumnBlobResolver* parameter
  and SupportsFilterV4() opt-in. Default delegates to FilterV3.
- CompactionBlobResolver (new class) implements WideColumnBlobResolver
  for the compaction path with stats tracking.
- ExtractLargeColumnValuesIfNeeded extracts inline columns to blob files
  during compaction (entities without existing blob columns only).
- GarbageCollectEntityBlobsIfNeeded relocates blob values from old blob
  files to new ones during compaction GC, with helpers FetchBlobsNeedingGC,
  RelocateBlobValues, and SerializeEntityAfterGC.
- PrepareOutput unified entity deserialization: single DeserializeV2 call
  reused by both filter and GC/extraction paths via entity_deserialized_
  flag, avoiding redundant parsing.

**Merge path -- V2 entity base value resolution:**
- MergeHelper::MergeUntil, GetContext::MergeWithWideColumnBaseValue, and
  DBIter::MergeWithWideColumnBaseValue resolve V2 blob columns before
  calling TimedFullMerge, using ResolveEntityForMerge.

**Blob garbage accounting:**
- BlobGarbageMeter tracks blob file in/out flow for V2 entity blob
  columns via ForEachBlobFileNumber, used for accurate GC decisions.
- FileMetaData::UpdateBoundaries tracks oldest_blob_file_number for
  V2 entities, ensuring blob files referenced by entities are not
  prematurely deleted.

**Serialization improvements:**
- WideColumnSerialization::SerializeV2Impl allocates serialized_blob_indices
  only for actual blob columns (not all columns) and uses autovector for
  name/value sizes.
- Add ForEachBlobFileNumber for lightweight blob file number extraction
  without full deserialization.
- Add ResolveEntityForMerge helper for merge-path resolution.
- Add section-size validation in DeserializeV2Impl.
- Add empty blob index and column type validation.
- blob_column_resolver_util.h -- shared helpers (FindBlobColumn, FindInCache,
  CacheInlinedBlob) used by both ReadPathBlobResolver and CompactionBlobResolver.

**Testing:**
- db_blob_direct_write_test: end-to-end PutEntity with BDW before/after flush,
  verifying Get, GetEntity, MultiGetEntity, and Iterator.
- db_blob_index_test: ~1550 lines covering V2 entity blob resolution through
  Get, GetEntity, MultiGet, Iterator, compaction filter (V3 compat and V4 lazy),
  merge with blob base, and compaction GC/extraction.
- compaction_iterator_test: ~950 lines testing entity blob GC, extraction,
  filter interaction, and combined GC+filter scenarios.
- db_wide_basic_test: ~1200 lines for wide-column lazy blob resolution through
  all read paths plus compaction round-trips.
- db_open_with_config_test: ~450 lines for BDW entity config validation.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14386

Reviewed By: anand1976

Differential Revision: D99739701

Pulled By: xingbowang

fbshipit-source-id: 6badd89b577f3054802eaaa654738468efb9dbdb
2026-04-08 18:58:40 -07:00
Anand Ananthabhotla 9d609dd6fe Fix MultiScanIndexIterator crash on reseek after exhaustion (#14581)
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14581

In `MultiScanIndexIterator::Seek()` Case 3, when re-entering a scan range
after all ranges were exhausted, `block_idx = std::max(cur_scan_start_idx,
cur_idx_)` could produce an out-of-bounds value because `cur_idx_` was left
at `block_handles_.size()` from previous exhaustion. `SeekToBlockIdx()`
unconditionally set `valid_ = true` without checking bounds, causing the
subsequent `value()` call to hit the assertion
`cur_idx_ < block_handles_.size()`.

Added bounds check before `SeekToBlockIdx()` in Case 3 to correctly report
exhaustion instead of crashing.

Reviewed By: joshkang97

Differential Revision: D99604049

fbshipit-source-id: 9d5d91afde7c0984a7b4c2f62604f27f19b07922
2026-04-08 15:25:02 -07:00
Xingbo Wang 4d8a0acd76 Disable read-path range tombstone synthesis when ReadOptions.table_filter is set (#14586)
Summary:
**Root cause:** DBIter's read-path range tombstone conversion assumes the iterator can observe *every* interior live key between point tombstones. When `ReadOptions.table_filter` is active, whole SSTs can be hidden from the scan. DBIter may then mistake non-contiguous deletes for a contiguous run and synthesize a range tombstone that covers real, still-live keys — silently corrupting the read path.

This is not UDT-specific; UDT just changes the failure surface. The stress test failures that triggered investigation were from `db_stress_tool/no_batched_ops_stress.cc` applying an SQFC-backed `table_filter` for range queries even when `total_order_seek` is forced.

## Fix

`db/db_iter.cc`: When initializing `min_tombstones_for_range_conversion_`, set it to 0 (disabling range conversion) whenever `read_options.table_filter` is non-null. This is the conservative and correct behavior — filtered scans don't provide the full key visibility the optimization requires.

## Test

`db/db_iterator_test.cc` — new test `ReadPathRangeTombstoneTest.TableFilterHiddenInteriorKey`:

- Constructs five flushed SSTs so that the live interior key `"b"` lives in a 2-entry SST that the `table_filter` hides.
- Keeps the active memtable non-empty (key `"zz"`) so range conversion has a valid insertion point — without this the test is a false negative.
- Asserts `inserted_ranges_.size() == 0` (no synthesis occurred).
- Verifies `"b"` is still readable via a normal `Get`.
- Runs both with and without UDT (user-defined timestamps) via `SCOPED_TRACE`.

On unpatched HEAD this test fails with `inserted_ranges_.size() == 1`; with the fix it passes.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14586

Reviewed By: joshkang97

Differential Revision: D100023487

Pulled By: xingbowang

fbshipit-source-id: 5ef885c89a6cfd7a97b814e38bdcc48d3a1ab349
2026-04-08 10:28:02 -07:00
Josh Kang 3efe9460ba Fix Range Tombstone Entry Accounting Bug (#14579)
Summary:
Crash test T263619547 exposed a flush input accounting bug in
`FragmentedRangeTombstoneList`.

When the constructor sees that the incoming range tombstones are not sorted, it
falls back to rereading all tombstones into `keys`/`values` and sorting them via
`VectorIterator` before fragmentation. However, `num_unfragmented_tombstones_`
was left with the partial count from the aborted first pass. In timestamp
stripping flushes, this stale count could make flush verification report
`Expected X entries in memtables, but read Y` even though all tombstones were
still processed.

Fix the slow path by resetting `num_unfragmented_tombstones_` from
`keys.size()` after the second pass. Add a regression test that feeds unsorted
range tombstones into the fragmenter and verifies the full tombstone count is
preserved.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14579

Test Plan: - New unit test that fails without the fix.

Reviewed By: xingbowang

Differential Revision: D99872374

Pulled By: joshkang97

fbshipit-source-id: 38e2bb73c68dc1c677870b8973c93b933ded9038
2026-04-07 16:34:24 -07:00
Xingbo Wang 4dd05023c0 Rocksdb Crash Test failed: unknown (#14584)
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/14584

Reviewed By: mszeszko-meta

Differential Revision: D98721280

fbshipit-source-id: 0a0f0da678312424b43dfab449e636cb1782d9cf
2026-04-07 16:32:33 -07:00
Maciej Szeszko 6a618d0686 Rocksdb Warm Storage Test failed: assertion failed - iters[i]->status().ok() (#14583)
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/14583

Reviewed By: joshkang97, xingbowang

Differential Revision: D99576714

fbshipit-source-id: 71f6cccfc2c5f9043e656e6a81e794d555919fe6
2026-04-07 16:28:02 -07:00
Josh Kang ec832ff42c Minimize WAL and and tracer gap (#14578)
Summary:
https://github.com/facebook/rocksdb/pull/14575 addresses an existing issue where trace record before WAL lead to inconsistent expected state.

However, placing the trace record after the WAL is still problematic because. The trace may not have all the records in the WAL, resulting in `Error restoring historical expected values -> Trace ended before replaying all expected write ops` on recovery.

This change minimizes the gap to reduce stress test failures, but it is still not a full solution. It is however still better than previously writing the trace before the WAL because the error is much more obvious when writing the trace after WAL.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14578

Test Plan: Existing CI passes.

Reviewed By: xingbowang

Differential Revision: D99870545

Pulled By: joshkang97

fbshipit-source-id: 024ee3b15e258afd2dd8ff98db7cbe6a01906be7
2026-04-07 14:05:21 -07:00
Josh Kang 91fa765f3a Rocksdb Crash Test failed: assertion failed - corruption: Sequence number is being set backwards dur (#14567) (#14567)
Summary:
Handle the case where a transaction commit with `commit_bypass_memtable` succeeds in writing the commit marker to WAL, but then `IngestWBWIAsMemtable` fails (e.g., WAL creation failure during `SwitchMemtable` or invalid column family). Previously this failure path was not distinguished from a direct WBWI ingest, so the DB could continue operating with committed data durable in WAL but not published to memtables.

This resulted in a higher seqno in the WAL that what the DB's latest seqno was.

Now `IngestWBWIAsMemtable` takes an `ingest_wbwi_for_commit` parameter. When true and the ingest fails (without a prior memtable update), the DB sets a fatal background error, stopping all writes until the DB is closed and reopened for WAL recovery. A close and reopen is required because the auto-recovery mechanism works by flushing memtables to disk, but here the committed data was never published to memtables in the first place — it only exists as prepare + commit records in the WAL. Only a full WAL replay during `DB::Open` can reconstruct the committed transaction data and insert it into memtables.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14567

Test Plan:
- Added `CommitBypassMemtableTest.SwitchMemtableFailureStopsDBUntilReopen`:
  - Injects an IO error during `SwitchMemtable` WAL creation via `TEST_SYNC_POINT_CALLBACK`
  - Verifies commit returns `Corruption` status
  - Verifies DB enters fatal background error state and rejects further writes
  - Verifies that after close and reopen, committed data is recovered from WAL and the DB resumes normal operation

Reviewed By: xingbowang

Differential Revision: D98638705

Pulled By: joshkang97

fbshipit-source-id: f116b1f257b19984b0413f6aeba0ba4a7c0a5b25
2026-04-06 19:47:36 -07:00
Peter Dillinger 92cedd58af Fix table cache leak when flush install fails (#14577)
Summary:
When BuildTable succeeds during flush, it caches the output SST file in the table cache for subsequent user reads (builder.cc:460). If the flush then fails to install — e.g., LogAndApply MANIFEST I/O error, CF dropped, or shutdown — the cache entry was never evicted. BuildTable only cleans up its own failures (builder.cc:511), not failures in the install path.

The FindObsoleteFiles full-scan backstop would normally catch this by finding the orphan file on disk and evicting the cache entry. However, under crash test metadata read fault injection
(open_metadata_read_fault_one_in), GetChildren fails and the orphan is never found, causing the TEST_VerifyNoObsoleteFilesCached assertion to fire during Close().

This is the same class of bug previously fixed in the compaction path by https://github.com/facebook/rocksdb/issues/14469 (Run failure) and https://github.com/facebook/rocksdb/issues/14549 (Install failure), but in the flush path which had no analogous cleanup.

Fix: call TableCache::ReleaseObsolete in FlushJob::Run() when the flush fails after BuildTable succeeded (meta_.fd.GetFileSize() > 0).

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14577

Test Plan:
New unit test DBFlushTest.LeakedTableCacheEntryOnFlushInstallFailure:
- Injects failure after BuildTable via sync point, deactivates filesystem to prevent backstop cleanup
- Without fix: assertion fires ("Leaked table cache entry")
- With fix: passes
```
COMPILE_WITH_ASAN=1 make -j db_flush_test
./db_flush_test --gtest_filter="DBFlushTest.LeakedTableCacheEntry*"
[  PASSED  ] 1 test.
```

Existing compaction leak tests still pass:
```
COMPILE_WITH_ASAN=1 make -j db_compaction_test
./db_compaction_test --gtest_filter="*LeakedTableCacheEntry*"
[  PASSED  ] 2 tests.
```

Reviewed By: xingbowang

Differential Revision: D99692601

Pulled By: pdillinger

fbshipit-source-id: ff5aa1ad165b3abec915844f97f6af9e59d85774
2026-04-06 15:21:58 -07:00
Xingbo Wang 3dd6d060e5 Upgrade CI Docker images: clang-18→clang-21, ubuntu24 24.0→24.1 (#14576)
Summary:
Upgrade the Ubuntu 24 CI Docker image to include clang-21 (from clang-18), and bump all workflow references to the new image tags. Also adds ccache to both images and renames all clang-18 job references to clang-21.

This is mainly for upgrading the clang version later used for generating C API automatically. See PR https://github.com/facebook/rocksdb/issues/14572

### Changes

**`build_tools/ubuntu24_image/Dockerfile`**
- Add clang-21 installation from LLVM snapshot repo (`apt.llvm.org/noble/llvm-toolchain-noble-21`)
- Add ccache
- Add comment pointing Meta employees to internal devvm build guide

**`build_tools/ubuntu22_image/Dockerfile`**
- Add ccache
- Add comment pointing Meta employees to internal devvm build guide

**`.github/workflows/pr-jobs.yml`**
- Rename `build-linux-clang-18-no_test_run` → `build-linux-clang-21-no_test_run`
- Rename `build-linux-clang18-asan-ubsan` → `build-linux-clang21-asan-ubsan`
- Rename `build-linux-clang18-mini-tsan` → `build-linux-clang21-mini-tsan`
- Update all `clang-18`/`clang++-18` references to `clang-21`/`clang++-21`
- Update ccache key prefixes: `clang18-asan-ubsan` → `clang21-asan-ubsan`, `clang18-tsan` → `clang21-tsan`
- Bump `rocksdb_ubuntu:24.0` → `rocksdb_ubuntu:24.1`

**`.github/workflows/nightly.yml`**
- Rename `build-linux-clang-18-asan-ubsan-with-folly` → `build-linux-clang-21-asan-ubsan-with-folly`
- Update clang-18 → clang-21 compiler references
- Bump `rocksdb_ubuntu:24.0` → `rocksdb_ubuntu:24.1`

**`.github/workflows/clang-tidy.yml`**
- Update clang-18 → clang-21
- Bump `rocksdb_ubuntu:24.0` → `rocksdb_ubuntu:24.1`

### New Docker Images

Both images have been built and tested locally. They will be pushed to `ghcr.io/facebook/rocksdb_ubuntu` before this PR is merged.

- `ghcr.io/facebook/rocksdb_ubuntu:22.2` — adds ccache, adds devvm build note
- `ghcr.io/facebook/rocksdb_ubuntu:24.1` — adds clang-21 from LLVM snapshot repo, adds ccache

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14576

Test Plan: Built both images locally and verified CI job names/compiler flags are consistent.

Reviewed By: joshkang97

Differential Revision: D99694293

Pulled By: xingbowang

fbshipit-source-id: c23c27f5cf870fb2c8b4e3d1cba281d0ce63f9d6
2026-04-06 15:00:53 -07:00
Josh Kang 1b817cff45 Trace writes only after successful write (#14575)
Summary:
Fix trace ordering for `preserve_write_order` mode to only record writes that actually succeeded.

Previously, when `preserve_write_order = true`, trace records were emitted **before** confirming the write succeeded — at a point where the write could still fail (e.g., WAL I/O error). This meant the trace could contain records for writes that never actually committed to the DB. When `db_stress` crash testing replays the trace to reconstruct expected state, these phantom records cause the expected state to diverge from reality, leading to false verification failures.

This fix moves the `preserve_write_order` tracing to happen **after** the write is confirmed successful (after memtable insert, right before `SetLastSequence`), in all three write paths: `WriteImpl`, `PipelinedWriteImpl`, and `WriteImplWALOnly`.
default and pipelined write modes. Replays the trace into a fresh DB and confirms only the successful write is present.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14575

Test Plan:
- New unit test: `TracePreserveWriteOrderSkipsFailedWrite` in `db/db_test2.cc`
  - Tests both `enable_pipelined_write = false` and `true`
  - Uses `FaultInjectionTestEnv` to force a write failure
  - Verifies the failed write is absent from both the original DB and the replayed trace

Reviewed By: xingbowang

Differential Revision: D99624672

Pulled By: joshkang97

fbshipit-source-id: 5793727c91cb01a12fbb4b2cd59615802ae3d394
2026-04-06 14:50:51 -07:00
Anand Ananthabhotla fc85a700cf Fix memory accounting leak in IODispatcher ReadIndex() (#14569)
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14569

ReadSet::ReadIndex() moves block values out of pinned_blocks_ via std::move,
but never releases the associated prefetch memory accounting. This causes
ReleaseBlock() and the destructor to skip ReleaseMemory() since they check
pinned_blocks_.GetValue() which returns null after the move. Over time, the
memory budget is exhausted and no further prefetches can be dispatched when
max_prefetch_memory_bytes is set. The bug was introduced in https://github.com/facebook/rocksdb/pull/14401.

The fix releases memory accounting in ReadIndex() when moving values out
(both for Case 1: block already available, and Case 2: after async IO
polling), and zeros block_sizes_ to prevent double-release.

Also adds multiscan_max_prefetch_memory_bytes option to db_stress/crashtest
for stress testing this code path.

Reviewed By: hx235

Differential Revision: D99488961

fbshipit-source-id: 5ddd1f50e2f6ebb357f86e013d781a790e7e558a
2026-04-06 12:35:18 -07:00
Anand Ananthabhotla 06699cd0fb Add IODispatcher memory limiting and multiscanrandom benchmark to db_bench (#14570)
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14570

Add support for benchmarking MultiScan with IODispatcher memory limiting in db_bench. This includes:

1. **New flags for existing `multiscan` benchmark:**
   - `--io_dispatcher_max_prefetch_memory_bytes`: Sets the global memory budget for IODispatcher prefetching across all ReadSets. When this limit is reached, `SubmitJob()` blocks until memory is released. 0 means unlimited (no IODispatcher created).
   - `--multiscan_max_prefetch_size`: Sets the per-file prefetch size limit (`MultiScanArgs::max_prefetch_size`).

2. **New `multiscanrandom` benchmark with randomized workload:**
   - Random batch sizes (1-64 ranges per MultiScan call)
   - Random range sizes (up to ~1MB worth of keys per range)
   - Generates sorted, non-overlapping ranges each iteration
   - Supports all IODispatcher flags above
   - `--use_multiscan` flag: when true (default), uses MultiScan API; when false, uses normal iterators (Seek + Next) for A/B comparison

## Benchmark Results

Setup: 5M keys, key=16B, value=256B, 5-level LSM (267 SSTs), direct reads, 8MB block cache, 30s duration.

### Normal Iterators vs MultiScan (`multiscanrandom`)

```
| Mode                                    | ops/sec |
|-----------------------------------------|---------|
| Normal iterators (use_multiscan=false)  |       6 |
| MultiScan sync (async=off)              |      15 |
| MultiScan async (async=on)              |      25 |
```

MultiScan sync is **2.5x** faster than normal iterators. MultiScan async is **4.2x** faster.

### MultiScan async=off with Memory Limits (P=4MB peak concurrent memory)

```
| Config          | Limit  | ops/sec | blocked |
|-----------------|--------|---------|---------|
| Baseline (10GB) | unlim  |      15 |       0 |
| Fits budget     | 4MB    |      15 |     246 |
| Exceeds 10%     | 3.6MB  |      14 |     698 |
| Exceeds 50%     | 2.67MB |      14 |   1,314 |
| Exceeds 100%    | 2MB    |      15 |  10,724 |
```

Sync IO is disk-bound so memory limiting has negligible throughput impact, even at 2x oversubscription.

### MultiScan async=on with Memory Limits (P=35MB peak concurrent memory)

```
| Config          | Limit  | ops/sec | blocked |
|-----------------|--------|---------|---------|
| Baseline (10GB) | unlim  |      25 |       0 |
| Fits budget     | 35MB   |      24 | 204,525 |
| Exceeds 10%     | 31.8MB |      19 | 361,444 |
| Exceeds 50%     | 23.3MB |      14 | 289,495 |
| Exceeds 100%    | 17.5MB |      19 | 464,878 |
```

Async IO uses ~9x more peak concurrent memory (35MB vs 4MB) due to multiple in-flight reads. At ~50% oversubscription, async throughput degrades to sync-level, negating the async benefit.

Reviewed By: xingbowang

Differential Revision: D99489650

fbshipit-source-id: e88d3a980f592e4e86628604e6389a1f2f71cfef
2026-04-06 11:59:33 -07:00
Peter Dillinger 0186937710 Skip AllocateTest block-count checks on btrfs (#14553)
Summary:
btrfs accepts fallocate without error but uses copy-on-write, so preallocated extents are not reflected in st_blocks. This caused AllocateTest to fail spuriously on btrfs filesystems. Detect btrfs via statfs and skip only the block-count assertions, keeping the rest of the test (write, flush, close, size checks) intact.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14553

Test Plan: env_test AllocateTest passes on btrfs (skips with message) and would still enforce preallocation checks on ext4.

Reviewed By: joshkang97

Differential Revision: D99307962

Pulled By: pdillinger

fbshipit-source-id: 8f40c0109aa397e9ca7d9ee4496fa0614e971970
2026-04-06 11:14:19 -07:00
Peter Dillinger 8ec2177bd1 Fix missing sync point in DeleteScheduler MarkAsTrash failure path (#14556)
Summary:
When MarkAsTrash fails (e.g., rename error due to filesystem conditions), AddFileToDeletionQueue falls back to deleting the file directly via fs_->DeleteFile(). This bypasses DeleteFileImmediately() and its TEST_SYNC_POINT("DeleteScheduler::DeleteFile") callback, causing tests that count deletions via sync points to undercount. This explains flaky failures in DBSSTTest.DeleteSchedulerMultipleDBPaths where bg_delete_file is 4 instead of the expected 8.

Fix by calling DeleteFileImmediately() in the error path instead of fs_->DeleteFile() directly. DeleteFileImmediately already handles the sync point, OnDeleteFile tracking, and FILES_DELETED_IMMEDIATELY stat, so the manual bookkeeping is also removed to avoid duplication.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14556

Test Plan:
- Existing tests: db_sst_test (DeleteSchedulerMultipleDBPaths, DestroyDBWithRateLimitedDelete) and delete_scheduler_test all pass.
- Ran DeleteSchedulerMultipleDBPaths 100x with COERCE_CONTEXT_SWITCH=1.

Reviewed By: joshkang97

Differential Revision: D99308177

Pulled By: pdillinger

fbshipit-source-id: 3918c4800c2420ff1ae667beb321da3f27406420
2026-04-06 11:13:20 -07:00
Josh Kang 156328107d Fix range deletion crash test errors (#14573)
Summary:
Fix two bugs in read-path range tombstone conversion (`min_tombstones_for_range_conversion`):

1. **SeekToLast stale saved_key_**: `SeekToLast()` did not clear `saved_key_` before calling `PrevInternal()`. A stale key from a prior `Seek()` would be swapped into `range_tomb_end_key_`, corrupting the tombstone tracking bounds and triggering an assertion failure when `range_tomb_first_key_ > end_key`.

2. **Tombstone inserted at wrong sequence**: The current behavior uses the "max" seqno of the tombstones as the insertion seqno. This is not correct because it does not take into account the seqno range deletions seen throughout the iteration. Unfortunately, range deletions are not visible to the db_iter as the are filtered at the merging iterator level. A future refactor may try to expose this, but currently the simplest solution is to just use the iterator seqno.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14573

Test Plan: Unit tests for both bugs that fail without the fix.

Reviewed By: xingbowang

Differential Revision: D99569668

Pulled By: joshkang97

fbshipit-source-id: bb1b154beccced7438831cd5cf2780ad1e11a4cc
2026-04-06 09:05:42 -07:00
Josh Kang 3f51c0a185 Convert sequential single deletes into range tombstones (#14448)
Summary:
Add a read-path optimization that converts contiguous point tombstones into range tombstones during forward/reverse iteration. When a configurable threshold of consecutive point deletions (kTypeDeletion, kTypeDeletionWithTimestamp, kTypeSingleDeletion — with no live keys between them) is detected, a range tombstone covering `[first_tombstone_key, next_live_key)` is inserted into the active mutable memtable. This benefits future iterators by enabling efficient skipping via range tombstone fragmentation.

If there is a memtable switch during the read iteration, then the range deletion entry is discarded.

The inserted range tombstones are logically redundant (they don't delete anything that isn't already deleted by point tombstones), skip WAL (they're a derived optimization regenerated by future reads on crash), and use the max tombstone sequence number so they don't interfere with newer writes.

## Key changes
- **New option `min_tombstones_for_range_conversion`** (`AdvancedColumnFamilyOptions`): Threshold of contiguous point tombstones before converting to a range tombstone. Default 0 (disabled). Dynamically changeable via `SetOptions()`.
- **`DBIter` tracking logic** (`db/db_iter.cc`): Tracks contiguous tombstones during `FindNextUserEntryInternal()` (forward) and `PrevInternal()` (reverse). When a live key terminates a run that meets the threshold, `MaybeInsertRangeTombstone()` inserts `[first_tombstone, live_key)` into the active memtable.
- **`FindValueForCurrentKey` `found_visible` output** (`db/db_iter.cc`): Distinguishes "key deleted at this snapshot" from "no visible entries" so reverse tracking doesn't treat post-snapshot keys as tombstones.
- **`IterKey::Swap()`** (`db/dbformat.h`): Efficiently tracks reverse tombstone run end keys without extra allocations.
- **`MemTable::AddLogicallyRedundantRangeTombstone()`** (`db/memtable.cc`): Concurrent-safe range tombstone insertion into the active memtable. Range tombstone skiplist always uses concurrent inserts.
- **`ConstructFragmentedRangeTombstones` race fix** (`db/db_impl/db_impl_write.cc`): Moved after `MarkImmutable()` to prevent lost entries.
- **`MarkImmutable` ordering fix** (`db/memtable_list.cc`): Called before `current_->Add()` to close a race window.
- **Prefix filter awareness** (`db/db_iter.cc`): Tombstone tracking scoped to the seek prefix when prefix filtering is active. See dedicated section below.
- **Transaction awareness** (`db/db_iter.cc`): Tombstones with `seq > snapshot` excluded from tracking. `min_uncommitted` guard uses `insert_seq` (which may be bumped to `earliest_seq`) instead of `range_tomb_max_seq_`. See dedicated section below.
- **Duplicate range check**: Skips insertion if the memtable already covers `[start, end)`.
- **New statistics**: `READ_PATH_RANGE_TOMBSTONES_INSERTED` and `READ_PATH_RANGE_TOMBSTONES_DISCARDED`.
- **Memtable MultiGet batch lookup** (`memtable/inlineskiplist.h`, `db/memtable.cc`): `InlineSkipList::MultiGet()` with cached search path ("finger") for sorted key lookups.
- **New option `memtable_batch_lookup_optimization`** (`AdvancedColumnFamilyOptions`): Enables batch lookup for memtable MultiGet. Default false. Immutable.

## Deciding Range Tombstone Seqno
- The range tombstone is inserted with `insert_seq = max(range_tomb_max_seq_, earliest_seq)` where `range_tomb_max_seq_` is the maximum sequence number across all point tombstones in the contiguous run, and `earliest_seq` is the memtable's earliest sequence number. This preserves the memtable's `earliest_seqno_` invariant.
- If the iterator's snapshot sequence (`sequence_`) predates the memtable's `earliest_seq`, insertion is skipped entirely to avoid unintentionally covering entries between `sequence_` and `earliest_seq`.

## ConstructFragmentedRangeTombstones Race Fix
- `MarkImmutable()` and `ConstructFragmentedRangeTombstones()` are now called before `mutex_.Lock()` in `SwitchMemtable`, keeping this work outside the DB mutex. `MarkImmutable()` blocks concurrent `AddLogicallyRedundantRangeTombstone()` calls via `immutable_mutex_`, ensuring no range tombstones are inserted after the fragmented list is built. `MarkImmutable()` is idempotent, so `MemTableList::Add()` calling it again inside the mutex is harmless.

## Prefix Filter Safety
- When prefix filtering is active, the BBTI bloom filter may reject SST files outside the seek prefix, but the memtable (no bloom filter) returns keys across prefix boundaries. Tombstone tracking is scoped to the seek prefix so that converted range tombstones cannot cover live keys hidden in filtered files.
- `total_order_seek=true` disables prefix filtering — all files are visible, so tombstones safely span prefix boundaries.

- **Behavior change**: Seeking to an out-of-domain key with `total_order_seek=false` now treats it as total-order (prefix_ not set). When `prefix_same_as_start=true`, iterating past an out-of-domain key cleanly invalidates the iterator instead of calling `Transform()` on it (which was UB in release builds with `FixedPrefixTransform`). This is a requirement because an incorrect iterator scan could lead to a range tombstone covering a live key.

## Transaction Support
- Tombstones written by the transaction's own uncommitted writes (sequence > snapshot) are now excluded from contiguous tombstone tracking entirely at the tracking site in `FindNextUserEntryInternal()` and `PrevInternal()`. Previously, tracking relied on a `min_uncommitted` check at insertion time, but this was insufficient — a transaction's own Delete with `seq > snapshot` could extend a run of committed tombstones, and the resulting range tombstone would cover data visible to other snapshots.
- The fix skips any tombstone with `ikey_.sequence > sequence_` during tracking. If a transaction-owned tombstone appears mid-run, it flushes the accumulated committed run first, then resets tracking. This ensures only tombstones visible to the current snapshot are ever converted.
- Both WritePrepared and WriteUnprepared transactions are supported with dedicated test coverage:
  - **WritePrepared**: When tombstones are committed before `Prepare()`, their seqnos are below `min_uncommitted` and insertion proceeds safely. When `Prepare()` happens first, tombstone seqnos exceed `min_uncommitted` and insertion is blocked.
  - **WriteUnprepared**: Multiple unprepared batches with different seqno ranges are handled correctly. Own transaction Deletes that extend a committed tombstone run block insertion of the entire run. After rollback, data correctness is verified.

## UDT support
- When user-defined timestamps (UDT) are enabled, keys include an 8-byte timestamp suffix. The comparator, Put/Delete APIs, and ReadOptions all require timestamps.
- Forward exhaustion with UDT: `iterate_upper_bound_` is a plain user key without a timestamp suffix. It is padded with min timestamp via `AppendKeyWithMinTimestamp()` so it sorts after all entries with this user key, preserving the exclusive bound semantics.
- Reverse exhaustion with UDT: The end key comes from either the previous live key (which already has a proper timestamp suffix) or the seek target set by `SetSavedKeyToSeekForPrevTarget()` (which appends a timestamp via `SetInternalKey(..., timestamp_ub_)` and `UpdateInternalKey(..., ts)`). In both cases, the end key is already properly timestamped, so no additional padding is needed unlike forward exhaustion.
- Contiguous tombstone detection works correctly with UDT because the underlying `kTypeDeletionWithTimestamp` entries are tracked the same way as `kTypeDeletion`.

## Concurrent Iterators
- Should concurrent iterators happen to read the range at the same time, both will produce the same range and seqno entry. Only one will be accepted by the skip list and the others will be rejected. Future iterators will read the range and not even attempt to insert the range.
- There is nothing preventing similar ranges from being inserted however. Two iterators can produce overlapping ranges, but this protection would be complicated to implement and there is no evidence that it is a likely scenario yet.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14448

Test Plan:
- **Unit tests** (`db/db_iterator_test.cc`): `ReadPathRangeTombstoneTest` parameterized by forward/reverse with cases for basic insertion, non-contiguous (below threshold), memtable switch, exhausted iterator with/without bounds, direction change, mixed Delete/SingleDelete, single-delete-only runs, snapshot predating memtable, block cache tier incomplete, skip when covered by existing range, UDT basic scan, UDT exhaustion, prefix filter cross-prefix scan (`PrefixFilterCrossPrefixScanCoversLiveKey` with default/total_order_seek/prefix_same_as_start variants), stale ikey from forward-then-reverse scan (`StaleIkeyFromForwardThenReverse`), and reseek stale ikey (`ReseekStaleIkey`).
- **Concurrency test** (`db/db_test2.cc`): `DBTestConcurrentRangeTombstoneConversions` parameterized by `(allow_concurrent_memtable_write, min_tombstones_for_range_conversion)` with mixed writers, deleters, range deleters, and concurrent forward/reverse readers.
- **Transaction tests** (`utilities/transactions/write_prepared_transaction_test.cc`, `write_unprepared_transaction_test.cc`): Tests for WritePrepared (insertion allowed when tombstones committed before prepare, blocked when after; seqno bump shadowing prepared writes `RangeTombstoneSeqnoBumpShadowsPreparedWrite`) and WriteUnprepared (multiple batches, extended visibility with CalcMaxVisibleSeq, own deletions with rollback).
- **IterKey::Swap tests** (`db/dbformat_test.cc`): `IterKeySwapTest` parameterized over `(key_len, copy, use_secondary)` × 2 covering all inline/heap/pinned/secondary combinations.
- **InlineSkipList MultiGet tests** (`memtable/inlineskiplist_test.cc`): Basic, exact matches, empty, single key, randomized validation against `std::set::lower_bound`, duplicate keys with callback walk, and concurrent MultiGet with read-after-write consistency.
- **Memtable MultiGet tests** (`db/db_basic_test.cc`): Batch lookup, overwrite, flush, merge, disabled by default, paranoid checks, and snapshot tests.
- **Stress test coverage**: `min_tombstones_for_range_conversion` and `memtable_batch_lookup_optimization` options added to `db_crashtest.py` and `db_stress` flags.
- `make check` passes all tests.

### Benchmark results

Tombstones scattered randomly in clusters via `seek_nexts_to_delete` for realistic workloads.

**DB Setup A (scattered deletes, 100 per seek)**:
```
# Step 1: Create and compact
./db_bench --benchmarks=fillseq,compact --seed=1 --compression_type=none --num=1000000 --db=<DB>
# Step 2: Scatter tombstones (5000 seeks × 100 deletes ≈ 500k tombstones)
./db_bench --benchmarks=seekrandom,flush --seed=1 --compression_type=none --num=2000 \
  --seek_nexts=0 --seek_nexts_to_delete=100 --use_existing_db=1 --threads=1 --db=<DB>
```

**DB Setup A2 (scattered deletes, 8 per seek)**:
```
# Step 1: Create and compact
./db_bench --benchmarks=fillseq,compact --seed=1 --compression_type=none --num=1000000 --db=<DB>
# Step 2: Scatter tombstones (5000 seeks × 8 deletes ≈ 40k tombstones)
./db_bench --benchmarks=seekrandom,flush --seed=1 --compression_type=none --num=2000 \
  --seek_nexts=0 --seek_nexts_to_delete=8 --use_existing_db=1 --threads=1 --db=<DB>
```

**DB Setup B (no deletes)**:
```
./db_bench --benchmarks=fillseq,compact --seed=1 --compression_type=none --num=1000000 \
  [--key_size=100] --db=<DB>
```

**Read workload** (same for all):
```
./db_bench --benchmarks=seekrandom --seek_nexts=100 --threads=8 \
  --reverse_iterator={true,false} --seed=1 --use_existing_db=1 \
  --compression_type=none --num=1000000 --duration=10 \
  --disable_auto_compactions \
  [--key_size=100] [--min_tombstones_for_range_conversion=X] --db=<DB_COPY>
```

Each workload averaged over 3 runs.

**Table 1: seekrandom forward, scattered deletes (2000 seeks × 100 deletes/seek)**

| Variant | avg ops/s | % vs main |
|---------|-----------|-----------|
| main | 2,895 | - |
| threshold=0 | 2,869 | -0.9% |
| threshold=8 | 287,334 | +9,824% |

**Table 2: seekrandom reverse, scattered deletes (2000 seeks × 100 deletes/seek)**

| Variant | avg ops/s | % vs main |
|---------|-----------|-----------|
| main | 544 | - |
| threshold=0 | 548 | +0.7% |
| threshold=8 | 206,491 | +37,860% |

**Table 3: seekrandom forward, scattered deletes (2000 seeks × 8 deletes/seek)**

| Variant | avg ops/s | % vs main |
|---------|-----------|-----------|
| main | 194,049 | - |
| threshold=0 | 195,703 | +0.9% |
| threshold=8 | 310,740 | +60.1% |

**Table 4: seekrandom reverse, scattered deletes (2000 seeks × 8 deletes/seek)**

| Variant | avg ops/s | % vs main |
|---------|-----------|-----------|
| main | 63,854 | - |
| threshold=0 | 69,266 | +8.5% |
| threshold=8 | 218,101 | +241.6% |

**Table 5: seekrandom forward, no deletes (regression check)**

| Variant | key=16B avg ops/s | % vs main | key=100B avg ops/s | % vs main |
|---------|-------------------|-----------|---------------------|-----------|
| main | 330,901 | - | 236,048 | - |
| threshold=0 | 328,398 | -0.8% | 238,055 | +0.9% |
| threshold=8 | 332,539 | +0.5% | 233,776 | -1.0% |

**Table 6: seekrandom reverse, no deletes (regression check)**

| Variant | key=16B avg ops/s | % vs main | key=100B avg ops/s | % vs main |
|---------|-------------------|-----------|---------------------|-----------|
| main | 261,445 | - | 192,177 | - |
| threshold=0 | 265,020 | +1.4% | 191,616 | -0.3% |
| threshold=8 | 250,881 | -4.0% | 189,239 | -1.5% |

Reviewed By: xingbowang

Differential Revision: D96203950

Pulled By: joshkang97

fbshipit-source-id: 06ba66ebde3c355f04671d1e681f1b1586e8751d
2026-04-03 22:47:51 -07:00
Jialiang Tan 65e77282ce Add block_decompress_count to PerfContext (#14557)
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14557

RocksDB's PerfContext tracks block_decompress_time but has no counter for the
number of block decompressions. The global Statistics ticker
NUMBER_BLOCK_DECOMPRESSED exists but is not accessible through PerfContext,
which is the thread-local, per-operation metric system used by benchmarks.

Add block_decompress_count as a new PerfContext counter (gated at
kEnableCount level) incremented in DecompressBlockData alongside the existing
NUMBER_BLOCK_DECOMPRESSED ticker. This enables benchmarks and applications to
observe how many blocks required decompression per operation, complementing
the existing block_read_count.

Reviewed By: anand1976

Differential Revision: D99233514

fbshipit-source-id: 40a68c1d9321f560cebdb7c30a544a0c62ae64f0
2026-04-03 16:48:09 -07:00
Jialiang Tan a0ff5b143f Add num_data_blocks_compression_rejected/bypassed to TableProperties (#14551)
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14551

Add two new per-SST TableProperties fields that track how many data blocks
were stored uncompressed:

- `num_data_blocks_compression_rejected`: blocks where compression was
  attempted but the compressed output exceeded the ratio limit set by
  `CompressionOptions::max_compressed_bytes_per_kb`.
- `num_data_blocks_compression_bypassed`: blocks where compression was
  never attempted (e.g., kNoCompression type, no compressor available).

Together with `num_data_blocks`, these give a full decomposition:
  num_data_blocks = compressed + rejected + bypassed

Previously this information was only available via global Statistics tickers
(NUMBER_BLOCK_COMPRESSION_REJECTED / BYPASSED) which aggregate across all
SSTs. Now it is persisted per-SST in the properties block and visible via
sst_dump --show_properties.

Reviewed By: anand1976

Differential Revision: D99229339

fbshipit-source-id: 8333f51acff30a759d725e5e94bd80f4fcb18b57
2026-04-03 16:48:09 -07:00
Rajat Jain 64f8d5765b Add blob_compression_opts to ColumnFamilyOptions (#14568)
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14568

Port of D98326754 to internal_repo_rocksdb (GitHub-synced repo).

BlobDB blob file compression currently uses hardcoded default
CompressionOptions{} in BlobFileBuilder, ignoring any user-specified
compression options like level, window_bits, or strategy. This means
ZSTD always uses level 3 (doubleFast strategy with two hash tables),
even when a lower level would significantly reduce CPU usage.

This diff adds a new `blob_compression_opts` field to
AdvancedColumnFamilyOptions (and MutableCFOptions) and plumbs it into
BlobFileBuilder::GetCompressor(). The option is registered as mutable
and dynamically changeable via SetOptions().

For example, setting level=1 with ZSTD switches from "doubleFast"
(two hash tables per thread) to "fast" (single hash table), reducing
L3 cache pressure on flush-heavy workloads.

There was an existing TODO in blob_file_builder.cc requesting exactly
this feature — this diff fulfills it.

This change is safe as the default value is always dFast. So this will not impact the current code.

Reviewed By: xingbowang

Differential Revision: D99478562

fbshipit-source-id: e79b847a854f60dea552442b18fd3f1384efac70
2026-04-03 16:42:16 -07:00
Xingbo Wang 4cf28450a5 Add customizable blob partition strategy for blob direct write (#14565)
Summary:
This PR adds a `BlobFilePartitionStrategy` interface that allows users to plug in custom partition selection logic for blob direct writes. The default behavior (round-robin across partitions) is preserved as the built-in `RoundRobinBlobFilePartitionStrategy`.

### Motivation

The existing blob direct write implementation distributes writes across partitions using a fixed round-robin strategy (atomic counter mod num_partitions). While this works well for uniform workloads, some use cases benefit from custom routing:

- **Key-based routing**: co-locate related keys in the same blob partition for locality-aware reads.
- **CF-based routing**: direct writes for different column families to dedicated partitions.
- **Value-size routing**: send large vs. small values to different partitions.
- **Application-defined affinity**: any domain-specific grouping that the default round-robin cannot express.

### Design

A new pure-virtual interface `BlobFilePartitionStrategy` is added to `include/rocksdb/advanced_options.h`:

```cpp
class BlobFilePartitionStrategy {
 public:
  virtual ~BlobFilePartitionStrategy() = default;

  // Select a partition for the given blob direct write.
  // The return value can be any uint32_t; the caller applies
  // modulo num_partitions internally.
  // Implementations must be thread-safe.
  virtual uint32_t SelectPartition(uint32_t num_partitions,
                                   uint32_t column_family_id,
                                   const Slice& key,
                                   const Slice& value) = 0;
};
```

The strategy receives:
- `num_partitions`: the configured partition count (for implementations that want to be partition-count-aware).
- `column_family_id`: useful for CF-based routing.
- `key` / `value`: the key and value being written.

The return value is taken modulo `num_partitions` internally, so implementations can return any `uint32_t` without worrying about bounds.

A new column family option `blob_direct_write_partition_strategy` (type `std::shared_ptr<BlobFilePartitionStrategy>`, default `nullptr`) wires the strategy into `BlobFilePartitionManager`. When `nullptr`, the built-in `RoundRobinBlobFilePartitionStrategy` is used, preserving existing behavior exactly.

### Changes

| File | Change |
|---|---|
| `include/rocksdb/advanced_options.h` | New `BlobFilePartitionStrategy` interface; new `blob_direct_write_partition_strategy` option |
| `db/blob/blob_file_partition_manager.h/.cc` | Accept strategy in constructor; replace atomic counter with strategy call; move `RoundRobinBlobFilePartitionStrategy` here as default |
| `options/cf_options.h/.cc` | Thread strategy through `ImmutableCFOptions` |
| `options/options_helper.cc` | Propagate strategy in `UpdateColumnFamilyOptions` |
| `options/options_settable_test.cc` | Register new option field in options settability test |
| `db/db_impl/db_impl.cc` | Pass strategy when constructing `BlobFilePartitionManager` |
| `db/blob/db_blob_direct_write_test.cc` | New test with `FixedBlobDirectWritePartitionStrategy` |

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14565

Test Plan:
New test `DirectWriteCustomPartitionStrategyRoutesWritesToOneBlobFile` verifies that a `FixedBlobDirectWritePartitionStrategy` that always returns partition 3 correctly routes all writes to a single blob file across 4 configured partitions.

```
make -j128 db_blob_direct_write_test && ./db_blob_direct_write_test --gtest_filter='*CustomPartition*'
make -j128 db_blob_direct_write_test && ./db_blob_direct_write_test
```

Reviewed By: joshkang97

Differential Revision: D99458813

Pulled By: xingbowang

fbshipit-source-id: 54cfbb0f75e24b58a8db61ad8755204d0db6ece7
2026-04-03 15:27:57 -07:00
Peter Dillinger ae0c9faaef Fix transaction lock timeout in crash test (#14540)
Summary:
When BatchedOpsStressTest runs on a TransactionDB, db_->Write() creates internal transactions using default_lock_timeout (1 second), which is too short under heavy contention with many threads. This causes sporadic "Timeout waiting to lock key" errors printed to stderr, which the crash test framework treats as fatal failures.

Increase default_lock_timeout to 600000ms (10 minutes) to match the lock_timeout already used for explicit transactions in NewTxn().

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14540

Test Plan: Manual `make crash_test_with_wc_txn` as a sanity check

Reviewed By: mszeszko-meta

Differential Revision: D98944474

Pulled By: pdillinger

fbshipit-source-id: 13b9e13fddc6332da010b01f7c41a51748f75624
2026-04-03 12:10:18 -07:00
Omkar Gawde 51561fc2cd Add explicit Close() call in WriteStringToFile (#14566)
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14566

**What**: Adds an explicit `file->Close()` call in `WriteStringToFile()` before
the function returns, instead of relying on the `unique_ptr` destructor to close
the file.

**How**: In `file_system.cc`, after the optional `Sync()` and before the
error-cleanup path, calls `file->Close()`. If `Close()` fails, the file is
deleted just like any other write failure. Adds two unit tests in `env_test.cc`:
- `WriteStringToFileClosesFile`: uses `CountedFileSystem` to verify `Close()` is
  called and content is written correctly.
- `WriteStringToFileCloseFailureDeletesFile`: uses a custom `CloseFailFS`
  wrapper to inject a `Close()` failure and verify the file is deleted on error.

**Why**: The previous implementation never called `Close()` explicitly — it
relied on the `unique_ptr` destructor which may silently swallow write errors
that occur during close (e.g., flushing buffered data). Explicit `Close()`
ensures such errors are detected and propagated to the caller.

Reviewed By: archang19

Differential Revision: D99462173

fbshipit-source-id: 525b5489bd0dbfc3159b007a13f9474f84d3c84e
2026-04-03 12:10:15 -07:00
Xingbo Wang ebd118711c Temporarily disable trie UDI index in stress test (#14559)
Summary:
**Summary:**
Disable `use_trie_index` in `db_crashtest.py` to avoid trie UDI stress test failures.

The default param was previously set to randomly enable trie index ~12.5% of the time (`random.choice([0, 0, 0, 0, 0, 0, 0, 1])`). Setting it to `0` until the underlying issues are resolved.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14559

Test Plan: Stress test runs without trie UDI failures.

Reviewed By: hx235

Differential Revision: D99343747

Pulled By: xingbowang

fbshipit-source-id: 6f86b5dd5ddb2869d797e93ddbda521981614bae
2026-04-02 15:01:36 -07:00
Hui Xiao ffe58fa976 Fix NULL persisted_seqno_ in AnonExpectedState (#14523) (#14523)
Summary:
AnonExpectedState::Open() never initialized the base class
persisted_seqno_ pointer, leaving it as nullptr. Any call to
SetPersistedSeqno() or GetPersistedSeqno() on an AnonExpectedState
(used when expected_values_dir is empty) would dereference a null
pointer. Fix by allocating and assigning it in Open(), matching
FileExpectedState's pattern.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14523

Reviewed By: mszeszko-meta

Differential Revision: D98556119

Pulled By: hx235

fbshipit-source-id: c4c3103272735d36c1d05b96f7d009b6b750bf53
2026-04-02 13:30:10 -07:00
Xingbo Wang d6b03d9c96 Add pre-push git hook to auto-format code (#14558)
Summary:
Add a git pre-push hook that automatically runs `make format-auto` before every push. If formatting issues are found, the hook fixes them, creates a new commit, and retries the push — all in one shot.

The hook is auto-configured on the first `make` build via `core.hooksPath`, so there is no manual install step needed.

## Changes

**`githooks/pre-push`** — pre-push hook that:
1. Runs `make check-format` to detect formatting issues
2. If issues found, runs `make format-auto` to fix them
3. Creates a new "format" commit with the fixes
4. Retries the push automatically with the format commit included
5. Stashes/restores any uncommitted work so it is not affected
6. Skips in non-interactive environments (CI) unless `ROCKSDB_FORMAT_HOOK` is set
7. Can be skipped with `git push --no-verify`

**`Makefile`**:
- `setup-hooks` target: auto-sets `core.hooksPath=githooks` in the local repo config. Runs as a dependency of `all`, so any `make` build activates the hook with zero manual setup.
- `install-hooks` / `uninstall-hooks` targets: manual copy-based alternative for those who prefer it

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14558

Reviewed By: archang19

Differential Revision: D99324330

Pulled By: xingbowang

fbshipit-source-id: b824e56d572ad423fab7de7e15ead5fa8fd8e847
2026-04-02 12:34:19 -07:00
Peter Dillinger abc1f7ced5 Fix table cache leak when InstallCompactionResults fails (#14549)
Summary:
When CompactionJob::Run() succeeds but Install() fails (e.g., LogAndApply MANIFEST I/O error), compact_->status was never updated with the install failure. CleanupCompaction() passed the stale OK status to SubcompactionState::Cleanup(), which skipped ReleaseObsolete -- leaking table cache entries for output files that were cached by VerifyOutputFiles but never installed into any Version.

This is the same class of bug fixed in https://github.com/facebook/rocksdb/issues/14469 (where Run() failed after VerifyOutputFiles), but in the Install() failure path. The FindObsoleteFiles full-scan backstop would normally catch this, but fails under crash test metadata read fault injection
(--open_metadata_read_fault_one_in), causing the
TEST_VerifyNoObsoleteFilesCached assertion to fire during Close().

Fix: propagate Install()'s local status back to compact_->status before CleanupCompaction(), so Cleanup() sees the failure and calls ReleaseObsolete on the output files.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14549

Test Plan:
New unit test DBCompactionTest.LeakedTableCacheEntryOnInstallFailure:
- Without fix (ASAN): assertion fires -- "File 12 is not live nor quarantined"
- With fix (ASAN): passes -- ReleaseObsolete properly cleans up the entry
```
COMPILE_WITH_ASAN=1 make -j db_compaction_test
./db_compaction_test --gtest_filter="DBCompactionTest.LeakedTableCacheEntry*"
[  PASSED  ] 2 tests.
```

Reviewed By: joshkang97

Differential Revision: D99155908

Pulled By: pdillinger

fbshipit-source-id: ed5374a38d7903866a38a0fe0f5539e12321bc84
2026-04-02 10:39:22 -07:00
Xingbo Wang 01ec15f328 Remove accidentally committed .claude/settings.local.json (#14554)
Summary:
This file was accidentally included in https://github.com/facebook/rocksdb/issues/14535. It contains local Claude Code permission settings that are user-specific and should not be in the repo.

Also adds it to `.gitignore` to prevent future accidental commits.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14554

Reviewed By: pdillinger

Differential Revision: D99296922

Pulled By: xingbowang

fbshipit-source-id: e229dd85179ae25bb15df1633bba7197dd2fa1f9
2026-04-02 09:49:27 -07:00
Xingbo Wang af4e32945b Blob direct write v1: write-path blob separation with partitioned files (reduced scope) (#14535)
Summary:
This PR introduces **blob direct write v1**, a reduced-scope write-path optimization where large values (>= `min_blob_size`) are written directly to blob files during `Put()` and replaced in the memtable with compact `BlobIndex` references. This avoids holding full values in memory until flush time.

### Motivation

In the existing BlobDB architecture, values are written to the WAL and memtable in their full form and separated into blob files only at flush time. This means:
- Large values are held in memory twice (raw in memtable + blob file at flush)
- Blob I/O is serialized through a single flush thread per column family

Blob direct write addresses both: values leave the write path as small `BlobIndex` references, and multiple **partitions** (configurable via `blob_direct_write_partitions`) allow concurrent blob writes with independent locks.

### Design (v1 — single-writer, WAL-disabled, reduced scope)

The v1 design intentionally keeps scope narrow for correctness and reviewability:

- **Single writer thread assumption**: no concurrent writes to the same partition file. One logical writer serializes the batch.
- **WAL-disabled**: direct-write blob files are only registered in MANIFEST at flush time. WAL replay cannot recover unregistered blob references, so WAL is disabled for this v1.
- **Flush-on-write**: each `AddRecord` call flushes to the OS immediately.
- **FIFO generation batching**: each memtable switch creates one generation batch. Direct-write files for that memtable are sealed and registered atomically when the batch is flushed to MANIFEST.
- **Round-robin partitions**: blob writes are distributed across `blob_direct_write_partitions` files using an atomic counter.

### New components

| Component | Description |
|---|---|
| `BlobFilePartitionManager` | Owns N partition files per CF. Manages open/seal/register lifecycle tied to memtable generations. |
| `BlobWriteBatchTransformer` | A `WriteBatch::Handler` that rewrites qualifying `Put` values as `BlobIndex` entries before the batch enters the write group. |

### Write path integration

1. `DBImpl::WriteImpl` calls `BlobWriteBatchTransformer::TransformBatch` before entering the writer group (for default write path), or before joining the batch group (for pipelined/unordered write).
2. Values >= `min_blob_size` are written to a partition file; the key is stored with a `BlobIndex` in the transformed batch. A rollback guard marks blob bytes as initial garbage if the write fails.
3. On `SwitchMemtable`, `RotateCurrentGeneration` moves active partitions into the next immutable batch.
4. `FlushMemTableToOutputFile` / `AtomicFlushMemTablesToOutputFiles` call `PrepareFlushAdditions` to seal partition files and collect `BlobFileAddition` + `BlobFileGarbage` entries registered to MANIFEST alongside the flush.
5. Shutdown paths (`CancelAllBackgroundWork`, `WaitForCompact` with `close_db=true`) force-flush all CFs with active direct-write managers to ensure blob files are registered before close.

### Read path

- **Get/MultiGet**: `MaybeResolveBlobForWritePath` resolves `BlobIndex` references found in memtable or immutable memtable via `BlobFilePartitionManager::ResolveBlobDirectWriteIndex`, which first checks manifest-visible state and falls back to direct blob-file reads via `BlobFileCache`.
- **Iterator**: `DBIter::BlobReader` is extended with a `BlobFilePartitionManager*` to resolve direct-write blob indexes during iteration. The unified `ResolveBlobDirectWriteIndex` path handles both manifest-visible and not-yet-flushed files.

### New options

| Option | Default | Description |
|---|---|---|
| `enable_blob_direct_write` | `false` | Enable write-path blob separation for this CF. Requires `enable_blob_files = true`. Not dynamically changeable. |
| `blob_direct_write_partitions` | `1` | Number of parallel partition files per CF. Not dynamically changeable. |

### Feature incompatibilities (reduced v1 scope)

The following features are *not supported* when `enable_blob_direct_write = true`, and are enforced both in `db_stress_tool` validation and `db_crashtest.py` sanitization:

**Write model constraints:**
- `threads` must be 1 (single writer assumption)
- `allow_concurrent_memtable_write` = 0
- `enable_pipelined_write` = 0 (transformation done before batch group, but pipelined path supported with pre-transform)
- `two_write_queues` = 0
- `unordered_write` = 0 (transformation done before batch group, but unordered path supported with pre-transform)

**WAL and recovery:**
- `disable_wal` = 1 (required — WAL replay of unregistered blob files is out of v1 scope)
- `best_efforts_recovery` = 0
- `reopen` = 0 (no crash-restart with WAL replay)
- All WAL-related stress features disabled: `manual_wal_flush_one_in`, `sync_wal_one_in`, `lock_wal_one_in`, `get_sorted_wal_files_one_in`, `get_current_wal_file_one_in`, `track_and_verify_wals`, `rate_limit_auto_wal_flush`, `recycle_log_file_num`

**Blob GC and dynamic options:**
- `use_blob_db` = 0 (stacked BlobDB not supported)
- `allow_setting_blob_options_dynamically` = 0
- `enable_blob_garbage_collection` = 0
- `blob_compaction_readahead_size` = 0
- `blob_file_starting_level` = 0

**Unsupported value types and APIs:**
- Merge (`use_merge`, `use_full_merge_v1`) — merge values pass through untransformed
- Entity APIs (`use_put_entity_one_in`, `use_get_entity`, `use_multi_get_entity`, `use_attribute_group`)
- `use_timed_put_one_in`
- User-defined timestamps (`user_timestamp_size`, `persist_user_defined_timestamps`, `create_timestamped_snapshot_one_in`)
- Transactions (`use_txn`, `use_optimistic_txn`, `test_multi_ops_txns`, `commit_bypass_memtable_one_in`) — though `WriteCommittedTxn::CommitInternal` falls back from bypass-memtable to normal path when BDW is active
- `IngestWriteBatchWithIndex` returns `NotSupported`
- `inplace_update_support` = 0

**Fault injection:**
- All write/read/metadata fault injection disabled (`sync_fault_injection`, `write_fault_one_in`, `metadata_write_fault_one_in`, `read_fault_one_in`, `metadata_read_fault_one_in`, `open_*_fault_one_in`)

**Infrastructure/snapshot APIs:**
- `remote_compaction_worker_threads` = 0
- `test_secondary` = 0
- `backup_one_in` = 0
- `checkpoint_one_in` = 0
- `get_live_files_apis_one_in` = 0
- `ingest_external_file_one_in` = 0
- `ingest_wbwi_one_in` = 0

### Tests

- `db/blob/db_blob_basic_test.cc`: ~660 lines of new direct-write unit tests covering basic put/get, multi-partition, flush/compaction, recovery, and error injection.
- `db/blob/blob_file_cache_test.cc`: ~96 lines of new tests for direct-write blob file cache behavior.
- `db/write_batch_test.cc`: ~96 lines of tests for WriteBatch with blob index entries.
- `utilities/transactions/transaction_test.cc`: verifies transaction commit path falls back correctly with direct write enabled.
- `db_stress_tool/`: full stress test support with `--enable_blob_direct_write` and `--blob_direct_write_partitions` flags, integrated into `db_crashtest.py` with 10% random selection alongside regular blob params.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14535

Test Plan:
```
make -j128 db_blob_basic_test && ./db_blob_basic_test
make -j128 blob_file_cache_test && ./blob_file_cache_test
make -j128 write_batch_test && ./write_batch_test
make -j128 transaction_test && ./transaction_test
make -j128 check
```

Stress test:
```
python3 tools/db_crashtest.py blackbox --enable_blob_direct_write=1 \
  --enable_blob_files=1 --blob_direct_write_partitions=4 \
  --disable_wal=1 --threads=1
```

Reviewed By: pdillinger

Differential Revision: D98766843

Pulled By: xingbowang

fbshipit-source-id: 1577653826913a59d05680a87bce5534ac5a5e69
2026-04-02 07:31:56 -07:00
Hemal Shah b611aa7309 Export header to add log data on a transaction (#14543)
Summary:
Transactions in rockdb supported adding log data using the `put_log_data` api, but this was not exported for other language bindings. Exported this binding allows other languages like rust, go, etc add log data on an transaction

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14543

Reviewed By: joshkang97

Differential Revision: D99171663

Pulled By: xingbowang

fbshipit-source-id: 24c94d71668a41cc7b2913972427255ab67d0863
2026-04-01 16:19:11 -07:00
Andrew Chang 830347c765 Fix null sqe crash in ReadAsync when io_uring submission queue is full (#14521)
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14521

ReadAsync calls io_uring_get_sqe() without checking for nullptr. When the
io_uring submission queue is full (outstanding completions not yet reaped),
io_uring_get_sqe returns NULL and the subsequent io_uring_prep_readv
dereferences it, causing a segfault.

MultiRead already handles this correctly by using io_uring_sq_space_left()
to cap submissions. ReadAsync submits exactly one SQE per call so a simple
null check with error return is sufficient.

On null sqe, clean up the already-allocated Posix_IOHandle and return
IOStatus::Busy so the caller can retry after reaping completions.

The io_uring queue depth is kIoUringDepth (256), and each thread gets its
own io_uring instance via thread-local storage. In practice the SQ rarely
fills because ReadAsync calls io_uring_submit() after each io_uring_get_sqe(),
immediately flushing the SQE to the kernel. The null SQE would only occur
under unusual kernel backpressure where the kernel cannot consume from the
SQ ring fast enough.

IOStatus::Busy was chosen (over IOError) because this is a transient
condition. The caller has two options:
1. Call Poll() to reap outstanding completions from the CQ, then retry
   ReadAsync. This mirrors how MultiRead handles queue pressure internally
   by capping submissions and reaping between batches.
2. Fall back to synchronous Read(). Existing callers (FilePrefetchBuffer,
   IODispatcher) already have synchronous fallback paths for non-OK
   ReadAsync status, so IOStatus::Busy naturally triggers that fallback
   without additional code changes. Given the rarity of this condition,
   the synchronous fallback is pragmatic and avoids adding retry complexity.

Also adds a TEST_SYNC_POINT_CALLBACK on io_uring_get_sqe to enable test
injection, and a new ReadAsyncQueueFull unit test that uses SyncPoint to
force a null SQE and verifies the Busy return, handle cleanup, and no crash.

Reviewed By: xingbowang

Differential Revision: D98533853

fbshipit-source-id: f6d181e5c0d5154b570ff6da39e2f52e2a6aea84
2026-04-01 15:54:30 -07:00
Andrew Chang de6cdbb62d Fix SupportedOps to verify per-thread io_uring before advertising kAsyncIO (#14514)
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14514

SupportedOps previously checked only that the ThreadLocalPtr container existed (non-null), which was set during construction based on a one-time probe on the main thread. However, CreateIOUring() can fail on other threads due to kernel resource limits or flag incompatibilities, causing ReadAsync to hit "failed to init io_uring" at runtime.

Now SupportedOps eagerly initializes the thread-local io_uring instance via Get() + CreateIOUring() and only advertises kAsyncIO if it succeeds. Also logs to stderr (with thread id) when io_uring init fails, and adds a TEST_SYNC_POINT_CALLBACK for testing simulated CreateIOUring failures.

Reviewed By: mszeszko-meta, xingbowang

Differential Revision: D98409140

fbshipit-source-id: efa92d9ac920860e95a46710c4a87e36bacbb466
2026-04-01 14:52:54 -07:00
Maciej Szeszko b8bbee0f3e Log IO uring init failures to stdout (instead of stderr) (#14548)
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/14548

Reviewed By: archang19

Differential Revision: D98667574

fbshipit-source-id: 4f01c9d8f2ae12051d4b3e4138258091b3c9eb3d
2026-04-01 14:06:03 -07:00
Xingbo Wang 24fac74206 docs: add end-to-end flow documentation (read_flow, write_flow) (#14486)
Summary:
Add comprehensive flow documentation tracing data through the full RocksDB system.

## Contents

### read_flow/ (11 chapters)
Get/MultiGet/Iterator paths through SuperVersion, MemTable, block cache, SST files (L0→Ln), bloom filters, range deletions, prefetch, async I/O.

### write_flow/ (10 chapters)
Put/Delete/Merge through WAL, MemTable, flush, compaction, delete cleanup, flow control, crash recovery, performance.

## Documentation Pattern
Each component follows a consistent structure:
- `index.md` — short overview (40-80 lines) with chapter table and key characteristics
- `NN_topic.md` — deep-dive chapters with source file references and workflow descriptions

All content validated against current codebase. No code snippets — refers to header files and struct/function names.

Part 1 of 11 in the RocksDB AI documentation series.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14486

Reviewed By: anand1976

Differential Revision: D97795679

Pulled By: xingbowang

fbshipit-source-id: 07c3be1a7f27e0a9cc1d82ccad16e8e4084f4ab0
2026-04-01 13:02:17 -07:00
Hui Xiao f12465cb93 Add COERCE_CONTEXT_SWITCH and ASSERT_STATUS_CHECKED guidance to CLAUDE.md (#14522)
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14522

Add two verification steps to RocksDB CLAUDE.md:

1. **Unit Test flakiness testing**: After writing a test, stress-test for
   flakiness with `COERCE_CONTEXT_SWITCH=1 make {test_binary}` followed by
   `--gtest_repeat=5`. This catches race conditions and context-switch-dependent
   failures early.

2. **Status object verification**: Run `ASSERT_STATUS_CHECKED=1 make check`
   to catch missing error handling that can lead to silent data corruption.
   These are compile-time checks enforced via a special build mode.

These are existing RocksDB best practices that both the stress test agent
and human developers should follow when writing or fixing code.

Reviewed By: xingbowang

Differential Revision: D98249994

fbshipit-source-id: 25ca6a628a82ecf968d5ed86aaa5c2f4b1060471
2026-04-01 11:58:43 -07:00
Xingbo Wang 7832ee886f db_stress: fix prefix scan correctness and crashtest stability (#14512)
Summary:
This PR fixes several correctness and stability issues in the stress test and crash test infrastructure, plus adds regression coverage for trie UDI iterators.

### db_stress: constrain batched prefix scans
`BatchedOpsStressTest::TestPrefixScan` opens 10 iterators with different prefixes and compares results in lockstep. Without `prefix_same_as_start`, unconstrained iterators could return keys beyond their seek prefix, producing more entries than bounded iterators and causing spurious assertion failures. Fix: set `prefix_same_as_start = true` on all iterators so every iterator stops at the seek prefix boundary.

### db_stress: skip invalid cross-prefix iterator checks
When prefix iteration is enabled, `ReadOptions` requires that the seek key and `iterate_lower_bound` share the same prefix. Previously, stress test verification could run against configurations where they don't (e.g. when `lower_bound` spans a prefix boundary), producing false positives. Fix: detect this invalid configuration and mark the check as diverged (skip verification) rather than asserting.

### db_crashtest: disable BlobDB in best-efforts recovery
BlobDB is not compatible with best-efforts recovery mode. Fix: explicitly disable blob file options when `best_efforts_recovery=True` in db_crashtest.py to avoid spurious failures.

### db_crashtest: preserve expected-state dirs across restarts
The expected-state directory was being wiped on certain restart paths, causing verification failures on the next run. Fix: preserve expected-state dirs across db_crashtest restarts. Adds a new `db_crashtest_test.py` test to verify the behavior, runnable via `make db_crashtest_test`.

### Add trie UDI iterator regression coverage
Adds regression tests for trie-based UserDefinedIndex (UDI) iterators covering snapshot-based reads, lower/upper bound iteration, `auto_refresh_iterator_with_snapshot`, and multi-version key handling. Also fixes an MSVC C4244 warning (int→char implicit narrowing) in the test.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14512

Reviewed By: hx235

Differential Revision: D98302018

Pulled By: xingbowang

fbshipit-source-id: 84f5878665e5aeb61338ec2b6bfb2d2b077bb9f2
2026-03-31 22:48:24 -07:00
Danny Chen ae322e0a73 Add SortedRunBuilder: use RocksDB as an external… (#14499)
Summary:
CONTEXT: MyRocks and other third-party users currently need to understand deep RocksDB internals to sort unsorted data into SST files. The existing pattern requires: opening a full DB instance, configuring VectorRepFactory with universal compaction, disabling auto-compaction, writing unsorted data, manually triggering CompactRange with kForceOptimized, collecting output via GetColumnFamilyMetaData, and finally ingesting via IngestExternalFile with allow_db_generated_files. This is error-prone and requires ~30 lines of RocksDB configuration knowledge.

WHAT: This adds a new utility class, SortedRunBuilder, that wraps all of the above into a simple Create/Add/Finish API. Callers feed in unsorted key-value pairs (in any order, from any number of threads) and receive sorted SST files with seqno=0 ready for ingestion -- or can iterate sorted output directly.

The name SortedRunBuilder was chosen over alternatives like UnsortedSstFileWriter because this utility targets a broad audience: anyone wanting to use RocksDB as an external sort engine, not just users migrating from SstFileWriter. That said, this explicitly removes the sorted-input requirement that SstFileWriter enforces -- callers no longer need to pre-sort their data before writing SST files.

KEY DESIGN DECISIONS:
- Zero changes to core RocksDB internals. This is a pure utility layer that exclusively calls existing public APIs:
  * DB::Open(), DB::Put(), DB::Write(), DB::Flush(), DB::CompactRange(), DB::GetColumnFamilyMetaData(), DB::NewIterator(), DestroyDB()
  * VectorRepFactory (sort-on-flush memtable)
  * BottommostLevelCompaction::kForceOptimized (zero seqnos)
- No modifications to: compaction logic, memtable implementation, SST file format, ingestion logic, or DB open/close paths
- All new code lives in utilities/sorted_run_builder/ and the public header include/rocksdb/utilities/sorted_run_builder.h
- Build file changes are limited to registering the new source/test files

API SURFACE:
  SortedRunBuilderOptions opts; opts.temp_dir = "/tmp/sort_work"; std::unique_ptr<SortedRunBuilder> builder; SortedRunBuilder::Create(opts, &builder);
  builder->Add(key, value);   // any order, thread-safe
  builder->AddBatch(&batch);  // WriteBatch for throughput
  builder->Finish();           // flush + compact + collect
  builder->GetOutputFiles();   // sorted SSTs for ingestion
  builder->NewIterator(ro);    // iterate sorted output

FILES CHANGED:
  New: include/rocksdb/utilities/sorted_run_builder.h (public header) New: utilities/sorted_run_builder/sorted_run_builder.cc (implementation) New: utilities/sorted_run_builder/sorted_run_builder_test.cc (12 tests) New: docs/plans/sorted_run_builder_plan.md (design plan) New: docs/plans/sorted_run_builder_usage_guide.md (usage guide) Modified: src.mk, CMakeLists.txt, Makefile (register new files only)

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14499

Test Plan:
$ make clean && make -j$(nproc) sorted_run_builder_test $ ./sorted_run_builder_test [==========] Running 12 tests from 1 test case. [  PASSED  ] 12 tests. (688 ms total)

  Tests cover: basic sort correctness, empty builder, WriteBatch path, concurrent multi-threaded writes, ingestion into a target DB, large random dataset (10K keys), entry/size counters, error cases (Add after Finish, iterator before Finish, empty temp_dir), cleanup verification, and duplicate key handling.

Reviewed By: xingbowang

Differential Revision: D98935972

Pulled By: dannyhchen

fbshipit-source-id: e3bb7f1ea5f6004aab697e4da667fa21292ca250
2026-03-31 14:23:35 -07:00
Maciej Szeszko 1e23ee053f skip stats if setup-ccache did not run or failed (#14542)
Summary:
`teardown-ccache` runs with if: always(), but in folly jobs `setup-folly` precedes `setup-ccache`. When `setup-folly` fails, `setup-ccache` is skipped, so `CCACHE_DIR` is unset and ccache is not on `PATH`. This is exactly what happened [here](https://github.com/facebook/rocksdb/actions/runs/23660261947/job/68928337162). Fix guards `teardown-ccache` to exit gracefully when `CCACHE_DIR` is unset, and tolerate missing ccache binary for stats.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14542

Reviewed By: joshkang97

Differential Revision: D98964757

Pulled By: mszeszko-meta

fbshipit-source-id: 8e83c1b62ef66130ba479142f002897d0a97f6c0
2026-03-31 13:42:34 -07:00
Maciej Szeszko effac3a918 Update CLANGTIDY to latest stable & secure version (#14541)
Summary:
As advertised.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14541

Reviewed By: xingbowang

Differential Revision: D98961628

Pulled By: mszeszko-meta

fbshipit-source-id: f0519089bb53649668e791b18428a65fc666b71c
2026-03-31 13:30:39 -07:00
Peter Dillinger c7f0e1fa93 Store UniqueIdVerifier file in expected_values_dir on local filesystem (#14539)
Summary:
The warm_storage_crash_test was failing because UniqueIdVerifier stored its .unique_ids bookkeeping file in the DB directory, which lives on warm storage. After a crash, warm storage's weaker durability guarantees could cause flushed-but-not-synced data to be lost, making the file appear shorter on a second open within the same constructor. This caused CopyFile to return Corruption and trigger assert(false).

Move the file to expected_values_dir (which is always on local filesystem) and always use Env::Default() for file operations, so that POSIX flush semantics are sufficient for read consistency without needing Sync.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14539

Test Plan: Full local run of `make blackbox_crash_test`

Reviewed By: mszeszko-meta

Differential Revision: D98934451

Pulled By: pdillinger

fbshipit-source-id: 5497f23c2382cb5ac2c3d7f238c6c4ca1dd29f5b
2026-03-31 12:30:54 -07:00
Xingbo Wang 141a1f0abf Add regression tests and diagnostic logging for trie UDI post-seek seqno correction (#14524)
Summary:
Adds regression tests and failure-diagnostic tooling for the trie UDI post-seek seqno correction bug (T258590238). The core fix itself already landed in https://github.com/facebook/rocksdb/issues/14466.

## What's in this PR

### Regression tests

- `TrieIndexFactoryTest.ZeroSeqMustNotSkipLeafForSmallerUserKey` — minimal unit test proving the original bug; exercises the seqno-based block advancement with a smaller user key that should NOT trigger advancement
- `TrieIndexDBTest.AutoRefreshSnapshotNextAcrossSameUserKeyBoundaries` — DB-level test for snapshot-refresh iteration across same-user-key block boundaries
- `TrieIndexDBTest.AutoRefreshSnapshotNextAfterCompactionAcrossSameUserKeyBoundaries` — same scenario after compaction reshapes SST layout
- `TrieIndexDBTest.AutoRefreshSnapshotStressLikeSingleCfCoalescingIterator` — stress-like test exercising snapshot-refresh + coalescing-iterator interaction

### Diagnostic logging in db_stress

Extracts the iterator verification failure dump into a `DumpIteratorVerificationFailure()` helper in `NonBatchedOpsStressTest`. On verification failure, logs:
- Expected-state window (pre/post read values, raw state, pending flags)
- Iterator config (UDI, trie, snapshot, multi-CF)
- Replay comparison: creates fresh iterators with standard vs trie index, direct vs coalescing, seek-to-failure-key vs replay-from-mid — making it possible to identify which index/iterator combination diverges

This logging was instrumental in triaging the original bug and will help with future trie UDI stress failures.

## Tests

All existing trie index tests pass. New tests listed above all pass.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14524

Reviewed By: hx235

Differential Revision: D98583342

Pulled By: xingbowang

fbshipit-source-id: 57e099055d6beae9b5f03f4e6605f0af6e65b94a
2026-03-31 11:38:13 -07:00
zaidoon 41e5eb1a4d Support reverse iteration in UDI/trie index and optimize hot paths (#14466)
Summary:
The following performance optimizations are included in this PR:
- Inline NextSetBit/PrevSetBit into header (called on every trie level)
- Unroll Rank1 popcount loop (the single hottest function)
- Advance/Retreat: replace path stack top in-place instead of pop+push
- std::swap for prev_key_scratch_ instead of O(n) string copy
- assign() instead of ToString() to reuse buffer capacity
- Cache IndexValue in wrapper to avoid repeated virtual dispatch
- Mark TrieIndexIterator/TrieIndexBuilder as final for devirtualization

Part of https://github.com/facebook/rocksdb/issues/12396

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14466

Reviewed By: anand1976

Differential Revision: D97979699

Pulled By: xingbowang

fbshipit-source-id: 7ec56ecc8b6d68548a336dd1aaccfb215189eff0
2026-03-31 07:35:11 -07:00
Hui Xiao 875b8544df Revert D98425822: Rocksdb Warm Storage Test failed: not implemented: SyncWAL() is not supported for this implementatio
Differential Revision:
D98425822

Original commit changeset: ec17a15c837c

Original Phabricator Diff: D98425822

fbshipit-source-id: 52c7c86d260bb7c239962c299841d62408ee5d88
2026-03-30 19:40:55 -07:00
generatedunixname3846135475516776 e96295a288 Rocksdb Warm Storage Test failed: not implemented: SyncWAL() is not supported for this implementatio (#14515)
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/14515

Reviewed By: xingbowang

Differential Revision: D98425822

fbshipit-source-id: ec17a15c837c2b3e2fbe7ae6bc62aee3685e7bcc
2026-03-30 13:00:44 -07:00
Josh Kang f897c1b2ef Add uniform block tracking stats (#14513)
Summary:
Add tracking of uniform index blocks as a table property (`num_uniform_blocks`). When `uniform_cv_threshold` is set, the block builder detects uniformly distributed keys via coefficient of variation of key gaps. This information is now surfaced end-to-end: from block building through index builders to SST table properties.

## Key changes
- **BlockBuilder**: Persist the uniformity result from `ScanForUniformity()` as a member (`is_uniform_`) exposed via `IsUniform()`, rather than a local variable discarded after `Finish()`.
- **Index builders**: All three index builder types (`ShortenedIndexBuilder`, `HashIndexBuilder`, `PartitionedIndexBuilder`) implement `NumUniformIndexBlocks()`. For partitioned indexes, the count accumulates across partition `Finish()` calls.
- **Table property serialization**: Added `TablePropertiesNames::kNumUniformBlocks` (`"rocksdb.num.uniform.blocks"`) with full serialization/deserialization support. Without this, the property was computed in memory but never persisted to SST files.
- **Table property aggregation**: `num_uniform_blocks` included in `Add()` and `GetAggregatablePropertiesAsMap()` for correct cross-SST aggregation.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14513

Test Plan:
- **Unit tests**: `block_test` (7504 passed), `table_test` (6917 passed), `table_properties_collector_test` (8 passed).
- **End-to-end verification via db_bench + sst_dump**:
  - Positive: `./db_bench --benchmarks=fillseq,compact --num=10000 --uniform_cv_threshold=0.2 --index_shortening_mode=0 --compression_type=none` produces SST with `# uniform blocks: 1`.
  - Negative: Same with `--uniform_cv_threshold=-1` (disabled) produces `# uniform blocks: 0`.

Reviewed By: xingbowang

Differential Revision: D98343170

Pulled By: joshkang97

fbshipit-source-id: ed08e83b2dcfb70f074bfb0f7bb5c31d75dc6da9
2026-03-30 11:58:45 -07:00
Xingbo Wang dae450b78e Improve Claude review reliability (#14526)
Summary:
## Problems

**(1) Claude review times out on large PRs**
The `claude-code-base-action` defaults to `timeout_minutes=10`. Large PRs (e.g. https://github.com/facebook/rocksdb/issues/14499) get killed with exit code 124 after 600 seconds mid-review.

**(2) Exported PRs never get reviewed**
PRs exported from Meta's internal pipeline (e.g. https://github.com/facebook/rocksdb/issues/14515) trigger CI within milliseconds of PR creation. The `workflow_run` payload has `pull_requests=[]`, and the SHA-based fallback also misses because GitHub hasn't registered the PR yet — so Claude review never fires.

## Fixes

- Set `timeout_minutes: "60"` on both auto-review and manual-review `Run Claude` steps
- Retry the SHA→PR lookup up to 5 times with a 10s delay, giving GitHub up to ~50s to register the PR. Also bumped `per_page` from 30 to 100.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14526

Reviewed By: pdillinger

Differential Revision: D98636217

Pulled By: xingbowang

fbshipit-source-id: bba19095ab2ddf468c5c19cf5c77d19536f498b0
2026-03-30 11:12:08 -07:00
Andrew Chang 2af38206ad Log errno and thread ID on CreateIOUring failure (#14520)
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14520

CreateIOUring() silently returns nullptr on failure, discarding the errno
from io_uring_queue_init. This makes it impossible to diagnose why
io_uring initialization fails on specific threads (e.g. ENOMEM from
memlock limits, EINVAL from unsupported flags, EMFILE from fd exhaustion).

Add a fprintf(stderr, ...) that logs strerror, errno, and pthread thread
ID when io_uring_queue_init fails, so failures are diagnosable from logs
without needing to reproduce.

Reviewed By: xingbowang

Differential Revision: D98526792

fbshipit-source-id: 1eb5042c8b62663c4d24c09f29ef7c55b90032f0
2026-03-28 15:36:15 -07:00
Josh Kang 5db0603613 Read-triggered compactions (#14426)
Summary:
Add read-triggered compaction, a new feature that reduces read amplification by compacting SST files that receive high read traffic. When an SST file's read frequency (`num_reads_sampled / file_size`) exceeds a configurable threshold, it is marked for compaction to a lower level.

The feature introduces two new options: a CF option `read_triggered_compaction_threshold` (default 0, disabled) and a DB option `max_periodic_compaction_trigger_seconds` (default 43200s) that controls how often the background thread re-evaluates compaction scores on quiet databases. Both options are dynamically changeable.

Lowering `max_periodic_compaction_trigger_seconds` does add some overhead, but generally is minimal, so running this every couple of minutes in a production environment seems fairly reasonable.

## Key changes

- **New CF option `read_triggered_compaction_threshold`** (`advanced_options.h`): When positive, files with `reads_per_byte > threshold` are marked for compaction. Files at the last non-empty level are skipped (bottommost compaction handles those separately). Marked files are sorted by hotness (reads_per_byte descending).
- **New DB option `max_periodic_compaction_trigger_seconds`** (`options.h`): Replaces the hardcoded 12-hour ceiling in `ComputeTriggerCompactionPeriod()`. Essential for read-triggered compaction on quiet DBs since there are no writes to trigger score re-evaluation.
- **Leveled compaction picker** (`compaction_picker_level.cc`): Adds read-triggered as the lowest-priority compaction reason in `SetupInitialFiles()`, using the existing `PickFileToCompact` helper.
- **Universal compaction picker** (`compaction_picker_universal.cc`): Adds `PickReadTriggeredCompaction` as lowest priority. Refactors shared "find output level + compute overlapping inputs + create Compaction" logic from both `PickDeleteTriggeredCompaction` and `PickReadTriggeredCompaction` into `BuildCompactionToNextLevel`, handling both single-level and multi-level universal cases.
- **Periodic trigger integration** (`db_impl.cc`): `TriggerPeriodicCompaction` now also fires for CFs with `read_triggered_compaction_threshold > 0`, even without time-based compaction configured.
- **Stress test & db_bench support**: Both `db_stress` and `db_bench` support the new options. `db_crashtest.py` randomly enables read-triggered compaction and sets a short periodic trigger interval when enabled.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14426

Test Plan:
**Unit tests**:
- `compaction_picker_test` — 7 new tests: `ReadTriggeredCompactionDisabled`, `ReadTriggeredCompactionBelowThreshold`, `ReadTriggeredCompactionAboveThreshold`, `NeedsCompactionReadTriggered`, `ReadTriggeredPicksFile`, `UniversalReadTriggeredCompaction`, `ReadTriggeredSkipsLastLevel`, `UniversalReadTriggeredNoPickWhenNotMarked`
- `db_compaction_test` — `ReadTriggeredCompaction` integration test verifying end-to-end behavior with sync points
- Stress test coverage

**Stress test**:
```
make V=1 -j "CRASH_TEST_EXT_ARGS=--duration=600 --max_key=2500000 --max_compaction_trigger_wakeup_seconds=10
  --read_triggered_compaction_threshold=0.0001 --interval=600" blackbox_crash_test
```
- confirmed read triggered compactions from LOGS

**Benchmark** (`db_bench`):

Setup: 5M keys (100B values, 16B keys), leveled compaction, 5 levels, 4MB target file size. DB fully compacted, then 2M overlapping keys written without compaction to create L0/L1 overlap (82 files, ~294MB).

LSM shape change during readrandom with read-triggered compaction:
```
BEFORE: L0=9 files (15MB), L1=4 (16MB), L2=20 (69MB), L3=49 (194MB) — 82 files, 294MB
AFTER:  L3=66 files (223MB)
```

| Benchmark | Config | avg ops/s | % change |
|-----------|--------|-----------|----------|
| readrandom (8 threads, 5M reads) | baseline (threshold=0) | 1,086,965 | — |
| readrandom (8 threads, 5M reads) | threshold=0.000001, trigger=5s | 1,453,697 | **+33.7%** |

Reviewed By: xingbowang

Differential Revision: D97838716

Pulled By: joshkang97

fbshipit-source-id: a21fcb270c7fadd4f78d98b9c821982f220dd3f0
2026-03-27 14:43:52 -07:00
Xingbo Wang a965706e73 blob: skip empty values even with min_blob_size=0 (#14517)
Summary:
When `min_blob_size=0`, the existing guard condition:

```cpp
if (value.size() < min_blob_size_) {
    return Status::OK();  // skip blob creation
}
```

is **false** for empty values (`0 < 0`), so empty values proceed to blob creation. This writes a 0-byte blob to disk and creates a `BlobContents` object with an empty `Slice`.

When that `BlobContents` is later evicted from the primary blob cache to `CompressedSecondaryCache`, the eviction handler calls `SaveTo(value, 0, 0, out)`, which hits:

```cpp
// typed_cache.h:230
assert(from_offset < slice.size());  // 0 < 0 → CRASH
```

This crash was found by the stress test with `--min_blob_size=0` and `--blob_cache` + secondary cache enabled (T261142690).

## Fix

Add an explicit `value.empty()` check before the blob path:

```cpp
if (value.empty() || value.size() < min_blob_size_) {
    return Status::OK();
}
```

Empty values are now always stored inline in the SST, regardless of `min_blob_size`. This is also correct on principle: a `BlobIndex` reference is larger than an empty value, so storing an empty value as a blob is pure overhead with no benefit.

## Root Cause Chain

1. User writes `Put("key", "")` with `min_blob_size=0`
2. `BlobFileBuilder::Add()` — `0 < 0` is false, empty value proceeds to blob creation
3. 0-byte blob written to blob file; `BlobContents` created with 0-size slice
4. `BlobContents` inserted into primary blob cache (LRU)
5. Cache eviction triggers `CacheWithSecondaryAdapter::EvictionHandler()`
6. `CompressedSecondaryCache::InsertInternal()` → `SaveTo(value, 0, 0, out)`
7. `assert(from_offset < slice.size())` → `assert(0 < 0)` → **** assertion failure

## Test

Added `DBBlobBasicTest.EmptyValueNotStoredAsBlob` which:
- Writes an empty value and a non-empty value with `min_blob_size=0`
- Verifies both are readable
- Confirms the empty value is stored **inline** (readable from `kBlockCacheTier` without blob I/O)
- Confirms the non-empty value is stored as a **blob** (returns `IsIncomplete()` from `kBlockCacheTier`)

## Related

- Task: T261142690

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14517

Reviewed By: hx235

Differential Revision: D98499174

Pulled By: xingbowang

fbshipit-source-id: 5713923daec83db6491d00ce58acdf8231fabeba
2026-03-27 13:55:18 -07:00
Xingbo Wang 1a4b1e42cc Add include_blob_files option to GetApproximateSizes (#14501)
Summary:
**Summary:**
Add a new boolean flag `include_blob_files` (default: `false`) to `SizeApproximationOptions` and a corresponding `INCLUDE_BLOB_FILES` enum value to `SizeApproximationFlags`. When set to `true`, the returned size includes an approximation of blob file data in the queried key range.

**Algorithm:**
The blob file size contribution is prorated using the SST size ratio:
```
blob_size_in_range ≈ total_blob_size * (sst_size_in_range / total_sst_size)
```
The blob-to-SST ratio (`total_blob_size / total_sst_size`) is computed once before the per-range loop, so iterating levels and blob files only happens once per `GetApproximateSizes` call regardless of how many ranges are queried. The per-range SST size (`ApproximateSize`) is computed once and shared between `include_files` and `include_blob_files`.

**Limitations:**
- Assumes blob data is distributed proportionally to SST data across the key space. May be inaccurate if blob value sizes vary significantly across different key ranges (e.g., one range has large blobs while another has small ones).
- If there are no SST files (all data in memtables), the blob size contribution will be 0 even if blob files exist on disk.

**Changes:**
- `include/rocksdb/options.h`: New `include_blob_files` field in `SizeApproximationOptions`; updated doc comments for `include_memtables`/`include_files`
- `include/rocksdb/db.h`: New `INCLUDE_BLOB_FILES` in `SizeApproximationFlags` enum, updated flags-to-options mapping
- `include/rocksdb/c.h`: New `rocksdb_size_approximation_flags_include_blob_files` C API enum value
- `java/`: Added `INCLUDE_BLOB_FILES` to `SizeApproximationFlag.java` and JNI flag mapping in `rocksjni.cc`
- `db/db_impl/db_impl.cc`: Blob-to-SST ratio computed once before loop, SST range size computed once per range and shared
- `db_stress_tool/db_stress_test_base.cc`: Randomized `include_blob_files` in stress test

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14501

Test Plan:
- New `DBBlobBasicTest.GetApproximateSizesIncludingBlobFiles` — verifies:
  - Size with blobs > without (full range)
  - Non-overlapping range returns 0
  - Partial range returns proportionally less than full range
  - `SizeApproximationFlags` API works
  - Multi-range query: two sub-ranges sum approximately to the full-range result
- Stress test now exercises the new option randomly

Reviewed By: hx235

Differential Revision: D97984211

Pulled By: xingbowang

fbshipit-source-id: e9127eac3308687fd4f0b17a771fd61fba6a8380
2026-03-27 13:53:21 -07:00
Andrew Chang 47de3a3a2c Fix io_uring kernel resource leak in DeleteIOUring() (#14518)
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14518

When RocksDB creates an io_uring instance via `CreateIOUring()`, it calls
`io_uring_queue_init()` under the hood. This function asks the Linux kernel
to set up a submission queue and completion queue for async I/O — the kernel
allocates file descriptors and memory-mapped ring buffers to make this work.

To properly release those kernel resources, you must call
`io_uring_queue_exit()` before freeing the `io_uring` struct. This function
tells the kernel "I'm done with this io_uring" — it unmaps the shared
memory regions and closes the kernel file descriptors. Without it, those
resources leak every time an io_uring instance is destroyed.

`DeleteIOUring()` (the ThreadLocalPtr destructor callback) was only doing
`delete iu` — freeing the C++ struct's heap memory but never telling the
kernel to clean up. This meant every thread exit leaked kernel resources.

The same bug also existed in the `PosixFileSystem` constructor, which
creates a temporary test io_uring to check kernel support and then did
`delete new_io_uring` without cleanup.

Fix:
1. Add `io_uring_queue_exit(iu)` before `delete iu` in `DeleteIOUring()`.
2. Replace bare `delete new_io_uring` in fs_posix.cc with
   `DeleteIOUring(new_io_uring)` to reuse the now-correct helper.

Note: the error-recovery path in io_posix.cc (~line 907) already correctly
calls `io_uring_queue_exit()` before `delete`, confirming this was the
intended pattern that was missed in these two spots.

Reviewed By: anand1976

Differential Revision: D98500559

fbshipit-source-id: bd6c10c19d9fd67cf537dd7f100ef9dc49bfe77e
2026-03-27 12:00:06 -07:00
Xingbo Wang 926e2b589f Fix nightly CI regressions, flaky tests, and Windows ccache from #14478 (#14508)
Summary:
Fix build failures, flaky tests, and Windows ccache issues exposed by PR https://github.com/facebook/rocksdb/issues/14478.

### 1. Release build (NDEBUG) compile error
**Job**: `build-linux-release-with-folly`

The SyncPoint cleanup listener in testharness.cc referenced `SyncPoint::GetInstance()` unconditionally, but SyncPoint is only declared in debug builds. Wrapped with `#ifndef NDEBUG`.

### 2. Folly-lite link error
**Job**: `build-linux-cmake-with-folly-lite`

The `USE_FOLLY_LITE` cmake path unconditionally linked `-lglog`, which fails with lld (added in https://github.com/facebook/rocksdb/issues/14478) when glog isn't installed. Use `find_library()` to link only when available.

### 3. Flaky tests — leaked `Env::Default()` thread pool state

Sharded execution runs multiple tests per process. Several tests in `db_compaction_test.cc` modified the global `Env::Default()` BOTTOM thread pool without resetting. With a leaked BOTTOM pool, subsequent tests' last-level compactions get forwarded to the bottom pool, freeing the LOW thread to pick additional compactions unexpectedly.

Fix: add `TearDown()` override to `DBCompactionTest` that captures default thread pool sizes in the constructor and restores them after every test. This is more robust than per-test cleanup because:
- It runs even when a test fails (gtest calls TearDown after assertion failures)
- It catches all current and future leakers without per-test maintenance

### 4. Windows ccache — 0.26% hit rate → should be ~99%

The Windows nightly build takes 57 minutes because ccache has near-zero hit rate. Two issues:
- **Cache key**: The `hendrikmuhs/ccache-action` used a timestamp-based key, so each nightly run created a unique key and never found the previous run's cache (`No cache found.`). Fixed by using a stable key `ccache-windows-<workflow>` with prefix-based restore.
- **Compiler check**: Missing `compiler_check=content` setting, so MSVC path/version changes between runners invalidated all cache entries. Added `compiler_check=content` (same fix as macOS in https://github.com/facebook/rocksdb/issues/14478).

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14508

Test Plan:
- Release and debug builds compile cleanly
- 70 consecutive shuffled runs of db_compaction_test (367 tests each) pass — 50 on full cores + 20 on 4 cores
- Format check passes
- Windows ccache fix requires CI run to verify (first run populates cache, second run should see high hit rate)

Reviewed By: anand1976

Differential Revision: D98380757

Pulled By: xingbowang

fbshipit-source-id: d74079b75786ba3299e335b145a6c5fdc81fd5c1
2026-03-27 10:54:08 -07:00
Andrew Chang 453ebd6f37 Fix MultiScan not respecting SupportedOps for async IO (#14510)
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14510

When a filesystem does not return kAsyncIO in SupportedOps(), MultiScan
still sends ReadAsync/Poll/AbortIO requests. Regular iterators check via
CheckFSFeatureSupport in ArenaWrappedDBIter::Init and ForwardIterator,
but MultiScan blindly trusted the caller-provided
MultiScanArgs::use_async_io flag.

Fix: In the MultiScan constructor, after scan_opts_ is initialized,
check CheckFSFeatureSupport and disable use_async_io if the FS doesn't
support it. Also pass the (potentially modified) scan_opts_ to Prepare()
instead of the original scan_opts parameter.

Reviewed By: mszeszko-meta

Differential Revision: D97995735

fbshipit-source-id: 331639d950fd3cb9f491feed996d5820294beb4e
2026-03-26 09:24:04 -07:00
Andrew Chang 24dc8306f8 Add atomic_flush to CreateBackupOptions (#14511)
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14511

Add `atomic_flush` option to `CreateBackupOptions` that plumbs through
`CheckpointImpl::CreateCustomCheckpoint` to `LiveFilesStorageInfoOptions`.

When `flush_before_backup=true` and `atomic_flush=true`, the backup engine
atomically flushes all column families before creating the backup. This
ensures cross-CF consistency without needing WAL files. Combined with
`BackupEngineOptions::backup_log_files=false`, this allows safely skipping
WAL backup for multi-CF databases, reducing backup size and complexity.

Changes:
- `backup_engine.h`: Add `bool atomic_flush = false` to `CreateBackupOptions`
- `checkpoint_impl.h/.cc`: Add `bool atomic_flush` parameter to
  `CreateCustomCheckpoint`, plumb to `LiveFilesStorageInfoOptions::atomic_flush`
- `backup_engine.cc`: Pass `options.atomic_flush` through to
  `CreateCustomCheckpoint`
- `checkpoint_test.cc`: Add `BackupWithAtomicFlushSkipsWAL` test verifying
  backup with atomic flush + no WAL creates a valid restorable multi-CF backup

Reviewed By: xingbowang

Differential Revision: D98208696

fbshipit-source-id: e818ba8669ac52a206e30e9dd56f1d7573cc0175
2026-03-26 08:49:18 -07:00
Xingbo Wang 0921ec1200 Improve Claude Code review: comprehensive prompt, incremental findings, recovery flow (#14507)
Summary:
Improves the Claude Code review CI workflow to produce deeper, more reliable reviews.

**Motivation:** The previous 30-turn limit caused reviews to silently produce "no output" on complex PRs (e.g. https://github.com/facebook/rocksdb/issues/14477). The review prompt was also too shallow — single-pass with no codebase context phase.

**Changes:**

**1. Comprehensive multi-agent review prompt** (`claude_md/ci_review_prompt.md`)
- 9 specialized review agents: design, correctness, cross-component, invariant-adversary, caller-audit, performance, API, serialization, test coverage
- Deep codebase context phase before agents spawn: caller-chain analysis (3-5 levels up), callee side-effect tracing, cross-component data consumer analysis, execution context verification, assumption stress-testing
- Inter-agent debate with round-robin critique assignments
- Final report quality rules: disproven findings removed, no stream-of-consciousness

**2. Incremental findings + recovery flow**
- `Write` tool added so Claude saves findings to `review-findings.md` after each phase
- If the review hits the turn limit, a recovery step launches a Sonnet session to format partial findings into the standard output
- Recovery file existence check prevents crash if recovery step fails
- `getLastAssistantText` fallback truncated to 50KB to avoid enormous PR comments

**3. Prompts extracted to files** (`claude_md/ci_*.md`)
- `ci_review_prompt.md` — full review methodology
- `ci_query_prompt.md` — `/claude-query` system prompt
- `ci_recovery_prompt.md` — recovery formatting prompt
- Secure: checkout is from base branch (main), not PR head

**4. `max_turns` increased from 30 to 300**
- Orchestrator budget for multi-agent workflow; sub-agents get their own turns
- Recovery flow ensures partial results if limit is still hit

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14507

Reviewed By: archang19

Differential Revision: D98170111

Pulled By: xingbowang

fbshipit-source-id: 390626a53e7a7f91c2d3e91ed4403494532425ed
2026-03-25 22:11:29 -07:00
Josh Kang f620a6d039 Support Pinnable Reads In SstFileReader (#14500)
Summary:
Add `SstFileReader::Get` (single-key) and `SstFileReader::MultiGet` (PinnableSlice) overloads to enable zero-copy point lookups directly from SST files. The existing `MultiGet(std::string*)` is refactored to delegate to the new `MultiGet(PinnableSlice*)`, which writes results directly into caller-provided `PinnableSlice` values instead of copying through an intermediate buffer. The single-key `Get` uses `TableReader::Get` with a `GetContext` for efficient single-key lookups without the overhead of MultiGet's sorting and batching machinery.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14500

Test Plan: - New unit tests

Reviewed By: xingbowang

Differential Revision: D97825648

Pulled By: joshkang97

fbshipit-source-id: 17f3edd59bbf4747d17309c44ef12f0d952ea4eb
2026-03-25 15:11:15 -07:00
anand76 c78144e452 Update version to 11.2.0 and add 11.1.fb to format compat (#14509)
Summary:
- Bump version.h from 11.1.0 to 11.2.0
- Add `11.1.fb` to `check_format_compatible.sh`

## Remaining manual steps
- [x] Cherry-pick HISTORY.md update from release branch
- [ ] Update folly hash in Makefile to latest - Follow up in another PR

Part of 11.1 release workflow.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14509

Reviewed By: archang19

Differential Revision: D98173862

Pulled By: anand1976

fbshipit-source-id: 020dbb30126d054788ed979f42fabd1be8d97abb
2026-03-25 12:41:45 -07:00
Xingbo Wang 056a0f9664 Add atomic_flush option for checkpoint (#14477)
Summary:
Add `force_atomic_flush` to `FlushOptions` so any flush caller (Checkpoint, `DB::Flush()`, etc.) can force atomic flush of all column families even when `DBOptions::atomic_flush` is disabled.

## Changes

### Public API
- Added `bool force_atomic_flush = false` to `FlushOptions`
- Added `bool atomic_flush = false` to `LiveFilesStorageInfoOptions`

### Flush dispatch
- `FlushAllColumnFamilies()` — uses `immutable_db_options_.atomic_flush || flush_options.force_atomic_flush`
- `DB::Flush()` (single-CF and multi-CF) — respects `force_atomic_flush`
- `FlushForGetLiveFiles()` — constructs `FlushOptions` with `force_atomic_flush` from `LiveFilesStorageInfoOptions::atomic_flush`

### Per-request atomic flush in pipeline
- `FlushRequest.atomic_flush` and `BGFlushArg.atomic_flush_` carry the per-request flag
- `GenerateFlushRequest`, `EnqueuePendingFlush`, `PopFirstFromFlushQueue`, `BackgroundFlush`, `FlushMemTablesToOutputFiles` all use the per-request flag

### Stress test
- Added `--checkpoint_atomic_flush` flag to db_stress, randomly enabled in db_crashtest.py

### Unit tests (6 new)
- End-to-end checkpoint with atomic flush override
- Negative test (default = no atomic flush)
- Single-CF edge case
- Interaction with `DBOptions::atomic_flush=true`
- Mixed non-atomic then atomic flushes in queue
- Mixed atomic then non-atomic flushes in queue

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14477

Reviewed By: joshkang97

Differential Revision: D97639233

Pulled By: xingbowang

fbshipit-source-id: b924c044d6179e68b38c6604eb46c9140516d42a
2026-03-25 09:54:00 -07:00
Hui Xiao f8ff77db6a Fix FIFO crash test false verification failures by disabling file dropping (#14503)
Summary:
FIFO compaction drops old SST files when total size exceeds `max_table_files_size`
or `max_data_files_size`. The stress test's expected state can't track these drops,
causing false "GetEntity returns NotFound" failures. Confirmed by
`rocksdb.fifo.max.size.compactions COUNT: 7` in an asan_crash_test whitebox run
(`compaction_style=2`, `fifo_compaction_max_data_files_size_mb=100`).

This diff adds a `fifo_compaction_max_table_files_size_mb` flag and sets it to
100GB in db_crashtest.py. Since `max_table_files_size` is always active (default
1GB), it must always be set very high. `fifo_compaction_max_data_files_size_mb`
is randomized between 0 (disabled, defers to `max_table_files_size`) and 100GB
(overrides `max_table_files_size`) to exercise both code paths while preventing
drops in either case. FIFO intra-L0 compaction (`fifo_allow_compaction`) is
still exercised.

Long-term TODO: handle FIFO drops in expected state via `OnCompactionBegin`
listener calling `SetPendingDel()` on affected key ranges, treating drops as
concurrent deletes.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14503

Reviewed By: joshkang97

Differential Revision: D97601382

Pulled By: hx235

fbshipit-source-id: 1a3badfa4de47efd73f965807982a966ebe135ef
2026-03-24 19:32:11 -07:00
Josh Kang dcf3db95a6 Make concurrent Memtable stats more robust (#14506)
Summary:
Fix thread-safety issues in memtable stats tracking and `MaybeUpdateNewestUDT` to prepare for PR https://github.com/facebook/rocksdb/issues/14448, which introduces concurrent range tombstone insertion from the read path into the mutable memtable. Currently the non-concurrent write path updates memtable counters (`num_entries_`, `num_deletes_`, `num_range_deletes_`, `data_size_`) using non-atomic load/store pairs. While this is safe today PR https://github.com/facebook/rocksdb/issues/14448 will make range_tombstone memtable concurrent, but the main write path can still be non-concurrent. This fix ensures stats are tracked correctly.

Similarly, `MaybeUpdateNewestUDT` was not thread-safe and was not called in the concurrent write path at all. This PR also fixes the `post_process_info` delete counter which missed `kTypeSingleDeletion` and `kTypeDeletionWithTimestamp`.

## Key changes
- Switch non-concurrent counter updates from `LoadRelaxed`/`StoreRelaxed` pairs to `FetchAddRelaxed` to avoid races with concurrent range tombstone inserts (e.g. `AddLogicallyRedundantRangeTombstone` calling `BatchPostProcess`).
- Make `MaybeUpdateNewestUDT` thread-safe by replacing `Slice newest_udt_` with `RelaxedAtomic<const char*> newest_udt_data_` and using a CAS loop. The pointed-to memory lives in the arena and remains valid for the memtable's lifetime.
- Call `MaybeUpdateNewestUDT` in the concurrent write path (was previously skipped with a TODO).
- Fix `post_process_info->num_deletes++` to also count `kTypeSingleDeletion` and `kTypeDeletionWithTimestamp`, matching the non-concurrent path.
- Change `GetNewestUDT()` return type from `const Slice&` to `Slice` (returning by value) to avoid dangling reference issues with the new atomic pointer storage. Updated across `MemTable`, `ReadOnlyMemTable`, `MemTableListVersion`, and `WBWIMemTable`.
- Remove unused `Slice newest_udt_` member from `WBWIMemTable`.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14506

Test Plan:
- Added `ConcurrentWriteMemTableProperties` test in `db_properties_test.cc`: 4 threads writing puts, deletes, and single deletes concurrently, then verifying `rocksdb.num-entries-active-mem-table` and `rocksdb.num-deletes-active-mem-table` properties are correct. Flush with `flush_verify_memtable_count=true` validates integrity.
- Added `ConcurrentGetTableNewestUDT` test in `memtable_list_test.cc`: 4 threads concurrently inserting entries with UDTs via `allow_concurrent=true`, verifying the newest UDT is correctly tracked.
- Extended `DuplicateSeq` and `ConcurrentMergeWrite` tests in `db_memtable_test.cc` to verify stat counters after `BatchPostProcess`.
- Benchmark results show no performance regression (3 runs each, averaged):

**Workload 1: fillrandom** (pure Put workload)
```
./db_bench -benchmarks=fillrandom -seed=1 -compression_type=none -threads=8 -db=<DB>
```

**Workload 2: readrandomwriterandom** (mixed read/write/delete)
```
# Setup: fillrandom,compact to populate DB (single-threaded)
./db_bench -benchmarks=readrandomwriterandom -seed=1 -compression_type=none \
  -use_existing_db=1 -readwritepercent=50 -deletepercent=20 -threads=8 -db=<DB>
```

**Workload 3: fillrandom with range tombstones** (puts interleaved with range deletes)
```
./db_bench -benchmarks=fillrandom -seed=1 -compression_type=none \
  -writes_per_range_tombstone=100 -range_tombstone_width=10 \
  -max_num_range_tombstones=1000 -threads=8 -db=<DB>
```

```
| Benchmark                          | avg ops/s (main) | avg ops/s (feature) | % change |
|------------------------------------|-----------------|--------------------:|----------|
| fillrandom                         |          531,986 |             532,099 |   +0.02% |
| readrandomwriterandom (50r/30w/20d)|          786,400 |             815,143 |   +3.65% |
| fillrandom (range tombstones)      |          541,656 |             531,008 |   -1.97% |
```

Reviewed By: xingbowang

Differential Revision: D97974456

Pulled By: joshkang97

fbshipit-source-id: ea95f23953fe37771a599aed61ece1a504b12c2e
2026-03-24 17:16:14 -07:00
Xingbo Wang ec31d0074f Accelerate build and test infrastructure (#14478)
Summary:
Reduce `make check` time from ~9.4 minutes to ~3.7 minutes (2.6x) on a 192-core machine, speed up CI by 1.8x across all jobs, and fix ccache hit rates (macOS cmake: 0.51% → 99.80%).

## Changes

### 1. Auto-detect lld linker (`build_detect_platform`)
- Probe for `lld` at configure time; fall back to default `ld.bfd` if unavailable
- Linux-only guard (`TARGET_OS = Linux`)
- **Measured: 12x faster linking** (7.4s → 0.6s per test binary)
- Add `-L/usr/local/lib` for lld library resolution on CI
- Install lld in CI pre-steps action
- Opt out with `ROCKSDB_NO_FAST_LINKER=1`

### 2. Sharded test execution (`Makefile`)

**Before:** `make check` enumerated every individual gtest case via `--gtest_list_tests`, then created a separate shell script for each case with `--gtest_filter=TestName`. For example, `block_based_table_reader_test` has 9,788 parameterized test cases — each spawned its own process, loading a ~50MB binary, initializing gtest, registering all ~10K tests, running exactly ONE, then tearing down. The per-process overhead was ~1.5s, so 9,788 × 1.5s ≈ 15,000s wasted on overhead alone. Across all 302 test binaries this produced 39,467 individual processes.

**After:** Use gtest's built-in sharding (`GTEST_TOTAL_SHARDS`/`GTEST_SHARD_INDEX`) to group ~10 test cases per process. Each shard loads the binary once and runs multiple tests sequentially — eliminating the per-process overhead. The 9,788 cases in `block_based_table_reader_test` become ~980 shards. Total process count drops from 39,467 to ~4,036. Same tests, same coverage, just fewer process spawns.

The shard count adapts to machine size: `min(ceil(test_count / GTEST_SHARD_SIZE), NCORES * 8)`. This ensures large machines (192 cores) get many small shards for parallelism, while small CI runners (4 cores) get fewer larger shards to avoid excessive overhead. Both `GTEST_SHARD_SIZE` (default 10) and `NCORES` can be overridden.

CI sharding uses round-robin distribution across 3 shards to balance heavy tests (db_test, db_compaction_test, etc.) instead of contiguous alphabetical ranges.

Note: gtest continues running all tests after a failure (`GTEST_THROW_ON_FAILURE=0`), so grouping tests does not mask failures — all failures within a shard are reported.

**Measured: 3.6x CPU reduction** (76,121s → 21,033s)

### 3. Fix SyncPoint leaks for test isolation (5 files)
Sharded execution exposed 43 test files that set SyncPoint callbacks but never clean them up. Stale callbacks with captured local variables cause segfaults or data corruption when subsequent tests in the same process trigger them.

**Systematic fix:** Global gtest `TestEventListener` in `testharness.cc` that calls `SyncPoint::DisableProcessing()` + `ClearAllCallBacks()` + `ClearTrace()` + `LoadDependency({})` after every test case. This cleans up all SyncPoint state: callbacks, dependency maps, cleared points, and the point filter. Registered via static initialization — no changes needed to individual test `main()` functions.

**SyncPoint infrastructure fixes:**
- `DisableProcessing()` now calls `cv_.notify_all()` to wake threads blocked in `Process()`. Previously, threads waiting for predecessor sync points that would never fire (because the test ended) would hang forever.
- `Process()` now rechecks `enabled_` after waking from `cv_.wait()`, so threads exit promptly when processing is disabled instead of looping forever.

**Specific fixes (defense-in-depth):**
- `SstFileReaderTest::VerifyNumEntriesCorruption` — leaked `PropertyBlockBuilder::AddTableProperty:Start` callback that corrupted SST entry counts
- `WritePreparedTransactionTest::CommitAndSnapshotDuringCompaction` — leaked `CompactionIterator:AfterInit` callback causing segfault via dangling pointers
- `TransactionTestBase` destructor — cleanup for 26 SyncPoint uses across transaction tests
- `RetriableLogTest` destructor — cleanup for log reader SyncPoint callbacks

### 4. Fix ccache for reliable cross-run caching (`setup-ccache`, `CMakeLists.txt`)
- **`CCACHE_COMPILERCHECK=content`**: Default `mtime` compared compiler binary modification time, which differs on every fresh CI runner → 0% hit rate. `content` hashes compiler output instead — stable across runner instances
- **`CCACHE_SLOPPINESS`**: CMake generates varying `-MF` paths and timestamps. Without sloppiness settings, ccache treated these as different compilations
- **`CMAKE_C/CXX_COMPILER_LAUNCHER`** instead of `RULE_LAUNCH_COMPILE`: Avoids double-wrapping ccache when also injected via PATH. Removes `RULE_LAUNCH_LINK` since ccache cannot cache link operations
- **macOS cmake hit rate: 0.51% → 99.80%**

### 5. Align GTEST_THROW_ON_FAILURE (`Makefile`)
- Set `GTEST_THROW_ON_FAILURE=0` in Makefile default to match CI's `pre-steps/action.yml`
- `=1` caused `std::terminate` in multi-threaded stress tests (e.g., `point_lock_manager_stress_test`, `rate_limiter_test`) when assertions fail — the gtest exception propagates across thread boundaries, triggering undefined behavior and heap-use-after-free under ASAN

### 6. Portability fixes
- `[[maybe_unused]]` instead of `__attribute__((unused))` for MSVC compatibility
- Remove blanket `-fPIC` from `COMMON_FLAGS` — only apply via `PLATFORM_SHARED_CFLAGS` for shared builds, preserving optimal codegen for release/static builds
- `make -s list_all_tests` to suppress make noise in test enumeration

## Results

### Local (192-core devvm, clean build)

| Metric | Before | After | Improvement |
|--------|--------|-------|-------------|
| Build time (clean) | 2m43s | ~2m | lld linker |
| Test wall clock | 400s | 200s | 2.0x |
| Test CPU total | 76,121s | 21,033s | 3.6x |
| Test failures | 0 | 0 | — |
| **Total make check** | **~9.4min** | **3m41s** | **2.6x** |

### CI (GitHub Actions)

| Metric | Before | After | Improvement |
|--------|--------|-------|-------------|
| Aggregate CI time | 15,220s | 8,425s | 1.8x |
| build-linux-arm | 763s | 221s | 3.5x |
| build-linux-release | 616s | 244s | 2.5x |
| build-windows-vs2022 | 2,868s | 1,205s | 2.4x |
| build-linux-java-static | 858s | 335s | 2.6x |

### ccache hit rates (all jobs)

| Job | Before | After |
|-----|--------|-------|
| macOS cmake (0-3) | 0.51% | **99.80%** |
| build-linux | 99.68% | **99.84%** |
| build-linux-clang18-asan-ubsan | 98.58% | **99.84%** |
| build-linux-clang18-mini-tsan | 98.74% | **99.84%** |
| build-windows-vs2022 | 99.22% | **99.83%** |
| All 27 ccache jobs | — | **>94%** |

## Remaining bottleneck

The critical path is now dominated by genuinely slow integration tests:
- `external_sst_file_test` shards: 97–113s each
- `db_test` shards: 109–124s each
- `db_bloom_filter_test`: 103s (non-parallel)

Further improvement would require test optimization or a `make check-fast` target.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14478

Test Plan:
- `make check -j192 SKIP_FORMAT_BUCK_CHECKS=1` — all 4,045 shards pass (ASAN+UBSAN), zero failures
- Verified SyncPoint fixes: each crash/corruption reproduces before fix, passes after
- Verified SyncPoint::DisableProcessing wakes blocked threads (previously caused indefinite hangs)
- Verified gtest does NOT stop after failure with `GTEST_THROW_ON_FAILURE=0` — all tests in a shard run and all failures are reported
- CI: all 34 checks pass across two consecutive runs
- ccache: validated hit rates >94% on second run after cache seeding

Reviewed By: hx235

Differential Revision: D97623305

Pulled By: xingbowang

fbshipit-source-id: b299e6d43c8713a9ef2b5659e8bbd74958afe155
2026-03-24 15:39:27 -07:00
Josh Kang 304ae7190a Support ExternalTable PinnableSlice Get (#14497)
Summary:
The ExternalTableReader `Get` API has been modified to use PinnableSlice instead of std::string, this will allow implementations to utilize zero-copy Gets (e.g. reading from mmap or a cache). This is not a compatible change, but the API is marked as experimental, so should be allowed.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14497

Test Plan: - New unit tests

Reviewed By: xingbowang

Differential Revision: D97782011

Pulled By: joshkang97

fbshipit-source-id: 8a9e8c5bc5ff5e8dee6c0f2ee745521f09042cef
2026-03-24 11:10:42 -07:00
Hui Xiao 9de0ea32e1 Fix crash_test_with_ts failure: disable use_trie_index with timestamps (#14502)
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14502

The asan_crash_test_with_ts randomly selected use_trie_index=1 alongside
user_timestamp_size=8. TrieIndexFactory requires plain BytewiseComparator,
but user-defined timestamps wrap it as BytewiseComparator.u64ts, causing
Status::NotSupported errors during flush/compaction.

Fix:
1. Add use_trie_index=0 to ts_params in db_crashtest.py to prevent the
   incompatible combination (matches existing pattern for other features).
2. Add early validation in db_stress_tool.cc to reject this combination
   with a clear error message (defense in depth).

Reviewed By: xingbowang

Differential Revision: D97592648

fbshipit-source-id: 2259f82d95ea8410c4278b52f60983a4b5e84ec2
2026-03-24 09:28:35 -07:00
Xingbo Wang 77a0e347bb Add Claude code review CI workflow (#14480)
Summary:
Add GitHub Actions workflows for AI-powered code review on PRs using Claude (Anthropic). Follows the clang-tidy security pattern with two separate workflows for privilege separation.

## Trigger Modes

**1. Auto** — Runs after `pr-jobs` workflow completes successfully via `workflow_run`. Safe for fork PRs (runs from default branch, never executes PR code).

**2. Manual** — Maintainers comment `/claude-review [focus area]` or `/claude-query <question>` on any PR. Restricted to 15 authorized team members.

**3. workflow_dispatch** — For manual testing.

## Security Model (Two-Workflow Separation)

Same pattern as clang-tidy:

**`claude-review.yml`** (analysis):
- Runs Claude with `ANTHROPIC_API_KEY`
- Has ONLY `contents: read` — no PR write, no issue write
- Saves review markdown + metadata as artifact

**`claude-review-comment.yml`** (posting):
- Triggers on `workflow_run` completion
- Downloads artifact and posts/updates PR comment
- Has `pull-requests: write` but never runs AI

This separation prevents a crafted PR from tricking Claude into exfiltrating write tokens.

## Review Methodology

Review prompt in `claude_md/code_review.md` (shared with local Claude Code reviews). Five perspectives:
- Call-chain analysis (3-5 levels up/down)
- Correctness & edge cases
- Cross-component & adversarial (10 execution contexts)
- Performance
- API compatibility & test coverage

## Shared Scripts

- `.github/scripts/post-pr-comment.js` — Create-or-update PR comment with marker-based dedup. Now used by both clang-tidy and Claude review.
- `.github/scripts/parse-claude-review.js` — Parses `claude-code-base-action` execution log into markdown.

## Files Changed

| File | Description |
|------|-------------|
| `.github/workflows/claude-review.yml` | Analysis workflow (476 lines) |
| `.github/workflows/claude-review-comment.yml` | Comment posting workflow (146 lines) |
| `.github/scripts/post-pr-comment.js` | Shared PR comment utility (57 lines) |
| `.github/scripts/parse-claude-review.js` | Execution log parser (78 lines) |
| `.github/workflows/clang-tidy-comment.yml` | Updated to use shared script |
| `claude_md/code_review.md` | Review methodology (104 lines) |

## Setup Required

Add `ANTHROPIC_API_KEY` secret to the repo settings.

## Testing

Tested end-to-end on `xingbowang/rocksdb` fork — both auto and manual triggers, artifact upload/download, comment posting, and duplicate detection all verified working.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14480

Reviewed By: omkarhgawde

Differential Revision: D97832666

Pulled By: xingbowang

fbshipit-source-id: f80c7d8683ac980614dc4ca66c1e545deb3be504
2026-03-23 16:20:01 -07:00
Andrew Chang 89e384ca6f Support GetLiveFiles on secondary DB instances (#14475)
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14475

GetLiveFiles was previously blocked on secondary instances with
Status::NotSupported, even though the operation is safe to perform.

The only reason GetLiveFiles was originally blocked is that it defaults
to flushing the memtable (flush_memtable=true), which is a write
operation. However, the actual implementation in DBImpl::GetLiveFiles
(db_filesnapshot.cc) does two things:

1. Optionally flush the memtable — which we skip by passing
   flush_memtable=false. The secondary already overrides
   FlushForGetLiveFiles() as a no-op, so even if true were passed
   it would not actually flush.

2. Read live file state from versions_ under the mutex — this is
   purely read-only. It iterates the ColumnFamilySet to collect live
   table and blob file numbers, builds relative file paths for SST
   files, blob files, CURRENT, MANIFEST, and OPTIONS, and reads the
   manifest file size.

The secondary maintains its own VersionSet which it keeps up to date
via MANIFEST replay in TryCatchUpWithPrimary(). So all of this state
is valid and accurate — it reflects exactly which files the secondary
considers live at its current replay point.

This is the same approach used by DBImplReadOnly and CompactedDBImpl,
which both delegate to DBImpl::GetLiveFiles with flush_memtable=false.

Reviewed By: xingbowang

Differential Revision: D97563143

fbshipit-source-id: 8d5b52e26a478ef190eba598819de6527817bcfc
2026-03-23 12:09:10 -07:00
Josh Kang 3d99378791 Fix table handle leak (#14469)
Summary:
Fix leaked table cache entries that cause `TEST_VerifyNoObsoleteFilesCached` assertion failure during `DB::Close()` in ASAN crash test builds (T258745630):
```
File 126519 is not live nor quarantined
Assertion `cached_file_is_live_or_quar' failed.
```

When a compaction fails *after* `VerifyOutputFiles` succeeds (e.g., at `VerifyCompactionRecordCounts`), the overall `compact_->status` is set to error but each subcompaction's individual `status` remains OK. `SubcompactionState::Cleanup` only checked the individual subcompaction status, so it skipped calling `ReleaseObsolete` on the output files' table cache entries — leaking them.

Normally, `Close()`'s backstop (`FindObsoleteFiles(force=true)` + `PurgeObsoleteFiles`) would catch this by finding the orphan file on disk, evicting the cache entry, and deleting the file. However, `FindObsoleteFiles` calls `FaultInjectionTestFS::GetChildren` which can fail under metadata read fault injection (`--open_metadata_read_fault_one_in=8`). The error is silently ignored (`s.PermitUncheckedError()` in db_impl_files.cc:199), so the orphan file is never found and the leaked cache entry is never evicted.

The `Cleanup` bug is latent and predates recent changes. PR https://github.com/facebook/rocksdb/issues/14433 added `verify_output_flags` randomization to the crash test, and PR https://github.com/facebook/rocksdb/issues/14456 fixed false-positive corruptions that https://github.com/facebook/rocksdb/issues/14433 caused. Before https://github.com/facebook/rocksdb/issues/14456, `VerifyOutputFiles` would produce false corruption errors that *accidentally prevented the leak* by setting the subcompaction status to non-OK.

### How it triggers

**Step 1 — VerifyOutputFiles adds cache entries for compaction output files:**
```
CompactionJob::Run()
  RunSubcompactions()              // all subcompactions succeed
                                   // output file 12 written to disk
                                   // each sub_compact.status = OK
  SyncOutputDirectories()          // status = OK
  VerifyOutputFiles()
    for each output_file:
      table_cache()->NewIterator(output_file.meta)
        FindTable()
          cache->Insert(file_number=12)   // <<< ENTRY ADDED TO TABLE CACHE
        return iterator holding handle
      delete iter                          // releases handle, entry stays in LRU
    // status = OK
```

**Step 2 — A post-verification step fails, overall status set but NOT subcompaction status:**
```
CompactionJob::Run()
  VerifyCompactionRecordCounts()   // returns Status::Corruption(...)
  FinalizeCompactionRun(status=Corruption)
    compact_->status = Corruption  // <<< OVERALL status = error
    // each sub_compact.status is still OK!
```

**Step 3 — Install skips InstallCompactionResults (file 12 never enters a Version):**
```
CompactionJob::Install()
  status = compact_->status        // Corruption
  if (status.ok())                 // FALSE
    InstallCompactionResults()     // <<< SKIPPED — file 12 not in any version
```

**Step 4 — CleanupCompaction skips ReleaseObsolete (THE BUG):**
```
CompactionJob::CleanupCompaction()
  for each sub_compact:
    sub_compact.Cleanup(table_cache)
      if (!status.ok())            // checks sub_compact.status = OK
                                   // <<< FALSE — individual status is OK!
        ReleaseObsolete(...)       // <<< NEVER CALLED — cache entry leaked!
```

**Step 5 — Metadata read fault prevents backstop from finding the orphan:**

`FindObsoleteFiles(force=true)` calls `GetChildren()` to scan the DB directory.
`FaultInjectionTestFS::GetChildren` injects a metadata read error
(`--open_metadata_read_fault_one_in=8`). The error is silently ignored
(`s.PermitUncheckedError()`), so the directory listing is empty and the orphan
file is never found. This happens both in the post-compaction cleanup
(`BackgroundCallCompaction`) and in `Close()`'s backstop:
```
FindObsoleteFiles(force=true)
  fs->GetChildren(path, ...)       // returns IOError (fault injected)
  s.PermitUncheckedError();        // error silently ignored
  // files vector is empty — orphan file 12 not found
PurgeObsoleteFiles()               // nothing to do — no Evict called
```

**Step 6 — Assertion fires during Close():**
```
CloseHelper()
  FindObsoleteFiles(force=true)    // GetChildren fails again → orphan missed
  PurgeObsoleteFiles()             // nothing to evict
  TEST_VerifyNoObsoleteFilesCached()
    for each cache entry:
      file_number = 12
      live_and_quar_files.find(12) == end()  // NOT in any version!
      >>> assert(cached_file_is_live_or_quar) FAILS <<<
```

**Crash test call stack:**
```
frame https://github.com/facebook/rocksdb/issues/9:  __assert_fail_base("cached_file_is_live_or_quar", "db_impl_debug.cc", 389)
frame https://github.com/facebook/rocksdb/issues/11: DBImpl::TEST_VerifyNoObsoleteFilesCached()::lambda  // finds leaked entry
frame https://github.com/facebook/rocksdb/issues/18: LRUCacheShard::ApplyToSomeEntries(...)              // iterating cache shard
frame https://github.com/facebook/rocksdb/issues/19: ShardedCache::ApplyToAllEntries(...)                // iterating all shards
frame https://github.com/facebook/rocksdb/issues/20: DBImpl::TEST_VerifyNoObsoleteFilesCached()          // the verification
frame https://github.com/facebook/rocksdb/issues/21: DBImpl::CloseHelper()                               // during Close
frame https://github.com/facebook/rocksdb/issues/22: DBImpl::CloseImpl()
frame https://github.com/facebook/rocksdb/issues/23: DBImpl::Close()
frame https://github.com/facebook/rocksdb/issues/24: StressTest::Reopen()                                // crash test reopen
frame https://github.com/facebook/rocksdb/issues/25: StressTest::OperateDb()                             // worker thread
```

### Fix

Pass the overall `compact_->status` to `SubcompactionState::Cleanup` and call
`ReleaseObsolete` when *either* the subcompaction status or the overall status
is non-OK:
```cpp
// Before:
if (!status.ok()) { ReleaseObsolete(...); }

// After:
if (!status.ok() || !overall_status.ok()) { ReleaseObsolete(...); }
```

## Key changes

- **`SubcompactionState::Cleanup`**: Now takes an `overall_status` parameter and calls `ReleaseObsolete` when *either* the subcompaction status or the overall compaction status is non-OK.
- **`CompactionJob::CleanupCompaction`**: Passes `compact_->status` (the overall status) to each subcompaction's `Cleanup`.
- **Sync point**: Added `CompactionJob::Run():AfterVerifyOutputFiles` for error injection in tests.
- **Unit test**: `DBCompactionTest.LeakedTableCacheEntryOnCompactionFailure` uses `FaultInjectionTestFS` to reproduce the crash test scenario — injects error after `VerifyOutputFiles` and deactivates the filesystem so `GetChildren` fails in `FindObsoleteFiles`, preventing the backstop from evicting the leaked cache entry.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14469

Test Plan:
- `DBCompactionTest.LeakedTableCacheEntryOnCompactionFailure`:
  - **Without fix (ASAN)**: assertion fires during `Close()` — `File 12 is not live nor quarantined`
  - **With fix (ASAN)**: passes — `ReleaseObsolete` properly cleans up the cache entry
  - **With fix (non-ASAN)**: passes
```
$ COMPILE_WITH_ASAN=1 make -j db_compaction_test
$ ./db_compaction_test --gtest_filter="DBCompactionTest.LeakedTableCacheEntryOnCompactionFailure"
[  PASSED  ] 1 test.
```

Reviewed By: xingbowang

Differential Revision: D97190944

Pulled By: joshkang97

fbshipit-source-id: fdfd481cc1e192803cfb7d64052ccb9162c21b94
2026-03-23 10:50:43 -07:00
Hui Xiao f455ab7bd6 Fix flaky BackgroundJobPressure test (#14476)
Summary:
**Context/Summary:**
Remove assertions on flush_running/flush_scheduled from the BackgroundJobPressure listener test. These checks are inherently racy because Flush() can return before the background thread fires the pressure callback.

The race: when BackgroundCallFlush re-acquires the DB mutex after cleanup (line 3661), COERCE_CONTEXT_SWITCH=1 or others may call bg_cv_->SignalAll() and sleep before actually acquiring the lock. This spurious signal wakes WaitForFlushMemTables(), which sees the flush is already installed and returns — so Flush() "completes" seen as foreground in test. The test then starts the next Put()+Flush(), scheduling new flush work. When the previous background thread finally wakes and captures the pressure snapshot, it sees the newly scheduled flush (flush_scheduled > 0). If the race happens on the last flush, its callback may not have fired at all by the time the test reads GetSnapshots(), so snapshots.back() can also show stale data.

The remaining assertions (compaction counts, speedup, write stall proximity) are not affected: compaction is blocked by sleeping_task in Phase 1-2, and Phase 3 uses TEST_WaitForCompact() whose wait condition (bg_*_scheduled_ counts) is only satisfied after the mutex is re-acquired and callbacks have fired.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14476

Test Plan:
COERCE_CONTEXT_SWITCH=1 make -j56 listener_test
  Before: ~22/50 failures at line 1715 (flush_running != 0)
  After:  50/50 pass

Reviewed By: xingbowang

Differential Revision: D97572545

Pulled By: hx235

fbshipit-source-id: e8a57a7a00e41d17bf47ef8e04cea977225a5909
2026-03-20 18:58:34 -07:00
Hui Xiao 63c86160cd Add OnBackgroundJobPressureChanged listener callback (#14474)
Summary:
**Summary:**
Add a new EventListener callback `OnBackgroundJobPressureChanged` that fires after every flush or compaction background job completes. The callback delivers a `BackgroundJobPressure` snapshot containing:
- Compaction scheduling counters (scheduled/running, combined and per-priority LOW/BOTTOM breakdown)
- Flush scheduling counters (scheduled/running)
- Write stall proximity percentage (0=healthy, 100=at stall threshold, can exceed 100 when stalling)
- Whether compaction speedup is active

`CaptureBackgroundJobPressure()` reads scheduling counters and computes write stall proximity from L0 sorted run count and pending compaction bytes (same inputs as `RecalculateWriteStallConditions()`). TODO: add memory-related write stall triggers later.

Introduces `num_running_bottom_compactions_` counter to track BOTTOM- priority compactions separately from LOW, enabling per-pool breakdown in the pressure snapshot.

The callback fires on the background thread after counter decrements and `MaybeScheduleFlushOrCompaction()`, so the snapshot reflects post- completion state. Uses the same mutex unlock/lock pattern as `NotifyOnFlushCompleted`. A `bg_pressure_callback_in_progress_` counter ensures destructor safety since the callback fires after `bg_flush_scheduled_`/`bg_compaction_scheduled_` are decremented.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14474

Test Plan:
- listener_test BackgroundJobPressure: 3-phase test verifying no pressure, pressure build-up (speedup, scheduling, proximity), and pressure relief after compaction completes
- db_compaction_test CompactRangeBottomPri: verifies num_running_bottom_compactions_ via sync point
- db_stress_tool exercises the callback with RandomSleep()

Reviewed By: xingbowang

Differential Revision: D97423623

Pulled By: hx235

fbshipit-source-id: 07003c8de226ec29d32b8a88e2d86e5de85cd2cc
2026-03-20 15:02:53 -07:00
Jay Huh 89322fdd9e Skip Wal Recovery on SecondaryDB Open if for Remote Compaction (#14462)
Summary:
Skip WAL recovery when opening a secondary DB instance in OpenAndCompact() for remote compaction. WAL replay is unnecessary in this flow since only LSM state from MANIFEST is needed.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14462

Test Plan:
- make -j db_secondary_test && ./db_secondary_test — 35/35 passed
- make -j compaction_service_test && ./compaction_service_test — 43/43 passed (includes new SkipWALRecoveryInOpenAndCompact test)
- make -j options_settable_test && ./options_settable_test --gtest_filter="*DBOptionsAllFieldsSettable*" — 1/1 passed
- Removed temporary hack in stress test that disables WAL

Reviewed By: hx235

Differential Revision: D96788211

Pulled By: jaykorean

fbshipit-source-id: f91a2f861f2450ebc83423ed4c6f5b70da7d9e8b
2026-03-19 15:48:16 -07:00
Yoshinori Matsunobu b23fc77aca Add per-block-type block read byte perf counters (#14473)
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14473

Add separate PerfContext byte counters for data, index, filter, compression-dictionary, and metadata block reads while preserving block_read_byte as the aggregate. Wire the new counters through the block fetch and multi-read data block paths, expose them via the C perfcontext API, and extend table tests to verify byte attribution and that the classified counters sum back to block_read_byte.

Reviewed By: xingbowang

Differential Revision: D97333746

fbshipit-source-id: 3411844d7fa9c76c9ff28af477b3a72a5d6e5d9b
2026-03-19 14:54:24 -07:00
Danny Chen b6f498b2c9 Add verify_manifest_content_on_close option (#14451)
Summary:
Add a new mutable DB option `verify_manifest_content_on_close` (default: false).
When enabled, on DB close the MANIFEST file is read back and all records are
validated (CRC checksums via log::Reader and logical content via
VersionEdit::DecodeFrom). If corruption is detected, a fresh MANIFEST is written
from in-memory state using the existing LogAndApply recovery path.

This complements the existing size validation in VersionSet::Close() with content
validation, reusing the same manifest reading pattern as VersionSet::Recover().

Implementation plan:

## Part 1: New DB Option — verify_manifest_content_on_close
- A new mutable bool DB option (default: false) that can be dynamically toggled
  via SetDBOptions() at runtime, following the pattern of other mutable manifest
  options like max_manifest_file_size.
- Propagation: SetDBOptions() -> DBImpl::mutable_db_options_ ->
  versions_->UpdatedMutableDbOptions() -> VersionSet::verify_manifest_content_on_close_

## Part 2: Core Implementation — Content Validation in VersionSet::Close()
- Inserted after existing size check, before closed_ = true
- Opens manifest as SequentialFileReader, creates log::Reader with checksum=true
- Loops ReadRecord with WALRecoveryMode::kAbsoluteConsistency, decodes each
  record as VersionEdit
- On corruption: fires OnIOError listeners, logs error, calls LogAndApply with
  empty edit to trigger manifest rewrite from in-memory state
- If manifest can't be opened for reading: logs warning, doesn't fail close

## Part 3: Unit Tests (in version_set_test.cc)
- ManifestContentValidationOnClose_Clean: enable option, normal close, verify
  no manifest rotation
- ManifestContentValidationOnClose_CorruptRecord: enable option, corrupt manifest
  via SyncPoint, verify rotation occurs and DB reopens cleanly
- ManifestContentValidationOnClose_Disabled: default off, verify content
  validation does not run
- ManifestContentValidationOnClose_SizeCheckFails: truncate manifest so size
  check fails first, verify recovery via size-check path

## What Happens If a Corruption is Detected
If corruption was detected, four things happen:
1. **Notify listeners** — Fires `OnIOError` on all registered event listeners
   (from db_options_->listeners) so monitoring/alerting systems can observe
   the corruption event. Uses `FileOperationType::kVerify` to categorize it.
2. **Permit unchecked errors** — `PermitUncheckedError()` silences RocksDB's
   debug-mode assertion that every `IOStatus` must be inspected. These statuses
   are informational-only here; the real recovery is via `LogAndApply`.
3. **Log the error** — Writes a `ROCKS_LOG_ERROR` message with the filename
   for operational visibility (grep-able in production logs).
4. **Rewrite the manifest via `LogAndApply`** — This is the actual recovery.
   `LogAndApply` is called with an empty `VersionEdit` (no changes). Internally,
   `LogAndApply` detects that the current `descriptor_log_` is null (it was
   reset at line 5551, or by the previous `LogAndApply` in the size-check
   path) and creates a brand-new MANIFEST file. It serializes the entire
   current in-memory LSM state — all column families, all levels, all file
   metadata, sequence numbers, etc. — into this new file. It then atomically
   updates the `CURRENT` file pointer to reference the new MANIFEST.
   This works because the in-memory state was built from the original manifest
   during `DB::Open()` and has been kept fully up to date through all
   subsequent operations (flushes, compactions, etc.) during the DB's lifetime.
   The on-disk manifest is essentially a journal of changes; `LogAndApply`
   with an empty edit produces a fresh, compacted snapshot of that state.

## Flow Diagram of Manifest Content Validation

VersionSet::Close()
│
├─ Close descriptor_log_ and check size
│  └─ Size mismatch? → LogAndApply (rewrite manifest)
│
├─ Content validation (if s.ok() && option enabled)
│  ├─ Open manifest for sequential reading
│  │  └─ Can't open? → WARN log, continue
│  │
│  ├─ For each record:
│  │  ├─ ReadRecord (CRC32 check, kAbsoluteConsistency)
│  │  └─ DecodeFrom (VersionEdit logical check)
│  │
│  └─ Corruption detected?
│     ├─ Notify OnIOError listeners
│     ├─ LOG_ERROR
│     └─ LogAndApply (rewrite manifest from in-memory state)
│
└─ closed_ = true; return s;

## How This Relates to the Existing Size Check
The existing size check (lines 5556-5582) and the new content validation are
complementary:
| Check          | What it catches                         | How it checks              |
|----------------|-----------------------------------------|----------------------------|
| Size check     | Truncation, partial writes, extra bytes | Compare expected vs actual file size |
| Content check  | Bit-rot, silent corruption, bad records | CRC32 + VersionEdit decode |
The size check catches gross corruption (file too short or too long). The
content check catches subtle corruption where the file is the right size but
individual bytes have been flipped (e.g., storage media bit-rot, buggy
filesystem, incomplete block write).
Both recovery paths use the same mechanism: `LogAndApply` with an empty
`VersionEdit` to rewrite the manifest from in-memory state.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14451

Reviewed By: xingbowang

Differential Revision: D96004906

Pulled By: dannyhchen

fbshipit-source-id: 0b0ecdada3a74e97d2cadbba2091b8b577f1d684
2026-03-19 12:01:23 -07:00
Xingbo Wang bcaf2794dc Fix TSAN data race in InjectedErrorLog by suppressing benign races (#14467)
Summary:
InjectedErrorLog is a lock-free circular ring buffer designed to be safe to call from signal handlers (which cannot use locks). It has an intentional benign data race between Record() (called by worker threads) and PrintAll() (called by the main thread from a signal/termination handler). The code documents this trade-off in comments, but was missing TSAN suppression annotations.

Simply adding TSAN_SUPPRESSION (__attribute__((no_sanitize("thread")))) is insufficient because TSAN still intercepts libc functions like vsnprintf/snprintf -- accesses through these interceptors are still tracked even when the calling function is annotated.

The fix:
1. Add TSAN_SUPPRESSION to both Record() and PrintAll() to suppress direct field reads/writes in the function body.
2. Restructure both functions to use local stack buffers for vsnprintf/snprintf operations instead of operating directly on shared entry data. This avoids passing shared memory through TSAN-intercepted libc functions.

Also adds fault_injection_fs_test with a ConcurrentRecordAndPrintAll test that exercises the concurrent Record() + PrintAll() pattern and verifies no TSAN race is reported.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14467

Test Plan:
- fault_injection_fs_test passes under TSAN (buck2 test fbcode//mode/dbg-tsan)
- Reverted fix, re-ran: Fatal (6 TSAN warnings) -- round-trip confirmed
- fault_injection_fs_test passes under debug mode (no regression)

Reviewed By: mszeszko-meta

Differential Revision: D96948483

Pulled By: xingbowang

fbshipit-source-id: efdd5eafa12a5a82f973e40aa327901cc5f95033
2026-03-19 09:40:02 -07:00
Andrew Chang 1cb7594b87 Set correct file_type in BackupInfo::file_details (#14464)
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14464

SetBackupInfoFromBackupMeta passes file_ptr->filename (which may contain
directory components like "private/1/000008.log") directly to
ParseFileName. ParseFileName expects a bare filename, so it fails and
file_type stays at the default kTempFile for all files in file_details.

Fix by extracting the basename before calling ParseFileName.

Reviewed By: mszeszko-meta

Differential Revision: D96793040

fbshipit-source-id: 7bb6b633eb07bb7ebda06edbc039a96b9b77b410
2026-03-18 12:01:29 -07:00
Dmitry Vinnik 166b4a4487 Remove Support Ukraine banner from RocksDB website (#14463)
Summary:
Remove the Support Ukraine socialBanner div from the RocksDB docs homepage layout.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14463

Reviewed By: hx235

Differential Revision: D96559640

Pulled By: xingbowang

fbshipit-source-id: 508c7bd660760e39cc73977c5e5c1bda204efedd
2026-03-17 10:15:24 -07:00
anand76 03bded03b4 Fix finger.prev_[0] assertion failure in MultiGet finger search (#14465)
Summary:
The callback loop in InlineSkipList::MultiGet updated finger.prev_[0] as it walked forward through entries (e.g., merge operands). When the MultiGet batch contained duplicate user keys, the next lookup for the same key would find finger.prev_[0] pointing to an entry that sorts AFTER the lookup key in internal key order (because the lookup key has a high sequence number which sorts first), violating the FindSpliceForLevel precondition: before == head_ || KeyIsAfterNode(key, before).

Fix: stop updating finger.prev_[0] in the callback loop. Only finger.next_[0] needs advancing to track the walk-forward position. The prev_[0] from FindGreaterOrEqualWithFinger is always a valid lower bound for any subsequent key, whether it uses kMaxSequenceNumber or a snapshot sequence number.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14465

Reviewed By: xingbowang

Differential Revision: D96882549

Pulled By: anand1976

fbshipit-source-id: a733fa9d4f23f8b55a027c257a114c4cf35abe2b
2026-03-17 09:57:24 -07:00
zaidoon ec22903914 Support all operation types in User Defined Index (UDI) interface (#14399)
Summary:
Remove the restriction that limited UDI to ingest-only, Puts-only use cases. This enables UDI plugins (including the trie index from https://github.com/facebook/rocksdb/issues/14310) to work with all operation types: Put, Delete, Merge, SingleDelete, PutEntity, etc.

Part of https://github.com/facebook/rocksdb/issues/12396

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14399

Reviewed By: pdillinger

Differential Revision: D96392636

Pulled By: xingbowang

fbshipit-source-id: 0f1e6c38531fa72539a0e2c6a3dffff333392b4c
2026-03-16 15:25:05 -07:00
Josh Kang 0c6f741422 Rename write_prepared_transaction_test_seqno to write_prepared_transaction_seqno_test (#14453)
Summary:
write_prepared_transaction_test_seqno keeps showing up in git changes

test binaries need to end with _test so they are ignored by .gitignore

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14453

Reviewed By: xingbowang

Differential Revision: D96202671

Pulled By: joshkang97

fbshipit-source-id: 91642226bedde37ca72c6af03f300990dca8ff3a
2026-03-16 14:33:13 -07:00
anand76 c66c14258f Add memtable MultiGet finger search optimization (#14428)
Summary:
Add memtable batch lookup optimization with finger search

Optimize memtable MultiGet by using a finger search on the skip list. After finding key[i], the search path (Splice) is retained as a "finger" for key[i+1]. The next search walks up the finger until the forward pointer overshoots, then descends -- costing O(log d) where d is the distance between consecutive sorted keys, rather than O(log N) from the head each time.

Controlled by the new `memtable_batch_lookup_optimization` column family option (default: false).

## Changes

- Add `FindGreaterOrEqualWithFinger()` and `MultiGet()` to `InlineSkipList` with optional paranoid validation support (key ordering checks and per-key checksum verification via default parameters)
- Add virtual `MultiGet()` to `MemTableRep`, override in `SkipListRep`
- Add `memtable_batch_lookup_optimization` CF option
- Integrate finger search into `MemTable::MultiGet` -- sorts keys, performs batched finger search, then unscrambles results
- Add `db_bench`, `db_stress`, and crash test support
- Add unit tests for `InlineSkipList::MultiGet` (5 tests) and integration tests at the DB level (25 `BatchLookup` tests), including paranoid validation tests

## Benchmark Results

Setup: 2M keys in memtable, release build (`DEBUG_LEVEL=0`).

### Single-threaded `multireadrandom`

| Batch Size | Baseline | BatchOpt | vs Base |
|:---:|---:|---:|---:|
| 2 | 363,593 | 376,562 | **+3.6%** |
| 8 | 385,204 | 386,082 | **+0.2%** |
| 32 | 360,339 | 375,105 | **+4.1%** |
| 64 | 352,696 | 378,497 | **+7.3%** |

### Multithreaded `multireadrandom` (batch_size=64)

| Threads | Baseline | BatchOpt | vs Base |
|:---:|---:|---:|---:|
| 4 | 171,356 | 185,633 | **+8.3%** |
| 8 | 161,478 | 171,392 | **+6.1%** |

### `multireadwhilewriting` (batch_size=64)

| Threads | Baseline | BatchOpt | vs Base |
|:---:|---:|---:|---:|
| 1 | 163,938 | 175,068 | **+6.8%** |
| 4 | 109,698 | 120,790 | **+10.1%** |
| 8 | 116,623 | 123,658 | **+6.0%** |

No regression at any batch size or thread count. The finger is stack-allocated per `MultiGet` call, so there is no shared state or cache-line contention between threads. Concurrent writes don't invalidate the finger's bracket since it only reads via acquire-load `Next()` pointers.

Small Batch Size Regression Check: memtable_batch_lookup_optimization
=====================================================================

All values are ops/sec. Measured with db_bench (release build).
2M keys in memtable, value_size=100, duration=30s per run.

Multithreaded multireadrandom — small batch sizes
--------------------------------------------------
Threads | Batch Size | Baseline   | BatchOpt   | Change
   4    |     2      |   532,302  |   540,682  |  +1.6%
   8    |     2      | 1,044,046  | 1,046,920  |  +0.3%
   4    |     8      |   633,733  |   630,039  |  -0.6%
   8    |     8      | 1,260,818  | 1,241,792  |  -1.5%

multireadwhilewriting — small batch sizes
-----------------------------------------
Threads | Batch Size | Baseline   | BatchOpt   | Change
   1    |     2      |   107,284  |   105,217  |  -1.9%
   4    |     2      |   428,508  |   423,747  |  -1.1%
   8    |     2      |   854,654  |   864,923  |  +1.2%
   1    |     8      |   114,788  |   118,496  |  +3.2%
   4    |     8      |   467,995  |   473,437  |  +1.2%
   8    |     8      |   935,636  |   960,791  |  +2.7%

No regression at small batch sizes. Variations at batch_size=2 are
within noise (~1-2%). At batch_size=8, modest positive trend (+1-3%
in read-while-writing).

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14428

Test Plan:
- `make check` -- all tests pass
- `ASSERT_STATUS_CHECKED=1 make check` -- all tests pass
- 2-hour `db_crashtest.py blackbox` with `--memtable_batch_lookup_optimization=1` -- passed
- 2-hour `db_crashtest.py blackbox` with `--memtable_batch_lookup_optimization=1 --paranoid_memory_checks=1` -- passed

Reviewed By: pdillinger

Differential Revision: D95813786

Pulled By: anand1976

fbshipit-source-id: b182a023e1026021f1b9682d1cc024c7729326c7
2026-03-16 10:45:49 -07:00
Omkar Gawde e43171d6ca Fix memory leak in ExportColumnFamily for empty column families (#14458)
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14458

In `CheckpointImpl::ExportColumnFamily()`, the assignment
`*metadata = result_metadata` is inside the `for` loop over
`db_metadata.levels`. If the column family has no levels, the loop body
never executes, so the `new ExportImportFilesMetaData()` is leaked and
the caller receives a nullptr despite a success status.

Fix: Move `*metadata = result_metadata` outside the loop.

Reviewed By: anand1976

Differential Revision: D95303457

fbshipit-source-id: 6a9be47bcca257803969eb3daac7b91e95143ebf
2026-03-13 21:14:48 -07:00
Omkar Gawde 1e1097d8d3 Fix close(-1) on failed open in btrfs rename fsync path (#14443)
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14443

In `PosixDirectory::FsyncWithDirOptions()`, when handling btrfs file rename
syncing, if `open()` fails and `fd` is -1, the code unconditionally calls
`close(fd)`. Calling `close(-1)` is undefined behavior per POSIX (returns
EBADF on Linux), and overwrites the original meaningful open error with a
misleading "While closing file after fsync" error message.

Fix: Guard the `close()` call with `fd >= 0`.

Reviewed By: xingbowang

Differential Revision: D95303407

fbshipit-source-id: 9b64b45a09e6ba41d87d164a1c094b4d22f7c186
2026-03-13 12:26:44 -07:00
Xingbo Wang e44a99f560 Fix false positive corruption in compaction output verification (#14456)
Summary:
Fix a bug where `VerifyOutputFiles()` produces false positive "Key-value checksum of compaction output doesn't match what was computed when written" errors when `verify_output_flags` includes `kVerifyIteration` but `paranoid_file_checks` is false.

The root cause is a hash enable flag mismatch between writing and verification:

- During compaction writing (`OpenCompactionOutputFile`), the `OutputValidator` hash computation was gated solely by `paranoid_file_checks_`. When false, `enable_hash=false` and the hash stays at 0.
- During verification (`VerifyOutputFiles`), a new `OutputValidator` is always created with `enable_hash=true`, computing a non-zero hash.
- `CompareValidator()` then compares 0 vs non-zero, producing a false positive corruption.

This was exposed by the crash test randomization of `verify_output_flags` added in https://github.com/facebook/rocksdb/issues/14433. Before that change, `verify_output_flags` was always 0, so `kVerifyIteration` was only exercised via `paranoid_file_checks=true` (which correctly enabled the hash during writing).

The fix ensures hash computation is enabled during writing whenever either `paranoid_file_checks_` is true OR `verify_output_flags` includes `kVerifyIteration`.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14456

Test Plan:
- Added `DBCompactionTest.VerifyIterationWithoutParanoidFileChecks`
- Added `DBCompactionTest.VerifyAllOutputFlagsWithoutParanoidFileChecks`
- Round-trip verified: tests FAIL without fix, PASS with fix

Reviewed By: jaykorean

Differential Revision: D96371769

Pulled By: xingbowang

fbshipit-source-id: 2f7406327496c7b541e4fa2668894df89eb813e8
2026-03-12 16:26:30 -07:00
Jay Huh 5494bc1d76 Add option to verify file checksum of output files (#14433)
Summary:
One of the follow ups from https://github.com/facebook/rocksdb/pull/14103. Users will have the option to verify file checksums for all compaction output files before they are installed. This feature helps prevent corrupted SST files from being added to the LSM tree.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14433

Test Plan: Unit test added

Reviewed By: archang19

Differential Revision: D95648000

Pulled By: jaykorean

fbshipit-source-id: 512f3c1a7449b96a660865531f3537624c89a9cc
2026-03-11 21:14:28 -07:00
Xingbo Wang 91f227d9be Add injected error log ring buffer for fault injection diagnostics (#14431)
Summary:
Add a circular ring buffer (InjectedErrorLog) to FaultInjectionTestFS that records the last 1000 injected errors. Each entry captures the timestamp, thread ID, FS API name with all arguments (file path, offset, buffer size, first 8 bytes of data in hex), and the injected error status. For example:

  Append("/path/035354.sst", size=4096, head=[1a 2b 3c ...]) -> IOError (Retryable): injected write error
  RenameFile("/path/tmp.sst", "/path/035355.sst") -> IOError: injected metadata write error
  Read(offset=16384, size=4096) -> IOError (Retryable): injected read error

The ring buffer is printed to a file automatically:
- On any fatal signal (SIGABRT, SIGSEGV, SIGTERM, SIGINT, SIGHUP, SIGFPE, SIGBUS, SIGILL, SIGQUIT, SIGXCPU, SIGXFSZ, SIGSYS) via a registered crash callback
- At the end of db_stress main(), for diagnostic visibility even when the test completes normally

This addresses a key debugging gap: when write fault injection causes secondary failures (e.g., the builder error propagation issue in T257612259), the injected errors were previously completely silent with no logging trail. The ring buffer provides the missing diagnostic context to correlate fault injection with downstream failures.

Changes:
- port/stack_trace.h/.cc: Add RegisterCrashCallback() API; extend InstallStackTraceHandler() to catch all catchable termination signals
- utilities/fault_injection_fs.h: Add InjectedErrorLog class with printf-style Record(), HexHead() for data bytes, and signal-safe PrintAll()
- utilities/fault_injection_fs.cc: Record full API arguments and error status at all 31 fault injection call sites
- db_stress_tool/db_stress_tool.cc: Register crash callback and print ring buffer at end of main()

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14431

Reviewed By: hx235

Differential Revision: D95435430

Pulled By: xingbowang

fbshipit-source-id: 6c18e1b072044575d6c8c3f198070127b0f80608
2026-03-11 18:15:04 -07:00
Omkar Gawde cb8bc56d14 Fix memory leak on error in C API create_column_family (#14447)
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14447

`rocksdb_create_column_family()` and
`rocksdb_transactiondb_create_column_family()` allocate a
`rocksdb_column_family_handle_t` but always return it even when
`CreateColumnFamily()` fails. This leaks the handle and returns an object
with an indeterminate `rep` pointer. Other similar functions like
`rocksdb_create_column_family_with_import()` correctly delete the handle
and return nullptr on error.

Fix: Initialize `handle->rep = nullptr`, check `SaveError()` return value,
and on error delete the handle and return nullptr.

Reviewed By: anand1976

Differential Revision: D95303444

fbshipit-source-id: 5fde7a3ed588794d78429d5cb3d9f621f0fb6388
2026-03-10 22:56:54 -07:00
Maciej Szeszko 1ed5052486 Prepopulate block cache during compaction (#14445)
Summary:
When RocksDB operates with tiered or remote storage (e.g., Warm Storage, HDFS, S3), reading recently compacted data incurs high-latency remote reads because compaction output files are not present in the block cache. The existing `prepopulate_block_cache = kFlushOnly` avoids this for flush output but leaves compaction output cold until first access.

Add a new `PrepopulateBlockCache::kFlushAndCompaction` enum value that warms all block types (data, index, filter, compression dict) into the block cache during both flush and compaction. Flush-warmed blocks use `LOW` priority (unchanged from kFlushOnly behavior), while compaction-warmed blocks use `BOTTOM` priority — compaction data is less temporally local than freshly flushed data, so it should be the first to be evicted when the cache is full. This gives the remote-read avoidance benefit without risking cache thrashing.

The enum uses `kFlushAndCompaction` rather than separate `kCompactionOnly` + `kFlushAndCompaction` values because there is no practical use case for warming compaction output without also warming flush output. Flush output is by definition the hottest data (just written by the user), so if a workload benefits from warming the colder compaction output, it would always benefit from warming flush output too.

The implementation reuses the existing `InsertBlockInCacheHelper` / `WarmInCache` infrastructure in `BlockBasedTableBuilder`. The only internal change is adding a `warm_cache_priority` field to `Rep` alongside the existing `warm_cache` bool, and plumbing it through to the `WarmInCache` call instead of the previously hardcoded `Cache::Priority::LOW`.

### Key changes
- New `PrepopulateBlockCache::kFlushAndCompaction` enum value in table.h
- `Rep::warm_cache_priority` field in BlockBasedTableBuilder for per-reason priority control
- Serialization support ("kFlushAndCompaction" in string map)
- db_bench support (--prepopulate_block_cache=2)
- Crash test coverage (random choice includes new value)

**NOTE:** Unlike flush output (which is inherently hot — just written by the user), it is hard to distinguish hot from cold blocks in compaction output. Warming all compaction output therefore risks polluting the block cache and evicting genuinely hot entries. The kFlushAndCompaction mode is recommended only for use cases where most or all of the database is expected to reside in cache (e.g., the working set fits in cache). For workloads where only a fraction of the data is hot, kFlushOnly remains the safer choice.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14445

Test Plan:
- New `WarmCacheWithDataBlocksDuringCompaction` test: verifies data blocks from compaction output are present in the block cache and served without misses
- Extended `DynamicOptions` test: verifies dynamic switching through kDisable -> kFlushAndCompaction -> kFlushOnly -> kDisable via SetOptions
- Existing `WarmCacheWithDataBlocksDuringFlush` and parameterized `WarmCacheWithBlocksDuringFlush` tests continue to pass (kFlushOnly behavior unchanged)
- db_block_cache_test: 81/81 passed
- options_test: 74/74 passed
- table_test: 6910/6910 passed

Reviewed By: xingbowang

Differential Revision: D95997952

Pulled By: mszeszko-meta

fbshipit-source-id: 4ad568264992532053947df298e63e343821ddb5
2026-03-10 16:19:33 -07:00
Peter Dillinger db9449a154 Fix crash test failures when use_trie_index conflicts with txn options (#14446)
Summary:
The trie UDI sanitization in db_crashtest.py disables use_txn because TransactionDB ROLLBACK writes DELETE entries that violate UDI's Put-only restriction. However, it was not clearing use_optimistic_txn or test_multi_ops_txns, which both require use_txn to be true.

Since use_trie_index is randomly enabled with 1/8 probability, the optimistic_txn and multiops_txn crash tests would intermittently fail with:
- "You cannot set use_optimistic_txn true while use_txn is false"
- "-use_txn must be true if -test_multi_ops_txns"

Fix by also setting use_optimistic_txn=0 and test_multi_ops_txns=0 in the trie UDI sanitization block alongside the existing use_txn=0.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14446

Test Plan: Watch crash test CI results

Reviewed By: xingbowang

Differential Revision: D95986370

Pulled By: pdillinger

fbshipit-source-id: 0751dcb4e99425ba9eb9aa34c2a99efa8eb3a194
2026-03-10 14:35:28 -07:00
Xingbo Wang 26aff32db3 Rename IsExpectedTxnLockTimeout to IsExpectedTxnError (#14441)
Summary:
The function handles more than just lock timeouts — it also covers TryAgain from optimistic transactions. Rename it and update the comment to reflect its broader scope.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14441

Reviewed By: hx235

Differential Revision: D95877799

Pulled By: xingbowang

fbshipit-source-id: 0651ffd39ac9d3a7979750be6dfe5b293eab7a64
2026-03-10 13:33:31 -07:00
Omkar Gawde 2afb387917 Fix missed condition variable signal in DeleteScheduler (#14442)
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14442

In `BackgroundEmptyTrash()`, the bucket counter uses post-decrement
(`iter->second--`), which assigns the pre-decrement value to
`pending_files_in_bucket`. When a bucket transitions from 1 to 0 files,
`pending_files_in_bucket` receives the value 1 (not 0), so the check
`pending_files_in_bucket == 0` fails. This means `WaitForEmptyTrashBucket()`
is never woken up when a specific bucket empties, unless all global pending
files are also zero.

Fix: Use pre-decrement (`--iter->second`) so the decremented value correctly
triggers the condition variable signal.

Reviewed By: xingbowang

Differential Revision: D95303385

fbshipit-source-id: 3c5f0978ff33600acaf406b3f839cf13d9983055
2026-03-10 10:26:45 -07:00
Omkar Gawde a066318a31 Fix out-of-bounds access in BlobFileReader::MultiGetBlob (#14427)
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14427

In `MultiGetBlob()`, the `adjustments` autovector is populated only for blob
requests that pass validation. Requests that fail validation are skipped via
`continue`, so `adjustments` has fewer entries than `blob_reqs`. When consuming
results, `adjustments[i]` uses the `blob_reqs` index instead of the
`read_reqs`/`adjustments` index, causing an out-of-bounds read when any request
fails validation.

Fix: Replace `adjustments[i]` with `adjustments[j - 1]`, since `j` tracks the
position in `read_reqs`/`adjustments` and has already been post-incremented.

Reviewed By: hx235

Differential Revision: D95303356

fbshipit-source-id: a264ae6481f74ce33f64e40624441d666135bcd0
2026-03-09 16:57:24 -07:00
Josh Kang 42eff8b632 Add new heauristic 'num_collapsible_entry_reads_sampled' (#14434)
Summary:
Add per-file sampling of "collapsible" entry reads (single deletions, merges, and kNotFound results) that may later be used to help inform read-triggered compactions. This is a better metric than `num_reads_sampled` as it is more targeted towards reads that could be avoided via compaction.

The existing behavior of `num_reads_sampled` is that reads only gets sampled on iterator creation for a file. It is problematic because next/prev() calls are not sampled, nor are additional seeks().

This PR moves sampling to per-seek/next granularity within `LevelIterator` and adds a new `num_collapsible_entry_reads_sampled` counter that tracks how often a file serves entries that could be eliminated by compaction.

 Note only L1+ files have iterator seeks/nexts/prevs sampled. Introducing this at L0 would require wrapping table reader iterators, introducing a performance cost.

## Key changes

- **New counter `num_collapsible_entry_reads_sampled`** in `FileSampledStats` tracks sampled reads that encounter deletions, single deletions, merges, or kNotFound results in both Get and Iterator paths.
- **Moved sampling from file-open to per-operation** in `LevelIterator`: sampling now happens in `SampleRead()` called from `Seek()`, `SeekForPrev()`, `SeekToFirst()`, `SeekToLast()`, `Next()`, `NextAndGetResult()`, and `Prev()`. The `should_sample` parameter was removed from `LevelIterator`'s constructor.
- **Differentiated sampling rate for Next() vs Seek()**: `should_sample_file_read_next()` uses a 64x lower sampling rate (`kFileReadSampleRate * 64`) since Next() is cheaper than Seek() and called more frequently.
- **Collapsible tracking in Get path**: `Version::Get()` now increments the collapsible counter when `GetContext::State()` is `kNotFound`, `kMerge`, or `kDeleted`.
- **Collapsible tracking in MultiGet path**: `MultiGetFromSST` also increments the collapsible counter for the same states.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14434

Test Plan:
- Added new DB tests for both num_reads_sampled and num_collapsible_entry_reads_sampled

### Benchmark results (readrandom, readseq)

Setup: 1M keys, 16-byte keys, 100-byte values, no compression, fillrandom+compact

| Benchmark  | Params             | ops/s (main) | ops/s (feature) | % change |
|------------|--------------------|-------------|--------------------------|----------|
| readrandom | seed=1, threads=1  | 387,194     | 389,449                  | +0.6%    |
| readseq    | seed=1, threads=1  | 5,598,371   | 5,572,975                | -0.5%    |

No meaningful performance regression observed — differences are within run-to-run noise.

Reviewed By: xingbowang

Differential Revision: D95613793

Pulled By: joshkang97

fbshipit-source-id: 9dd09c9b7527b148424bde5686f4157c7a9e1214
2026-03-09 16:42:41 -07:00
Hui Xiao cb43abb1f1 Fix incorrect input file expansion in SetupOtherFilesWithRoundRobinExpansion (#14436)
Summary:
### Context/Summary:
**_See below for an example of the bug:_**

L1 has 5 SST files at indices [0, 1, 2, 3, 4] where files at indices 1/2/3
share user key boundaries (e.g., file[1].largest and file[2].smallest have
the same user key).

Round-robin picks file[1] at start_index=1.

Before fix:
  1. ExpandInputsToCleanCut expands {file[1]} → {file[1], file[2], file[3]}
  2. Loop starts at i=2 (start_index+1), adds file[2] → duplicate!
  3. start_level_inputs_ = {file[1], file[2], file[3], file[2]}
  4. GetRange uses back()=file[2], returns [file[1].smallest, file[2].largest]
  5. ExpandInputsToCleanCut converges on {file[1], file[2]}, missing file[3]
  6. AssertCleanCut crashes (debug) or data corruption (release)

After fix:
  1. ExpandInputsToCleanCut expands {file[1]} → {file[1], file[2], file[3]}
  2. Find last file = file[3] at index 3, loop starts at i=4
  3. start_level_inputs_ = {file[1], file[2], file[3], file[4]} (no duplicates)
  4. GetRange correctly returns [file[1].smallest, file[4].largest]

_**More details:**_

When round-robin compaction picks a file, PickFileToCompact calls ExpandInputsToCleanCut which may expand start_level_inputs_ from 1 file to N files (when adjacent files share user key boundaries). Then SetupOtherFilesWithRoundRobinExpansion loops from start_index + 1 to add more files, not knowing the expansion already happened. This re-adds files already in start_level_inputs_, creating duplicates.

The duplicates corrupt GetRange, which trusts inputs.back()->largest for non-L0 levels. With a duplicate earlier file at back(), GetRange returns a truncated range. ExpandInputsToCleanCut then converges on an incomplete file set, violating the clean-cut invariant.

In debug builds this crashes at AssertCleanCut. In release builds the compaction proceeds with a non-clean-cut input, causing data corruption (newer data in a later level while older data remains in an earlier level).

The fix finds the position of the last file already in start_level_inputs_ and starts the loop after it, avoiding duplicates. When start_level_inputs_ has only 1 file (no expansion happened), the behavior is unchanged.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14436

Test Plan:
New test RoundRobinCleanCutWithSharedBoundary:
- Without fix: crashes at AssertCleanCut in SetupOtherFilesWithRoundRobinExpansion
```
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from DBCompactionTest
[ RUN      ] DBCompactionTest.RoundRobinCleanCutWithSharedBoundary
db_compaction_test: db/compaction/compaction_picker.cc:81: void rocksdb::AssertCleanCut(const rocksdb::InternalKeyComparator*, rocksdb::VersionStorageInfo*, rocksdb::CompactionInputFiles*, int, rocksdb::Logger*): Assertion `false' failed.
Received signal 6 (Aborted)
Invoking GDB for stack trace...
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/local/fbcode/platform010/lib/libthread_db.so.1".
0x00007fd9ba8f0e13 in __GI___wait4 (pid=1004850, stat_loc=0x7fd9b8bfb2ac, options=0, usage=0x0) at ../sysdeps/unix/sysv/linux/wait4.c:30
30      ../sysdeps/unix/sysv/linux/wait4.c: No such file or directory.
https://github.com/facebook/rocksdb/issues/4  __pthread_kill_internal (signo=6, threadid=<optimized out>) at pthread_kill.c:45
45      pthread_kill.c: No such file or directory.
https://github.com/facebook/rocksdb/issues/5  __GI___pthread_kill (threadid=<optimized out>, signo=6) at pthread_kill.c:62
62      in pthread_kill.c
https://github.com/facebook/rocksdb/issues/6  0x00007fd9ba8444ad in __GI_raise (sig=6) at ../sysdeps/posix/raise.c:26
26      ../sysdeps/posix/raise.c: No such file or directory.
https://github.com/facebook/rocksdb/issues/7  0x00007fd9ba82c433 in __GI_abort () at abort.c:79
79      abort.c: No such file or directory.
https://github.com/facebook/rocksdb/issues/8  0x00007fd9ba83bc28 in __assert_fail_base (fmt=0x7fd9ba9e11d8 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=0x7fd9bcc04819 "false", file=0x7fd9bcc04700 "db/compaction/compaction_picker.cc", line=81, function=<optimized out>) at assert.c:92
92      assert.c: No such file or directory.
https://github.com/facebook/rocksdb/issues/9  0x00007fd9ba83bc93 in __GI___assert_fail (assertion=0x7fd9bcc04819 "false", file=0x7fd9bcc04700 "db/compaction/compaction_picker.cc", line=81, function=0x7fd9bcc04780 "void rocksdb::AssertCleanCut(const rocksdb::InternalKeyComparator*, rocksdb::VersionStorageInfo*, rocksdb::CompactionInputFiles*, int, rocksdb::Logger*)") at assert.c:101
101     in assert.c
https://github.com/facebook/rocksdb/issues/10 0x00007fd9bc2f8181 in rocksdb::AssertCleanCut (icmp=0x7fd9b9a7a840, vstorage=0x7fd9b9b6d040, inputs=0x7fd9b8bfc690, level=1, logger=0x7fd9b9aa2790) at db/compaction/compaction_picker.cc:81
81            assert(false);
https://github.com/facebook/rocksdb/issues/11 0x00007fd9bc2f8fa6 in rocksdb::CompactionPicker::ExpandInputsToCleanCut (this=0x7fd9b9b32900, vstorage=0x7fd9b9b6d040, inputs=0x7fd9b8bfc690, next_smallest=0x0) at db/compaction/compaction_picker.cc:310
310       AssertCleanCut(icmp_, vstorage, inputs, level, ioptions_.logger);
https://github.com/facebook/rocksdb/issues/12 0x00007fd9bc30cbfb in rocksdb::(anonymous namespace)::LevelCompactionBuilder::SetupOtherFilesWithRoundRobinExpansion (this=0x7fd9b8bfc910) at db/compaction/compaction_picker_level.cc:423
```
- With fix: compaction completes, data correctness verified

Reviewed By: joshkang97

Differential Revision: D95718659

Pulled By: hx235

fbshipit-source-id: 6ef125455ef5ae8a07c323835ff25588dbbb3634
2026-03-09 16:04:17 -07:00
Jay Huh e27114b135 Skip flush during recovery if read_only (#14440)
Summary:
The whitebox_crash_test crashes with Assertion `checking_set_.count(cfd) == 0` failed in `FlushScheduler::ScheduleWork()` during a read-only DB open (e.g., StressTest::TestBackupRestore).

## Root cause:

Commit 3aa706c2b ("Enforce WriteBufferManager during WAL recovery") added logic to schedule flushes when `WriteBufferManager::ShouldFlush()` is true during WAL recovery. However, the drain logic in `MaybeWriteLevel0TableForRecovery()` was gated by !read_only, so in read-only mode the flush
scheduler queue was never cleared. On subsequent WAL records, `ScheduleWork()` is called again for the same CFD still in the queue, triggering the duplicate assertion.

## Fix:

Add a `read_only` parameter to `InsertLogRecordToMemtable()` and skip WBM flush scheduling entirely in read-only mode. Flushes cannot be performed during read-only recovery, so scheduling them is pointless and causes the assertion failure when the scheduler is never drained.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14440

Test Plan:
- flush_job_test — all 15 tests pass
  - db_basic_test --gtest_filter="*ReadOnly*:*Recovery*:*WAL*" — all 9 tests pass
  - db_write_buffer_manager_test — all 18 tests pass
  - `python3 tools/db_crashtest.py whitebox --simple --duration=600 --interval=30`

Reviewed By: xingbowang

Differential Revision: D95829493

Pulled By: jaykorean

fbshipit-source-id: bdfef9ce66d507a1169381064344afd612a4b318
2026-03-09 15:17:31 -07:00
Josh Kang 3ad23b2d94 Support automated interpolation search (#14383)
Summary:
Add automatic per-block interpolation search selection (`kAuto` mode) for index blocks. During SST construction, each index block's key distribution is analyzed using the coefficient of variation (CV) of gaps between restart-point keys. Blocks with uniformly distributed keys are flagged via a new bit in the data block footer, and at read time, `kAuto` resolves to interpolation search for uniform blocks and binary search otherwise.

## Key changes

- **New `BlockSearchType::kAuto` enum value**: Resolves per-block at read time to either `kInterpolation` or `kBinary` based on the block's uniformity flag. Falls back to `kBinary` on older versions that don't recognize it.
- **Write-path uniformity analysis**: `BlockBuilder::ScanForUniformity()` uses Welford's online algorithm to incrementally compute the CV of key gaps at restart points. The result is stored in a new bit (bit 30) of the data block footer's packed restart count.
- **New table option `uniform_cv_threshold`** (default: -1 `disabled`): Controls how strict the uniformity check is. Set to negative to disable. Exposed in C++, Java (JNI), and `db_bench`.
- **Code reorganization**: Block entry decode helpers (`DecodeEntry`, `DecodeKey`, `DecodeKeyV4`, `ReadBe64FromKey`) moved from `block.cc` to a new shared header `block_util.h` so they can be reused by `BlockBuilder` on the write path.
- **New histogram `BLOCK_KEY_DISTRIBUTION_CV`**: Records the CV (scaled by 10000) of each index block's key distribution for observability.
- **Java bindings**: `IndexSearchType.kAuto`, `uniformCvThreshold` getter/setter, JNI portal constructor signature updated, and `HistogramType.BLOCK_KEY_DISTRIBUTION_CV` added.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14383

Test Plan:
- `IndexBlockTest.IndexValueEncodingTest` parameterized to include `kAuto` search type alongside `kBinary` and `kInterpolation`, verifying correct seek/iteration behavior across all combinations of key distributions, restart intervals, and key lengths.
- Uniformity detection validated: blocks with uniform key distribution correctly set `is_uniform = true`, blocks with clustered/non-uniform keys set `is_uniform = false`.
- Stress test coverage
- Updated check_format_compatible to also include a "uniform" dataset. By default using uniform_cv_threshold=-1 does not result in an incompatibility issues. When manually changing the threshold (e.g. `uniform_cv_threshold=1000`), I see `bad block contents`, which is expected

## Benchmark

readrandom with `fillrandom,compact -seed=1 --statistics`:

| Benchmark | Branch | Params | avg ops/s | % change vs main | CV P50 |
|-----------|--------|--------|-----------|------------------|--------|
| readrandom | main | `binary_search, shortening=1` | 335,791 | baseline | N/A |
| readrandom | feature | `binary_search, shortening=1` (default) | 335,749 | -0.0% | 1,500 |
| readrandom | feature | `auto_search, shortening=1` (kAuto) | 366,832 | **+9.2%** | 1,500 |
| readrandom | feature | `interpolation_search, shortening=1` | 366,598 | **+9.2%** | 1,500 |
| readrandom | feature | `auto_search, shortening=2` (kAuto) | 344,631 | **+2.6%** | 1,030,000 |
| readrandom | feature | `interpolation_search, shortening=2` | 201,178 | **-40.1%** | 1,030,000 |

As seen with shortening=2, a non-uniform distribution produces a high CV, which does not use interpolation search.

## Write benchmark

There is a write overhead which scans each restart entry for a block upon Finish. In practice this is very low because currently it is only applied to index blocks.

See cpu profile (https://fburl.com/strobelight/io5hwj9h) here of `-benchmarks=fillseq,compact -compression_type=none -disable_wal=1`. Only 0.08% attributed to `ScanForUniformity`.

Reviewed By: pdillinger

Differential Revision: D94738890

Pulled By: joshkang97

fbshipit-source-id: 9661ac593c5fef89d49f3a8a027f1338a0c96766
2026-03-06 10:13:51 -08:00
Xingbo Wang 8a86620653 Disable more features in stress test when trie UDI is enabled (#14432)
Summary:
Address 2 compatibility issue of UDI:

A:
Trie UDI (UserDefinedIndex) does not support SeekToFirst/SeekToLast. The crash test already disabled prefix scanning (prefixpercent=0) when use_trie_index=1, but iteration (iterpercent) was still enabled.

During iteration, LevelIterator::SkipEmptyFileForward() internally calls file_iter_.SeekToFirst() when Next() crosses SST file boundaries within a level. This propagates to UserDefinedIndexIteratorWrapper::SeekToFirst() which returns NotSupported, causing "Iterator diverged from control iterator" / "VerifyIterator failed" errors across many crash test variants.

B:
BlobDB is incompatible with trie UDI (user-defined index). When BlobDB is enabled (`enable_blob_files=1`), the flush job stores large values in separate blob files and writes `kTypeBlobIndex` entries in the SST instead of `kTypeValue`. The UDI builder (`UserDefinedIndexBuilderWrapper::OnKeyAdded()`) rejects any entry whose type is not `kTypeValue`, causing the flush to fail with `"user_defined_index_factory only supported with Puts"`.

Once the flush fails, the DB enters background error state, and all subsequent `Write()` calls fail with the same error -- printed as the repeated `"multiput error: ..."` messages from
`BatchedOpsStressTest::TestPut`. Eventually, `ProcessStatus` encounters this error and triggers `assert(false)`.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14432

Reviewed By: pdillinger

Differential Revision: D95457462

Pulled By: xingbowang

fbshipit-source-id: bf2bc47bd1ed75926765b2e3ff068f99f89a7793
2026-03-06 08:56:16 -08:00
Maciej Szeszko 071b5eff7a Fix MemPurge memtable ID ordering assertion (#14424)
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14424

## Context

`MemPurge` releases db_mutex_ to do its merge work and re-acquires it before adding the output memtable to the immutable list. During this window, concurrent writers can fill the active memtable and trigger a switch, adding a new immutable memtable with a higher ID. `MemPurge` then assigns its output memtable the stale ID from mems_.back() (the newest memtable in the original flush batch), which is now lower than the front of the immutable list, violating the ordering assertion in `MemTableListVersion::AddMemTable `.

## Fix

After re-acquiring db_mutex_, use `std::max(mems_.back()->GetID(), imm()->GetLatestMemTableID())` so the output memtable's ID is never lower than any existing immutable memtable. Two `TEST_SYNC_POINT`s are added to the `MemPurge` success path to enable deterministic repro.

Reviewed By: pdillinger

Differential Revision: D94756638

fbshipit-source-id: 2e9e15b4285dc6b996c8744795228180dbd73ed3
2026-03-05 22:32:47 -08:00
Jay Huh 3aa706c2bf Enforce WriteBufferManager during WAL recovery (#14305)
Summary:
Add a new immutable DB option `enforce_write_buffer_manager_during_recovery` to control whether WriteBufferManager buffer_size is enforced during WAL recovery. When multiple RocksDB instances share a WriteBufferManager, a recovering instance could exceed the global memory limit by replaying large amounts of WAL data into memtables. This can cause OOM, especially when other instances are actively using the shared memory budget. When this option is enabled (and a WriteBufferManager is configured), RocksDB will check `WriteBufferManager::ShouldFlush()` after each batch insertion during WAL recovery and schedule flushes when needed to keep memory bounded.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14305

Test Plan:
Two unit tests added - `WriteBufferManagerLimitDuringWALRecoverySingleDB` and `WriteBufferManagerLimitDuringWALRecoveryMultipleDBs`
```
./db_write_buffer_manager_test --gtest_filter="*WriteBufferManagerLimitDuringWALRecovery*"
```
Basic stress test added with options toggled

## To follow up

Multiple DB scenario in stress test

Reviewed By: hx235

Differential Revision: D92533792

Pulled By: jaykorean

fbshipit-source-id: 35ca70e2300a8bfd6b81a6646a65ef284fb90a9a
2026-03-05 15:56:46 -08:00
Xingbo Wang 2dc6d6f765 Propagate builder error when flush produces empty output (#14418)
Summary:
Propagate builder error when flush produces empty output

When write fault injection causes the table builder to enter an error
state during flush, all subsequent Add() calls return early (ok() is
false), leaving the builder empty (IsEmpty() == true). Previously,
BuildTable() would call builder->Abandon() but not propagate the
builder's error status to 's', leaving it OK. This caused the downstream
key count validation in flush_job.cc to fire a misleading Corruption
error ('Number of keys in flush output SST files does not match...'),
which the stress test harness couldn't identify as a retryable injected
fault error, leading to SafeTerminate().

This started failing recently because ('Separate keys and
values in data blocks', ) introduced a new SST
block format (separate_key_value_in_data_block) that stores keys and
values in separate sections within data blocks. This format requires
additional write operations during Flush() inside the table builder,
increasing the probability that write fault injection
(--write_fault_one_in=128) hits a data block write and puts the builder
into an error state before any entries are committed. The bug in
BuildTable() existed before, but was rarely triggered because the old
interleaved block format had fewer write points susceptible to fault
injection during the critical Add() path.

Fix: After builder->Abandon(), propagate the builder's error status to
's' when the builder is empty due to an internal error. This ensures the
actual IOError from write fault injection is reported, which the stress
test can properly handle via IsErrorInjectedAndRetryable().

The analysis was based on stack trace. However, it would be great
if we could get direct evidence from fault injection.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14418

Test Plan: Unit test

Reviewed By: hx235

Differential Revision: D95121070

Pulled By: xingbowang

fbshipit-source-id: cfb513bd744ac34ac90cda11c1cbe49a9d0a7c6c
2026-03-05 14:18:08 -08:00
Peter Dillinger ef5b145140 Fix ldb dump swallowing errors, but ldb scan for compatibility test (#14422)
Summary:
This fixes a longstanding bug in which `ldb dump` swallows iterator errors. This can affect check_format_compatible.sh test results; if lucky, it will misleadingly look like a data mismatch instead of an outright failure. If unlucky, it could cause a test false negative.

However the compatibility test uses old versions of ldb, so the best way to improve the test (for the foreseeable future) is to replace `ldb dump` with `ldb scan`.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14422

Test Plan: manual

Reviewed By: joshkang97

Differential Revision: D95332577

Pulled By: pdillinger

fbshipit-source-id: bef1b427dd8aaa2cabbd23b7ad9f3cad1f67a349
2026-03-05 09:41:58 -08:00
Anand Ananthabhotla 3b5cb114e3 Refactor MultiScan to use MultiScanIndexIterator (#14401)
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14401

Unify the MultiScan and regular iterator codepaths in BlockBasedTableIterator by introducing a MultiScanIndexIterator that implements InternalIteratorBase<IndexValue>. During Prepare(), the original index iterator is swapped out for a MultiScanIndexIterator that wraps the prefetched block handles and scan range metadata. This allows SeekImpl() and FindBlockForward() to use the same code flow for both regular and MultiScan operations, eliminating the need for separate MultiScan-specific methods (SeekMultiScan, FindBlockForwardInMultiScan, MultiScanSeekTargetFromBlock, MultiScanUnexpectedSeekTarget, MultiScanLoadDataBlock, MarkPreparedRangeExhausted).

Key changes:
- New MultiScanIndexIterator class that manages scan range tracking, block handle iteration, forward-only seek enforcement, and wasted block counting
- InitDataBlock() loads blocks from ReadSet when MultiScan is active
- FindBlockForward() detects scan range boundaries via IsScanRangeExhausted() after index_iter_->Next()
- Disabled reseek optimization for MultiScan so MultiScanIndexIterator::Seek() is always called to update scan range tracking state
- Removed MultiScanState struct and all MultiScan-specific methods from BlockBasedTableIterator
- No changes to CheckDataBlockWithinUpperBound or CheckOutOfBound — they work as-is through iterate_upper_bound
- multi_scan_status_ intentionally not checked in Valid() hot path to avoid performance regression; when status is non-OK, block_iter_points_to_real_block_ is already false
- Fixed pre-existing bug in ReadSet::SyncRead() that used the base decompressor without compression dictionary, causing ZSTD data corruption when blocks with dictionary compression needed synchronous fallback reads

Reviewed By: xingbowang

Differential Revision: D93300655

fbshipit-source-id: 231059208e0cc512bc2ec43ff7055fcb2a2dc72d
2026-03-04 21:09:54 -08:00
zaidoon 16c81a27e9 Extend UDI trie index with seqno side-table for same-user-key block boundaries (#14412)
Summary:
When the same user key spans adjacent data blocks with different sequence
numbers, the trie index cannot distinguish them because it only stores
user keys. This adds a side-table that stores per-leaf sequence numbers
and overflow block metadata, enabling correct post-seek correction using
seqno comparison.

Design:
- Trie always stores user-key-only separators (unchanged)
- Duplicate separators from same-key boundaries are de-duplicated
- Side-table stores leaf_seqnos[], leaf_block_counts[], and overflow
  arrays (offsets, sizes, seqnos) serialized after the trie data
- Post-seek correction: after trie Seek lands on a leaf, compare
  target_seq vs leaf_seqno to decide whether to advance through
  overflow blocks or to the next trie leaf
- Zero overhead when no same-user-key boundaries exist (no side-table
  serialized, no seqno checks at seek time)

Public API changes (user_defined_index.h):
- AddIndexEntry: single pure virtual with IndexEntryContext parameter
  (replaces old 4-arg version). Context carries last_key_seq and
  first_key_seq for block boundaries.
- SeekAndGetResult: single pure virtual with SeekContext parameter
  (replaces old 2-arg version). Context carries target_seq.
- Fix trailing semicolons on NewBuilder/NewReader default overrides.

Wrapper layer (user_defined_index_wrapper.h):
- Extract sequence numbers from parsed internal keys and pass via
  context structs to UDI builder/iterator.
- Fix CurrentIndexSizeEstimate() to delegate to internal builder
  (was returning 0).
- Fix ApproximateMemoryUsage() to include UDI reader memory.
- Fix typos: lof_err_key -> log_err_key, COuld -> Could,
  "Bad index name" -> "Bad index name: ".

Trie builder (trie_index_factory.cc):
- Buffer separator entries during building; at Finish(), detect whether
  any same-user-key boundary was seen (sticky flag, same strategy as
  ShortenedIndexBuilder::must_use_separator_with_seq_).
- When seqno encoding is needed, re-encode all separators with seqno
  side-table metadata in the trie.
- Default null comparator to BytewiseComparator() to prevent crash.

Trie iterator (trie_index_factory.cc):
- Post-seek correction: compare target_seq vs leaf_seqno to advance
  through overflow runs.
- Overflow run state tracking: overflow_run_index_, overflow_run_size_,
  overflow_base_idx_ for O(1) access to overflow block handles.
- NextAndGetResult advances within overflow runs before moving to next
  trie leaf.
- Return kOutOfBound instead of kUnknown when Seek/Next finds no blocks.

LOUDS trie (louds_trie.h, louds_trie.cc):
- Seqno side-table: builder AddKeyWithSeqno/AddOverflowBlock methods,
  BFS reordering of seqno/block_count arrays, serialization/deserialization
  of per-leaf seqnos, block counts, overflow handles/seqnos, and
  overflow_base_ prefix sum for O(1) access.
- AppendKeySlot() helper with debug bounds assert on all key-append sites.

Dead code removal:
- LoudsTrieBuilder::NumKeys() (never called externally)
- sparse_leaf_count_ (serialized but never read by the reader)
- DenseChildNodeNum(pos), DenseLeafIndex(pos), DenseNodeNum(pos)
  (superseded by FromRank variants that avoid redundant Rank1 calls)

Deserialization hardening (all in InitFromData, cold path only):
- Move num_keys_ validation before any dependent arithmetic.
- Validate dense bitvector sizes match dense_node_count_ (d_labels
  must be node_count*256, d_has_child must equal d_labels.NumOnes(),
  d_is_prefix_key must equal node_count).
- Validate sparse bitvector sizes match s_labels_size_ (s_has_child
  and s_louds must have the same number of bits as the label array).
- Validate child position table values within s_labels_size_ bounds.
- Validate chain suffix offset+length within suffix data blob.
- Validate chain end child indices < num_internal or == UINT32_MAX.
- EliasFano: add count_ upper-bound check (<=2^30) to prevent
  count_*low_bits_ integer overflow.
- Bitvector: validate select hint values < num_rank_samples_ to
  prevent OOB in FindNthOneBit/FindNthZeroBit.
- EliasFano: custom move ctor/assignment to re-seat low_words_
  after owned_low_data_ move (fixes dangling pointer for SSO strings).

https://github.com/facebook/rocksdb/issues/14406

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14412

Reviewed By: anand1976

Differential Revision: D95160933

Pulled By: xingbowang

fbshipit-source-id: f2c3681c1059c03d540ce1cb9cc14cc79cd9730c
2026-03-04 15:31:17 -08:00
Xingbo Wang 3b66de3fac Fix out of disk unit test issue (#14425)
Summary:
It contains 3 commits.

  Commit 1: Fix /dev/shm exhaustion during make check

  Fix /dev/shm exhaustion during make check

  1. Add space-heavy tests to NON_PARALLEL_TEST: perf_context_test (1GB
     write_buffer_size), obsolete_files_test (~1GB), backup_engine_test
     (1GB), prefetch_test (1GB), and db_io_failure_test (256MB).

  2. Add per-test cleanup on success: each parallel test script now
     removes its test directory after passing. Failed test directories
     are preserved for debugging.

  3. Add age-based stale directory cleanup: at the start of make check,
     remove /dev/shm/rocksdb.* directories older than 3 hours from
     previous failed runs, using age-based filtering to avoid disturbing
     concurrent runs.

  Commit 2: Fix SIGSEGV in prefetch_test due to stale SyncPoint callbacks

  Both FilePrefetchBufferTest and FSBufferPrefetchTest fixtures set up
  SyncPoint callbacks capturing local variables by reference and enable
  processing, but neither fixture's TearDown() clears them. When a
  subsequent test runs, the stale callbacks fire with dangling
  references, causing memory corruption and SIGSEGV.

  Fixed by adding DisableProcessing() and ClearAllCallBacks() to both
  fixtures' TearDown() methods.

  Commit 3: Fix OpenFilesAsyncTest using excessive disk space

  Fix OpenFilesAsyncTest using excessive disk space in /dev/shm

  OpenFilesAsyncTest::SetupData() set write_buffer_size to SIZE_MAX to
  prevent automatic flushes. This caused each of its ~20 parallel
  instances to consume 6-43 GB in /dev/shm (validated: a single
  Shutdown/3 instance used 43 GB), totaling ~233 GB and filling the
  disk. This was the primary cause of "No space left on device" errors
  in make check. The sst flushed is small, but it causes issue in WAL
  space reservation, which bloated the disk usage.

  The SIZE_MAX is unnecessary — the test writes only 4 tiny key-value
  pairs and does explicit Flush() after each. The default 64 MB
  write_buffer_size will never auto-flush for such small writes.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14425

Test Plan: Unit Test

Reviewed By: joshkang97

Differential Revision: D95253518

Pulled By: xingbowang

fbshipit-source-id: 03907df59b3d89d90413a0e33996ec205944d48b
2026-03-04 13:48:48 -08:00
Xingbo Wang 9a9cc1e2ef Fix uninitialized wal_in_db_path_ in read-only and compacted DB open paths (#14419)
Summary:
wal_in_db_path_ was declared without an in-class initializer in DBImpl. While DB::Open, OpenAsSecondary, and OpenAsFollower all explicitly set this field, OpenForReadOnly and CompactedDBImpl::Open did not.

This was harmless until recently because CloseHelper() only calls PurgeObsoleteFiles when opened_successfully_ is true, and read-only DBs previously did not set opened_successfully_. After https://github.com/facebook/rocksdb/issues/14322 added opened_successfully_ = true to the read-only path, CloseHelper now executes PurgeObsoleteFiles -> DeleteObsoleteFileImpl, which reads wal_in_db_path_ to decide whether WAL files should be deleted in the foreground. Reading an uninitialized bool is undefined behavior, caught by UBSan as "invalid-bool-load" in the secondary_cache_crash_test.

Fixed by adding an in-class initializer (= false) to wal_in_db_path_. The default of false means WAL deletions use foreground deletion, which is safe for read-only DBs that don't write WALs. The existing DB::Open path continues to set the correct value explicitly.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14419

Test Plan:
- Added regression test ReadOnlyDBWalInDbPathInitialized in db_test2
- Ran test 5 times without failure
- Ran all OpenForReadOnly tests (3 tests pass)

Task:
T257988006

Reviewed By: joshkang97

Differential Revision: D95122268

Pulled By: xingbowang

fbshipit-source-id: 7f65fe1f09e7e0b42ba68f44f246615abc0757d4
2026-03-04 13:26:19 -08:00
Xingbo Wang ea2f33c81a Disable prefix scanning when use_trie_index is enabled in crash test (#14421)
Summary:
The trie-based User Defined Index (UDI) has a bug in its iterator implementation that causes "Not implemented: SeekToFirst not supported" errors during prefix scanning. This causes assertion failures in BatchedOpsStressTest::TestPrefixScan and may cause silent wrong results in other prefix scan tests.

Until the trie index is fixed, disable prefix scanning when use_trie_index is enabled by setting prefixpercent=0 and redistributing the percentage to reads.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14421

Test Plan:
- Reproduced: db_stress with --use_trie_index=1 --prefixpercent=5 crashes with assertion failure. Instrumentation revealed iterator status "Not implemented: SeekToFirst not supported" on prefix 0x35.
- Verified: db_stress with --use_trie_index=1 --prefixpercent=0 completes cleanly.

Reviewed By: anand1976

Differential Revision: D95135250

Pulled By: xingbowang

fbshipit-source-id: 81540b2426aa1a855a58e6ca9e68a035d53aa2d8
2026-03-04 13:11:33 -08:00
Xingbo Wang 15f2f2bf6f Relax option sanitization for kv ratio compaction (#14397)
Summary:
The kv ratio compaction option sanitization is too strict. Sometimes, it blocks engine start up. Relax the validation and convert it to runtime check.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14397

Test Plan: Unit test

Reviewed By: pdillinger

Differential Revision: D94709471

Pulled By: xingbowang

fbshipit-source-id: cc076c397b3acfa426112063224771a196684798
2026-03-03 21:40:36 -08:00
Xingbo Wang 9d3d3c4b61 Handle TryAgain from optimistic transactions in stress test (#14410)
Summary:
When using optimistic transactions (--use_optimistic_txn=1), the transaction commit can fail with Status::TryAgain() if the memtable history is insufficient for conflict detection (controlled by max_write_buffer_size_to_maintain). ExecuteTransaction retries up to 10 times, and if all retries fail, it returns TryAgain.

Previously, this TryAgain status was not handled in TestPut, TestDelete, and TestSingleDelete, causing the stress test to call SafeTerminate() and crash with "put or merge error" / "delete error".

This is an expected condition with optimistic transactions and should be handled gracefully by rolling back the pending expected value.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14410

Test Plan:
- make -j db_stress && run crash_test_with_optimistic_txn
- Stress test with --use_optimistic_txn=1 --use_txn=1 with small max_write_buffer_size_to_maintain to verify no crash on TryAgain.

Reviewed By: hx235

Differential Revision: D95077227

Pulled By: xingbowang

fbshipit-source-id: 15c343cc1c5f888fb79e95a87cfe46145f98b3a1
2026-03-03 17:16:17 -08:00
Xingbo Wang cc19bd7555 Disable mmap_read with trie index to fix SIGSEGV (#14409)
Summary:
The trie-based UDI (LoudsTrie/Bitvector) stores zero-copy reinterpret_cast pointers into the serialized UDI block data. When mmap_read is enabled, the block data resides in memory-mapped file pages. If the SST file's mmap mapping is later invalidated (e.g. through table cache eviction or DB reopen), the trie's internal pointers become dangling, causing SIGSEGV.

For now, disable mmap_read when use_trie_index is enabled:
- db_crashtest.py: force mmap_read=0 when use_trie_index=1
- db_stress_tool.cc: reject the combination with an error message

The proper fix, involve loading index then fix the pointer address.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14409

Reviewed By: pdillinger

Differential Revision: D94971965

Pulled By: xingbowang

fbshipit-source-id: 6abbd4bfbfd26e7ba8725a93a6c68beb26f6393e
2026-03-03 08:50:44 -08:00
Xingbo Wang c2580e2b7a Fix use-after-free in db_stress with UDI factory (#14411)
Summary:
In StressTest::OperateDb(), read_opts.table_index_factory is set to udi_factory_.get() (a raw pointer) once at function entry. When Reopen() is called during the stress test loop, Open() recreates udi_factory_ via std::make_shared<TrieIndexFactory>(), destroying the old factory. But read_opts.table_index_factory still holds the dangling pointer.

When MultiGet subsequently calls UserDefinedIndexReaderWrapper::NewIterator() and accesses read_options.table_index_factory->Name(), it dereferences a dangling pointer, causing a segmentation fault.

Fix: After each reopen, update read_opts.table_index_factory to point to the newly created udi_factory_ instance.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14411

Test Plan: - Ran db_stress with --use_trie_index=1 --reopen=20 --use_multiget=1 4 times without failure (previously crashed with SIGSEGV)

Reviewed By: pdillinger

Differential Revision: D95064144

Pulled By: xingbowang

fbshipit-source-id: 7139baacda659d8a3cb84fdcc4822a428542bf0c
2026-03-03 08:43:40 -08:00
Josh Kang f25fb41da6 Add option to validate sst files in the background on DB open (#14322)
Summary:
Add `open_files_async` option for faster DB startup. When enabled, SST file opening and validation is deferred to a background thread after `DB::Open` returns, reducing startup latency for databases with many SST files. WAL recovery remains synchronous.

To support this, `FindTable` is extended with a pinning mechanism that stores the cache handle directly on `FileMetaData` via a new `PinnedTableReader` class, and sets the table reader atomically so subsequent reads skip cache lookups. `FileDescriptor::table_reader` is replaced with `PinnedTableReader pinned_reader` which wraps a `std::atomic<TableReader*>` with acquire/release ordering to safely handle concurrent access between the background opener and read threads.

Should validations fail, the background opener sets a `kAsyncFileOpen` background error. Future read requests will look up the table reader again via the cache, and if any validations fail there it will get propagated to the user (existing behavior when `max_open_files > 0`).

This feature is most useful when `max_open_files=-1`, because otherwise file opening is already capped at 16 files and DB open should be fast.

## Restrictions
- This feature also is incompatible with fifo compaction because fifo compaction requires reading table properties under DB mutex. When table reader is unpinned, this may cause a DB hang.
- This feature is also incompatible with `skip_stats_update_on_db_open=false` because it will result in even longer DB open

## Key changes

- New `open_files_async` DB option with C, Java, and `db_bench` bindings
- `BGWorkAsyncFileOpen` background worker that opens all SST files post-`DB::Open`, with shutdown awareness via `shutting_down_` flag
- New `PinnedTableReader` class in `version_edit.h` — thread-safe wrapper holding `std::atomic<TableReader*>` and `Cache::Handle*` with proper acquire/release ordering. Replaces the old `FileDescriptor::table_reader` raw pointer and `FileMetaData::table_reader_handle`
- Extract `LoadTableHandlersHelper` into `db/version_util.cc` — shared between `VersionBuilder::LoadTableHandlers` (for version edits during recovery) and `BGWorkAsyncFileOpen` (for base storage post-open)
- `FindTable` extended with `pin_table_handle` and `out_table_reader` params — when pinning is enabled, the table reader is stored on `FileMetaData` so Get/MultiGet/Iterator skip redundant cache lookups. `FindTable` now performs the pinned-reader fast-path check internally instead of requiring callers to check `fd.table_reader` beforehand
  - Note: pinning is explicit (not default) because some callers create temporary `FileMetaData`s that would need to properly clean up table handles
- `CompactedDBImpl` updated to use `FindTable` + pinning instead of raw `fd.table_reader` access for Get/MultiGet
- New `kAsyncFileOpen` background error reason in `listener.h` and `error_handler.cc`
- Add a check in ~DBImpl to ensure async file open task has not been forgotten to be scheduled in (future) subclasses of DBImpl. Certain subclasses that never use it will need to explicitly mark it.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14322

Test Plan:
- `OpenFilesAsyncTest` parameterized over `num_flushes` (1, 20), `ReadType` (Get, MultiGet, Iterator), `max_open_files` (-1, 10), and `read_only` (true, false)
  - **ConcurrentFileAccess**: concurrent reads and compactions race with async opener
  - **AfterRead**: reads happen before async opener, verifying lazy open and that the opener sees already-pinned readers
  - **BeforeRead**: async opener completes first, verifying reads use pre-loaded table readers
  - **Shutdown**: DB closes before async opener starts, verifying clean cancellation with 0 file opens
  - **Error**: corrupted SST files, verifying `kAsyncFileOpen` background error is set and reads return corruption
  - **DropColumnFamily**: CF dropped before async opener runs, verifying the opener gracefully skips dropped CFs
- Added to crash test

### Benchmark

To simulate a high-latency remote filesystem, I set up a virtual filesystem with dm-delay using 10ms reads, 0 ms writes.

```
# Generate a DB with many L0 files

TEST_TMPDIR=/data/users/jkangs/dm-delay-test/mnt ./db_bench -benchmarks=fillseq -disable_auto_compactions=true -write_buffer_size=1000 -num=1000000
```

```
./db_bench -use_existing_db=true -db=/data/users/jkangs/dm-delay-test/mnt/dbbench -benchmarks=readrandom -reads=1 -report_open_timing=true -open_files_async=true -use_direct_reads -file_opening_threads=1 -skip_stats_update_on_db_open

OpenDb:     25.1419 milliseconds
```

```
./db_bench -use_existing_db=true -db=/data/users/jkangs/dm-delay-test/mnt/dbbench -benchmarks=readrandom -reads=1 -report_open_timing=true -open_files_async=false -use_direct_reads -file_opening_threads=1 -skip_stats_update_on_db_open

OpenDb:     23109.4 milliseconds
```

### No read regressions

On main branch
```
./db_bench -use_existing_db=true -db=/dev/shm/dbbench -benchmarks=readrandom -seed=1 -threads=8 -duration=30

readrandom   :       4.827 micros/op 1657100 ops/sec 30.005 seconds 49720992 operations;  183.3 MB/s (6198999 of 6198999 found)
```

On this branch
```
./db_bench -use_existing_db=true -db=/dev/shm/dbbench -benchmarks=readrandom -seed=1 -threads=8 -duration=30

readrandom   :       4.863 micros/op 1644808 ops/sec 30.007 seconds 49354992 operations;  182.0 MB/s (6099999 of 6099999 found)

./db_bench -use_existing_db=true -db=/dev/shm/dbbench -benchmarks=readrandom -seed=1 -threads=8 -duration=30 -open_files_async=true

readrandom   :       4.803 micros/op 1665392 ops/sec 30.004 seconds 49968992 operations;  184.2 MB/s (6222999 of 6222999 found)
```

Reviewed By: pdillinger, xingbowang

Differential Revision: D93538033

Pulled By: joshkang97

fbshipit-source-id: 32ac70c112cd733b7c1e1c1e2e7ce6422318a5ae
2026-03-02 16:18:14 -08:00
Xingbo Wang b48d6f2c12 Block TransactionDB when UDI (use_trie_index) is enabled (#14408)
Summary:
User-Defined Index (UDI) only supports kTypeValue (Put) entries. TransactionDB is incompatible because transaction ROLLBACK operations write DELETE entries to the WAL to undo uncommitted changes.

Evidence from crash DB analysis (T257683723):
- All SST files had 0 deletions, 0 merges, 0 range deletions
- WAL file 012653.log contained DELETE entries from ROLLBACK: 322982,DELETE(2) ROLLBACK(0x7869643334333837) 322987,DELETE(8) ROLLBACK(0x7869643334333935) 322989,DELETE(8) ROLLBACK(0x7869643334343031) 322993,DELETE(9) ROLLBACK(0x7869643334343035) 322996,DELETE(4) ROLLBACK(0x7869643334333939)

Root cause chain:
1. TransactionDB ROLLBACK writes DELETE entries to WAL
2. During recovery/Open(), WAL entries are replayed into memtable
3. During flush, memtable (containing DELETEs) triggers UDI builder
4. UDI builder rejects DELETE: pkey.type != kTypeValue
5. Error: "user_defined_index_factory only supported with Puts"

This change adds:
1. Validation in db_stress_tool.cc to reject use_trie_index + use_txn
2. Sanitization in db_crashtest.py to set use_txn=0 when use_trie_index=1

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14408

Test Plan:
./db_stress --use_trie_index=1 --use_txn=1 --delpercent=0 \
      --delrangepercent=0 --readpercent=30 --prefixpercent=20 \
      --writepercent=40 --iterpercent=10 --customopspercent=0
      # Now correctly rejects with clear error message

Reviewed By: joshkang97

Differential Revision: D94928704

Pulled By: xingbowang

fbshipit-source-id: bdaa1e719f158e4a3fd76c64cf097ac3f41c8512
2026-03-02 15:54:27 -08:00
Xingbo Wang aa9160e8d7 Fix stress test failure with trie index (#14407)
Summary:
Fix stress test to check UDI usage for unsupported iterator ops

    UserDefinedIndexIterator only supports Seek(target) + Next() - it requires
    a target key for all seeks. The following operations are not supported:
    - SeekToFirst() - no target key
    - SeekToLast() - no target key
    - SeekForPrev() - not in UDI interface
    - Prev() - not in UDI interface

    This fix checks for UDI usage at both levels:
    1. ReadOptions level: ro.table_index_factory != nullptr
    2. CF/table level: udi_factory_ != nullptr (BlockBasedTableOptions)

    This is future-proof for any UDI implementation, not just trie index.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14407

Reviewed By: joshkang97

Differential Revision: D94925001

Pulled By: xingbowang

fbshipit-source-id: bf2ceacc7acebbc2347be445e2f208820a97b817
2026-03-02 11:25:57 -08:00
Peter Dillinger da2c3c0ee6 Fix & improve compaction trigger on a "quiet" DB (#14396)
Summary:
As a follow-up to https://github.com/facebook/rocksdb/issues/13736, allow a "quiet" DB to react much sooner to time-based compaction triggers. For details see DBImpl::ComputeTriggerCompactionPeriod() implementation.

Also based on review feedback, fixing a bug where only column families setting periodic compaction would be triggered, rather than any time-based compaction.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14396

Test Plan: extended+added unit tests to cover much of the logic

Reviewed By: xingbowang

Differential Revision: D94626166

Pulled By: pdillinger

fbshipit-source-id: de9ca19e46bdba5d9715474efbc61805354d730d
2026-02-28 13:43:55 -08:00
zaidoon 8707abfc11 Fix crash in UserDefinedIndexBuilderWrapper with buffered data blocks (#14404)
Summary:
When the UDI wrapper's OnKeyAdded() encountered a non-Put key type (e.g. Delete or Merge), it set its internal status_ to non-OK and stopped forwarding OnKeyAdded() to the wrapped internal index builder. However, AddIndexEntry() was always forwarded unconditionally. This asymmetry left the internal ShortenedIndexBuilder's current_block_first_internal_key_ empty, triggering an assertion failure in GetFirstInternalKey() during the buffered-block replay in MaybeEnterUnbuffered().

The crash required three conditions to co-occur:
1. UDI enabled (use_trie_index=1)
2. Compression dictionary enabled (triggering kBuffered mode)
3. Non-Put entries in the data (Delete, Merge, etc.)

Fix: move the internal_index_builder_->OnKeyAdded() call before the status_ guard so the internal builder always receives every key, matching the unconditional forwarding in AddIndexEntry().

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14404

Reviewed By: archang19

Differential Revision: D94791493

Pulled By: xingbowang

fbshipit-source-id: 882e409f61ae9aca084e9511794ca32ba4ff090f
2026-02-28 12:58:21 -08:00
zaidoon ce0fff9f41 Add trie-based User Defined Index (UDI) plugin (#14310)
Summary:
Implement a Fast Succinct Trie (FST) index based on LOUDS encoding as a User Defined Index plugin for RocksDB's block-based tables. First step toward trie-based indexing per https://github.com/facebook/rocksdb/issues/12396.
The trie uses hybrid LOUDS-Dense (upper levels, 256-bit bitmaps) + LOUDS-Sparse (lower levels, label arrays) encoding inspired by the [SuRF paper](https://www.pdl.cmu.edu/PDL-FTP/Storage/surf_sigmod18.pdf) (Zhang et al., SIGMOD 2018). The boundary between dense and sparse levels is automatically chosen to minimize total space.

Key Components
- **Bitvector** with O(1) rank and O(log n) select using a rank LUT sampled every 256 bits with popcount intrinsics. Uses uint32_t rank LUT entries (halving memory vs uint64_t). Includes word-level `AppendWord()` for efficient dense bitmap construction and `AppendMultiple()` optimized at word granularity for bulk bit fills.
- **Streaming trie builder** using flat per-level arrays with deferred internal marking, handle migration for prefix keys, and lazy node creation. Infers trie structure directly from sorted keys via LCP analysis in a single pass (no intermediate tree).
- **LoudsTrie** for immutable querying with BFS-ordered handle reordering built into single-pass level-by-level serialization. Move-only semantics with correct pointer re-seating after `std::string` move.
- **LoudsTrieIterator** with rank-based traversal, key reconstruction from trie path, and stack-based backtracking for `Next()`. Uses packed 8-byte `LevelPos` (is_dense flag in bit 63) and `autovector<LevelPos, 24>` to avoid heap allocation. Key reconstruction uses a raw char buffer allocated once to `MaxDepth()+1` bytes.
- **TrieIndexFactory/Builder/Reader/Iterator** implementing the `UserDefinedIndexFactory` interface.
- **Zero-copy block handle loading** using two fixed-width uint64_t arrays (offsets + sizes) with 8-byte alignment, enabling O(1) initialization via direct pointer assignment.

Seek Hot Path Optimizations
- **Fanout-1 sparse fast path**: Most sparse nodes in tries built from zero-padded numeric keys have exactly one child. Detected via `start_pos + 1 == end_pos` and inlined as a single byte comparison, avoiding the full `SparseSeekLabel` call.
- **Linear scan for small sparse nodes**: `SparseSeekLabel` uses sequential scan for nodes with ≤16 labels instead of binary search. Faster for common 10-child digit nodes where branch misprediction cost outweighs linear scan cost.
- **Rank reuse**: `DenseLeafIndexFromRankAndHasChildRank` and `SparseLeafIndexFromHasChildRank` overloads accept pre-computed `has_child_rank` from the Seek descent, avoiding redundant `Rank1` calls.
 General Performance Optimizations
- **Select-free sparse traversal**: Precomputed child position lookup tables (`s_child_start_pos_`/`s_child_end_pos_`) eliminate `Select1` calls during Seek. Sparse traversal tracks `(start_pos, end_pos)` directly, using only `Rank1` (O(1)) + array lookup (O(1)) for child descent.
- **Cached label_rank pattern**: Eliminates redundant `Rank1` calls in hot paths (Seek, Next, Advance all cache and reuse the label_rank computed for has_child checking).
- **Leaf index fast path**: When no prefix keys exist (common case), `SparseLeafIndex` and `DenseLeafIndexFromRank` skip the prefix-key `Rank1` calls entirely, reducing from 3 to 1 `Rank1` call.
- **Popcount-based Select64** via 6-step binary search within 64-bit words.
- MSVC portability using RocksDB's `BitsSetToOne`/`CountTrailingZeroBits`.

Benchmark Results

Trie Seek at 32K keys (16-byte keys, 5M lookups, median of 5 runs):
| Configuration | ns/op |
|---|---:|
| Trie (optimized) | **118** |
| Binary search (native) | 134 |
Trie Seek is **~12% faster** than native binary search index at 32K keys per block.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14310

Reviewed By: anand1976

Differential Revision: D93921511

Pulled By: xingbowang

fbshipit-source-id: ba18604bc6bd4f575311a1ae3047a541274869b6
2026-02-26 21:20:48 -08:00
Hui Xiao 372995470a Pass statistics and fix null clock in SstFileReader::MultiGet (#14393)
Summary:
**Summary:**
This is to sync an internal change of passing `Statistics*` to the GetContext constructor and collecting more stats as well as fix a bug this change created.

SstFileReader::MultiGet was passing nullptr for both `SystemClock*` and `Statistics*` to the GetContext constructor.  After  `Statistics*` was passed to the GetContext constructor (the internal change), this caused a segfault when a merge operation was triggered with statistics enabled, because the merge helper's StopWatchNano attempted to dereference the null clock pointer. Fix by passing `r->ioptions.clock` from the reader's options.

Additionally, add `assert(clock_)` guards to `StopWatchNano::Start()` and `ElapsedNanos()` to catch null clock bugs in debug builds. Can't do so in release build because it's on hot path.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14393

Test Plan:
- `./sst_file_reader_test --gtest_filter='SstFileReaderTableMultiGetTest.Basic'` exercises merge with statistics enabled, previously segfaulted without the fix with the clock, now passes.
- `./sst_file_reader_test` - all tests pass.

Reviewed By: xingbowang

Differential Revision: D94599343

Pulled By: hx235

fbshipit-source-id: 0a748bb00ee27bb202d01d410b52657101c05de0
2026-02-26 19:16:12 -08:00
xingbowang 1a8471b17e stop using portable folly build for nightly job (#14391)
Summary:
The nightly build had a flag for portable build on folly, but rocksdb does not. This causes link error. Fix this by removing portable build flag in folly build in nightly run. Nightly run will always build without cache using native flag. Only PR jobs uses cache.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14391

Test Plan: nightly run

Reviewed By: joshkang97

Differential Revision: D94520182

Pulled By: xingbowang

fbshipit-source-id: 7c7d3b089744c7e2e8ea073158f1b5b80db420d4
2026-02-26 12:43:28 -08:00
Xingbo Wang c311362ab6 Reapply "Fix flaky unit test in mempurge. (#14377)" (#14381) (#14385)
Summary:
This reverts commit 4213f9e14a.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14385

Reviewed By: archang19

Differential Revision: D94367585

Pulled By: xingbowang

fbshipit-source-id: 6c125b23ffa74e4113ef04847ccdefa015d7db35
2026-02-25 12:39:45 -08:00
xingbowang 5cb88bae30 fix portable build for cmake pr job (#14388)
Summary:
-march=native in cmake builds causes SIGILL after cache hits
Issue: PORTABLE=1 env var only works for Makefile builds. cmake ignores it and injects -march=native via its own CPU detection. Since ccache hashes the flag string literally, a cache compiled on an AVX-512 runner and restored on a non-AVX-512 runner produces a SIGILL crash.
Fix: Added -DPORTABLE=ON to build-linux-cmake-with-benchmark-no-thread-status and build-linux-cmake-with-folly-coroutines cmake commands.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14388

Test Plan: Github CI

Reviewed By: joshkang97

Differential Revision: D94367557

Pulled By: xingbowang

fbshipit-source-id: 5aa39cc36fc004d0ee636cd3fa197880d6cb773a
2026-02-25 10:47:20 -08:00
Hui Xiao 300fb8cace Add claude md file for deprecated option removal (#14360)
Summary:
**Context/Summary:**

It's very easy to make mistake in removing deprecated option such as deleting the corresponding entry in type info that breaks backward compatibility or missing tests to test backward compatibility. Example: https://github.com/facebook/rocksdb/pull/14350#discussion_r2834554287. Added a claude md file for that.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14360

Test Plan: In claude code, prompted for deprecated option removal with this claude md file in a repo separated from my previous deprecation efforts as much as possible and received correct change at one try.

Reviewed By: pdillinger

Differential Revision: D93920127

Pulled By: hx235

fbshipit-source-id: 1977ce7404b0188f84258552b4843e70394a5aea
2026-02-24 23:04:22 -08:00
Josh Kang 5db2eb0f3c Fix missing corruption check for key overflows (#14384)
Summary:
The refactoring in https://github.com/facebook/rocksdb/pull/14287/changes#diff-9f58e06172d3c8b15496fb4e8216a9ceee07ea2648e4c93d6a105d578697aa07L101-L103 unintentionally removed a corruption check for a key existing past its limit, but does account for a value existing past its limit.

This fix adds an explicit key check as well

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14384

Test Plan: CI

Reviewed By: pdillinger

Differential Revision: D94275062

Pulled By: joshkang97

fbshipit-source-id: ba793bbb966f0bae33df4c98e498e3d6a2b04087
2026-02-24 22:34:28 -08:00
Peter Dillinger 1ba60ee9ee Restore blob_dump_tool compression support (partially revert PR #14266) (#14382)
Summary:
PR https://github.com/facebook/rocksdb/issues/14266 ("Remove compression support") removed compression-related functionality from blob_dump_tool and ldb commands. While this was valid for the legacy stacked BlobDB (which no longer supports compression), the integrated blob storage (`enable_blob_files`) uses the same blob file format and fully supports compression via `blob_compression_type`.

This partial revert restores the ability to view uncompressed blob data from compressed blob files, which is essential for debugging and analysis of integrated blob storage.

Restored functionality:
- `blob_dump --show_uncompressed_blob` option
- `ldb dump --dump_uncompressed_blobs` option
- `ldb dump_live_files --dump_uncompressed_blobs` option
- Related ldb_test.py test coverage

The decompression implementation in utilities/blob_db/blob_dump_tool.cc has been updated to use the modern
`GetBuiltinV2CompressionManager()->GetDecompressorOptimizeFor()` API.

Suggested follow-up:
* Tests for blob_dump (or integrate it into sst_dump/ldb?)

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14382

Test Plan:
Restored ldb_test.py
Some manual testing

Reviewed By: mszeszko-meta

Differential Revision: D94257588

Pulled By: pdillinger

fbshipit-source-id: c6d8a556a51ec9422df208ae161806ccc1f20b36
2026-02-24 14:10:13 -08:00
xingbowang e95f3759e9 Fix clang tidy workflow (#14380)
Summary:
Fix clang tidy workflow

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14380

Test Plan: Unit test

Reviewed By: joshkang97

Differential Revision: D94237335

Pulled By: xingbowang

fbshipit-source-id: ed794cc13e684b3d6d11e20ce33edb4d6de79c9b
2026-02-24 13:16:18 -08:00
Hui Xiao de3616dc3b Prepare for 11.1.0 development (#14365)
Summary:
**Context/Summary:** history, version.h, format checking script, folly hash update

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14365

Test Plan: CI

Reviewed By: pdillinger

Differential Revision: D93952349

Pulled By: hx235

fbshipit-source-id: deeb7237cb51bbe3f3255f8eaa6a4ddda65248c2
2026-02-24 12:09:55 -08:00
xingbowang 4213f9e14a Revert "Fix flaky unit test in mempurge. (#14377)" (#14381)
Summary:
This reverts commit e7c3391640.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14381

Reviewed By: archang19

Differential Revision: D94244056

Pulled By: xingbowang

fbshipit-source-id: 0fffa8ce15dab4ece7b37e3839368fb399cd5734
2026-02-24 10:56:10 -08:00
Xingbo Wang 49a4165e3a Fix deadlock error false positive in stress test (#14376)
Summary:
Fix deadlock error false positive in stress test. TestDelete could trigger similar deadlock issue found earlier.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14376

Test Plan: Stress test

Reviewed By: hx235

Differential Revision: D94149457

Pulled By: xingbowang

fbshipit-source-id: 53f19c3816bcaaba0b7c01fe4e3f5f3e09a5dd6e
2026-02-24 07:14:30 -08:00
xingbowang bd5b299b92 2026 02 21 accelerate ci (#14368)
Summary:
Accelerate CI: ccache integration, replace scan-build, upgrade runners

    1. ccache integration via reusable composite action (setup-ccache)
       - Created .github/actions/setup-ccache/action.yml
       - Applied to 13+ Linux compilation jobs in pr-jobs.yml
       - PORTABLE=1 by default to avoid illegal instruction errors on
         heterogeneous runners; disabled for jobs linking pre-built Folly
       - On cache hit, reducing compilation from 20+ min to ~2-5 min per job
       - ccache placed before Folly build so Folly compilation also
         benefits from cache on Folly cache misses

    2. Replace scan-build with clang-tidy (30+ min -> seconds)
       - Removed build-linux-clang18-clang-analyze job from pr-jobs.yml
       - Expanded .clang-tidy to enable all clang-analyzer-* checks,
         matching scan-build's full coverage
       - Existing clang-tidy-comment.yml workflow now handles both
         clang-tidy and clang-analyzer checks on changed files only

    3. Upgrade Folly job runners for faster builds and tests
       - build-linux-make-with-folly: 16-core -> 32-core, -j32 -> -j64
       - build-linux-cmake-with-folly-coroutines: 16-core -> 32-core,
         -j20 -> -j64 (both make and ctest)

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14368

Reviewed By: joshkang97

Differential Revision: D94166095

Pulled By: xingbowang

fbshipit-source-id: 3dfbd71aace3ba9cb2e790873bdbedba20215657
2026-02-24 04:47:32 -08:00
Xingbo Wang e7c3391640 Fix flaky unit test in mempurge. (#14377)
Summary:
Add proper synchronization in mempurge unit test to fix flaky test

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14377

Test Plan: Unit test

Reviewed By: nmk70

Differential Revision: D94153290

Pulled By: xingbowang

fbshipit-source-id: 2a2cdcc6f8b5500bb5e68d2d223255cd1a5660b3
2026-02-23 22:55:04 -08:00
Josh Kang e704352e5d Sanitize crashtest invalid arg for interpolation search (#14374)
Summary:
Sanitize crash test arg for interpolation search when UDTs are used. Currently its not supported and returns invalid arg on DB open.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14374

Test Plan: CI

Reviewed By: archang19

Differential Revision: D94123220

Pulled By: joshkang97

fbshipit-source-id: 89b63f14a721bc692465620298161af23eec2446
2026-02-23 14:06:32 -08:00
Josh Kang 901c88e37b Separate keys and values in data blocks (#14287)
Summary:
Introduce new table option with separated key-value storage in data blocks.

This PR implements a new SST block format where keys and values are stored in separate sections within data blocks, rather than interleaved. Keys are stored first, followed by all values in a contiguous section. The motivation is better cpu cache hit rate during seeks and potentially better compression.

The additional storage cost is a varint per restart point, and 4 bytes additional block footer. For a data block with a restart interval of 16, it is approximately 1 bit of overhead per entry. But compression actually performs better, resulting in ~3% storage savings from benchmark.

For now I've opted to not separate kvs in non-data blocks since restart interval for those blocks is typically 1, and values are typically small and probably better inlined.

### New block layout

```
+------------------+
| Keys Section     |  <- Key entries with delta encoding
+------------------+
| Values Section   |  <- (new) Values stored contiguously
+------------------+
| Restart Array    |  <- Fixed32 offsets to restart points
+------------------+
| Values Offset    |  <- (new) 4 bytes: offset to values section
| Footer           |  <- 4 bytes: packed index_type + num_restarts
+------------------+
```

### Entry Format

**At restart points**
```
+--------------+------------------+----------------+-----------------+-----------+
| shared (v32) | non_shared (v32) | value_sz (v32) | value_off (v32) | key_delta |
+--------------+------------------+----------------+-----------------+-----------+
```

**At non-restart points**
```
+--------------+------------------+----------------+-----------+
| shared (v32) | non_shared (v32) | value_sz (v32) | key_delta |
+--------------+------------------+----------------+-----------+
```

- `value_offset` is only stored at restart points to save space
- For non-restart entries, value offset is computed as: `prev_value_offset + prev_value_size`

### Forward Compatibility
- We make use of reserved block footer bits to mark if a block has separated kv format. Should an older version read this, it will assume a very large block restart interval and result in a corruption error.

### Key Changes
- **BlockBuilder**: Accumulates values in a separate buffer; value offsets are stored only at restart points (other entries derive offset from previous value's position). There is an additional memcpy cost to place the value data after the key data.
- **Block iteration**: Iteration now needs to know if we are at a restart point. This will rely on `cur_entry_idx_`, which was previously only used for per-kv checksum purposes. In this new format, we also need to know the block_restart_interval, which was previously also only calculated for per-kv checksums.
- **Table properties**: Store `data_block_restart_interval`, `index_block_restart_interval`, and `separate_key_value_in_data_block` in table properties

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14287

Test Plan:
- Extended block_test, table_test, compaction_test to contain new separated_kv param
- Added new parameter to crash test

 ---

## Benchmark

### Varying Value Size

**Write:** `db_bench --num=1000000 --separate_key_value_in_data_block=<bool> --value_size=<X> --benchmarks=fillrandom,compact`
**Read:** `db_bench --db=$DB --use_existing_db --benchmarks=readrandom,readseq`

| Value Size | FillRandom (ops/s) | | Compact (s) | | ReadRandom (ops/s) | | ReadSeq (ops/s) | | SST Size (MB) | |
|----------:|-------------------:|-----:|----------:|-----:|-------------------:|-----:|----------------:|-----:|-------------:|-----:|
| | baseline | sep_kv | baseline | sep_kv | baseline | sep_kv | baseline | sep_kv | baseline | sep_kv |
| 1 | 253,280 | 264,977 | 0.516 | 0.497 (**-3.7%**) | 596,328 | 586,625 (**-1.6%**) | 5,904,681 | 6,069,581 (**+2.8%**) | 5.1 | 4.2 (**-18.6%**) |
| 16 | 235,757 | 242,367 | 0.572 | 0.533 (**-6.8%**) | 570,751 | 572,815 (**+0.4%**) | 5,371,138 | 5,604,908 (**+4.4%**) | 14.7 | 13.4 (**-8.5%**) |
| 100 | 264,299 | 265,427 | 0.461 | 0.454 (**-1.5%**) | 323,696 | 332,790 (**+2.8%**) | 4,239,725 | 4,232,416 (**-0.2%**) | 38.8 | 37.6 (**-3.2%**) |
| 1,000 | 238,992 | 242,764 | 2.349 | 2.329 (**-0.9%**) | 244,608 | 261,403 (**+6.9%**) | 1,285,394 | 1,265,868 (**-1.5%**) | 342.1 | 342.0 (**-0.0%**) |

### Varying Block Restart Interval

**Write:** `db_bench --num=1000000 --separate_key_value_in_data_block=<bool> --block_restart_interval=<X> --benchmarks=fillrandom,compact`
**Read:** `db_bench --db=$DB --use_existing_db --benchmarks=readrandom,readseq`

| BRI | FillRandom (ops/s) | | Compact (s) | | ReadRandom (ops/s) | | ReadSeq (ops/s) | | SST Size (MB) | |
|----:|-------------------:|-----:|----------:|-----:|-------------------:|-----:|----------------:|-----:|-------------:|-----:|
| | baseline | sep_kv | baseline | sep_kv | baseline | sep_kv | baseline | sep_kv | baseline | sep_kv |
| 1 | 251,654 | 263,707 | 0.453 | 0.485 (**+7.1%**) | 334,653 | 328,708 (**-1.8%**) | 4,194,342 | 3,954,291 (**-5.7%**) | 40.6 | 42.4 (**+4.5%**) |
| 4 | 253,797 | 252,394 | 0.476 | 0.476 (**+0.0%**) | 332,719 | 341,676 (**+2.7%**) | 4,135,691 | 4,051,151 (**-2.0%**) | 39.2 | 39.3 (**+0.3%**) |
| 8 | 260,143 | 262,273 | 0.496 | 0.460 (**-7.3%**) | 330,859 | 337,567 (**+2.0%**) | 4,144,081 | 4,187,389 (**+1.0%**) | 38.9 | 38.1 (**-2.1%**) |
| 16 | 252,875 | 263,176 | 0.464 | 0.455 (**-1.9%**) | 323,783 | 335,418 (**+3.6%**) | 4,127,310 | 4,217,028 (**+2.2%**) | 38.8 | 37.6 (**-3.2%**) |
| 32 | 260,224 | 269,422 | 0.464 | 0.451 (**-2.8%**) | 304,001 | 314,989 (**+3.6%**) | 4,310,162 | 4,247,248 (**-1.5%**) | 38.8 | 37.3 (**-3.8%**) |

### Varying Compression

**Write:** `db_bench --num=1000000 --separate_key_value_in_data_block=<bool> --compression_type=<X> [--compression_level=<N>] --benchmarks=fillrandom,compact`
**Read:** `db_bench --db=$DB --use_existing_db --benchmarks=readrandom,readseq`

| Compression | FillRandom (ops/s) | | Compact (s) | | ReadRandom (ops/s) | | ReadSeq (ops/s) | | SST Size (MB) | |
|------------|-------------------:|-----:|----------:|-----:|-------------------:|-----:|----------------:|-----:|-------------:|-----:|
| | baseline | sep_kv | baseline | sep_kv | baseline | sep_kv | baseline | sep_kv | baseline | sep_kv |
| None | 252,494 | 260,552 | 0.413 | 0.419 (**+1.5%**) | 356,290 | 371,535 (**+4.3%**) | 4,479,261 | 4,507,133 (**+0.6%**) | 73.5 | 73.6 (**+0.2%**) |
| LZ4 | 246,010 | 256,360 | 0.477 | 0.455 (**-4.6%**) | 342,497 | 345,882 (**+1.0%**) | 4,400,570 | 4,268,102 (**-3.0%**) | 38.3 | 37.6 (**-2.0%**) |
| ZSTD (L3) | 254,748 | 258,556 | 1.067 | 1.055 (**-1.1%**) | 176,724 | 177,566 (**+0.5%**) | 2,736,841 | 2,717,739 (**-0.7%**) | 32.9 | 31.3 (**-4.7%**) |
| ZSTD (L6) | 256,459 | 259,388 | 1.556 | 1.462 (**-6.0%**) | 177,390 | 176,691 (**-0.4%**) | 2,754,336 | 2,688,682 (**-2.4%**) | 32.8 | 31.1 (**-5.1%**) |

### Varying Block Size

**Write:** `db_bench --num=1000000 --separate_key_value_in_data_block=<bool> --block_size=<X> --benchmarks=fillrandom,compact`
**Read:** `db_bench --db=$DB --use_existing_db --benchmarks=readrandom,readseq`

| Block Size | FillRandom (ops/s) | | Compact (s) | | ReadRandom (ops/s) | | ReadSeq (ops/s) | | SST Size (MB) | |
|-----------:|-------------------:|-----:|----------:|-----:|-------------------:|-----:|----------------:|-----:|-------------:|-----:|
| | baseline | sep_kv | baseline | sep_kv | baseline | sep_kv | baseline | sep_kv | baseline | sep_kv |
| 4 KB | 263,203 | 260,362 | 0.469 | 0.461 (**-1.7%**) | 324,178 | 332,249 (**+2.5%**) | 4,231,537 | 4,217,763 (**-0.3%**) | 38.8 | 37.6 (**-3.1%**) |
| 16 KB | 252,742 | 263,161 | 0.426 | 0.428 (**+0.5%**) | 227,805 | 222,873 (**-2.2%**) | 5,146,997 | 5,081,080 (**-1.3%**) | 38.1 | 36.7 (**-3.6%**) |
| 64 KB | 257,490 | 260,225 | 0.423 | 0.414 (**-2.1%**) | 86,807 | 91,586 (**+5.5%**) | 5,380,403 | 5,372,372 (**-0.1%**) | 36.3 | 35.0 (**-3.5%**) |

### Varying Key Size

**Write:** `db_bench --num=1000000 --separate_key_value_in_data_block=<bool> --min_key_size=10 --max_key_size=100 --benchmarks=fillrandom,compact`
**Read:** `db_bench --db=$DB --use_existing_db --benchmarks=readrandom,readseq`

| Key Size | FillRandom (ops/s) | | Compact (s) | | ReadRandom (ops/s) | | ReadSeq (ops/s) | | SST Size (MB) | |
|---------:|-------------------:|-----:|----------:|-----:|-------------------:|-----:|----------------:|-----:|-------------:|-----:|
| | baseline | sep_kv | baseline | sep_kv | baseline | sep_kv | baseline | sep_kv | baseline | sep_kv |
| 10–100 | 243,740 | 255,183 | 0.618 | 0.622 (**+0.6%**) | 284,304 | 307,569 (**+8.2%**) | 3,738,921 | 3,686,676 (**-1.4%**) | 41.5 | 41.2 (**-0.8%**) |

### CPU Profile Notes
- No compression: DataBlock::SeekForGet uses less cpu (13.2% vs 13.9%)
  - https://fburl.com/strobelight/6mwwebft with separated KV
  - https://fburl.com/strobelight/m9m798ka without
- ZSTD compression: rocksdb::DecompressSerializedBlock uses more CPU (45.8% vs 44.9%), while DataBlock::SeekForGet uses less cpu (5.09% vs 6.52%)
  - https://fburl.com/strobelight/3x5nw1k4 with separated KV
  - https://fburl.com/strobelight/e7809046 without

 ---

Reviewed By: xingbowang, pdillinger

Differential Revision: D92103024

Pulled By: joshkang97

fbshipit-source-id: 47cfeb656ff3c20d34975f0b6c4c0462935a83dc
2026-02-23 12:42:05 -08:00
anand76 eb5d12e744 Fix race condition causing flaky hang in WritePreparedTransactionSeqnoTest (#14361)
Summary:
Summary

  - Fix a race condition in three WritePreparedTransactionSeqnoTest tests (SeqnoGoesBackwardsDuringErrorRecovery, SeqnoDiscrepancyDuringErrorRecovery, ConcurrentWritesDuringErrorRecovery) that could cause permanent hangs.
  - The tests inject a filesystem error during flush via a WriteManifest sync point callback, then wait for background error recovery to complete. The bug was in the ordering of operations after recovery starts: SetFilesystemActive(true) was called before ClearCallBack, allowing a window where recovery's ResumeImpl could trigger the callback and re-disable the filesystem. This left the filesystem permanently disabled, causing all recovery retries to fail and exit without firing the RecoverSuccess sync point, leaving the test thread blocked forever.
  - The fix swaps the order so ClearCallBack is called before SetFilesystemActive(true), ensuring the filesystem cannot be re-disabled by a late callback firing.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14361

Test Plan:
- Stress tested with gtest_parallel (500 iterations, 32 workers, 60s timeout) with no hangs observed.
  - Previously reproduced the hang at ~7% rate under stress with 15s timeout before the fix.

Reviewed By: pdillinger

Differential Revision: D93929251

Pulled By: anand1976

fbshipit-source-id: 9cb6844ed20146c754091575156b08d5551b3034
2026-02-23 12:01:28 -08:00
Xingbo Wang 4e11dd79e1 V2 serialization format for wide columns with blob references (#14314)
Summary:
Introduce a new V2 serialization format for wide column entities that supports storing individual column values in blob files. The V2 format adds a column type section that marks each column as either inline or blob-index, enabling per-column blob storage for large values.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14314

Reviewed By: pdillinger

Differential Revision: D92832066

Pulled By: xingbowang

fbshipit-source-id: 13c24347e1f481a059d67eef987d2d2b184b4a51
2026-02-21 06:19:45 -08:00
xingbowang aa7571c3ad Run clang-tidy in github CI (#14347)
Summary:
RocksDB has been using clang-tidy for a long time inside Meta. However, it is not efficient for external contributor, as the result from clang-tidy has to be ferried back through internal contributor. This PR added support to run clang-tidy on external github CI. It added .clang-tidy file based on internal version. It run clang-tidy in a separate pr job and a workflow step would post the pr job result to the PR itself. See example below.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14347

Test Plan: Github CI

Reviewed By: archang19

Differential Revision: D93862467

Pulled By: xingbowang

fbshipit-source-id: bb4330241036894deb619470efd73a7041a8b62f
2026-02-20 14:58:21 -08:00
Hui Xiao 29819f37e1 Remove deprecated ReadOptions::managed, `ColumnFamilyOptions::snap_refresh_nanos (#14350)
Summary:
**Context/Summary:**
Remove deprecated, unused APIs and options:
- ReadOptions::managed: This option was not used anymore. The functionality it controlled has been removed long ago.
- ColumnFamilyOptions::snap_refresh_nanos: Deprecated and unused option.

Corresponding C API (rocksdb_readoptions_set_managed) and Java API (ReadOptions.managed/setManaged) are also removed. All related checks an db_impl and db_impl_secondary iterators are cleaned up.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14350

Test Plan: make check

Reviewed By: pdillinger

Differential Revision: D93812438

Pulled By: hx235

fbshipit-source-id: e4a9d21c65f83294b6d0878286ba14024f049bac
2026-02-20 14:00:41 -08:00
Andrew Chang 6d6f7d825b Check io_uring probe result in SupportedOps (#14355)
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14355

SupportedOps advertised kAsyncIO based only on the IsIOUringEnabled() weak symbol check, without verifying that the constructor's io_uring probe actually succeeded. Add a thread_local_async_read_io_urings_ null check so kAsyncIO is only reported when the probe passed. Also update the constructor to probe with the same IORING_SETUP_SINGLE_ISSUER | IORING_SETUP_DEFER_TASKRUN flags that ReadAsync and MultiRead use at runtime.

Reviewed By: anand1976

Differential Revision: D93780065

fbshipit-source-id: 6f51f544b267cb39d09b49949a9485f55eeae12e
2026-02-20 10:43:30 -08:00
Hui Xiao 4c89ff1102 Remove deprecated SstFileWriter::Add() and skip_filters parameter (#14352)
Summary:
**Context/Summary:**
Remove `SstFileWriter::Add()` (deprecated in favor of `Put()`) and the `skip_filters` parameter from `SstFileWriter` constructors (deprecated in favor of setting `BlockBasedTableOptions::filter_policy` to `nullptr`).

Both APIs have zero active callers. The `skip_filters` field is also removed from `TableBuilderOptions` (write-side only; the read-side `TableReaderOptions::skip_filters` is unchanged).

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14352

Test Plan: make check

Reviewed By: xingbowang

Differential Revision: D93812389

Pulled By: hx235

fbshipit-source-id: 236b36a6e664758ab5ad90e606bc195d0a6de70f
2026-02-19 22:10:47 -08:00
Peter Dillinger f1a6759b1f Fix flaky DBTestXactLogIterator.TransactionLogIteratorCheckWhenArchive (#14349)
Summary:
a couple recent failures in this test. Waiting for purge and disabling sync points before Close should resolve the issues.

Also fixing EventListenerTest.BlobDBOnFlushCompleted because it showed up as flaky in CI for this PR

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14349

Test Plan: watch CI

Reviewed By: mszeszko-meta

Differential Revision: D93619322

Pulled By: pdillinger

fbshipit-source-id: bb9fc7d3c0ecaaeaffe4305e1ad403cbcd597484
2026-02-19 20:26:38 -08:00
Peter Dillinger 520c3ecbf1 Prepare for 11.0 major release (#14357)
Summary:
In my last version bump, I forgot that the next release would be a major release. We can fix that now ahead of release cut.

I'm also updating folly now because I have experience resolving folly issues. Folly commit e04860553 changed libevent to build as static-only so required a change in our build.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14357

Test Plan: CI

Reviewed By: xingbowang

Differential Revision: D93797666

Pulled By: pdillinger

fbshipit-source-id: 22179da900f9dc6c5544163071079a4701c7c663
2026-02-19 18:51:34 -08:00
Hui Xiao 407f02da19 Remove deprecated SliceTransform::InRange() virtual method (#14353)
Summary:
**Summary/Context:**

Remove the `InRange()` virtual method from `SliceTransform` and all its overrides. This method was marked DEPRECATED, never called by RocksDB, and existed only for backward compatibility.

Also removes the `in_range` callback parameter from `rocksdb_slicetransform_create()` in the C API, which is a breaking change appropriate for a major version release.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14353

Test Plan: Make check

Reviewed By: xingbowang

Differential Revision: D93795070

Pulled By: hx235

fbshipit-source-id: 5eba23f1d038b19c494997a55e5d8ca379fbedcb
2026-02-19 16:45:51 -08:00
Josh Kang 98002215d0 Fix interpolation search target key less than shared prefix length. (#14343)
Summary:
There was an edge case missed in the implementation of interpolation search for target keys that had a length smaller than the shared prefix.

E.g. first_key = "aaaaaa", last_key = "aaaaaz", target_key = "aaz". In the existing setup, we will seek to position 0, but in reality is should be seeked to the end.

#### The fix
The solution here was to also do a bounds check on the first search iteration. We utilize memcmp on the target key with the shared_prefix to determine if the target key is outside the bounds. An edge case here is if the target key itself a prefix of the shared prefix (e.g. target = "aaaa"), in this case memcmp return return 0, but the target key is actually smaller.

### Minor optimizations
- cache left,right values so we don't need to re-compute it when left/right boundaries don't change
- In ReadBe64FromKey, utilize memcpy + swap for fast path
- since we have already computed a shared_prefix, every other comparison only needs to compare the non-shared suffixes

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14343

Test Plan:
Added new unit tests to test this case

### Benchmarks

No significant regressions due to additional memcmp.

#### Configuration
- **CPU:** 192 * AMD EPYC-Genoa Processor
- **RocksDB Version:** 10.12.0
- **Compression:** Snappy
- **Entries:** 1,000,000
- **Value Size:** 100 bytes
- **Index Search Type:** interpolation_search
- **Index Shortening Mode:** 1

#### Results

| Benchmark | Params | ops/s (main) | ops/s (feature) | % change |
|-----------|--------|-------------|-----------------|----------|
| readrandom | 16B keys, no prefix | 367,264 | 369,163 | +0.52% |
| readrandom | 100B keys, prefix_size=50 | 376,066 | 371,193 | -1.29% |

Reviewed By: pdillinger

Differential Revision: D93535267

Pulled By: joshkang97

fbshipit-source-id: beda182efce1e914ff587e697b927347cfa42656
2026-02-19 15:22:58 -08:00
xingbowang cfc2a523a3 Add clang-tidy-comment workflow (#14348)
Summary:
Add clang-tidy-comment workflow. This workflow allows pr clang tidy pr job to post the clang-tidy finding directly on the PR page.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14348

Test Plan: Will be tested with next clang-tidy PR

Reviewed By: joshkang97

Differential Revision: D93670150

Pulled By: xingbowang

fbshipit-source-id: 8245f9d5bde8cf800d88034c4339de9f387c5692
2026-02-19 14:59:37 -08:00
xingbowang 3556c22059 Remove deprecated option skip_checking_sst_file_sizes_on_db_open (#14346)
Summary:
Remove deprecated option skip_checking_sst_file_sizes_on_db_open

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14346

Test Plan: Unit test

Reviewed By: hx235

Differential Revision: D93602683

Pulled By: xingbowang

fbshipit-source-id: f576825cb107bb0aeb14f4ff29fef0df269b8728
2026-02-19 14:12:38 -08:00
Peter Dillinger 61b4edd15e Test compaction forward compatibility (#14344)
Summary:
Extending https://github.com/facebook/rocksdb/issues/14323 by testing scenarios for compaction after downgrade. Detail: we shouldn't need to test loading options with compaction, as options file inclusion is mostly a sanity check for "can you open the DB with options file?"

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14344

Test Plan: manual run of SHORT_TEST=1 J=140 tools/check

Reviewed By: xingbowang

Differential Revision: D93553897

Pulled By: pdillinger

fbshipit-source-id: ec08ae2a3d49971e24a215e38df9506fe1133096
2026-02-18 10:52:04 -08:00
Peter Dillinger d3817f058d Remove deprecated DB::Open raw pointer variants (and more) (#14335)
Summary:
and remove deprecated DB::MaxMemCompactionLevel(). In the process of pushing through a relatively clean refactoring of uses of the old functions, some other minor public APIs are also migrated from raw DB pointers to unique_ptr.

Claude did pretty much all the work, but requiring dozens of prompts to actually push through relatively clean phase out of raw DB pointers from what needed to be touched, and leaving that code in better shape. (Hundreds of `DB*` still remain all over the place even outside C and Java bindings.)

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14335

Test Plan: existing tests; no functional changes intended

Reviewed By: xingbowang, mszeszko-meta

Differential Revision: D93523820

Pulled By: pdillinger

fbshipit-source-id: e4ca22ad81cd2cfe91122d7507d7ca34fe03d043
2026-02-17 23:33:39 -08:00
Peter Dillinger 1f9d8ee302 Remove obsolete code for block-based format_version < 2 (after #14315) (#14327)
Summary:
After PR https://github.com/facebook/rocksdb/issues/14315 dropped support for block-based table format_version < 2, several code paths became obsolete. This change removes them.

Investigation findings:

1. Table properties are now a hard requirement for block-based SST files:
   - format_version >= 2 guarantees a properties block exists
   - Removed defensive conditionals like `if (rep_->table_properties)`
   - Missing properties block now returns Status::Corruption instead of just logging an error. This is important because some properties affect the semantic interpretation of the file.

2. Index type property (kIndexType) is now required:
   - kIndexType was introduced in Feb 2014 (commit 74939a9e1), ~11 months BEFORE format_version was introduced in Jan 2015
   - BlockBasedTablePropertiesCollector::Finish() has always written kIndexType unconditionally for all block-based tables
   - Therefore all format_version >= 2 files have this property
   - Now returns Status::Corruption if missing instead of silently defaulting to kBinarySearch

3. Removed SetOldTableOptions() from sst_file_dumper:
   - This fallback handled files without a properties block
   - Dead code since format_version >= 2 guarantees properties exist

4. Removed kPropertiesBlockOldName ("rocksdb.stats") fallback:
   - The properties block was renamed from "rocksdb.stats" to "rocksdb.properties" in RocksDB 2.7 (April 2014)
   - format_version 2 was introduced in RocksDB 3.10 (Oct 2015)
   - All table formats (block-based, plain, cuckoo) were created after the rename, so they all use "rocksdb.properties"
   - The backward compatibility fallback in FindOptionalMetaBlock() was dead code for all supported table formats

5. Removed obsolete assertion about format_version 0 checksum in BlockBasedTableBuilder::WriteFooter()

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14327

Test Plan: some tests updated for updated requirements. Mostly, CI including format compatible test

Reviewed By: mszeszko-meta

Differential Revision: D93124820

Pulled By: pdillinger

fbshipit-source-id: eb12cbdca0e69f34a08051d5160c282384128a4a
2026-02-17 23:31:30 -08:00
Peter Dillinger 641f4703ac Refactor data block footer to reserve metadata bits for future features (#14332)
Summary:
I'm implementing this intending it to be used for https://github.com/facebook/rocksdb/issues/14287

Refactor the data block footer encoding/decoding to use a struct-based Encode/Decode API (DataBlockFooter), reserving the top 4 bits of the footer for metadata:
- Bit 31: Hash index present (kDataBlockBinaryAndHash) - existing use
- Bits 28-30: Reserved for future features

Comments have some detail for why it is safe to assume no practical existing SST files would use these newly reserved bits. And for forward compatibility, existing versions detect (non-zero) use of these new bits as impossibly large num_restarts and report "bad block contents". Not perfect, but not bad.

Key changes:
- Replace PackIndexTypeAndNumRestarts/UnPackIndexTypeAndNumRestarts with DataBlockFooter::EncodeTo/DecodeFrom methods
- DecodeFrom returns a detailed error when reserved bits are set, enabling graceful failure on newer format versions
- Reduce kMaxNumRestarts from 2^31-1 to 2^28-1 (268M), which is adequate for the maximum possible restarts in a 4GiB block
- Add GetCorruptionStatus() to Block for detailed error messages (Note that we are sensitive to the size of Block objects, so have to avoid adding unnecessary new members.)
- Remove obsolete kMaxBlockSizeSupportedByHashIndex size checks

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14332

Test Plan:
- Existing unit tests and format compatibility test
- Add test for reserved bit detection (ReservedBitInDataBlockFooter)

Reviewed By: joshkang97

Differential Revision: D93293152

Pulled By: pdillinger

fbshipit-source-id: b65a83e96bb09a98fb9b8b2dd9f754653ca7ed4d
2026-02-17 17:28:54 -08:00
Peter Dillinger d693d5ae26 Blog post on CPU bug (#14078)
Summary:
see draft post

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14078

Test Plan: markdown preview (simple post)

Reviewed By: hx235

Differential Revision: D85475518

Pulled By: pdillinger

fbshipit-source-id: d7f7b0d68de3880fcffbdbbef27cd2c14fe51f96
2026-02-17 15:30:00 -08:00
anand76 653fd9c65b Bug fix for bg error recovery in TransactionDB (#14313)
Summary:
This PR fixes a bug in the interaction between WritePrepared/WriteUnprepared TransactionDB (with two_write_queues=true) and background error recovery. This bug caused crash tests to fail with a "sequence number going backwards" error during DB open.

Root Cause
------------
When two_write_queues=true, sequence numbers are allocated via FetchAddLastAllocatedSequence() before a write completes, but are only published via SetLastSequence() after the write succeeds. If a background error occurs (e.g., a MANIFEST write failure during flush), the error recovery path in DBImpl::ResumeImpl creates new memtables and WAL files. The new WAL's starting sequence number is based on LastSequence() (the published value), which can be lower than already-allocated sequence numbers that were written to the old WAL. On subsequent recovery, RocksDB detects that sequence numbers in the new WAL are lower than those in the old WAL and reports a "sequence number going backwards" corruption error, causing the DB to fail to open.

Fix
 ---
The fix adds a call to a new VersionSet::SyncLastSequenceWithAllocated() method at the beginning of DBImpl::ResumeImpl, before any new memtables or WALs are created. This method advances last_sequence_ to match last_allocated_sequence_ if the latter is higher, ensuring the new WAL starts with a sequence number that is at least as high as any previously allocated one.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14313

Test Plan:
---------
Add new unit tests in write_prepared_transaction_test_seqno

Reviewed By: pdillinger

Differential Revision: D92746944

Pulled By: anand1976

fbshipit-source-id: 34385fc13fd74435dd1c3283637eb118f45d887e
2026-02-17 14:52:00 -08:00
Peter Dillinger f065e1c95d Fix up authors.yml for blog entries (#14342)
Summary:
Fixes blog author display issues on rocksdb.org/blog by:

* Adding missing authors to authors.yml: pdillinger, alanpaxton, akankshamahajan15, anand1976, poojam23
* Standardizing on GitHub usernames: renamed sdong → siying
* Fixing typo in 2016-02-25-rocksdb-ama.markdown: yhchiang → yhciang
* A short note in CLAUDE.md

Authors were not showing on the blog because they were referenced in post frontmatter but not defined in the _data/authors.yml lookup file.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14342

Test Plan: push & see ;)

Reviewed By: mszeszko-meta

Differential Revision: D93523972

Pulled By: pdillinger

fbshipit-source-id: 757c33e80f3c1d99ff4134a37321f40634d6e294
2026-02-17 14:30:03 -08:00
Hui Xiao 49695ef868 Add debug assertion for clean cut invariant in expanding compaction inputs (#14333)
Summary:
**Context/Summary:**

Stress test recently encountered a one-off failure where input file selection for trivial move did not select all the files it should and left behind one adjacent file to the input file. This violated the clean cut invariant enforced through `ExpandInputsToCleanCut()` and caused `Get()` to return stale data.

While I had no luck reproducing it nor in code inspection to find the root cause, this debug assertion should help in two ways: 1. Fail fast if the invariant is violated, showing us the file boundary in memory 2. If the assertion doesn't trigger yet the same failure occurs, it points to metadata corruption bypassing this check and ExpandInputsToCleanCut() enforcement

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14333

Test Plan:
- Existing unit tests
- Manually trace through and run a stress test command that frequently exercises this check for 10 minutes
```
./db_stress --level0_file_num_compaction_trigger=2 --acquire_snapshot_one_in=10000 --adaptive_readahead=1 --allow_concurrent_memtable_write=0 --allow_data_in_errors=True --allow_setting_blob_options_dynamically=1 --async_io=1 --auto_readahead_size=0 --avoid_flush_during_recovery=0 --avoid_unnecessary_blocking_io=1 --backup_max_size=104857600 --backup_one_in=100000 --batch_protection_bytes_per_key=8 --blob_cache_size=1048576 --blob_compaction_readahead_size=4194304 --blob_compression_type=lz4 --blob_file_size=1048576 --blob_file_starting_level=1 --blob_garbage_collection_age_cutoff=1.0 --blob_garbage_collection_force_threshold=0.75 --block_protection_bytes_per_key=2 --block_size=16384 --bloom_before_level=0 --bloom_bits=16 --bottommost_compression_type=zstd --bottommost_file_compaction_delay=0 --bytes_per_sync=0 --cache_index_and_filter_blocks=0 --cache_size=8388608 --cache_type=auto_hyper_clock_cache --charge_compression_dictionary_building_buffer=1 --charge_file_metadata=1 --charge_filter_construction=0 --charge_table_reader=0 --checkpoint_one_in=1000000 --checksum_type=kXXH3 --clear_column_family_one_in=0 --column_families=1 --compact_files_one_in=1000000 --compact_range_one_in=1000000 --compaction_pri=3 --compaction_readahead_size=0 --compaction_ttl=2 --compression_checksum=1 --compression_max_dict_buffer_bytes=0 --compression_max_dict_bytes=0 --compression_parallel_threads=1 --compression_type=xpress --compression_use_zstd_dict_trainer=1 --compression_zstd_max_train_bytes=0 --continuous_verification_interval=0 --data_block_index_type=0 --db=/dev/shm/rocksdb_test/rocksdb_crashtest_whitebox --db_write_buffer_size=0 --delpercent=4 --delrangepercent=1 --destroy_db_initially=0 --detect_filter_construct_corruption=1 --disable_wal=0 --enable_blob_files=0 --enable_blob_garbage_collection=0 --enable_compaction_filter=0 --enable_pipelined_write=1 --enable_thread_tracking=1 --expected_values_dir=/dev/shm/rocksdb_test/rocksdb_crashtest_expected --fail_if_options_file_error=0 --fifo_allow_compaction=0 --file_checksum_impl=big --flush_one_in=1000000 --format_version=3 --get_current_wal_file_one_in=0 --get_live_files_one_in=1000000 --get_property_one_in=1000000 --get_sorted_wal_files_one_in=0 --index_block_restart_interval=2 --index_type=3 --ingest_external_file_one_in=0 --initial_auto_readahead_size=524288 --iterpercent=10 --key_len_percent_dist=1,30,69 --level_compaction_dynamic_level_bytes=1 --lock_wal_one_in=1000000 --long_running_snapshots=1 --manual_wal_flush_one_in=1000 --mark_for_compaction_one_file_in=0 --max_auto_readahead_size=0 --max_background_compactions=1 --max_bytes_for_level_base=1000 --max_key=25000000 --max_key_len=3 --max_manifest_file_size=16384 --max_write_batch_group_size_bytes=64 --max_write_buffer_number=3 --max_write_buffer_size_to_maintain=1000 --memtable_max_range_deletions=100 --memtable_prefix_bloom_size_ratio=0.01 --memtable_protection_bytes_per_key=4 --memtable_whole_key_filtering=0 --memtablerep=skip_list --min_blob_size=0 --min_write_buffer_number_to_merge=1 --mmap_read=1 --mock_direct_io=False --nooverwritepercent=1 --num_file_reads_for_auto_readahead=2 --open_files=-1 --open_metadata_write_fault_one_in=0 --open_read_fault_one_in=0 --open_write_fault_one_in=0 --ops_per_thread=100000000 --optimize_filters_for_memory=1 --paranoid_file_checks=0 --partition_filters=0 --partition_pinning=1 --pause_background_one_in=1000000 --periodic_compaction_seconds=1 --prefix_size=8 --prefixpercent=5 --prepopulate_blob_cache=0 --prepopulate_block_cache=0 --preserve_internal_time_seconds=0 --progress_reports=0 --read_fault_one_in=32 --readahead_size=0 --readpercent=45 --recycle_log_file_num=0 --reopen=0 --secondary_cache_fault_one_in=32 --secondary_cache_uri=compressed_secondary_cache://capacity=8388608 --set_options_one_in=0 --snapshot_hold_ops=100000 --sst_file_manager_bytes_per_sec=0 --sst_file_manager_bytes_per_truncate=0 --stats_dump_period_sec=10 --subcompactions=1 --sync=0 --sync_fault_injection=0 --target_file_size_base=1000 --target_file_size_multiplier=1 --test_batches_snapshots=0 --top_level_index_pinning=0 --unpartitioned_pinning=3 --use_blob_cache=0 --use_direct_io_for_flush_and_compaction=0 --use_direct_reads=0 --use_full_merge_v1=0 --use_get_entity=0 --use_merge=0 --use_multi_get_entity=0 --use_multiget=0 --use_put_entity_one_in=5 --use_shared_block_and_blob_cache=0 --user_timestamp_size=0 --value_size_mult=32 --verification_only=0 --verify_checksum=1 --verify_checksum_one_in=1000000 --verify_db_one_in=100000 --verify_file_checksums_one_in=1000000 --verify_iterator_with_expected_state_one_in=5 --verify_sst_unique_id_in_manifest=1 --wal_bytes_per_sync=524288 --wal_compression=none --write_buffer_size=1000 --write_dbid_to_manifest=1 --write_fault_one_in=0 --writepercent=35
```

Reviewed By: mszeszko-meta

Differential Revision: D93300664

Pulled By: hx235

fbshipit-source-id: b56f01c08a7348ba383110dd8f89b5b1b7961c55
2026-02-17 14:12:08 -08:00
Andrew Chang 09bda51c50 Propagate file_checksum through FileOptions on NewRandomAccessFile (#14321)
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14321

Add file_checksum and file_checksum_func_name fields to FileOptions so that downstream FileSystem implementations can access per-file checksum metadata when SST files are opened. The fields are populated from FileMetaData at all call sites where SST files are opened via NewRandomAccessFile: TableCache::GetTableReader, Version::GetTableProperties, and CompactionJob::ReadTablePropertiesDirectly. Also fixes the fallback path in TableCache::GetTableReader to use the local fopts (with temperature and checksum) instead of the original file_options.

Added a kNoFileChecksumFuncName which is distinct from  kUnknownFileChecksumFuncName:

 - kUnknownFileChecksumFuncName ("Unknown"): We have FileMetaData for this file, and the metadata says no checksum was computed (no factory was configured when the file was written). This is a property of the file itself.
- kNoFileChecksumFuncName ("Unavailable"): We don't even have FileMetaData — we're opening this file in a context where there's no checksum metadata to propagate at all (e.g., SstFileDumper, SstFileReader, checksum generation). It's a property of the call site, not the file.

So the assertion file_checksum.empty() is correct for both, but for different reasons — one says "the file has no checksum," the other says "we have no idea about this file's checksum."

Reviewed By: pdillinger

Differential Revision: D92728944

fbshipit-source-id: 8fd34ea22ca87090b26d0a55c921f354f97f1ffc
2026-02-17 13:05:44 -08:00
Maciej Szeszko 5f692d747c Fall back to sync read when async IO is unavailable (#14337)
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14337

## Context:

D91624185 changed `FilePrefetchBuffer::PollIfNeeded` from `void` to returning `Status`, correctly propagating `Poll` errors instead of silently swallowing them. A side effect is that when io_uring fails to initialize at runtime (e.g., sandcastle seccomp restrictions), `ReadAsync` returns `NotSupported` which now propagates through `PrefetchRemBuffers` and `HandleOverlappingAsyncData`, causing `PrefetchInternal` to return early before executing the synchronous `Read` that the caller actually depends on. This leaves iterators invalid with no recovery path — both in `db_stress` crash tests and in production. The filesystem advertises async IO support (`CheckFSFeatureSupport` passes), so the failure only surfaces at runtime when io_uring initialization fails. The prior behavior silently degraded to sync reads because `PollIfNeeded` swallowed the error.

## Changes

Add a sync fallback in `FilePrefetchBuffer::ReadAsync` — the single chokepoint for all async reads. When `reader->ReadAsync()` returns `NotSupported`, fall back to `reader->Read()` synchronously, populate the buffer inline, and return OK. Since `async_read_in_progress_` stays false, `PollIfNeeded` becomes a no-op (nothing to poll, data is already there). All callers — `PrefetchRemBuffers`, `HandleOverlappingAsyncData`, `PrefetchAsync` — work transparently without any per-site changes.

Reviewed By: archang19

Differential Revision: D93432284

fbshipit-source-id: daef185fc3535e347d182e75dd443ae921eeb495
2026-02-17 12:51:36 -08:00
Maciej Szeszko ebd1000008 Fix UB in ReadBe64FromKey shift by 64 (#14340)
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14340

`ReadBe64FromKey` pads its result with `val <<= (8 - len) * 8` to right-align partial reads. When the seek target's user key is shorter than shared_prefix_len, len is 0 and this becomes a shift by 64, which is undefined behavior for uint64_t. On x86 this happens to produce 0 (the correct result), but UBSan rightfully flags it. Guard the shift with `len > 0 && len < 8`.

Reviewed By: joshkang97

Differential Revision: D93435715

fbshipit-source-id: bab128e9a65ea18d401670268cbac77d45e11340
2026-02-17 12:28:31 -08:00
Maciej Szeszko 821bd37d09 Cap max table files size in db stress FIFO compactions (#14341)
Summary:
FIFO crash tests fail on DB open when `fifo_compaction_max_data_files_size_mb` is randomly set to 100 or 500 MB, because `max_table_files_size` defaults to 1GB and the validation requires `max_data_files_size` >= `max_table_files_size` when non-zero. Cap `max_table_files_size` to `max_data_files_size` in db_stress when the latter is set. `max_table_files_size` is ignored at runtime when `max_data_files_size` is non-zero, so this only satisfies the validation constraint.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14341

Reviewed By: pdillinger

Differential Revision: D93503113

Pulled By: mszeszko-meta

fbshipit-source-id: 5c3e7c9b568661244c71c548cb0fe5e55472c0ca
2026-02-17 11:49:20 -08:00
Maciej Szeszko 88ff4f6b12 Disable Interpolation Search in DB Stress (#14339)
Summary:
Temporarily disable interpolation search.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14339

Reviewed By: pdillinger

Differential Revision: D93497658

Pulled By: mszeszko-meta

fbshipit-source-id: 6ac826dd3fc354e18af0d928f87ed71e2cef3f14
2026-02-17 11:13:17 -08:00
Xingbo Wang b040ab83e1 Add a new picking algorithm in fifo compaction (#14326)
Summary:
Add a new kv ratio based compaction picking algorithm in fifo compaction

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14326

Test Plan: Unit test

Reviewed By: pdillinger

Differential Revision: D93257941

Pulled By: xingbowang

fbshipit-source-id: fd2d0e1356c7b54682a1197475a1bd26cb45c9d4
2026-02-15 10:04:58 -08:00
Josh Kang 9f47518676 Add interpolation search as an alternative to binary search (#14247)
Summary:
Interpolation search is an alternative algorithm to binary search, which performs better on uniformly distributed keys. Instead of binary search always computing the mid point of the left and right boundaries, interpolation search "interpolates" the mid point based on the distance to the target. Fortunately, we can re-use existing block format to support interpolation search.

For a given block, we compute the shared_prefix length of the first and last key. Interpolation search is usually done with numerical target values, so for a variable binary length key, we calculate the "value" as the first 8 non-shared bytes. This also means interpolation search would only really be effective for bytewise comparator (guarded via options validations).

#### Fallback to binary search
- if the the val(left_key) == val(right_key) then we fallback to classic binary search (to avoid divide by 0)
- interpolation search is significantly more computationally expensive than binary search, so when the search distance is small, we also fallback to binary search.
- if interpolation search does not make significant progress (i.e. reduces search space by more than half each iteration), we can assume data is non-uniform and fallback.

Interpolation search also performs best when there is minimal shortening, especially shortening of the last block, as it can heavily skew the distribution of the actual keys.

Note that each search algorithm is guaranteed to make progress because at each iteration the search space is guaranteed to be reduce by at least 1.

For now this change only applies to index block seeks, as data block seeks and other blocks do not have as many entries and would not require significant number of search rounds, but it could be easily extended to include that support.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14247

Test Plan:
Updated unit tests and crash test with new search option

### Benchmark
The default benchmark sets up keys in generally uniform distribution, so it was a good way to test performance improvements.

Setup: `./db_bench -benchmarks=fillseq,compact -index_shortening_mode=1`

#### Before this change
```
./db_bench -use_existing_db=true -benchmarks=readrandom -seed=1

readrandom   :       2.899 micros/op 344973 ops/sec 2.899 seconds 1000000 operations;   38.2 MB/s (1000000 of 1000000 found)
```

#### After this change

Notice how key comparison counts are the same between the two.
```
./db_bench -use_existing_db=true -benchmarks=readrandom -seed=1 -index_search_type=binary_search

readrandom   :       2.881 micros/op 347128 ops/sec 2.881 seconds 1000000 operations;   38.4 MB/s (1000000 of 1000000 found)
```

```
./db_bench -use_existing_db=true -benchmarks=readrandom -seed=1 -index_search_type=interpolation_search

readrandom   :       2.609 micros/op 383209 ops/sec 2.610 seconds 1000000 operations;   42.4 MB/s (1000000 of 1000000 found)
```

With a non-uniform distribution, `i.e. index_shortening_mode=2`

```
./db_bench -use_existing_db=true -benchmarks=readrandom -seed=1 -index_search_type=binary_search

readrandom   :       2.958 micros/op 338075 ops/sec 2.958 seconds 1000000 operations;   37.4 MB/s (1000000 of 1000000 found)
```

```
./db_bench -use_existing_db=true -benchmarks=readrandom -seed=1 -index_search_type=interpolation_search

readrandom   :       5.502 micros/op 181750 ops/sec 5.502 seconds 1000000 operations;   20.1 MB/s (1000000 of 1000000 found)
```

Reviewed By: pdillinger

Differential Revision: D91063163

Pulled By: joshkang97

fbshipit-source-id: 151d6aa76f8713740b714de6e406aff40d28ccbc
2026-02-13 17:15:10 -08:00
Peter Dillinger 871f79d6ef Reformat source files (#14331)
Summary:
probably something changed, maybe https://github.com/facebook/rocksdb/issues/14311

Full command:
```
git ls-files '*.cc' '*.h' | grep -v '^third-party/' | grep -v 'range_tree' | xargs clang-format -i
```

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14331

Test Plan: CI

Reviewed By: mszeszko-meta

Differential Revision: D93246992

Pulled By: pdillinger

fbshipit-source-id: 6bc5b97978fef8aee52823dadb6daa4bea57343d
2026-02-13 11:56:22 -08:00
Peter Dillinger 672389fd8c Remove obsolete compression code and some .h->.cc movement (#14325)
Summary:
In follow-up to https://github.com/facebook/rocksdb/issues/14315

Remove obsolete code replaced by new Compressor/Decompressor interface:
* OLD_CompressData and OLD_UncompressData
* Individual compression/decompression functions (Snappy_*, Zlib_*, BZip2_*, LZ4_*, LZ4HC_*, XPRESS_*, ZSTD_Compress, ZSTD_Uncompress)
* CompressionInfo and UncompressionInfo classes
* UncompressionDict class
* compression::PutDecompressedSizeInfo and GetDecompressedSizeInfo

The only small refactoring in this change that is not pure code removal or movement is in blob_file_builder_test.cc.

Move some function implementations etc. from compression.h to compression.cc:
* CompressionTypeToString, CompressionTypeFromString, CompressionOptionsToString
* ZSTD_TrainDictionary (both overloads), ZSTD_FinalizeDictionary
* DecompressorDict::Populate
* Most compression library includes

Also cleaned up other includes of compression.h, which caused some other files to need new includes.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14325

Test Plan: existing tests

Reviewed By: hx235

Differential Revision: D93120580

Pulled By: pdillinger

fbshipit-source-id: ab5c50db7379c0387a8c0e379642c9ea2799eae5
2026-02-13 11:18:05 -08:00
Hui Xiao c33a4989ad Default CompactionOptionsUniversal::reduce_file_locking to be true (#14329)
Summary:
**Context/Summary:**

Internal adoption has demonstrated stability and measurable improvements of this feature without much cost so we can turn it on by default. Eventually we'd like to remove this configuration and make this an expected behavior.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14329

Test Plan: Existing unit test

Reviewed By: mszeszko-meta

Differential Revision: D93210059

Pulled By: hx235

fbshipit-source-id: 04f77954e6624c8e60a2db030eb19eb341dd0fcf
2026-02-13 10:37:34 -08:00
Peter Dillinger 7ecc12110c Fix format compatibility issues, extend test (#14323)
Summary:
See https://github.com/facebook/rocksdb/issues/14240 which brought this to my attention. Here I've added range deletions and compactions to the format compatible test, and fixed or worked-around compatibility issues (likely longstanding).

The first fix was in Version::MaybeInitializeFileMetaData for an assertion failure simply from adding range deletions from some 5.x version.

The second fix is a broader work-around for older SST files with unreliable num_entries/num_range_deletions/num_deletions statistics in their table properties. We depend on them only for some paranoid checks for compaction, so in my assessment the best way to deal with those files is to exclude the paranoid checks when dealing with the files with unrelaible data. (Details in code comments.) The important part is that compacting old files is exceptionally rare, so we aren't really interefering with the paranoid checks doing thier job on an ongoing basis.

This depends on https://github.com/facebook/rocksdb/issues/14315 (just landed) because there is a remaining undiagnosed problem with some very early releases, but I'm not fixing that because its support is being dropped.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14323

Test Plan: test extended (ran locally excluding some releases)

Reviewed By: xingbowang

Differential Revision: D93032653

Pulled By: pdillinger

fbshipit-source-id: f90b32f30ba4764692e68d23705f42c778e0dc1d
2026-02-13 09:18:40 -08:00
Hui Xiao d72a471749 Replace resumable compaction job unit test with compaction service unit test (#14191)
Summary:
**Context/Summary**:
compaction_job_test does low level assertions on what keys were saved and to resume in https://github.com/facebook/rocksdb/commit/1e5fa69c99ac8765783f5ce8a3a065b08f5b08a7 before the integration of the feature is done in a separate PR https://github.com/facebook/rocksdb/commit/f7e4009de1d16421a254dd7e799dd91c522d832c. Such low-level test makes it difficult to assert data correctness, is hard to understand by being tied to implementation details.

Therefore they are now replaced with db-level tests  with data correctness check, which is what we ultimately care out of those details. I also expand the test to cover wide column and TimedPut() which associates a key with write time.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14191

Test Plan:
- Only test change; I also manually traced every test to ensure correct resumption point; also removing
```
if (c_iter->IsCurrentKeyAlreadyScanned()) {
    return false;
  }
```
correctly leads to the expected error with resumption at merge, single delete and deletion at bottom

Reviewed By: jaykorean

Differential Revision: D89492846

Pulled By: hx235

fbshipit-source-id: 6c6ab3cbd643ca1b15d049a062da2c76165ef9db
2026-02-12 22:30:55 -08:00
Hui Xiao c3184220b8 Fix an internal comment about resumable compaction (#14215)
Summary:
**Context/Summary:** as titled

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14215

Test Plan: no code change

Reviewed By: jaykorean

Differential Revision: D90037003

Pulled By: hx235

fbshipit-source-id: 8621a8dedef474b02bb16531e0de4ea399659d21
2026-02-12 21:29:10 -08:00
Peter Dillinger d8b1893c9d DROP support for block-based SST format_version < 2 (#14315)
Summary:
... and remove some old code and tech debt in the process.

This is arguably a great milestone and precendent in RocksDB history as for the first time we are explicitly dropping support for the ability to read source-of-truth data in old formats. (We previously dropped support for reading some old bloom filters, but those are performance optimizers not source-of-truth. https://github.com/facebook/rocksdb/issues/10184) However, DBs written with default settings since release 4.6.0, which is very nearly 10 years ago, can still be read. And by using compaction with intermediate versions, there's an upgrade path going back to (AFAIK) early releases of LevelDB (from which RocksDB was forked).

Some detail:
* The magic number for LevelDB SST files (0xdb4775248b80fb57, most recently called kLegacyBlockBasedTableMagicNumber) now only exists in the code to provide a good error message and to test that good error message.
* There is some notable refactoring and renaming around format_version handling. This is a bit of a messy area of code because the footer code being shared between different table formats (block-based, plain, cuckoo) means format_version in the footer is in ways tied to all of them, but in other ways is just tied to block-based table where we have been making updates. Hopefully code comments keep this clear.
* Now that there are old format_versions we can't read (and can't write authoritatively in tests), I've needed to split out kMinSupportedFormatVersion into a constant for reads and for writes, currently the same at format_version=2. Comments describe how to update these in the future.
* The idea of versioning the compression format is basically going away, though we're keeping BuiltinV2 in places just because it's already there. There's lots of room in the BuiltinV2 schema to expand to new built-in compression types, or new ways of handling existing compression algorithms. CompressionManager with CompatibilityName gives users the power to customize compression without the need for versions tied to format_version.

Immediate follow-up:
* Clean up compression loose ends like OLD_Compress, OLD_Uncompress

Suggested follow-up:
* Update plain table builder to migrate to new footer version so that we can drop support for legacy footer. We have to be careful that the (likely untested) forward compatibility path I put in place a while back works (or fix it and wait a while) before dropping support for plain table with legacy footer.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14315

Test Plan:
* Some tests updated / added
* A couple tests are obsolete: removed
* Also updated format compatible test, which now doesn't need to dig as far back into history building RocksDB.

Reviewed By: hx235

Differential Revision: D92577766

Pulled By: pdillinger

fbshipit-source-id: a23be846189d901ce087af4ca9a99cef18445cb7
2026-02-11 14:43:41 -08:00
Richard Barnes 3148c6cad4 Fix string-conversion issue in internal_repo_rocksdb/repo/table/block_based/index_builder.cc +1
Summary:
This could is triggering `-Wstring-conversion`, which presents as:
```
warning: implicit conversion turns string literal into bool: A to B
```
This is often a bug and what was intended. The most frequent cause is the code was:
```
void foo(bool) { ... }
void foo(std::string) { ... }
foo("this gets interpreted as a bool");
```

It is also possible the issue is innocuous as part of an assert:
```
assert(false && "this string is true, so the assertion is false");
EXPECT_FALSE("this string is true, so the expect fails");
```
in these cases the use is to "cute", so we modify the code to make it more obvious.
```
assert(false && "the compiler recognizes and doesn't complain about this pattern");
FAIL() << "much more obvious";
```

Differential Revision: D92886041

fbshipit-source-id: 6adfaa102f12e293491cc579ec92b48834d1d0a8
2026-02-11 14:09:12 -08:00
Anand Ananthabhotla 56cb88ec79 Fix racy assertion in AbortIOPartialHandlesBug test (#14319)
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14319

The test asserted h1->is_finished and h2->is_finished immediately after AbortIO({H0}), before calling Poll. This is invalid because AbortIO only guarantees that handles in its abort set are finalized. Non-aborted handles' CQEs may or may not be consumed during AbortIO depending on io_uring completion ordering. If H0's two CQEs (original read + cancel) arrive before H1/H2's CQEs, AbortIO breaks out of its wait loop without processing them. Move the H1/H2 is_finished assertions to after Poll, which correctly handles either case. Also remove the racy req_count checks for non-aborted handles since Poll does not increment req_count.

Reviewed By: jaykorean

Differential Revision: D92848827

fbshipit-source-id: 0c09b44ceada99877e8311cff799fa94f1056545
2026-02-10 15:05:12 -08:00
Richard Barnes 51feb25567 Fix string-conversion issue in internal_repo_rocksdb/repo/utilities/persistent_cache/block_cache_tier_file.cc +2 (#14312)
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14312

This could is triggering `-Wstring-conversion`, which presents as:
```
warning: implicit conversion turns string literal into bool: A to B
```
This is often a bug and what was intended. The most frequent cause is the code was:
```
void foo(bool) { ... }
void foo(std::string) { ... }
foo("this gets interpreted as a bool");
```

It is also possible the issue is innocuous as part of an assert:
```
assert(!"this string is true, so the assertion is false");
EXPECT_FALSE("this string is true, so the expect fails");
```
in these cases the use is to "cute", so we modify the code to make it more obvious.
```
assert(false && "the compiler recognizes and doesn't complain about this pattern");
FAIL() << "much more obvious";
```

Reviewed By: dmm-fb

Differential Revision: D92528316

fbshipit-source-id: 93fbb624e8731c4cdb559746b44c1aa71d786304
2026-02-09 09:21:12 -08:00
Josh Kang 6284a79847 Upgrade clang format in CI to 21.1.2 (#14311)
Summary:
To make CI consistent with internal meta clang version.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14311

Test Plan:
CI shows correct version
```
Successfully installed clang-format-21.1.2
clang-format version 21.1.2
```

Reviewed By: xingbowang

Differential Revision: D92535441

Pulled By: joshkang97

fbshipit-source-id: ea21ea97b13a35b286f0c2ce18b3f01ffbf49afd
2026-02-06 13:41:32 -08:00
Ryan Hancock 8f9cb1a708 Introduce Memory restrictions for IO Dispatcher. (#14300)
Summary:
Introduction of memory limiter for IO Dispatch.

Currently, the user has no way of enacting policy with IO dispatcher. One important policy is the ability to restrict the amount of memory a multiscan or set of multiscans is allowed to pin. This PR introduces the max_prefetch_memory_bytes in the IODispatcherOptions, allowing for users to specify bounds on block cache memory usage.

There seems to be a minor performance increase however, I have found the scans to be a bit noisy. Each benchmark is run with a stride size of 30000 keys. This was done to ensure we maintain parity with trunk.
```
Configuration: 10 concurrent scans, 1024B values, 5242880 byte SST files
Scan sizes: 1024 keys = 1MiB, 2048 keys = 2MiB, 4096 keys = 4MiB per scan

| Keys/Scan | Mode  | Main (ops/sec)   | Main (us/op)     | limiter              (ops/sec) | limiter            (us/op) | Delta ops/sec |
|-----------|-------|------------------|------------------|----------------------|--------------------|---------------|
| 1024      | sync  |   151.6 +/- 8.0   | 6591.14 +/- 343.30 |    170.6 +/- 4.0      | 5855.32 +/- 136.19   | +12.00%       |
| 1024      | async |   156.4 +/- 24.7  | 6589.64 +/- 1345.73 |    173.8 +/- 2.7      | 5744.51 +/- 91.35   | +11.00%       |
| 2048      | sync  |    77.8 +/- 1.6   | 12785.64 +/- 286.49 |     87.6 +/- 3.4      | 11354.01 +/- 441.71   | +12.00%       |
| 2048      | async |    85.6 +/- 4.7   | 11658.11 +/- 618.49 |     91.4 +/- 1.2      | 10873.63 +/- 143.49   | +6.00%        |
| 4096      | sync  |    43.2 +/- 1.5   | 22932.27 +/- 730.66 |     43.8 +/- 0.7      | 22563.90 +/- 320.93   | +1.00%        |
| 4096      | async |    45.4 +/- 0.8   | 21875.64 +/- 357.04 |     46.2 +/- 0.7      | 21416.95 +/- 311.89   | +1.00%        |

```

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14300

Reviewed By: anand1976

Differential Revision: D92316556

Pulled By: krhancoc

fbshipit-source-id: dc0b7958a33b8ef5fa5af82b1c6d960041837fc1
2026-02-06 11:29:30 -08:00
Anand Ananthabhotla 3695cb6767 Fix AbortIO consuming completions for non-aborted handles (#14301)
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14301

When AbortIO was called with a subset of outstanding async read handles,
it would consume io_uring completions for handles NOT in the abort set
but fail to finalize them. This caused subsequent Poll calls on those
handles to hang forever waiting for completions that had already been
consumed.

The fix adds an `is_being_aborted` flag to Posix_IOHandle that is set
when submitting the cancel request. When processing completions in
AbortIO, handles with this flag wait for req_count==2 (original + cancel),
while handles without the flag are finalized immediately at req_count==1.

Also refactored the completion finalization logic into a shared
FinalizeAsyncRead() helper function used by both Poll and AbortIO.

Reviewed By: mszeszko-meta, archang19

Differential Revision: D92230883

fbshipit-source-id: e6d11e009a4930e5608459771990f6cf7d46d827
2026-02-06 10:51:21 -08:00
anand76 a668dcbe8c Add Txn db support to ldb (#14304)
Summary:
This change adds the ability to open and operate on databases as TransactionDB in the ldb command-line tool.

  New Command-Line Options

  - --use_txn - Opens the database as a TransactionDB instead of a regular DB
  - --txn_write_policy=<0|1|2> - Sets the transaction write policy:
    - 0 = WRITE_COMMITTED (default)
    - 1 = WRITE_PREPARED
    - 2 = WRITE_UNPREPARED

  Use Case
                                                                                                                                                                                                                      This is needed to inspect or modify databases that were created with WritePrepared or WriteUnprepared transactions, which require opening via TransactionDB::Open() rather than the regular DB::Open().

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14304

Test Plan:
Tests (tools/ldb_test.py): Adds testTxnPutGet() covering:
    - Basic put/get/delete with TransactionDB
    - All three write policies
    - Validation that --use_txn and --ttl are mutually exclusive

Reviewed By: pdillinger

Differential Revision: D92323195

Pulled By: anand1976

fbshipit-source-id: 0a62b8ea4e2985feed977fad72595d6fff75db09
2026-02-06 10:47:19 -08:00
Andrew Chang 6ac0da313e Fix crash in GetLiveFilesStorageInfo on read-only DB (#14306)
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14306

GetLiveFilesStorageInfo crashes when called on a read-only RocksDB because
it calls FlushWAL(), which accesses logs_.back() on an empty deque.

Root cause: DBImplReadOnly overrides SyncWAL() to return NotSupported, but
does NOT override FlushWAL(). Read-only DBs have an empty logs_ deque
because they don't create WAL writers during recovery - there's nothing to
write, so no WAL infrastructure is initialized.

The reason SyncWAL was originally marked NotSupported is that these WAL
operations (SyncWAL syncs buffer to disk, FlushWAL flushes to OS buffer)
require an active WAL writer at logs_.back().writer. Since read-only DBs:
1. Cannot perform writes
2. Don't create WAL files for writing
3. Have an empty logs_ deque

...there's no WAL writer to sync or flush. The operations are semantically
meaningless, not just "forbidden write operations."

The fix adds a FlushWAL override matching the SyncWAL pattern. The caller
in db_filesnapshot.cc:403-405 already handles IsNotSupported() gracefully:
  if (s.IsNotSupported()) { s = Status::OK(); }

Reviewed By: pdillinger

Differential Revision: D92419557

fbshipit-source-id: 7079071209b3c7be41a2c98c9b691e68bc031595
2026-02-05 15:22:52 -08:00
Richard Barnes 47344a0feb Fix string-conversion issue in internal_repo_rocksdb/repo/utilities/persistent_cache/volatile_tier_impl.cc +5 (#14296)
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14296

This could is triggering `-Wstring-conversion`, which presents as:
```
warning: implicit conversion turns string literal into bool: A to B
```
This is often a bug and what was intended. The most frequent cause is the code was:
```
void foo(bool) { ... }
void foo(std::string) { ... }
foo("this gets interpreted as a bool");
```

It is also possible the issue is innocuous as part of an assert:
```
assert(!"this string is true, so the assertion is false");
EXPECT_FALSE("this string is true, so the expect fails");
```
in these cases the use is to "cute", so we modify the code to make it more obvious.
```
assert(false && "the compiler recognizes and doesn't complain about this pattern");
FAIL() << "much more obvious";
```

Reviewed By: anand1976

Differential Revision: D92013593

fbshipit-source-id: 0b4e00339bef3f76fc5b9ad35e2383c5e4f828f9
2026-02-05 11:22:28 -08:00
Peter Dillinger 48ec45d7bb Remove useless option CompressedSecondaryCacheOptions::compress_format_version (#14302)
Summary:
I don't think this option was ever useful. There was no compressed secondary cache compatibility issue that needed to accommodate compression format version 1. It was needlessly imported from legacy SST file formats. Version 1 is simply an inefficient format because it requires guessing the uncompressed size on decompression.

And as far as I know, we don't have any plans to make compressed secondary cache entries persistable across RocksDB versions. I.e. if persisting, we would simply tag the persistence layer with the version (perhaps major and minor) and throw out the cache whenever that changes. Then we don't have to deal with explicit schema versioning in persistenct caches. This is a workable approach because unlike SSTs, caches are not source-of-truth that need to survive version rollback.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14302

Test Plan: existing tests

Reviewed By: anand1976

Differential Revision: D92315003

Pulled By: pdillinger

fbshipit-source-id: 0b82cfdbd92bcd2b8fbddd6586824f53c88069c4
2026-02-04 15:11:09 -08:00
Facebook GitHub Bot c2fab4629b Re-sync with internal repository
The internal and external repositories are out of sync. This Pull Request attempts to brings them back in sync by patching the GitHub repository. Please carefully review this patch. You must disable ShipIt for your project in order to merge this pull request. DO NOT IMPORT this pull request. Instead, merge it directly on GitHub using the MERGE BUTTON. Re-enable ShipIt after merging.

fbshipit-source-id: 08b287a3f343f6ac5872c2a059d91d1bed9ff0a8
2026-02-03 14:32:21 -08:00
Anand Ananthabhotla 82ff0678d4 Add a cleanup target to crash_test.mk (#14286)
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14286

Add the db_c leanup target which can be used by CI test scripts to delete the db on failure. The db_crashtest.py doesn't automatically delete on error.

Reviewed By: jaykorean

Differential Revision: D91912877

fbshipit-source-id: d36ec0896fba64faaafe055d8673e437e85d0c3a
2026-02-03 12:22:09 -08:00
557 changed files with 70980 additions and 8514 deletions
+86
View File
@@ -0,0 +1,86 @@
# When making changes, verify the output of:
# clang-tidy -list-checks
---
Checks: "-*,\
bugprone-argument-comment,\
bugprone-dangling-handle,\
bugprone-fold-init-type,\
bugprone-forward-declaration-namespace,\
bugprone-forwarding-reference-overload,\
bugprone-shadow,\
bugprone-sizeof-*,\
bugprone-string-constructor,\
bugprone-undefined-memory-manipulation,\
bugprone-unused-return-value,\
bugprone-use-after-move,\
cert-env33-c,\
cert-err58-cpp,\
cert-msc30-c,\
cert-msc50-cpp,\
clang-analyzer-*,\
clang-diagnostic-*,\
-clang-diagnostic-missing-designated-field-initializers,\
concurrency-mt-unsafe,\
cppcoreguidelines-avoid-non-const-global-variables,\
cppcoreguidelines-missing-std-forward,\
cppcoreguidelines-pro-type-member-init,\
cppcoreguidelines-special-member-functions,\
cppcoreguidelines-virtual-class-destructor,\
google-build-using-namespace,\
google-explicit-constructor,\
google-readability-avoid-underscore-in-googletest-name,\
misc-definitions-in-headers,\
misc-redundant-expression,\
modernize-make-shared,\
modernize-use-emplace,\
modernize-use-noexcept,\
modernize-use-override,\
modernize-use-using,\
performance-faster-string-find,\
performance-for-range-copy,\
performance-implicit-conversion-in-loop,\
performance-inefficient-algorithm,\
performance-inefficient-string-concatenation,\
performance-inefficient-vector-operation,\
performance-move-const-arg,\
performance-move-constructor-init,\
performance-no-automatic-move,\
performance-no-int-to-ptr,\
performance-noexcept-move-constructor,\
performance-noexcept-swap,\
performance-trivially-destructible,\
performance-type-promotion-in-math-fn,\
performance-unnecessary-copy-initialization,\
performance-unnecessary-value-param,\
readability-braces-around-statements,\
readability-duplicate-include,\
readability-isolate-declaration,\
readability-operators-representation,\
readability-redundant-string-init"
WarningsAsErrors: "bugprone-use-after-move"
CheckOptions:
- key: bugprone-easily-swappable-parameters.MinimumLength
value: 4
- key: cppcoreguidelines-avoid-non-const-global-variables.AllowThreadLocal
value: true
- key: cppcoreguidelines-special-member-functions.AllowSoleDefaultDtor
value: true
- key: cppcoreguidelines-special-member-functions.AllowImplicitlyDeletedCopyOrMove
value: true
- key: modernize-use-using.IgnoreExternC
value: true
- key: performance-move-const-arg.CheckTriviallyCopyableMove
value: false
- key: performance-unnecessary-value-param.AllowedTypes
value: '[Pp]ointer$;[Pp]tr$;[Rr]ef(erence)?$'
- key: performance-unnecessary-copy-initialization.AllowedTypes
value: '[Pp]ointer$;[Pp]tr$;[Rr]ef(erence)?$'
- key: readability-operators-representation.BinaryOperators
value: '&&;&=;&;|;~;!;!=;||;|=;^;^='
- key: readability-redundant-string-init.StringNames
value: '::std::basic_string'
- key: readability-named-parameter.InsertPlainNamesInForwardDecls
value: true
...
+10 -1
View File
@@ -9,7 +9,16 @@ runs:
steps:
- name: Build folly and dependencies
if: ${{ inputs.cache-hit != 'true' }}
run: make build_folly
run: |
clean_path=()
IFS=: read -ra path_entries <<< "$PATH"
for entry in "${path_entries[@]}"; do
if [[ "$entry" != "/usr/lib/ccache" ]]; then
clean_path+=("$entry")
fi
done
export PATH="$(IFS=:; echo "${clean_path[*]}")"
make build_folly
shell: bash
- name: Skip folly build (using cached version)
if: ${{ inputs.cache-hit == 'true' }}
+1 -1
View File
@@ -30,4 +30,4 @@ runs:
# - The docker image, which may not always be specified/known
# - Hash of folly.mk, which includes the folly repository commit hash
# NOTE: this is still only intended for DEBUG folly builds
key: folly-build-${{ runner.os }}-${{ runner.arch }}-${{ github.job_container.image }}-${{ steps.extract-folly-hash.outputs.hash }}
key: folly-build-${{ runner.os }}-${{ runner.arch }}-${{ github.job_container.image }}-${{ steps.extract-folly-hash.outputs.hash }}-${{ env.PORTABLE == '1' && 'portable' || 'native' }}
+3
View File
@@ -2,6 +2,9 @@ name: pre-steps
runs:
using: composite
steps:
- name: Install lld linker for faster builds
run: apt-get update -y && apt-get install -y lld 2>/dev/null || true
shell: bash
- name: Setup Environment Variables
run: |-
echo "GTEST_THROW_ON_FAILURE=0" >> "$GITHUB_ENV"
+54
View File
@@ -0,0 +1,54 @@
name: setup-ccache
description: Setup ccache for faster C++ compilation caching
inputs:
cache-key-prefix:
description: Unique prefix for the cache key (e.g., 'build-linux')
required: true
portable:
description: Set PORTABLE=1 to disable -march=native (set to "false" for jobs linking pre-built Folly)
required: false
default: "true"
runs:
using: composite
steps:
- name: Set ccache environment variables
run: |
echo "CCACHE_DIR=${{ github.workspace }}/.ccache" >> $GITHUB_ENV
echo "CCACHE_BASEDIR=${{ github.workspace }}" >> $GITHUB_ENV
echo "CCACHE_NOHASHDIR=true" >> $GITHUB_ENV
echo "CCACHE_COMPILERCHECK=content" >> $GITHUB_ENV
echo "CCACHE_SLOPPINESS=clang_index_store,file_stat_matches,include_file_ctime,include_file_mtime,ivfsoverlay,pch_defines,modules,system_headers,time_macros" >> $GITHUB_ENV
echo "CCACHE_MAXSIZE=4G" >> $GITHUB_ENV
if [ "${{ inputs.portable }}" = "true" ]; then
echo "PORTABLE=1" >> $GITHUB_ENV
fi
shell: bash
- name: Restore ccache
uses: actions/cache@v4
with:
path: ${{ github.workspace }}/.ccache
key: ccache-${{ inputs.cache-key-prefix }}-${{ github.head_ref || github.ref }}-${{ github.sha }}
restore-keys: |-
ccache-${{ inputs.cache-key-prefix }}-${{ github.head_ref || github.ref }}-
ccache-${{ inputs.cache-key-prefix }}-refs/heads/main-
- name: Install ccache
run: |
if [ "$RUNNER_OS" = "macOS" ]; then
which ccache || brew install ccache
else
which ccache || (apt-get update && apt-get install -y ccache)
fi
shell: bash
- name: Add ccache to PATH
run: |
if [ "$RUNNER_OS" = "macOS" ]; then
echo "$(brew --prefix ccache)/libexec" >> $GITHUB_PATH
else
echo "/usr/lib/ccache" >> $GITHUB_PATH
fi
shell: bash
- name: Zero ccache stats and set build marker
run: |
ccache -z
touch "$CCACHE_DIR/.build_marker"
shell: bash
@@ -0,0 +1,15 @@
name: teardown-ccache
description: Trim stale ccache entries and print stats
runs:
using: composite
steps:
- name: Trim and print ccache stats
run: |
if [ -z "$CCACHE_DIR" ]; then
echo "teardown-ccache: CCACHE_DIR not set, skipping (setup-ccache may not have run)"
exit 0
fi
.github/scripts/ccache-trim.sh || true
ccache -s || echo "teardown-ccache: ccache not found, skipping stats"
if: always()
shell: bash
+34 -12
View File
@@ -1,19 +1,31 @@
name: windows-build-steps
inputs:
suite-run:
description: Comma-separated list of test suites to run (empty to skip C++ tests)
required: false
default: arena_test,db_basic_test,db_test,db_test2,db_merge_operand_test,bloom_test,c_test,coding_test,crc32c_test,dynamic_bloom_test,env_basic_test,env_test,hash_test,random_test
run-java:
description: Whether to run Java tests
required: false
default: "true"
runs:
using: composite
steps:
- name: Add msbuild to PATH
uses: microsoft/setup-msbuild@v1.3.1
- name: Cache ccache directory
id: ccache-cache
uses: actions/cache@v4
with:
path: C:\a\rocksdb\rocksdb\.ccache
key: rocksdb-build-${{ runner.os }}-${{ runner.arch }}-ccache-${{ hashFiles('CMakeLists.txt', 'cmake/**/*.cmake') }}-v1
- name: ccache
uses: hendrikmuhs/ccache-action@v1.2
with:
max-size: "10GB"
key: ccache-windows-${{ github.workflow }}
restore-keys: |
ccache-windows-
- name: Configure ccache
shell: pwsh
run: |
ccache --set-config=base_dir=C:\a\rocksdb\rocksdb
ccache --set-config=hash_dir=false
ccache --set-config=compiler_check=content
- name: Custom steps
env:
THIRDPARTY_HOME: ${{ github.workspace }}/thirdparty
@@ -56,12 +68,22 @@ runs:
msbuild build/rocksdb.sln /m:32 /p:LinkIncremental=false -property:Configuration=Debug -property:Platform=x64
if(!$?) { Exit $LASTEXITCODE }
echo ========================= Test RocksDB =========================
build_tools\run_ci_db_test.ps1 -SuiteRun arena_test,db_basic_test,db_test,db_test2,db_merge_operand_test,bloom_test,c_test,coding_test,crc32c_test,dynamic_bloom_test,env_basic_test,env_test,hash_test,random_test -Concurrency 16
if(!$?) { Exit $LASTEXITCODE }
echo ======================== Test RocksJava ========================
cd build\java
& ctest -C Debug -j 16
if(!$?) { Exit $LASTEXITCODE }
$suiteRun = "${{ inputs.suite-run }}"
if ($suiteRun -ne "") {
$suiteArray = $suiteRun -split ','
build_tools\run_ci_db_test.ps1 -SuiteRun $suiteArray -Concurrency 16
if(!$?) { Exit $LASTEXITCODE }
} else {
echo "Skipping C++ tests (suite-run is empty)"
}
if ("${{ inputs.run-java }}" -eq "true") {
echo ======================== Test RocksJava ========================
cd build\java
& ctest -C Debug -j 16
if(!$?) { Exit $LASTEXITCODE }
} else {
echo "Skipping Java tests"
}
shell: pwsh
- name: Show ccache stats
shell: pwsh
+39
View File
@@ -0,0 +1,39 @@
#!/bin/bash
# Trim ccache to keep only entries accessed during the current build.
#
# Usage:
# 1. Before build: touch "$CCACHE_DIR/.build_marker"
# 2. Run build (ccache updates mtime on hits, creates new files for misses)
# 3. After build: .github/scripts/ccache-trim.sh
#
# This removes cache files not accessed during the build (stale entries from
# previous commits). Only intended for CI where each run builds one commit.
# Do NOT use on local builds where multiple worktrees may share the cache.
set -e
CCACHE_DIR="${CCACHE_DIR:?CCACHE_DIR must be set}"
MARKER="$CCACHE_DIR/.build_marker"
if [ ! -f "$MARKER" ]; then
echo "ccache-trim: No .build_marker found, skipping (was the marker created before build?)"
exit 0
fi
# Count files before cleanup
before=$(find "$CCACHE_DIR" -type f \( -name '*R' -o -name '*M' \) | wc -l)
# Delete cache files (results and manifests) older than the marker
find "$CCACHE_DIR" -type f \( -name '*R' -o -name '*M' \) ! -newer "$MARKER" -delete
# Clean up empty directories
find "$CCACHE_DIR" -mindepth 2 -type d -empty -delete 2>/dev/null || true
# Recalculate size counters
ccache -c 2>/dev/null || true
after=$(find "$CCACHE_DIR" -type f \( -name '*R' -o -name '*M' \) | wc -l)
echo "ccache-trim: $before -> $after cache files (removed $((before - after)) stale entries)"
# Clean up marker
rm -f "$MARKER"
+32
View File
@@ -0,0 +1,32 @@
#!/bin/bash
# Compute test shard for parallel CI execution.
# Distributes tests round-robin across N shards for balanced load.
# Outputs ROCKSDBTESTS_SUBSET — the list of test binaries for this shard.
# The Makefile uses this to build and run only the assigned tests.
#
# Usage: compute-test-shard.sh <shard_index> <num_shards>
set -euo pipefail
shard=${1:?Usage: compute-test-shard.sh <shard_index> <num_shards>}
nshards=${2:?Usage: compute-test-shard.sh <shard_index> <num_shards>}
# Get sorted test list (db_test first since it's the heaviest, then alpha)
make -s list_all_tests 2>/dev/null | tr ' ' '\n' | grep '_test$' | sort -u > /tmp/sorted.txt
(echo db_test; grep -v '^db_test$' /tmp/sorted.txt) > /tmp/all_tests.txt
total=$(wc -l < /tmp/all_tests.txt)
# Round-robin: assign test i to shard (i % nshards).
# This spreads heavy tests (which are scattered alphabetically) evenly.
awk -v s="$shard" -v n="$nshards" 'NR > 0 && (NR - 1) % n == s' /tmp/all_tests.txt > /tmp/include.txt
included=$(wc -l < /tmp/include.txt)
first=$(head -1 /tmp/include.txt)
last=$(tail -1 /tmp/include.txt)
# Output space-separated list for ROCKSDBTESTS_SUBSET
subset=$(tr '\n' ' ' < /tmp/include.txt)
echo "subset=${subset}" >> "$GITHUB_OUTPUT"
echo "Shard $shard/$nshards: $included tests (round-robin), first=$first last=$last (total $total)"
+115
View File
@@ -0,0 +1,115 @@
// Parse Claude Code execution log and produce a markdown review comment.
//
// Usage from actions/github-script:
// const parse = require('./.github/scripts/parse-claude-review.js');
// const markdown = parse({ executionFile, conclusion, meta });
//
// Parameters:
// executionFile - path to the JSON execution log from claude-code-base-action
// conclusion - 'success' or 'failure' from the action output
// meta - { trigger, headSha, reviewer, isQuery, isPartial }
const fs = require('fs');
module.exports = function parseClaude({executionFile, conclusion, meta}) {
let responseBody = '';
try {
const executionLog = JSON.parse(fs.readFileSync(executionFile, 'utf8'));
if (!Array.isArray(executionLog)) {
throw new Error('Expected array format from claude-code-base-action');
}
const resultMessage = executionLog.find(m => m.type === 'result');
// Helper: extract the last substantial assistant text from the log.
// Used as a fallback when Claude ran out of turns and the recovery
// session also failed to produce a result.
function getLastAssistantText(log, minLength = 200) {
for (let i = log.length - 1; i >= 0; i--) {
const m = log[i];
if (m.type !== 'assistant') continue;
const content = m.message && m.message.content;
if (!Array.isArray(content)) continue;
for (let j = content.length - 1; j >= 0; j--) {
if (content[j].type === 'text' && content[j].text &&
content[j].text.trim().length >= minLength) {
const text = content[j].text.trim();
// Truncate to avoid enormous PR comments
return text.length > 50000 ? text.substring(0, 50000) +
'\n\n*[Truncated — full output in execution log artifact]*' :
text;
}
}
}
return null;
}
if (!resultMessage) {
responseBody = '⚠️ No result message found in execution log.';
} else if (resultMessage.subtype === 'success' && resultMessage.result) {
responseBody = resultMessage.result;
} else if (resultMessage.is_error || resultMessage.subtype === 'error') {
const errorInfo =
resultMessage.result || resultMessage.error || 'Unknown error';
responseBody = `❌ **Claude encountered an error:**\n\n${errorInfo}`;
} else if (resultMessage.subtype === 'error_max_turns') {
// The workflow runs a recovery session when this happens, so this
// branch is typically only hit if recovery wasn't attempted (e.g.,
// no findings file was written). Extract what we can.
const partial = getLastAssistantText(executionLog);
if (partial) {
responseBody =
`⚠️ **Review incomplete — Claude hit the turn limit.**\n\nBelow is the last partial output. You can request a fresh review with \`/claude-review\`.\n\n---\n\n${
partial}`;
} else {
responseBody =
'⚠️ **Review incomplete — Claude hit the turn limit before producing output.** You can request a fresh review with `/claude-review`.';
}
} else if (resultMessage.result) {
responseBody = `⚠️ **Completed with status: ${
resultMessage.subtype}**\n\n${resultMessage.result}`;
} else {
responseBody = '⚠️ Claude completed but produced no output.';
}
} catch (error) {
responseBody = `❌ Error parsing Claude response: ${error.message}`;
}
const isPartial = !!meta.isPartial;
const icon = isPartial ? '🟡' : (conclusion === 'success' ? '✅' : '⚠️');
const headerTitle = meta.isQuery ? 'Claude Response' : 'Claude Code Review';
const triggerLine = meta.trigger === 'auto' ?
`*Auto-triggered after CI passed — reviewing commit ${
meta.headSha.substring(0, 7)}*` :
`*Requested by @${meta.reviewer}*`;
return [
`## ${icon} ${headerTitle}`,
'',
triggerLine,
'',
'---',
'',
responseBody,
'',
'---',
'',
'<details>',
'<summary>️ About this response</summary>',
'',
'Generated by [Claude Code](https://github.com/anthropics/claude-code-action).',
'Review methodology: `claude_md/code_review.md`',
'',
'**Limitations:**',
'- Claude may miss context from files not in the diff',
'- Large PRs may be truncated',
'- Always apply human judgment to AI suggestions',
'',
'**Commands:**',
'- `/claude-review [context]` — Request a code review',
'- `/claude-query <question>` — Ask about the PR or codebase',
'</details>',
].join('\n');
};
+57
View File
@@ -0,0 +1,57 @@
// Shared PR comment posting utility.
// Used by both clang-tidy-comment.yml and claude-review-comment.yml.
//
// Usage from actions/github-script:
// const post = require('./.github/scripts/post-pr-comment.js');
// await post({ github, context, core, prNumber, body, marker });
//
// Parameters:
// github - octokit instance from actions/github-script
// context - GitHub Actions context
// core - @actions/core for logging
// prNumber - PR number to comment on
// body - comment body (markdown string)
// marker - HTML comment marker for dedup (e.g. '<!-- claude-review -->')
// If an existing comment with this marker is found, it is updated.
// If not found, a new comment is created.
module.exports = async function postPrComment(
{github, context, core, prNumber, body, marker}) {
if (!prNumber || !body) {
core.warning('Missing prNumber or body; skipping comment.');
return;
}
const owner = context.repo.owner;
const repo = context.repo.repo;
// Ensure marker is embedded in the body
const markedBody = body.includes(marker) ? body : `${marker}\n${body}`;
// Search for existing comment with this marker
const {data: comments} = await github.rest.issues.listComments({
owner,
repo,
issue_number: prNumber,
per_page: 100,
});
const existing = comments.find(c => c.body.includes(marker));
if (existing) {
await github.rest.issues.updateComment({
owner,
repo,
comment_id: existing.id,
body: markedBody,
});
core.info(`Updated existing comment ${existing.id}`);
} else {
await github.rest.issues.createComment({
owner,
repo,
issue_number: prNumber,
body: markedBody,
});
core.info('Created new PR comment');
}
};
+51
View File
@@ -0,0 +1,51 @@
name: Post clang-tidy PR comment
on:
workflow_run:
workflows: ["clang-tidy"]
types: [completed]
permissions:
pull-requests: write
jobs:
comment:
if: github.event.workflow_run.event == 'pull_request'
runs-on: ubuntu-latest
steps:
- name: Checkout scripts
uses: actions/checkout@v4
with:
sparse-checkout: .github/scripts
sparse-checkout-cone-mode: true
- name: Download clang-tidy results
id: download
uses: actions/download-artifact@v4.1.3
with:
name: clang-tidy-result
run-id: ${{ github.event.workflow_run.id }}
github-token: ${{ secrets.GITHUB_TOKEN }}
continue-on-error: true
- name: Post or update PR comment
if: steps.download.outcome == 'success'
uses: actions/github-script@v7
with:
script: |
const fs = require('fs');
if (!fs.existsSync('clang-tidy-comment.md') || !fs.existsSync('pr_number.txt')) {
core.info('No clang-tidy results found; skipping.');
return;
}
const body = fs.readFileSync('clang-tidy-comment.md', 'utf8');
const prNumber = parseInt(fs.readFileSync('pr_number.txt', 'utf8').trim());
const post = require('./.github/scripts/post-pr-comment.js');
await post({
github,
context,
core,
prNumber,
body,
marker: '<!-- clang-tidy-bot -->',
});
+74
View File
@@ -0,0 +1,74 @@
name: clang-tidy
on:
push:
pull_request:
permissions: {}
jobs:
clang-tidy:
if: github.repository_owner == 'facebook'
runs-on:
labels: 4-core-ubuntu
container:
image: ghcr.io/facebook/rocksdb_ubuntu:24.1
steps:
- uses: actions/checkout@v4.1.0
with:
fetch-depth: 2
- name: Mark workspace as safe for git
run: git config --global --add safe.directory $GITHUB_WORKSPACE
- name: Determine diff base
id: diff-base
run: |
if [ "${{ github.event_name }}" = "pull_request" ]; then
BASE="${{ github.event.pull_request.base.sha }}"
git fetch --depth=1 origin "$BASE"
else
BASE="${{ github.event.before }}"
if echo "$BASE" | grep -q '^0\{40\}$'; then
echo "skip=true" >> "$GITHUB_OUTPUT"
echo "New branch push; skipping clang-tidy."
exit 0
fi
fi
echo "ref=$BASE" >> "$GITHUB_OUTPUT"
echo "skip=false" >> "$GITHUB_OUTPUT"
- name: Install clang-tidy
if: steps.diff-base.outputs.skip != 'true'
run: apt-get update && apt-get install -y clang-tidy-21 && ln -sf /usr/bin/clang-tidy-21 /usr/local/bin/clang-tidy
- name: Generate compile_commands.json
if: steps.diff-base.outputs.skip != 'true'
run: |
mkdir build && cd build
cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=ON \
-DCMAKE_C_COMPILER=clang-21 \
-DCMAKE_CXX_COMPILER=clang++-21 ..
cd ..
ln -sf build/compile_commands.json compile_commands.json
- name: Run clang-tidy on changed files
id: clang-tidy
if: steps.diff-base.outputs.skip != 'true'
run: |
python3 tools/run_clang_tidy.py \
-j 4 \
--diff-base ${{ steps.diff-base.outputs.ref }} \
--github-annotations \
--github-step-summary \
--comment-output clang-tidy-comment.md
continue-on-error: true
- name: Save PR number
if: github.event_name == 'pull_request' && always()
run: echo "${{ github.event.pull_request.number }}" > pr_number.txt
- name: Upload clang-tidy results
if: always()
uses: actions/upload-artifact@v4.0.0
with:
name: clang-tidy-result
path: |
clang-tidy-comment.md
pr_number.txt
if-no-files-found: ignore
- name: Fail if clang-tidy found issues
if: steps.clang-tidy.outcome == 'failure'
run: exit 1
+146
View File
@@ -0,0 +1,146 @@
# Claude Code Review — Comment Posting Workflow
#
# Companion to claude-review.yml. Downloads the review artifact and posts
# it as a PR comment. This workflow has write permissions but never runs
# AI or processes untrusted code.
#
# Security: Separating analysis (has ANTHROPIC_API_KEY, no write perms)
# from posting (has GITHUB_TOKEN write perms, no AI) prevents a crafted
# PR from tricking Claude into exfiltrating tokens.
name: Post Claude Review Comment
on:
workflow_run:
workflows: ["Claude Code Review"]
types: [completed]
permissions:
pull-requests: write
issues: write
jobs:
comment:
if: github.event.workflow_run.conclusion == 'success'
runs-on: ubuntu-latest
steps:
- name: Checkout scripts
uses: actions/checkout@v4
with:
sparse-checkout: .github/scripts
sparse-checkout-cone-mode: true
- name: Download review artifact
id: download
uses: actions/download-artifact@v4
with:
name: claude-review-result
run-id: ${{ github.event.workflow_run.id }}
github-token: ${{ secrets.GITHUB_TOKEN }}
continue-on-error: true
- name: Post or update PR comment
if: steps.download.outcome == 'success'
uses: actions/github-script@v7
with:
script: |
const fs = require('fs');
if (!fs.existsSync('claude-review-comment.md') || !fs.existsSync('pr_number.txt')) {
core.info('No review results found; skipping.');
return;
}
const body = fs.readFileSync('claude-review-comment.md', 'utf8');
const prNumber = parseInt(fs.readFileSync('pr_number.txt', 'utf8').trim());
const trigger = fs.existsSync('trigger_type.txt')
? fs.readFileSync('trigger_type.txt', 'utf8').trim()
: 'auto';
// Use different markers for auto vs manual so they don't
// overwrite each other. Auto-reviews update in place (one
// per PR). Manual reviews always create new comments.
const marker = trigger === 'auto'
? '<!-- claude-review-auto -->'
: `<!-- claude-review-manual-${Date.now()} -->`;
const post = require('./.github/scripts/post-pr-comment.js');
await post({
github,
context,
core,
prNumber,
body,
marker,
});
- name: Add reaction to trigger comment
if: steps.download.outcome == 'success'
uses: actions/github-script@v7
with:
script: |
const fs = require('fs');
if (!fs.existsSync('trigger_type.txt')) return;
const trigger = fs.readFileSync('trigger_type.txt', 'utf8').trim();
if (trigger !== 'manual') return;
if (!fs.existsSync('comment_id.txt')) return;
const commentId = parseInt(fs.readFileSync('comment_id.txt', 'utf8').trim());
if (!commentId) return;
await github.rest.reactions.createForIssueComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: commentId,
content: 'rocket'
});
# Notify on failure (for manual triggers)
failure-notice:
if: github.event.workflow_run.conclusion == 'failure'
runs-on: ubuntu-latest
steps:
- name: Download artifact for PR number
id: download
uses: actions/download-artifact@v4
with:
name: claude-review-result
run-id: ${{ github.event.workflow_run.id }}
github-token: ${{ secrets.GITHUB_TOKEN }}
continue-on-error: true
- name: React with failure
if: steps.download.outcome == 'success'
uses: actions/github-script@v7
with:
script: |
const fs = require('fs');
if (!fs.existsSync('comment_id.txt')) return;
const commentId = parseInt(fs.readFileSync('comment_id.txt', 'utf8').trim());
if (!commentId) return;
await github.rest.reactions.createForIssueComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: commentId,
content: 'confused'
});
# Unauthorized user notice
unauthorized-notice:
runs-on: ubuntu-latest
if: >-
github.event.workflow_run.event == 'issue_comment' &&
github.event.workflow_run.conclusion == 'skipped'
steps:
- name: Check if unauthorized
id: check
uses: actions/github-script@v7
with:
script: |
// This job only runs when the analysis workflow was skipped,
// which happens when the user is not in the authorized list.
// We cannot easily get the original comment from workflow_run,
// so this is a best-effort notice via workflow logs.
core.info('Analysis workflow was skipped — likely unauthorized user.');
+592
View File
@@ -0,0 +1,592 @@
# Claude Code Review — Analysis Workflow
#
# This workflow runs Claude to review PRs. It produces a markdown review
# as an artifact but does NOT post comments (no write permissions).
# The companion workflow (claude-review-comment.yml) handles posting.
#
# Triggers:
# 1. workflow_run: Auto-review after pr-jobs passes
# 2. issue_comment: Manual /claude-review or /claude-query by maintainers
# 3. workflow_dispatch: Manual testing
#
# Security:
# - This job has contents:read ONLY (no PR write, no issue write)
# - ANTHROPIC_API_KEY is used here but never coexists with write tokens
# - Claude uses read-only tools (View, GlobTool, GrepTool) plus Write
# (only for incremental findings file, not repo modifications)
# - Review methodology: claude_md/code_review.md
name: Claude Code Review
on:
workflow_run:
workflows: ["facebook/rocksdb/pr-jobs"]
types: [completed]
issue_comment:
types: [created]
workflow_dispatch:
inputs:
pr_number:
description: PR number to review
required: true
type: number
model:
description: Claude model to use
required: false
type: choice
options:
- claude-opus-4-6
- claude-sonnet-4-20250514
default: claude-opus-4-6
permissions:
contents: read
jobs:
# ======================== Auto Review ======================== #
auto-review:
name: Auto Claude Review
runs-on: ubuntu-latest
permissions:
contents: read
actions: read
if: >-
github.event_name == 'workflow_run' &&
github.event.workflow_run.conclusion == 'success' &&
github.event.workflow_run.event == 'pull_request'
steps:
- name: Get PR info
id: pr_info
uses: actions/github-script@v7
with:
script: |
const headSha = context.payload.workflow_run.head_sha;
let prNumber = null;
const prs = context.payload.workflow_run.pull_requests;
if (prs && prs.length > 0) {
prNumber = prs[0].number;
} else {
// PR may not be registered yet if push and PR creation raced.
// Retry up to 5 times with 10s delay.
for (let attempt = 0; attempt < 5; attempt++) {
if (attempt > 0) {
core.info(`PR not found for SHA ${headSha}, retrying in 10s (attempt ${attempt + 1}/5)...`);
await new Promise(r => setTimeout(r, 10000));
}
const { data: pulls } = await github.rest.pulls.list({
owner: context.repo.owner,
repo: context.repo.repo,
state: 'open',
sort: 'updated',
direction: 'desc',
per_page: 100
});
const match = pulls.find(p => p.head.sha === headSha);
if (match) {
prNumber = match.number;
break;
}
}
}
if (!prNumber) {
core.warning(
`Could not find PR for SHA ${headSha} after retries. ` +
`This can happen when the PR head advanced before this ` +
`workflow_run-triggered job started. Skipping stale review run.`
);
core.setOutput('skip', 'true');
core.setOutput('skip_reason', `stale_sha:${headSha}`);
return;
}
const pr = await github.rest.pulls.get({
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: prNumber
});
core.setOutput('skip', 'false');
core.setOutput('pr_number', prNumber.toString());
core.setOutput('head_repo', pr.data.head.repo.clone_url);
core.setOutput('head_ref', pr.data.head.ref);
core.setOutput('head_sha', pr.data.head.sha);
core.setOutput('base_sha', pr.data.base.sha);
core.setOutput('base_ref', pr.data.base.ref);
core.setOutput('pr_title', pr.data.title);
core.setOutput('pr_body', pr.data.body || '');
core.setOutput('pr_author', pr.data.user.login);
- name: Skip stale run
if: steps.pr_info.outputs.skip == 'true'
run: |-
echo "Skipping stale auto-review run: ${{ steps.pr_info.outputs.skip_reason }}"
- name: Check for existing review
if: steps.pr_info.outputs.skip != 'true'
id: check_existing
uses: actions/github-script@v7
env:
PR_NUMBER: ${{ steps.pr_info.outputs.pr_number }}
HEAD_SHA: ${{ steps.pr_info.outputs.head_sha }}
with:
script: |
const comments = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: parseInt(process.env.PR_NUMBER),
per_page: 100
});
const shortSha = process.env.HEAD_SHA.substring(0, 7);
const existing = comments.data.find(c =>
c.body.includes('<!-- claude-review -->') &&
c.body.includes(shortSha)
);
core.setOutput('skip', existing ? 'true' : 'false');
- name: Checkout base repository
if: steps.pr_info.outputs.skip != 'true' && steps.check_existing.outputs.skip != 'true'
uses: actions/checkout@v4
with:
ref: ${{ steps.pr_info.outputs.base_ref }}
fetch-depth: 0
persist-credentials: false
- name: Generate diff and prompt
if: steps.pr_info.outputs.skip != 'true' && steps.check_existing.outputs.skip != 'true'
env:
HEAD_REPO: ${{ steps.pr_info.outputs.head_repo }}
HEAD_REF: ${{ steps.pr_info.outputs.head_ref }}
HEAD_SHA: ${{ steps.pr_info.outputs.head_sha }}
BASE_SHA: ${{ steps.pr_info.outputs.base_sha }}
PR_TITLE: ${{ steps.pr_info.outputs.pr_title }}
PR_BODY: ${{ steps.pr_info.outputs.pr_body }}
PR_AUTHOR: ${{ steps.pr_info.outputs.pr_author }}
run: |
git remote add fork "${HEAD_REPO}" || true
git fetch fork "${HEAD_REF}" --depth=100
git diff "${BASE_SHA}...${HEAD_SHA}" > /tmp/pr.diff
DIFF_STATS=$(git diff --stat "${BASE_SHA}...${HEAD_SHA}" | tail -1)
DIFF_SIZE=$(wc -c < /tmp/pr.diff)
IS_TRUNCATED=false
if [ "$DIFF_SIZE" -gt 100000 ]; then
head -c 100000 /tmp/pr.diff > /tmp/pr_truncated.diff
mv /tmp/pr_truncated.diff /tmp/pr.diff
IS_TRUNCATED=true
fi
cat claude_md/ci_review_prompt.md > /tmp/prompt.txt
cat >> /tmp/prompt.txt << PROMPT_EOF
## Pull Request Information
- **Title:** ${PR_TITLE}
- **Author:** ${PR_AUTHOR}
- **Changes:** ${DIFF_STATS}
PROMPT_EOF
if [ "${IS_TRUNCATED}" = "true" ]; then
echo "- **Note:** Diff truncated due to size." >> /tmp/prompt.txt
fi
printf '\n## PR Description\n' >> /tmp/prompt.txt
printf '%s\n' "${PR_BODY}" >> /tmp/prompt.txt
printf '\n---\n\n## Diff to Review\n\n' >> /tmp/prompt.txt
cat /tmp/pr.diff >> /tmp/prompt.txt
- name: Run Claude
if: steps.pr_info.outputs.skip != 'true' && steps.check_existing.outputs.skip != 'true'
id: claude
uses: anthropics/claude-code-base-action@beta
with:
prompt_file: /tmp/prompt.txt
model: claude-opus-4-6
max_turns: "300"
timeout_minutes: "60"
allowed_tools: "View,GlobTool,GrepTool,Write,Task"
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
- name: Recover partial review on turn limit
if: steps.pr_info.outputs.skip != 'true' && steps.check_existing.outputs.skip != 'true'
id: recover
uses: actions/github-script@v7
env:
EXECUTION_FILE: ${{ steps.claude.outputs.execution_file }}
with:
script: |
const fs = require('fs');
const log = JSON.parse(fs.readFileSync(process.env.EXECUTION_FILE, 'utf8'));
const result = log.find(m => m.type === 'result');
const hitLimit = result && result.subtype === 'error_max_turns';
const hasFindings = fs.existsSync('review-findings.md');
core.setOutput('hit_limit', hitLimit ? 'true' : 'false');
core.setOutput('has_findings', hasFindings ? 'true' : 'false');
core.setOutput('needs_recovery', (hitLimit && hasFindings) ? 'true' : 'false');
- name: Format partial findings
if: >-
steps.pr_info.outputs.skip != 'true' &&
steps.check_existing.outputs.skip != 'true' &&
steps.recover.outputs.needs_recovery == 'true'
id: format_recovery
uses: anthropics/claude-code-base-action@beta
with:
prompt_file: claude_md/ci_recovery_prompt.md
model: claude-sonnet-4-20250514
max_turns: "10"
allowed_tools: "View,GlobTool,GrepTool"
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
- name: Build review comment
if: steps.pr_info.outputs.skip != 'true' && steps.check_existing.outputs.skip != 'true'
uses: actions/github-script@v7
env:
EXECUTION_FILE: ${{ steps.claude.outputs.execution_file }}
RECOVERY_FILE: ${{ steps.format_recovery.outputs.execution_file }}
CONCLUSION: ${{ steps.claude.outputs.conclusion }}
HEAD_SHA: ${{ steps.pr_info.outputs.head_sha }}
NEEDS_RECOVERY: ${{ steps.recover.outputs.needs_recovery }}
with:
script: |
const parse = require('./.github/scripts/parse-claude-review.js');
const fs = require('fs');
let execFile = process.env.EXECUTION_FILE;
if (process.env.NEEDS_RECOVERY === 'true') {
const rf = process.env.RECOVERY_FILE;
if (rf && fs.existsSync(rf)) {
execFile = rf;
}
}
const body = parse({
executionFile: execFile,
conclusion: process.env.CONCLUSION,
meta: {
trigger: 'auto',
headSha: process.env.HEAD_SHA,
reviewer: '',
isQuery: false,
isPartial: process.env.NEEDS_RECOVERY === 'true',
}
});
fs.writeFileSync('claude-review-comment.md', body);
- name: Save PR number
if: steps.pr_info.outputs.skip != 'true' && steps.check_existing.outputs.skip != 'true'
run: echo "${{ steps.pr_info.outputs.pr_number }}" > pr_number.txt
- name: Save trigger metadata
if: steps.pr_info.outputs.skip != 'true' && steps.check_existing.outputs.skip != 'true'
run: echo "auto" > trigger_type.txt
- name: Upload review artifact
if: always() && steps.pr_info.outputs.skip != 'true' && steps.check_existing.outputs.skip != 'true'
uses: actions/upload-artifact@v4
with:
name: claude-review-result
path: |
claude-review-comment.md
pr_number.txt
trigger_type.txt
if-no-files-found: ignore
retention-days: 1
- name: Upload execution log
if: always() && steps.pr_info.outputs.skip != 'true' && steps.check_existing.outputs.skip != 'true'
uses: actions/upload-artifact@v4
with:
name: claude-review-execution-log
path: ${{ steps.claude.outputs.execution_file }}
retention-days: 7
if-no-files-found: warn
- name: Upload recovery execution log
if: always() && steps.pr_info.outputs.skip != 'true' && steps.recover.outputs.needs_recovery == 'true'
uses: actions/upload-artifact@v4
with:
name: claude-review-recovery-log
path: ${{ steps.format_recovery.outputs.execution_file }}
retention-days: 7
if-no-files-found: ignore
# ======================== Manual Review ======================== #
manual-review:
name: Manual Claude Review
runs-on: ubuntu-latest
permissions:
contents: read
if: >-
github.event_name == 'workflow_dispatch' ||
(
github.event_name == 'issue_comment' &&
github.event.issue.pull_request &&
(contains(github.event.comment.body, '/claude-review') || contains(github.event.comment.body, '/claude-query')) &&
contains(fromJSON('["pdillinger", "jaykorean", "hx235", "dannyhchen", "joshkang97", "zaidoon1", "omkarhgawde", "xingbowang", "anand1976", "virajthakur", "nmk70", "archang19", "mszeszko-meta", "yoshinorim", "cbi42"]'), github.event.comment.user.login)
)
steps:
- name: Detect command type
id: command
uses: actions/github-script@v7
with:
script: |
if (context.eventName === 'workflow_dispatch') {
core.setOutput('type', 'review');
core.setOutput('query', '');
return;
}
const body = context.payload.comment.body;
if (body.includes('/claude-review')) {
core.setOutput('type', 'review');
const match = body.match(/\/claude-review\s+([\s\S]*)/);
core.setOutput('query', match ? match[1].trim() : '');
} else if (body.includes('/claude-query')) {
const match = body.match(/\/claude-query\s+([\s\S]*)/);
const query = match ? match[1].trim() : '';
if (!query) {
core.setFailed('No query provided. Usage: /claude-query <your question>');
return;
}
core.setOutput('type', 'query');
core.setOutput('query', query);
} else {
core.setFailed('Unknown command');
}
- name: Get PR details
id: pr_info
uses: actions/github-script@v7
env:
INPUT_PR_NUMBER: ${{ inputs.pr_number }}
with:
script: |
const prNumber = context.eventName === 'workflow_dispatch'
? parseInt(process.env.INPUT_PR_NUMBER, 10)
: context.issue.number;
const pr = await github.rest.pulls.get({
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: prNumber
});
core.setOutput('pr_number', prNumber.toString());
core.setOutput('head_repo', pr.data.head.repo.clone_url);
core.setOutput('head_ref', pr.data.head.ref);
core.setOutput('head_sha', pr.data.head.sha);
core.setOutput('base_sha', pr.data.base.sha);
core.setOutput('base_ref', pr.data.base.ref);
core.setOutput('pr_title', pr.data.title);
core.setOutput('pr_body', pr.data.body || '');
core.setOutput('pr_author', pr.data.user.login);
- name: Checkout base repository
uses: actions/checkout@v4
with:
ref: ${{ steps.pr_info.outputs.base_ref }}
fetch-depth: 0
persist-credentials: false
- name: Generate diff and prompt
env:
HEAD_REPO: ${{ steps.pr_info.outputs.head_repo }}
HEAD_REF: ${{ steps.pr_info.outputs.head_ref }}
HEAD_SHA: ${{ steps.pr_info.outputs.head_sha }}
BASE_SHA: ${{ steps.pr_info.outputs.base_sha }}
PR_TITLE: ${{ steps.pr_info.outputs.pr_title }}
PR_BODY: ${{ steps.pr_info.outputs.pr_body }}
PR_AUTHOR: ${{ steps.pr_info.outputs.pr_author }}
DIFF_STATS: ""
COMMAND_TYPE: ${{ steps.command.outputs.type }}
ADDITIONAL_CONTEXT: ${{ steps.command.outputs.query }}
USER_QUERY: ${{ steps.command.outputs.query }}
run: |
git remote add fork "${HEAD_REPO}" || true
git fetch fork "${HEAD_REF}" --depth=100
git diff "${BASE_SHA}...${HEAD_SHA}" > /tmp/pr.diff
DIFF_STATS=$(git diff --stat "${BASE_SHA}...${HEAD_SHA}" | tail -1)
DIFF_SIZE=$(wc -c < /tmp/pr.diff)
IS_TRUNCATED=false
if [ "$DIFF_SIZE" -gt 100000 ]; then
head -c 100000 /tmp/pr.diff > /tmp/pr_truncated.diff
mv /tmp/pr_truncated.diff /tmp/pr.diff
IS_TRUNCATED=true
fi
if [ "${COMMAND_TYPE}" = "query" ]; then
# Query prompt
cat claude_md/ci_query_prompt.md > /tmp/prompt.txt
printf '\n---\n\n' >> /tmp/prompt.txt
cat >> /tmp/prompt.txt << PROMPT_EOF
## Pull Request Context
- **Title:** ${PR_TITLE}
- **Author:** ${PR_AUTHOR}
- **Changes:** ${DIFF_STATS}
## PR Description
PROMPT_EOF
printf '%s\n' "${PR_BODY}" >> /tmp/prompt.txt
printf '\n## Question\n' >> /tmp/prompt.txt
printf '%s\n' "${USER_QUERY}" >> /tmp/prompt.txt
printf '\n---\n## PR Diff (for reference)\n' >> /tmp/prompt.txt
cat /tmp/pr.diff >> /tmp/prompt.txt
else
# Review prompt
cat claude_md/ci_review_prompt.md > /tmp/prompt.txt
cat >> /tmp/prompt.txt << PROMPT_EOF
## Pull Request Information
- **Title:** ${PR_TITLE}
- **Author:** ${PR_AUTHOR}
- **Changes:** ${DIFF_STATS}
PROMPT_EOF
if [ "${IS_TRUNCATED}" = "true" ]; then
echo "- **Note:** Diff truncated due to size." >> /tmp/prompt.txt
fi
printf '\n## PR Description\n' >> /tmp/prompt.txt
printf '%s\n' "${PR_BODY}" >> /tmp/prompt.txt
if [ -n "${ADDITIONAL_CONTEXT}" ]; then
printf '\n## Additional Instructions from Reviewer\n' >> /tmp/prompt.txt
printf '%s\n' "${ADDITIONAL_CONTEXT}" >> /tmp/prompt.txt
fi
printf '\n---\n\n## Diff to Review\n\n' >> /tmp/prompt.txt
cat /tmp/pr.diff >> /tmp/prompt.txt
fi
- name: Run Claude
id: claude
uses: anthropics/claude-code-base-action@beta
with:
prompt_file: /tmp/prompt.txt
model: ${{ github.event_name == 'workflow_dispatch' && inputs.model || 'claude-opus-4-6' }}
max_turns: "300"
timeout_minutes: "60"
allowed_tools: "View,GlobTool,GrepTool,Write,Task"
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
- name: Recover partial review on turn limit
id: recover
uses: actions/github-script@v7
env:
EXECUTION_FILE: ${{ steps.claude.outputs.execution_file }}
COMMAND_TYPE: ${{ steps.command.outputs.type }}
with:
script: |
const fs = require('fs');
// Skip recovery for queries — only reviews use incremental findings
if (process.env.COMMAND_TYPE === 'query') {
core.setOutput('needs_recovery', 'false');
return;
}
const log = JSON.parse(fs.readFileSync(process.env.EXECUTION_FILE, 'utf8'));
const result = log.find(m => m.type === 'result');
const hitLimit = result && result.subtype === 'error_max_turns';
const hasFindings = fs.existsSync('review-findings.md');
core.setOutput('hit_limit', hitLimit ? 'true' : 'false');
core.setOutput('has_findings', hasFindings ? 'true' : 'false');
core.setOutput('needs_recovery', (hitLimit && hasFindings) ? 'true' : 'false');
- name: Format partial findings
if: steps.recover.outputs.needs_recovery == 'true'
id: format_recovery
uses: anthropics/claude-code-base-action@beta
with:
prompt_file: claude_md/ci_recovery_prompt.md
model: claude-sonnet-4-20250514
max_turns: "10"
allowed_tools: "View,GlobTool,GrepTool"
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
- name: Build review comment
uses: actions/github-script@v7
env:
EXECUTION_FILE: ${{ steps.claude.outputs.execution_file }}
RECOVERY_FILE: ${{ steps.format_recovery.outputs.execution_file }}
CONCLUSION: ${{ steps.claude.outputs.conclusion }}
HEAD_SHA: ${{ steps.pr_info.outputs.head_sha }}
REVIEWER: ${{ github.event_name == 'workflow_dispatch' && github.actor || github.event.comment.user.login }}
COMMAND_TYPE: ${{ steps.command.outputs.type }}
NEEDS_RECOVERY: ${{ steps.recover.outputs.needs_recovery }}
with:
script: |
const parse = require('./.github/scripts/parse-claude-review.js');
const fs = require('fs');
const isReview = process.env.COMMAND_TYPE === 'review';
let execFile = process.env.EXECUTION_FILE;
if (process.env.NEEDS_RECOVERY === 'true') {
const rf = process.env.RECOVERY_FILE;
if (rf && fs.existsSync(rf)) {
execFile = rf;
}
}
const body = parse({
executionFile: execFile,
conclusion: process.env.CONCLUSION,
meta: {
trigger: 'manual',
headSha: process.env.HEAD_SHA,
reviewer: process.env.REVIEWER,
isQuery: !isReview,
isPartial: process.env.NEEDS_RECOVERY === 'true',
}
});
fs.writeFileSync('claude-review-comment.md', body);
- name: Save PR number
run: echo "${{ steps.pr_info.outputs.pr_number }}" > pr_number.txt
- name: Save trigger metadata
env:
REVIEWER: ${{ github.event_name == 'workflow_dispatch' && github.actor || github.event.comment.user.login }}
COMMENT_ID: ${{ github.event.comment.id }}
run: |
echo "manual" > trigger_type.txt
echo "${REVIEWER}" > reviewer.txt
echo "${COMMENT_ID}" > comment_id.txt
- name: Upload review artifact
if: always()
uses: actions/upload-artifact@v4
with:
name: claude-review-result
path: |
claude-review-comment.md
pr_number.txt
trigger_type.txt
reviewer.txt
comment_id.txt
if-no-files-found: ignore
retention-days: 1
- name: Upload execution log
if: always()
uses: actions/upload-artifact@v4
with:
name: claude-review-execution-log
path: ${{ steps.claude.outputs.execution_file }}
retention-days: 7
if-no-files-found: warn
- name: Upload recovery execution log
if: always() && steps.recover.outputs.needs_recovery == 'true'
uses: actions/upload-artifact@v4
with:
name: claude-review-recovery-log
path: ${{ steps.format_recovery.outputs.execution_file }}
retention-days: 7
if-no-files-found: ignore
+26 -7
View File
@@ -41,22 +41,41 @@ jobs:
- uses: "./.github/actions/pre-steps"
- run: make V=1 -j32 check
- uses: "./.github/actions/post-steps"
build-linux-clang-18-asan-ubsan-with-folly:
build-linux-clang-21-asan-ubsan-with-folly:
if: ${{ github.repository_owner == 'facebook' }}
runs-on:
labels: 16-core-ubuntu
container:
image: ghcr.io/facebook/rocksdb_ubuntu:24.0
image: ghcr.io/facebook/rocksdb_ubuntu:24.1
options: --shm-size=16gb
env:
CC: clang-18
CXX: clang++-18
CC: clang-21
CXX: clang++-21
steps:
- uses: actions/checkout@v4.1.0
- uses: "./.github/actions/pre-steps"
- uses: "./.github/actions/cache-getdeps-downloads"
- uses: "./.github/actions/setup-folly"
- uses: "./.github/actions/build-folly"
- name: Build folly and dependencies
run: |
clean_path=()
IFS=: read -ra path_entries <<< "$PATH"
for entry in "${path_entries[@]}"; do
if [[ "$entry" != "/usr/lib/ccache" ]]; then
clean_path+=("$entry")
fi
done
export PATH="$(IFS=:; echo "${clean_path[*]}")"
ccache_bin="$(command -v ccache || true)"
if [[ -n "$ccache_bin" && -x "$ccache_bin" ]]; then
mv "$ccache_bin" "${ccache_bin}.disabled"
fi
export CC=gcc
export CXX=g++
export USE_CCACHE=0
export CCACHE_DISABLE=1
make build_folly
shell: bash
- run: LIB_MODE=static USE_CLANG=1 USE_FOLLY=1 COMPILE_WITH_UBSAN=1 COMPILE_WITH_ASAN=1 make -j32 check
- uses: "./.github/actions/post-steps"
build-linux-cmake-with-folly:
@@ -145,13 +164,13 @@ jobs:
runs-on:
labels: 4-core-ubuntu
container:
image: ghcr.io/facebook/rocksdb_ubuntu:24.0
image: ghcr.io/facebook/rocksdb_ubuntu:24.1
options: --shm-size=16gb
steps:
- uses: actions/checkout@v4.1.0
- uses: "./.github/actions/pre-steps"
- name: Build rocksdb lib
run: CC=clang-18 CXX=clang++-18 USE_CLANG=1 make -j4 static_lib
run: CC=clang-21 CXX=clang++-21 USE_CLANG=1 make -j4 static_lib
- name: Build fuzzers
run: cd fuzz && make sst_file_writer_fuzzer db_fuzzer db_map_fuzzer
- uses: "./.github/actions/post-steps"
+272 -95
View File
@@ -1,7 +1,18 @@
name: facebook/rocksdb/pr-jobs
on: [push, pull_request]
permissions: {}
env:
# Set to a job name to run only that job (on any repo), or leave empty for
# normal behavior (all jobs on facebook repo only).
ONLY_JOB: ''
jobs:
config:
runs-on: ubuntu-latest
outputs:
only_job: ${{ steps.set.outputs.only_job }}
steps:
- id: set
run: echo "only_job=$ONLY_JOB" >> "$GITHUB_OUTPUT"
# NOTE: multiple workflows would be recommended, but the current GHA UI in
# PRs doesn't make it clear when there's an overall error with a workflow,
# making it easy to overlook something broken. Grouping everything into one
@@ -19,6 +30,10 @@ jobs:
# increasing the risk of misconfiguration, especially on forks that might
# want to run with this GHA setup.
#
# SELECTIVE JOB EXECUTION: Set the ONLY_JOB env var at the top of this file
# to a job name (e.g. "build-linux-clang-tidy") to run only that job,
# bypassing the repository owner check. Leave it empty for normal behavior.
#
# DEBUGGING WITH SSH: Temporarily add this as a job step, either before the
# step of interest without the "if:" line or after the failing step with the
# "if:" line. Then use ssh command printed in CI output.
@@ -30,7 +45,8 @@ jobs:
# ======================== Fast Initial Checks ====================== #
check-format-and-targets:
if: ${{ github.repository_owner == 'facebook' }}
if: needs.config.outputs.only_job == 'check-format-and-targets' || (needs.config.outputs.only_job == '' && github.repository_owner == 'facebook')
needs: config
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4.1.0
@@ -44,6 +60,10 @@ jobs:
run: python -m pip install --upgrade pip
- name: Install argparse
run: pip install argparse
- name: Install clang-format
run: |
pip install https://files.pythonhosted.org/packages/fb/ac/3c04772acc0257f5730e83adb542b2603c1a62d1315010ab593a980af404/clang_format-21.1.2-py2.py3-none-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
clang-format --version
- name: Download clang-format-diff.py
run: wget https://rocksdb-deps.s3.us-west-2.amazonaws.com/llvm/llvm-project/release/12.x/clang/tools/clang-format/clang-format-diff.py
- name: Check format
@@ -52,6 +72,8 @@ jobs:
run: make check-buck-targets
- name: Simple source code checks
run: make check-sources
- name: Validate GitHub Actions YAML
run: make check-workflow-yaml
- name: Sanity check check_format_compatible.sh
run: |-
export TEST_TMPDIR=/dev/shm/rocksdb
@@ -62,7 +84,8 @@ jobs:
SANITY_CHECK=1 LONG_TEST=1 tools/check_format_compatible.sh
# ========================= Linux With Tests ======================== #
build-linux:
if: ${{ github.repository_owner == 'facebook' }}
if: needs.config.outputs.only_job == 'build-linux' || (needs.config.outputs.only_job == '' && github.repository_owner == 'facebook')
needs: config
runs-on:
labels: 16-core-ubuntu
container:
@@ -71,18 +94,27 @@ jobs:
steps:
- uses: actions/checkout@v4.1.0
- uses: "./.github/actions/pre-steps"
- uses: "./.github/actions/setup-ccache"
with:
cache-key-prefix: build-linux
- run: make V=1 J=32 -j32 check
- uses: "./.github/actions/teardown-ccache"
if: always()
- uses: "./.github/actions/post-steps"
build-linux-cmake-mingw:
if: ${{ github.repository_owner == 'facebook' }}
if: needs.config.outputs.only_job == 'build-linux-cmake-mingw' || (needs.config.outputs.only_job == '' && github.repository_owner == 'facebook')
needs: config
runs-on:
labels: 4-core-ubuntu
container:
image: ghcr.io/facebook/rocksdb_ubuntu:24.0
image: ghcr.io/facebook/rocksdb_ubuntu:24.1
options: --shm-size=16gb
steps:
- uses: actions/checkout@v4.1.0
- uses: "./.github/actions/pre-steps"
- uses: "./.github/actions/setup-ccache"
with:
cache-key-prefix: cmake-mingw
- run: update-alternatives --set x86_64-w64-mingw32-g++ /usr/bin/x86_64-w64-mingw32-g++-posix
- name: Build cmake-mingw
run: |-
@@ -91,11 +123,14 @@ jobs:
which java && java -version
which javac && javac -version
mkdir build && cd build && cmake -DJNI=1 -DWITH_GFLAGS=OFF .. -DCMAKE_C_COMPILER=x86_64-w64-mingw32-gcc -DCMAKE_CXX_COMPILER=x86_64-w64-mingw32-g++ -DCMAKE_SYSTEM_NAME=Windows && make -j4 rocksdb rocksdbjni
- uses: "./.github/actions/teardown-ccache"
if: always()
- uses: "./.github/actions/post-steps"
build-linux-make-with-folly:
if: ${{ github.repository_owner == 'facebook' }}
if: needs.config.outputs.only_job == 'build-linux-make-with-folly' || (needs.config.outputs.only_job == '' && github.repository_owner == 'facebook')
needs: config
runs-on:
labels: 16-core-ubuntu
labels: 32-core-ubuntu
container:
image: ghcr.io/facebook/rocksdb_ubuntu:22.1
options: --shm-size=16gb
@@ -104,15 +139,21 @@ jobs:
- uses: "./.github/actions/pre-steps"
- uses: "./.github/actions/cache-getdeps-downloads"
- uses: "./.github/actions/setup-folly"
- uses: "./.github/actions/setup-ccache"
with:
cache-key-prefix: make-with-folly
- uses: "./.github/actions/cache-folly"
id: cache-folly
- uses: "./.github/actions/build-folly"
with:
cache-hit: ${{ steps.cache-folly.outputs.cache-hit }}
- run: USE_FOLLY=1 LIB_MODE=static V=1 make -j32 check
- run: USE_FOLLY=1 LIB_MODE=static V=1 make -j64 check
- uses: "./.github/actions/teardown-ccache"
if: always()
- uses: "./.github/actions/post-steps"
build-linux-make-with-folly-lite-no-test:
if: ${{ github.repository_owner == 'facebook' }}
if: needs.config.outputs.only_job == 'build-linux-make-with-folly-lite-no-test' || (needs.config.outputs.only_job == '' && github.repository_owner == 'facebook')
needs: config
runs-on:
labels: 16-core-ubuntu
container:
@@ -123,12 +164,18 @@ jobs:
- uses: "./.github/actions/pre-steps"
- uses: "./.github/actions/cache-getdeps-downloads"
- uses: "./.github/actions/setup-folly"
- uses: "./.github/actions/setup-ccache"
with:
cache-key-prefix: make-folly-lite
- run: USE_FOLLY_LITE=1 EXTRA_CXXFLAGS=-DGLOG_USE_GLOG_EXPORT V=1 make -j32 all
- uses: "./.github/actions/teardown-ccache"
if: always()
- uses: "./.github/actions/post-steps"
build-linux-cmake-with-folly-coroutines:
if: ${{ github.repository_owner == 'facebook' }}
if: needs.config.outputs.only_job == 'build-linux-cmake-with-folly-coroutines' || (needs.config.outputs.only_job == '' && github.repository_owner == 'facebook')
needs: config
runs-on:
labels: 16-core-ubuntu
labels: 32-core-ubuntu
container:
image: ghcr.io/facebook/rocksdb_ubuntu:22.1
options: --shm-size=16gb
@@ -137,27 +184,48 @@ jobs:
- uses: "./.github/actions/pre-steps"
- uses: "./.github/actions/cache-getdeps-downloads"
- uses: "./.github/actions/setup-folly"
- uses: "./.github/actions/setup-ccache"
with:
cache-key-prefix: cmake-folly-coroutines
- uses: "./.github/actions/cache-folly"
id: cache-folly
- uses: "./.github/actions/build-folly"
with:
cache-hit: ${{ steps.cache-folly.outputs.cache-hit }}
- run: "(mkdir build && cd build && cmake -DUSE_COROUTINES=1 -DWITH_GFLAGS=1 -DROCKSDB_BUILD_SHARED=0 .. && make VERBOSE=1 -j20 && ctest -j20)"
- run: "(mkdir build && cd build && cmake -DUSE_COROUTINES=1 -DWITH_GFLAGS=1 -DROCKSDB_BUILD_SHARED=0 -DPORTABLE=ON .. && make VERBOSE=1 -j64 && ctest -j64)"
- uses: "./.github/actions/teardown-ccache"
if: always()
- uses: "./.github/actions/post-steps"
build-linux-cmake-with-benchmark-no-thread-status:
if: ${{ github.repository_owner == 'facebook' }}
if: needs.config.outputs.only_job == 'build-linux-cmake-with-benchmark-no-thread-status' || (needs.config.outputs.only_job == '' && github.repository_owner == 'facebook')
needs: config
runs-on:
labels: 16-core-ubuntu
strategy:
fail-fast: false
matrix:
shard: [0, 1, 2, 3]
container:
image: ghcr.io/facebook/rocksdb_ubuntu:22.1
options: --shm-size=16gb
steps:
- uses: actions/checkout@v4.1.0
- uses: "./.github/actions/pre-steps"
- run: mkdir build && cd build && cmake -DWITH_GFLAGS=1 -DWITH_BENCHMARK=1 -DCMAKE_CXX_FLAGS=-DNROCKSDB_THREAD_STATUS .. && make VERBOSE=1 -j20 && ctest -j20
- uses: "./.github/actions/setup-ccache"
with:
cache-key-prefix: cmake-benchmark
- name: Build
run: mkdir build && cd build && cmake -DWITH_GFLAGS=1 -DWITH_BENCHMARK=1 -DPORTABLE=ON -DCMAKE_CXX_FLAGS=-DNROCKSDB_THREAD_STATUS .. && make VERBOSE=1 -j20
- name: Test shard ${{ matrix.shard }} of 4
run: cd build && ctest -j20 -I ${{ matrix.shard }},,4
- uses: "./.github/actions/teardown-ccache"
if: always()
- uses: "./.github/actions/post-steps"
with:
artifact-prefix: "${{ github.job }}-shard${{ matrix.shard }}"
build-linux-encrypted_env-no_compression:
if: ${{ github.repository_owner == 'facebook' }}
if: needs.config.outputs.only_job == 'build-linux-encrypted_env-no_compression' || (needs.config.outputs.only_job == '' && github.repository_owner == 'facebook')
needs: config
runs-on:
labels: 16-core-ubuntu
container:
@@ -166,12 +234,18 @@ jobs:
steps:
- uses: actions/checkout@v4.1.0
- uses: "./.github/actions/pre-steps"
- uses: "./.github/actions/setup-ccache"
with:
cache-key-prefix: encrypted-env-no-compression
- run: ENCRYPTED_ENV=1 ROCKSDB_DISABLE_SNAPPY=1 ROCKSDB_DISABLE_ZLIB=1 ROCKSDB_DISABLE_BZIP=1 ROCKSDB_DISABLE_LZ4=1 ROCKSDB_DISABLE_ZSTD=1 make V=1 J=32 -j32 check
- run: "./sst_dump --help | grep -E -q 'Supported built-in compression types: kNoCompression$' # Verify no compiled in compression\n"
- uses: "./.github/actions/teardown-ccache"
if: always()
- uses: "./.github/actions/post-steps"
# ======================== Linux No Test Runs ======================= #
build-linux-release:
if: ${{ github.repository_owner == 'facebook' }}
if: needs.config.outputs.only_job == 'build-linux-release' || (needs.config.outputs.only_job == '' && github.repository_owner == 'facebook')
needs: config
runs-on:
labels: 16-core-ubuntu
container:
@@ -179,6 +253,9 @@ jobs:
options: --shm-size=16gb
steps:
- uses: actions/checkout@v4.1.0
- uses: "./.github/actions/setup-ccache"
with:
cache-key-prefix: release
- run: make V=1 -j32 LIB_MODE=shared release
- run: ls librocksdb.so
- run: "./trace_analyzer --version" # A tool dependent on gflags that can run in release build
@@ -195,9 +272,12 @@ jobs:
- run: USE_RTTI=1 make V=1 -j32 release
- run: ls librocksdb.a
- run: if ./trace_analyzer --version; then false; else true; fi
- uses: "./.github/actions/teardown-ccache"
if: always()
- uses: "./.github/actions/post-steps"
build-linux-clang-13-no_test_run:
if: ${{ github.repository_owner == 'facebook' }}
if: needs.config.outputs.only_job == 'build-linux-clang-13-no_test_run' || (needs.config.outputs.only_job == '' && github.repository_owner == 'facebook')
needs: config
runs-on:
labels: 8-core-ubuntu
container:
@@ -206,61 +286,61 @@ jobs:
steps:
- uses: actions/checkout@v4.1.0
- uses: "./.github/actions/pre-steps"
- uses: "./.github/actions/setup-ccache"
with:
cache-key-prefix: clang-13
# FIXME: get back to "all microbench" targets
- run: CC=clang-13 CXX=clang++-13 USE_CLANG=1 EXTRA_CXXFLAGS=-stdlib=libc++ EXTRA_LDFLAGS=-stdlib=libc++ make -j32 shared_lib
- run: make clean
# FIXME: get back to "release" target
- run: CC=clang-13 CXX=clang++-13 USE_CLANG=1 EXTRA_CXXFLAGS=-stdlib=libc++ EXTRA_LDFLAGS=-stdlib=libc++ DEBUG_LEVEL=0 make -j32 shared_lib
- uses: "./.github/actions/teardown-ccache"
if: always()
- uses: "./.github/actions/post-steps"
build-linux-clang-18-no_test_run:
if: ${{ github.repository_owner == 'facebook' }}
build-linux-clang-21-no_test_run:
if: needs.config.outputs.only_job == 'build-linux-clang-21-no_test_run' || (needs.config.outputs.only_job == '' && github.repository_owner == 'facebook')
needs: config
runs-on:
labels: 16-core-ubuntu
container:
image: ghcr.io/facebook/rocksdb_ubuntu:24.0
image: ghcr.io/facebook/rocksdb_ubuntu:24.1
options: --shm-size=16gb
steps:
- uses: actions/checkout@v4.1.0
- uses: "./.github/actions/pre-steps"
- run: CC=clang-18 CXX=clang++-18 USE_CLANG=1 make -j32 all microbench
- uses: "./.github/actions/setup-ccache"
with:
cache-key-prefix: clang-21
- run: CC=clang-21 CXX=clang++-21 USE_CLANG=1 make -j32 all microbench
- run: make clean
- run: CC=clang-18 CXX=clang++-18 USE_CLANG=1 DEBUG_LEVEL=0 make -j32 release
- run: CC=clang-21 CXX=clang++-21 USE_CLANG=1 DEBUG_LEVEL=0 make -j32 release
- uses: "./.github/actions/teardown-ccache"
if: always()
- uses: "./.github/actions/post-steps"
build-linux-gcc-14-no_test_run:
if: ${{ github.repository_owner == 'facebook' }}
if: needs.config.outputs.only_job == 'build-linux-gcc-14-no_test_run' || (needs.config.outputs.only_job == '' && github.repository_owner == 'facebook')
needs: config
runs-on:
labels: 16-core-ubuntu
container:
image: ghcr.io/facebook/rocksdb_ubuntu:24.0
image: ghcr.io/facebook/rocksdb_ubuntu:24.1
options: --shm-size=16gb
steps:
- uses: actions/checkout@v4.1.0
- uses: "./.github/actions/pre-steps"
- uses: "./.github/actions/setup-ccache"
with:
cache-key-prefix: gcc-14
- run: CC=gcc-14 CXX=g++-14 V=1 make -j32 all microbench
- uses: "./.github/actions/teardown-ccache"
if: always()
- uses: "./.github/actions/post-steps"
# ======================== Linux Other Checks ======================= #
build-linux-clang18-clang-analyze:
if: ${{ github.repository_owner == 'facebook' }}
runs-on:
labels: 16-core-ubuntu
container:
image: ghcr.io/facebook/rocksdb_ubuntu:24.0
options: --shm-size=16gb
steps:
- uses: actions/checkout@v4.1.0
- uses: "./.github/actions/pre-steps"
- run: CC=clang-18 CXX=clang++-18 ROCKSDB_DISABLE_ALIGNED_NEW=1 CLANG_ANALYZER="/usr/bin/clang++-18" CLANG_SCAN_BUILD=scan-build-18 USE_CLANG=1 make V=1 -j32 analyze
- uses: "./.github/actions/post-steps"
- name: compress test report
run: tar -cvzf scan_build_report.tar.gz scan_build_report
if: failure()
- uses: actions/upload-artifact@v4.0.0
with:
name: scan-build-report
path: scan_build_report.tar.gz
build-linux-unity-and-headers:
if: ${{ github.repository_owner == 'facebook' }}
if: needs.config.outputs.only_job == 'build-linux-unity-and-headers' || (needs.config.outputs.only_job == '' && github.repository_owner == 'facebook')
needs: config
runs-on:
labels: 4-core-ubuntu
container:
@@ -269,49 +349,101 @@ jobs:
steps:
- uses: actions/checkout@v4.1.0
- run: apt-get update -y && apt-get install -y libgflags-dev
- uses: "./.github/actions/setup-ccache"
with:
cache-key-prefix: unity-headers
- name: Unity build
run: make V=1 -j8 unity_test
- run: make V=1 -j8 -k check-headers
- uses: "./.github/actions/teardown-ccache"
if: always()
- uses: "./.github/actions/post-steps"
build-linux-mini-crashtest:
if: ${{ github.repository_owner == 'facebook' }}
if: needs.config.outputs.only_job == 'build-linux-mini-crashtest' || (needs.config.outputs.only_job == '' && github.repository_owner == 'facebook')
needs: config
runs-on:
labels: 4-core-ubuntu
strategy:
fail-fast: false
matrix:
include:
- crash_test_target: blackbox_crash_test_with_atomic_flush
crash_duration: 480
- crash_test_target: blackbox_crash_test
crash_duration: 240
container:
image: ghcr.io/facebook/rocksdb_ubuntu:22.1
options: --shm-size=16gb
steps:
- uses: actions/checkout@v4.1.0
- uses: "./.github/actions/pre-steps"
- run: ulimit -S -n `ulimit -H -n` && make V=1 -j8 CRASH_TEST_EXT_ARGS='--duration=960 --max_key=2500000' blackbox_crash_test_with_atomic_flush
- uses: "./.github/actions/setup-ccache"
with:
cache-key-prefix: mini-crashtest
- run: ulimit -S -n `ulimit -H -n` && make V=1 -j8 CRASH_TEST_EXT_ARGS='--duration=${{ matrix.crash_duration }} --max_key=2500000' ${{ matrix.crash_test_target }}
- uses: "./.github/actions/teardown-ccache"
if: always()
- uses: "./.github/actions/post-steps"
with:
artifact-prefix: "${{ github.job }}-${{ matrix.crash_test_target }}"
# ======================= Linux with Sanitizers ===================== #
build-linux-clang18-asan-ubsan:
if: ${{ github.repository_owner == 'facebook' }}
build-linux-clang21-asan-ubsan:
if: needs.config.outputs.only_job == 'build-linux-clang21-asan-ubsan' || (needs.config.outputs.only_job == '' && github.repository_owner == 'facebook')
needs: config
runs-on:
labels: 32-core-ubuntu
strategy:
fail-fast: false
matrix:
shard: [0, 1, 2]
container:
image: ghcr.io/facebook/rocksdb_ubuntu:24.0
image: ghcr.io/facebook/rocksdb_ubuntu:24.1
options: --shm-size=16gb
steps:
- uses: actions/checkout@v4.1.0
- uses: "./.github/actions/pre-steps"
- run: COMPILE_WITH_ASAN=1 COMPILE_WITH_UBSAN=1 CC=clang-18 CXX=clang++-18 ROCKSDB_DISABLE_ALIGNED_NEW=1 USE_CLANG=1 make V=1 -j40 check
- uses: "./.github/actions/setup-ccache"
with:
cache-key-prefix: clang21-asan-ubsan
- run: COMPILE_WITH_ASAN=1 COMPILE_WITH_UBSAN=1 CC=clang-21 CXX=clang++-21 ROCKSDB_DISABLE_ALIGNED_NEW=1 USE_CLANG=1 make V=1 -j40 check
env:
CI_SHARD_INDEX: ${{ matrix.shard }}
CI_TOTAL_SHARDS: 3
- uses: "./.github/actions/teardown-ccache"
if: always()
- uses: "./.github/actions/post-steps"
build-linux-clang18-mini-tsan:
if: ${{ github.repository_owner == 'facebook' }}
with:
artifact-prefix: "${{ github.job }}-shard${{ matrix.shard }}"
build-linux-clang21-mini-tsan:
if: needs.config.outputs.only_job == 'build-linux-clang21-mini-tsan' || (needs.config.outputs.only_job == '' && github.repository_owner == 'facebook')
needs: config
runs-on:
labels: 32-core-ubuntu
strategy:
fail-fast: false
matrix:
shard: [0, 1, 2]
container:
image: ghcr.io/facebook/rocksdb_ubuntu:24.0
image: ghcr.io/facebook/rocksdb_ubuntu:24.1
options: --shm-size=16gb
steps:
- uses: actions/checkout@v4.1.0
- uses: "./.github/actions/pre-steps"
- run: COMPILE_WITH_TSAN=1 CC=clang-18 CXX=clang++-18 ROCKSDB_DISABLE_ALIGNED_NEW=1 USE_CLANG=1 make V=1 -j32 check
- uses: "./.github/actions/setup-ccache"
with:
cache-key-prefix: clang21-tsan
- run: COMPILE_WITH_TSAN=1 CC=clang-21 CXX=clang++-21 ROCKSDB_DISABLE_ALIGNED_NEW=1 USE_CLANG=1 make V=1 -j32 check
env:
CI_SHARD_INDEX: ${{ matrix.shard }}
CI_TOTAL_SHARDS: 3
- uses: "./.github/actions/teardown-ccache"
if: always()
- uses: "./.github/actions/post-steps"
with:
artifact-prefix: "${{ github.job }}-shard${{ matrix.shard }}"
build-linux-static_lib-alt_namespace-status_checked:
if: ${{ github.repository_owner == 'facebook' }}
if: needs.config.outputs.only_job == 'build-linux-static_lib-alt_namespace-status_checked' || (needs.config.outputs.only_job == '' && github.repository_owner == 'facebook')
needs: config
runs-on:
labels: 16-core-ubuntu
container:
@@ -320,11 +452,17 @@ jobs:
steps:
- uses: actions/checkout@v4.1.0
- uses: "./.github/actions/pre-steps"
- uses: "./.github/actions/setup-ccache"
with:
cache-key-prefix: static-alt-namespace
- run: ASSERT_STATUS_CHECKED=1 TEST_UINT128_COMPAT=1 ROCKSDB_MODIFY_NPHASH=1 LIB_MODE=static OPT="-DROCKSDB_USE_STD_SEMAPHORES -DROCKSDB_NAMESPACE=alternative_rocksdb_ns" make V=1 -j24 check
- uses: "./.github/actions/teardown-ccache"
if: always()
- uses: "./.github/actions/post-steps"
# ========================= MacOS build only ======================== #
build-macos:
if: ${{ github.repository_owner == 'facebook' }}
if: needs.config.outputs.only_job == 'build-macos' || (needs.config.outputs.only_job == '' && github.repository_owner == 'facebook')
needs: config
runs-on: macos-15-xlarge
env:
ROCKSDB_DISABLE_JEMALLOC: 1
@@ -333,15 +471,21 @@ jobs:
- uses: maxim-lobanov/setup-xcode@v1.6.0
with:
xcode-version: 16.4.0
- uses: "./.github/actions/setup-ccache"
with:
cache-key-prefix: macos
- uses: "./.github/actions/increase-max-open-files-on-macos"
- uses: "./.github/actions/install-gflags-on-macos"
- uses: "./.github/actions/pre-steps-macos"
- name: Build
run: ulimit -S -n `ulimit -H -n` && make V=1 J=16 -j8 all
- uses: "./.github/actions/teardown-ccache"
if: always()
- uses: "./.github/actions/post-steps"
# ========================= MacOS with Tests ======================== #
build-macos-cmake:
if: ${{ github.repository_owner == 'facebook' }}
if: needs.config.outputs.only_job == 'build-macos-cmake' || (needs.config.outputs.only_job == '' && github.repository_owner == 'facebook')
needs: config
runs-on: macos-15-xlarge
strategy:
matrix:
@@ -351,6 +495,9 @@ jobs:
- uses: maxim-lobanov/setup-xcode@v1.6.0
with:
xcode-version: 16.4.0
- uses: "./.github/actions/setup-ccache"
with:
cache-key-prefix: macos-cmake
- uses: "./.github/actions/increase-max-open-files-on-macos"
- uses: "./.github/actions/install-gflags-on-macos"
- uses: "./.github/actions/pre-steps-macos"
@@ -370,83 +517,87 @@ jobs:
- name: Run shard 3 out of 4 test shards
run: ulimit -S -n `ulimit -H -n` && cd build && ctest -j8 -I 3,,4
if: ${{ matrix.run_sharded_tests == 3 }}
- uses: "./.github/actions/teardown-ccache"
if: always()
- uses: "./.github/actions/post-steps"
# ======================== Windows with Tests ======================= #
# NOTE: some windows jobs are in "nightly" to save resources
build-windows-vs2022:
if: ${{ github.repository_owner == 'facebook' }}
if: needs.config.outputs.only_job == 'build-windows-vs2022' || (needs.config.outputs.only_job == '' && github.repository_owner == 'facebook')
needs: config
runs-on: windows-8-core
strategy:
fail-fast: false
matrix:
include:
- test_shard: db_test
suite_run: db_test
run_java: "false"
- test_shard: other
suite_run: arena_test,db_basic_test,db_test2,db_merge_operand_test,bloom_test,c_test,coding_test,crc32c_test,dynamic_bloom_test,env_basic_test,env_test,hash_test,random_test
run_java: "false"
- test_shard: java
suite_run: ""
run_java: "true"
env:
CMAKE_GENERATOR: Visual Studio 17 2022
CMAKE_PORTABLE: 1
steps:
- uses: actions/checkout@v4.1.0
- uses: "./.github/actions/windows-build-steps"
with:
suite-run: ${{ matrix.suite_run }}
run-java: ${{ matrix.run_java }}
# ============================ Java Jobs ============================ #
build-linux-java:
if: ${{ github.repository_owner == 'facebook' }}
if: needs.config.outputs.only_job == 'build-linux-java' || (needs.config.outputs.only_job == '' && github.repository_owner == 'facebook')
needs: config
runs-on:
labels: 4-core-ubuntu
container:
image: ghcr.io/facebook/rocksdb_ubuntu:22.1
options: --shm-size=16gb
steps:
# The docker image is intentionally based on an OS that has an older GLIBC version.
# That GLIBC is incompatibile with GitHub's actions/checkout. Thus we implement a manual checkout step.
# NOTE: replaced evolvedbinary/rocksjava:centos7_x64-be with ghcr.io/facebook/rocksdb_ubuntu:22.1
# until a more appropriate docker image with C++20 support is made.
- name: Checkout
env:
GH_TOKEN: ${{ github.token }}
run: |
chown `whoami` . || true
git clone --no-checkout https://oath2:$GH_TOKEN@github.com/${{ github.repository }}.git .
git -c protocol.version=2 fetch --update-head-ok --no-tags --prune --no-recurse-submodules --depth=1 origin +${{ github.sha }}:${{ github.ref }}
git checkout --progress --force ${{ github.ref }}
git log -1 --format='%H'
- uses: actions/checkout@v4.1.0
- uses: "./.github/actions/pre-steps"
- uses: "./.github/actions/setup-ccache"
with:
cache-key-prefix: java
- name: Set Java Environment
run: |-
echo "JAVA_HOME=${JAVA_HOME}"
which java && java -version
which javac && javac -version
- name: Test RocksDBJava
# NOTE: replaced scl enable devtoolset-7 'make V=1 J=8 -j8 jtest'
run: make V=1 J=8 -j8 jtest
# post-steps skipped because of compatibility issues with docker image
- uses: "./.github/actions/teardown-ccache"
if: always()
build-linux-java-static:
if: ${{ github.repository_owner == 'facebook' }}
if: needs.config.outputs.only_job == 'build-linux-java-static' || (needs.config.outputs.only_job == '' && github.repository_owner == 'facebook')
needs: config
runs-on:
labels: 4-core-ubuntu
container:
image: ghcr.io/facebook/rocksdb_ubuntu:22.1
options: --shm-size=16gb
steps:
# The docker image is intentionally based on an OS that has an older GLIBC version.
# That GLIBC is incompatibile with GitHub's actions/checkout. Thus we implement a manual checkout step.
# NOTE: replaced evolvedbinary/rocksjava:centos7_x64-be with ghcr.io/facebook/rocksdb_ubuntu:22.1
# until a more appropriate docker image with C++20 support is made.
- name: Checkout
env:
GH_TOKEN: ${{ github.token }}
run: |
chown `whoami` . || true
git clone --no-checkout https://oath2:$GH_TOKEN@github.com/${{ github.repository }}.git .
git -c protocol.version=2 fetch --update-head-ok --no-tags --prune --no-recurse-submodules --depth=1 origin +${{ github.sha }}:${{ github.ref }}
git checkout --progress --force ${{ github.ref }}
git log -1 --format='%H'
- uses: actions/checkout@v4.1.0
- uses: "./.github/actions/pre-steps"
- uses: "./.github/actions/setup-ccache"
with:
cache-key-prefix: java-static
- name: Set Java Environment
run: |-
echo "JAVA_HOME=${JAVA_HOME}"
which java && java -version
which javac && javac -version
- name: Build RocksDBJava Static Library
# NOTE: replaced scl enable devtoolset-7 'make V=1 J=8 -j8 rocksdbjavastatic'
run: make V=1 J=8 -j8 rocksdbjavastatic
# post-steps skipped because of compatibility issues with docker image
- uses: "./.github/actions/teardown-ccache"
if: always()
build-macos-java:
if: ${{ github.repository_owner == 'facebook' }}
if: needs.config.outputs.only_job == 'build-macos-java' || (needs.config.outputs.only_job == '' && github.repository_owner == 'facebook')
needs: config
runs-on: macos-15-xlarge
env:
JAVA_HOME: "/Library/Java/JavaVirtualMachines/liberica-jdk-8.jdk/Contents/Home"
@@ -456,6 +607,9 @@ jobs:
- uses: maxim-lobanov/setup-xcode@v1.6.0
with:
xcode-version: 16.4.0
- uses: "./.github/actions/setup-ccache"
with:
cache-key-prefix: macos-java
- uses: "./.github/actions/increase-max-open-files-on-macos"
- uses: "./.github/actions/install-gflags-on-macos"
- uses: "./.github/actions/install-jdk8-on-macos"
@@ -467,9 +621,12 @@ jobs:
which javac && javac -version
- name: Test RocksDBJava
run: make V=1 J=16 -j16 jtest
- uses: "./.github/actions/teardown-ccache"
if: always()
- uses: "./.github/actions/post-steps"
build-macos-java-static:
if: ${{ github.repository_owner == 'facebook' }}
if: needs.config.outputs.only_job == 'build-macos-java-static' || (needs.config.outputs.only_job == '' && github.repository_owner == 'facebook')
needs: config
runs-on: macos-15-xlarge
env:
JAVA_HOME: "/Library/Java/JavaVirtualMachines/liberica-jdk-8.jdk/Contents/Home"
@@ -478,6 +635,9 @@ jobs:
- uses: maxim-lobanov/setup-xcode@v1.6.0
with:
xcode-version: 16.4.0
- uses: "./.github/actions/setup-ccache"
with:
cache-key-prefix: macos-java-static
- uses: "./.github/actions/increase-max-open-files-on-macos"
- uses: "./.github/actions/install-gflags-on-macos"
- uses: "./.github/actions/install-jdk8-on-macos"
@@ -489,9 +649,12 @@ jobs:
which javac && javac -version
- name: Build RocksDBJava x86 and ARM Static Libraries
run: make V=1 J=16 -j16 rocksdbjavastaticosx
- uses: "./.github/actions/teardown-ccache"
if: always()
- uses: "./.github/actions/post-steps"
build-macos-java-static-universal:
if: ${{ github.repository_owner == 'facebook' }}
if: needs.config.outputs.only_job == 'build-macos-java-static-universal' || (needs.config.outputs.only_job == '' && github.repository_owner == 'facebook')
needs: config
runs-on: macos-15-xlarge
env:
JAVA_HOME: "/Library/Java/JavaVirtualMachines/liberica-jdk-8.jdk/Contents/Home"
@@ -500,6 +663,9 @@ jobs:
- uses: maxim-lobanov/setup-xcode@v1.6.0
with:
xcode-version: 16.4.0
- uses: "./.github/actions/setup-ccache"
with:
cache-key-prefix: macos-java-static-universal
- uses: "./.github/actions/increase-max-open-files-on-macos"
- uses: "./.github/actions/install-gflags-on-macos"
- uses: "./.github/actions/install-jdk8-on-macos"
@@ -511,9 +677,12 @@ jobs:
which javac && javac -version
- name: Build RocksDBJava Universal Binary Static Library
run: make V=1 J=16 -j16 rocksdbjavastaticosx_ub
- uses: "./.github/actions/teardown-ccache"
if: always()
- uses: "./.github/actions/post-steps"
build-linux-java-pmd:
if: ${{ github.repository_owner == 'facebook' }}
if: needs.config.outputs.only_job == 'build-linux-java-pmd' || (needs.config.outputs.only_job == '' && github.repository_owner == 'facebook')
needs: config
runs-on:
labels: 4-core-ubuntu
container:
@@ -539,12 +708,20 @@ jobs:
name: maven-site
path: "${{ github.workspace }}/java/target/site"
build-linux-arm:
if: ${{ github.repository_owner == 'facebook' }}
if: needs.config.outputs.only_job == 'build-linux-arm' || (needs.config.outputs.only_job == '' && github.repository_owner == 'facebook')
needs: config
runs-on:
labels: 4-core-ubuntu-arm
steps:
- uses: actions/checkout@v4.1.0
- uses: "./.github/actions/pre-steps"
- run: sudo apt-get update && sudo apt-get install -y build-essential
- run: sudo apt-get update && sudo apt-get install -y build-essential ccache
- uses: "./.github/actions/setup-ccache"
with:
cache-key-prefix: arm
- run: ROCKSDBTESTS_PLATFORM_DEPENDENT=only make V=1 J=4 -j4 all_but_some_tests check_some
- name: Print ccache stats
run: ccache -s
if: always()
shell: bash
- uses: "./.github/actions/post-steps"
+8
View File
@@ -102,3 +102,11 @@ cmake-build-*
third-party/folly/
.cache
*.sublime-*
# Claude Code local settings
.claude/settings.local.json
tools/__pycache__/
# Keep documentation trackable even if broader ignores match names like "*_test".
!docs/
!docs/**
+9
View File
@@ -0,0 +1,9 @@
# Agent Instructions
This repository's authoritative agent instructions live in `CLAUDE.md`.
Read and follow [`CLAUDE.md`](./CLAUDE.md) in full before making changes or
reviewing code in this checkout.
If there is any ambiguity between this file and `CLAUDE.md`, `CLAUDE.md` takes
precedence.
+61 -4
View File
@@ -32,12 +32,14 @@ cpp_library_wrapper(name="rocksdb_lib", srcs=[
"db/blob/blob_file_cache.cc",
"db/blob/blob_file_garbage.cc",
"db/blob/blob_file_meta.cc",
"db/blob/blob_file_partition_manager.cc",
"db/blob/blob_file_reader.cc",
"db/blob/blob_garbage_meter.cc",
"db/blob/blob_log_format.cc",
"db/blob/blob_log_sequential_reader.cc",
"db/blob/blob_log_writer.cc",
"db/blob/blob_source.cc",
"db/blob/blob_write_batch_transformer.cc",
"db/blob/prefetch_buffer_collection.cc",
"db/builder.cc",
"db/c.cc",
@@ -106,8 +108,10 @@ cpp_library_wrapper(name="rocksdb_lib", srcs=[
"db/version_edit.cc",
"db/version_edit_handler.cc",
"db/version_set.cc",
"db/version_util.cc",
"db/wal_edit.cc",
"db/wal_manager.cc",
"db/wide/read_path_blob_resolver.cc",
"db/wide/wide_column_serialization.cc",
"db/wide/wide_columns.cc",
"db/wide/wide_columns_helper.cc",
@@ -207,6 +211,7 @@ cpp_library_wrapper(name="rocksdb_lib", srcs=[
"table/block_based/hash_index_reader.cc",
"table/block_based/index_builder.cc",
"table/block_based/index_reader_common.cc",
"table/block_based/multi_scan_index_iterator.cc",
"table/block_based/parsed_full_filter_block.cc",
"table/block_based/partitioned_filter_block.cc",
"table/block_based/partitioned_index_iterator.cc",
@@ -329,6 +334,7 @@ cpp_library_wrapper(name="rocksdb_lib", srcs=[
"utilities/secondary_index/simple_secondary_index.cc",
"utilities/simulator_cache/cache_simulator.cc",
"utilities/simulator_cache/sim_cache.cc",
"utilities/sorted_run_builder/sorted_run_builder.cc",
"utilities/table_properties_collectors/compact_for_tiering_collector.cc",
"utilities/table_properties_collectors/compact_on_deletion_collector.cc",
"utilities/trace/file_trace_reader_writer.cc",
@@ -362,6 +368,9 @@ cpp_library_wrapper(name="rocksdb_lib", srcs=[
"utilities/transactions/write_prepared_txn_db.cc",
"utilities/transactions/write_unprepared_txn.cc",
"utilities/transactions/write_unprepared_txn_db.cc",
"utilities/trie_index/bitvector.cc",
"utilities/trie_index/louds_trie.cc",
"utilities/trie_index/trie_index_factory.cc",
"utilities/ttl/db_ttl_impl.cc",
"utilities/types_util.cc",
"utilities/wal_filter.cc",
@@ -369,10 +378,10 @@ cpp_library_wrapper(name="rocksdb_lib", srcs=[
"utilities/write_batch_with_index/write_batch_with_index_internal.cc",
], deps=[
"//folly/container:f14_hash",
"//folly/experimental/coro:blocking_wait",
"//folly/experimental/coro:collect",
"//folly/experimental/coro:coroutine",
"//folly/experimental/coro:task",
"//folly/coro:blocking_wait",
"//folly/coro:collect",
"//folly/coro:coroutine",
"//folly/coro:task",
"//folly/synchronization:distributed_mutex",
], headers=glob(["**/*.h"]), link_whole=False, extra_test_libs=False)
@@ -4799,6 +4808,12 @@ cpp_unittest_wrapper(name="db_blob_corruption_test",
extra_compiler_flags=[])
cpp_unittest_wrapper(name="db_blob_direct_write_test",
srcs=["db/blob/db_blob_direct_write_test.cc"],
deps=[":rocksdb_test_lib"],
extra_compiler_flags=[])
cpp_unittest_wrapper(name="db_blob_index_test",
srcs=["db/blob/db_blob_index_test.cc"],
deps=[":rocksdb_test_lib"],
@@ -4937,6 +4952,12 @@ cpp_unittest_wrapper(name="db_merge_operator_test",
extra_compiler_flags=[])
cpp_unittest_wrapper(name="db_open_with_config_test",
srcs=["db/db_open_with_config_test.cc"],
deps=[":rocksdb_test_lib"],
extra_compiler_flags=[])
cpp_unittest_wrapper(name="db_options_test",
srcs=["db/db_options_test.cc"],
deps=[":rocksdb_test_lib"],
@@ -5027,6 +5048,12 @@ cpp_unittest_wrapper(name="db_wide_basic_test",
extra_compiler_flags=[])
cpp_unittest_wrapper(name="db_wide_blob_direct_write_test",
srcs=["db/wide/db_wide_blob_direct_write_test.cc"],
deps=[":rocksdb_test_lib"],
extra_compiler_flags=[])
cpp_unittest_wrapper(name="db_with_timestamp_basic_test",
srcs=["db/db_with_timestamp_basic_test.cc"],
deps=[":rocksdb_test_lib"],
@@ -5137,6 +5164,12 @@ cpp_unittest_wrapper(name="faiss_ivf_index_test",
extra_compiler_flags=[])
cpp_unittest_wrapper(name="fault_injection_fs_test",
srcs=["utilities/fault_injection_fs_test.cc"],
deps=[":rocksdb_test_lib"],
extra_compiler_flags=[])
cpp_unittest_wrapper(name="fault_injection_test",
srcs=["db/fault_injection_test.cc"],
deps=[":rocksdb_test_lib"],
@@ -5521,6 +5554,12 @@ cpp_unittest_wrapper(name="slice_transform_test",
extra_compiler_flags=[])
cpp_unittest_wrapper(name="sorted_run_builder_test",
srcs=["utilities/sorted_run_builder/sorted_run_builder_test.cc"],
deps=[":rocksdb_test_lib"],
extra_compiler_flags=[])
cpp_unittest_wrapper(name="sst_dump_test",
srcs=["tools/sst_dump_test.cc"],
deps=[":rocksdb_test_lib"],
@@ -5623,6 +5662,18 @@ cpp_unittest_wrapper(name="transaction_test",
extra_compiler_flags=[])
cpp_unittest_wrapper(name="trie_index_db_test",
srcs=["utilities/trie_index/trie_index_db_test.cc"],
deps=[":rocksdb_test_lib"],
extra_compiler_flags=[])
cpp_unittest_wrapper(name="trie_index_test",
srcs=["utilities/trie_index/trie_index_test.cc"],
deps=[":rocksdb_test_lib"],
extra_compiler_flags=[])
cpp_unittest_wrapper(name="ttl_test",
srcs=["utilities/ttl/ttl_test.cc"],
deps=[":rocksdb_test_lib"],
@@ -5725,6 +5776,12 @@ cpp_unittest_wrapper(name="write_controller_test",
extra_compiler_flags=[])
cpp_unittest_wrapper(name="write_prepared_transaction_seqno_test",
srcs=["utilities/transactions/write_prepared_transaction_seqno_test.cc"],
deps=[":rocksdb_test_lib"],
extra_compiler_flags=[])
cpp_unittest_wrapper(name="write_prepared_transaction_test",
srcs=["utilities/transactions/write_prepared_transaction_test.cc"],
deps=[":rocksdb_test_lib"],
+331
View File
@@ -0,0 +1,331 @@
# RocksDB Code Generation and Review Guidance
This document provides guidance for generating and reviewing code in the RocksDB project, derived from analysis of code review feedback across hundreds of complex merged Pull Requests. Use this as a reference when writing code with AI assistants or conducting code reviews.
---
## General Best Practices
### Code Quality and Maintainability
**Clarity and Readability:** Write clear, self-documenting code. Use meaningful variable names, add comments for complex logic, and structure code to minimize cognitive load. Avoid clever tricks that sacrifice readability for marginal performance gains unless absolutely necessary.
**Consistent Style:** Follow existing code style conventions. RocksDB uses `.clang-format` for formatting, specific naming conventions, and structural patterns. Deviations from these patterns are frequently flagged in reviews.
**Error Handling:** Ensure robust error handling throughout the codebase. Use RocksDB's `Status` type consistently, propagate errors appropriately, and avoid silently ignoring failures. Reviewers pay close attention to edge cases and failure modes.
### Testing Philosophy
**Comprehensive Coverage:** Every change should include appropriate test coverage. This includes unit tests for isolated functionality, integration tests for component interactions, and stress tests for concurrency and performance validation. Reviewers will ask for additional tests if coverage is insufficient.
**Edge Cases and Failure Modes:** Tests should explicitly cover edge cases, boundary conditions, and potential failure scenarios. This is especially important for changes affecting core database operations, compaction, or recovery logic.
**Platform-Specific Testing:** RocksDB supports multiple platforms (Linux, Windows, macOS) and compilers (GCC, Clang, MSVC). Changes should be tested across relevant platforms, particularly when touching platform-specific code or using compiler-specific features.
### Performance Considerations
**⚠️ PERFORMANCE IS CRITICAL:** RocksDB is a high-performance storage engine where every CPU cycle and memory access matters. When writing code, always evaluate from a performance perspective. This is not optional—performance-aware coding is a fundamental requirement for all contributions.
**Benchmarking and Profiling:** Performance claims should be backed by empirical evidence. Use RocksDB's benchmarking tools (e.g., `db_bench`) to validate improvements. Reviewers will request benchmark results for changes that could impact performance.
**Memory Allocation:** Minimize dynamic memory allocations, especially in hot paths. Prefer stack allocation over heap allocation. Reuse buffers when possible. Consider using arena allocators or memory pools for frequent small allocations. Every `new`, `malloc`, or container resize has a cost.
**Memory Copy:** Avoid unnecessary memory copies. Use move semantics, `std::string_view`, `Slice`, and pass-by-reference where appropriate. Be aware of implicit copies in STL containers and function returns. Prefer in-place operations over copy-and-modify patterns.
**CPU Cache Efficiency:** Design data structures and access patterns to be cache-friendly. Keep frequently accessed data together (data locality). Prefer sequential memory access over random access. Be mindful of cache line sizes (typically 64 bytes) and avoid false sharing in concurrent code. Consider struct packing and field ordering to improve cache utilization.
**Loop Optimization:** Look for opportunities to collapse nested loops, reduce loop overhead, and minimize branch mispredictions. Hoist invariant computations out of loops. Consider loop unrolling for tight inner loops. Batch operations when possible to amortize per-operation overhead.
**SIMD and Vectorization:** Leverage SIMD instructions (SSE, AVX) for data-parallel operations when appropriate. Structure data to enable auto-vectorization by the compiler. Consider explicit SIMD intrinsics for critical hot paths like checksum computation, encoding/decoding, and bulk data processing.
**Branch Prediction:** Minimize unpredictable branches in hot paths. Use `LIKELY`/`UNLIKELY` macros to hint branch prediction. Consider branchless alternatives for simple conditionals. Order switch cases and if-else chains by frequency.
**Memory and Resource Management:** Be mindful of memory allocations, especially in hot paths. Use RAII patterns, smart pointers, and RocksDB's memory management utilities appropriately.
**Hot Path Analysis:** When deciding how aggressively to optimize code, consider whether it's on a hot path:
- **Hot path** (executed thousands+ times, e.g., data access, iteration, compaction loops): Performance is paramount. Apply all optimization techniques—loop collapsing, SIMD, cache optimization, pre-allocation, etc. The cost of each operation is multiplied by execution frequency.
- **Cold path** (executed rarely, e.g., DB open, configuration parsing, error handling): Maintainability and clarity are more important. Prefer readable code over micro-optimizations. Complex optimizations here add maintenance burden with negligible performance benefit.
- **Warm path** (moderate frequency): Balance both concerns. Use profiling data to guide optimization decisions.
**Avoid Premature Optimization:** While performance is critical, focus on correctness first, then optimize based on profiling data. However, be performance-aware from the start—choosing the right algorithm and data structure upfront is not premature optimization. Use the hot path analysis above to decide how much optimization effort is warranted.
### API Design and Compatibility
**Backwards Compatibility:** RocksDB maintains strong backwards compatibility guarantees. Breaking changes are rare and require extensive justification. When deprecating features, follow the project's deprecation policy (typically spanning multiple releases).
**API Consistency:** New APIs should be consistent with existing patterns. Use similar naming conventions, parameter ordering, and return types. Reviewers will suggest changes to improve consistency with the broader codebase.
**Documentation:** Public APIs must be thoroughly documented. Include usage examples, parameter descriptions, and notes on thread safety, performance characteristics, and compatibility considerations.
---
## Component-Specific Guidance
### Database Core (`db`)
The database core handles write-ahead logging (WAL), memtables, compaction, and recovery. This component receives the most scrutiny in code reviews.
**Concurrency and Thread Safety:** Database operations are highly concurrent. Reviewers carefully examine locking strategies, atomic operations, and memory ordering. Document synchronization assumptions clearly. Use appropriate memory ordering semantics (`acquire`/`release` vs. `seq_cst`).
**Compaction Logic:** Changes to compaction are complex and high-risk. Ensure that compaction logic respects configured parameters, handles edge cases (empty databases, single-file compactions), and maintains correctness under concurrent operations.
**Error Propagation:** Database operations can fail in many ways (I/O errors, corruption, resource exhaustion). Ensure that errors are properly propagated, logged, and handled. Avoid assertions in production code paths.
**Testing:** Database core changes require extensive testing, including unit tests, integration tests, and stress tests. Test with various configurations, compaction styles, and concurrent workloads.
### Public Headers (`include`)
Public headers define RocksDB's API surface. Changes here have the highest compatibility impact.
**API Design:** New APIs should be intuitive, consistent with existing patterns, and well-documented. Consider how the API will be used in practice and avoid adding unnecessary complexity.
**Backwards Compatibility:** Breaking changes to public APIs require extensive justification and a deprecation plan. Maintain ABI compatibility for bug fixes and patch releases.
**Documentation:** Every public API must be thoroughly documented with usage examples, parameter descriptions, and notes on thread safety and performance characteristics.
**Deprecation:** When deprecating APIs, follow the project's policy. Mark deprecated APIs clearly, provide migration guidance, and maintain support for at least one major release.
### Internal Utilities (`util`)
Internal utilities provide common functionality used throughout the codebase.
**Code Reuse:** Utilities should be general-purpose and reusable. Avoid duplicating functionality that already exists elsewhere in the codebase.
**Error Handling:** Utility functions should handle errors robustly and propagate them appropriately. Consider edge cases like overflow, underflow, and invalid inputs.
**Testing:** Utility functions should have comprehensive test coverage, including edge cases and failure modes. Consider adding death tests for assertions.
**Performance:** Utilities are often used in hot paths. Ensure that implementations are efficient and avoid unnecessary allocations or copies.
### Table Management (`table`)
Table management handles SST file format, block-based tables, and table readers/writers.
**Block Format and Checksums:** Changes to block format require extreme care. Ensure that checksums are computed and verified correctly. Test with various compression algorithms and block sizes.
**Iterator Correctness:** Table iterators are used throughout the codebase. Ensure that iterator semantics (Seek, Next, Prev) are correct, especially at boundaries and with deletions.
**Caching and Prefetching:** Table readers interact with the block cache and prefetching logic. Ensure that cache keys are unique and that prefetching respects configured limits.
**Performance:** Table operations are performance-critical. Benchmark changes that could impact read or write performance.
### Utilities (`utilities`)
Utilities include optional features like transactions, backup engine, and checkpoint.
**Feature Isolation:** Utilities should be self-contained and not introduce unnecessary dependencies on core database internals.
**Deprecation and Cleanup:** Legacy features are being phased out. When removing deprecated code, ensure that migration paths are documented and that users have sufficient warning.
**Cross-Platform Compatibility:** Utilities often interact with OS-specific APIs. Ensure that code works on all supported platforms.
### Options and Configuration (`options`)
Options define RocksDB's configuration system.
**Type Safety:** Use appropriate types for options (e.g., `uint32_t` for flags, scoped enums for enumerated values).
**Deprecation Policy:** When deprecating options, follow the project's policy. Document the deprecation, provide migration guidance, and maintain support for at least one major release.
**Dynamic Configuration:** Some options can be changed dynamically. Ensure that dynamic changes are thread-safe and take effect correctly.
**Validation:** Validate option values and provide clear error messages for invalid configurations.
### Cache (`cache`)
Cache management is critical for RocksDB's performance.
**Concurrency:** Cache operations are highly concurrent. Ensure that implementations are thread-safe and use appropriate synchronization primitives.
**Performance:** Cache operations are in the hot path. Optimize for low latency and high throughput. Benchmark changes carefully.
**Memory Management:** Cache implementations must manage memory carefully to avoid leaks and excessive allocations.
**Eviction Policies:** Changes to eviction policies should be well-tested and benchmarked to ensure they improve overall performance.
---
## Code Review Checklist
When reviewing RocksDB code (or preparing code for review), use this checklist:
### Correctness
- [ ] Does the change preserve database semantics (e.g., snapshot isolation, key ordering)?
- [ ] Are all error cases handled appropriately?
- [ ] Is the change thread-safe? Are synchronization primitives used correctly?
- [ ] Are there any potential data races or deadlocks?
### Testing
- [ ] Does the change include appropriate test coverage?
- [ ] Are edge cases and failure modes tested?
- [ ] Have the tests been run on all supported platforms?
- [ ] Are stress tests passing?
### Performance
- [ ] Are there benchmark results for performance-sensitive changes?
- [ ] Does the change avoid unnecessary allocations or copies?
- [ ] Are hot paths optimized appropriately?
### API and Compatibility
- [ ] Is the change backwards compatible?
- [ ] Are new APIs consistent with existing patterns?
- [ ] Is the public API documented?
- [ ] Are deprecated features handled according to policy?
### Code Quality
- [ ] Does the code follow RocksDB's style conventions?
- [ ] Is the code clear and maintainable?
- [ ] Are comments and documentation sufficient?
- [ ] Are there any code smells or anti-patterns?
---
## Common Review Feedback Patterns
The following patterns emerged as frequent sources of review feedback:
1. **Test Coverage:** Reviewers frequently request additional tests for edge cases, platform-specific behavior, and failure modes. Complex changes require comprehensive test coverage including unit tests, integration tests, and stress tests.
2. **Error Handling:** Ensure proper error propagation using RocksDB's `Status` type. Avoid silent failures and provide clear error messages that include context about what failed and why.
3. **API Design:** New APIs should be consistent with existing patterns. Use descriptive names that follow established conventions. Avoid breaking changes without strong justification and a clear deprecation plan.
4. **Documentation:** Public APIs must be documented with usage examples and notes on thread safety, performance characteristics, and compatibility considerations. Complex internal logic should also be well-commented.
5. **Performance:** Performance-sensitive changes require benchmark results to validate improvements. Use `db_bench` and other profiling tools to measure impact. Avoid premature optimization that adds complexity without measurable benefit.
6. **Concurrency:** Thread safety is critical in RocksDB. Document synchronization assumptions clearly. Use appropriate memory ordering semantics. Consider potential race conditions and deadlocks.
7. **Code Style:** Follow existing conventions for naming, formatting, and structure. Use `.clang-format` for consistent formatting. Prefer scoped enums (`enum class`) over unscoped enums.
8. **Backwards Compatibility:** RocksDB maintains strong compatibility guarantees. Breaking changes require extensive justification. When deprecating features, provide migration guidance and maintain support across multiple releases.
9. **Refactoring:** Reviewers appreciate refactoring that improves code readability and maintainability. Look for opportunities to deduplicate code and simplify complex logic.
10. **Platform Compatibility:** Ensure changes work correctly on all supported platforms (Linux, Windows, macOS) and with all supported compilers (GCC, Clang, MSVC).
---
## Important tips
### Build system
* There are 3 build system. Make, CMake, BUCK(meta internal).
* When a new .cc file is added, update Makefile, CMakeLists.txt, src.mk, BUCK.
* Don't manually edit BUCK file, after updating src.mk, run
/usr/local/bin/python3 buckifier/buckify_rocksdb.py to update it
* Use make to build and run the test. CMake and BUCK are not used locally.
* Use `make dbg` command to build all of the unit test in debug mode.
* For -j in make command, use the number of CPU cores to decide it.
### Unit Test
* After all of the unit tests are added, review them and try to extract common
reusable utility functions to reduce code duplication due to copy past between
unit tests. This should be done every time unit test is updated.
* Don't use sleep to wait for certain events to happen. This will cause test to
be flaky. Instead, use sync point to synchronize thread progress.
* Cap unit test execution with 60 seconds timeout.
* When there are multiple unit tests need to be executed, try to use
gtest_parallel.py if available. E.g.
python3 ${GTEST_PARALLEL}/gtest_parallel.py ./table_test
* After writing a test, stress-test for flakiness:
```bash
COERCE_CONTEXT_SWITCH=1 make {test_binary}
./{test_binary} --gtest_filter="*YourTestName*" --gtest_repeat=5
```
### Unit test dedup guidelines
* Extract helper functions for repeated patterns such as object
construction, round-trip (encode → decode → verify), and common
assertion sequences.
* Use table-driven tests (struct array + loop) when multiple test cases
share the same logic but differ only in input/expected data.
* Prefer randomized tests over exhaustive parameter permutations. Use
`Random` from `util/random.h` (not `std::mt19937`). Use a time-based
seed with `SCOPED_TRACE("seed=" + std::to_string(seed))` so failures
are reproducible.
* Keep deterministic edge-case tests separate from randomized tests
(error paths, boundary conditions, format verification).
* Methods only used in tests should be private with `friend class` +
`TEST_F` fixture wrappers. In wrappers, always fully qualify the
target method to avoid infinite recursion.
### Adding new public API
Refer to claude_md/add_public_api.md
### Adding new option
Refer to claude_md/add_option.md
### Removing deprecated option
Refer to claude_md/remove_option.md
### Metrics
* When adding a new feature, evaluate whether there is opportunity to add
metrics. Try to avoid causing performance regression on hot path when adding
metrics.
### Stress test
* When adding a new feature, make sure stress test covers the new option.
### Component docs
* For component-level design notes and implementation walkthroughs, start with
`docs/components/index.md`.
* Documentation under `docs/components/` is organized by subsystem in
`docs/components/<area>/`.
* Each subsystem directory should have an `index.md` entry point plus focused
chapter files for deeper topics.
### DB bench update
* When adding a performance related feature, support it in db_bench
### Adding release note
* Release note should be kept short at high level for external user consumption.
### Blog posts (docs/_posts)
* Blog post authors must be defined in `docs/_data/authors.yml` to be displayed
### Final verification of the change
* Execute make clean to clean all of the changes.
* Execute make check to build all of the changes and execute all of the tests.
Note that executing all of the tests could take multiple minutes.
* Run `ASSERT_STATUS_CHECKED=1 make check` to verify all Status objects are
properly checked. This catches missing error handling that can lead to
silent data corruption.
### Monitoring make check progress
* Use `make check-progress` to get machine-parseable JSON progress while
`make check` is running. This is useful for Claude Code to monitor long
builds without timeout issues.
* Run `make check` in background, then poll progress:
```bash
make check &
# Poll periodically:
make check-progress
```
* The output shows current phase and progress:
```json
{"status":"running","phase":"compiling","completed":300,"total":919,...}
{"status":"running","phase":"testing","completed":1500,"total":29962,"failed":0,"percent":5,...}
{"status":"completed","phase":"testing","completed":29962,"total":29962,"failed":0,"percent":100,...}
```
* Phases: `compiling` -> `linking` -> `generating` -> `testing` -> `completed`
* Key fields: `status`, `phase`, `completed`, `total`, `failed`, `percent`
* When tests fail, `failed_tests` array shows details (up to 10 failures):
```json
{"status":"running",...,"failed":3,"failed_tests":[
{"test":"cache_test-CacheTest.Usage","exit_code":1,"signal":0,"output":"...test log..."},
{"test":"env_test-EnvTest.Open","exit_code":0,"signal":11,"output":"...Segmentation fault..."}
]}
```
* `exit_code`: non-zero means test assertion failed
* `signal`: non-zero means test was killed (e.g., 9=SIGKILL, 6=SIGABRT, 11=SIGSEGV)
* `output`: last 50 lines of test log including error messages and stack traces
### Executing benchmark using db_bench
* Since the goal is to measure performance, we need to build a release binary
using `make clean && DEBUG_LEVEL=0 make db_bench`. If there is an engine
crash due to bug, we need to switch back to debug build. Make sure to run
`make clean` before running `make dbg`.
### Formatting code
* After making change, use `make format-auto` to auto-apply formatting without
interactive prompts (Claude Code friendly).
+78 -4
View File
@@ -82,10 +82,15 @@ if(NOT CMAKE_BUILD_TYPE)
endif()
message(STATUS "CMAKE_BUILD_TYPE is set to ${CMAKE_BUILD_TYPE}")
# Use ccache for compilation if available. Use CMAKE_C/CXX_COMPILER_LAUNCHER
# instead of RULE_LAUNCH_COMPILE to avoid double-wrapping when ccache is also
# injected via PATH (e.g., /usr/lib/ccache or brew --prefix ccache/libexec).
# Note: we intentionally do NOT set RULE_LAUNCH_LINK because ccache cannot
# cache link operations -- it only adds overhead.
find_program(CCACHE_FOUND ccache)
if(CCACHE_FOUND)
set_property(GLOBAL PROPERTY RULE_LAUNCH_COMPILE ccache)
set_property(GLOBAL PROPERTY RULE_LAUNCH_LINK ccache)
set(CMAKE_C_COMPILER_LAUNCHER ccache)
set(CMAKE_CXX_COMPILER_LAUNCHER ccache)
endif(CCACHE_FOUND)
option(WITH_JEMALLOC "build with JeMalloc" OFF)
@@ -515,6 +520,15 @@ elseif(CMAKE_SYSTEM_NAME MATCHES "iOS")
add_definitions(-DOS_MACOSX -DIOS_CROSS_COMPILE)
elseif(CMAKE_SYSTEM_NAME MATCHES "Linux")
add_definitions(-DOS_LINUX)
# Use lld linker if available for faster linking (12x faster than ld.bfd)
if(NOT ROCKSDB_NO_FAST_LINKER)
execute_process(COMMAND ${CMAKE_CXX_COMPILER} -fuse-ld=lld -Wl,--version
OUTPUT_VARIABLE LLD_VERSION_OUTPUT ERROR_QUIET RESULT_VARIABLE LLD_RESULT)
if(LLD_RESULT EQUAL 0 AND LLD_VERSION_OUTPUT MATCHES "LLD")
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -fuse-ld=lld")
set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -fuse-ld=lld")
endif()
endif()
elseif(CMAKE_SYSTEM_NAME MATCHES "SunOS")
add_definitions(-DOS_SOLARIS)
elseif(CMAKE_SYSTEM_NAME MATCHES "kFreeBSD")
@@ -660,10 +674,38 @@ if(USE_FOLLY)
endif()
endif()
# Folly itself uses gflags transitively, but RocksDB tools and benchmarks
# also call gflags APIs directly, so they need an explicit link dependency.
if(WITH_GFLAGS)
if(NOT TARGET gflags::gflags AND NOT TARGET gflags_shared AND
NOT gflags_LIBRARIES)
find_package(gflags CONFIG QUIET)
if(NOT gflags_FOUND)
find_package(gflags REQUIRED)
endif()
endif()
if(TARGET gflags::gflags)
set(GFLAGS_LIB gflags::gflags)
elseif(TARGET gflags_shared)
set(GFLAGS_LIB gflags_shared)
elseif(DEFINED GFLAGS_TARGET AND TARGET ${GFLAGS_TARGET})
set(GFLAGS_LIB ${GFLAGS_TARGET})
elseif(gflags_LIBRARIES)
set(GFLAGS_LIB ${gflags_LIBRARIES})
else()
message(FATAL_ERROR
"WITH_GFLAGS is enabled, but no gflags library could be resolved")
endif()
list(APPEND THIRDPARTY_LIBS ${GFLAGS_LIB})
endif()
add_compile_definitions(USE_FOLLY FOLLY_NO_CONFIG HAVE_CXX11_ATOMIC)
list(APPEND THIRDPARTY_LIBS Folly::folly)
set(FOLLY_LIBS Folly::folly)
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -Wl,--copy-dt-needed-entries")
# --copy-dt-needed-entries is ld.bfd-specific; lld handles DT_NEEDED transitively
if(NOT CMAKE_EXE_LINKER_FLAGS MATCHES "fuse-ld=lld")
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -Wl,--copy-dt-needed-entries")
endif()
endif()
find_package(Threads REQUIRED)
@@ -690,6 +732,7 @@ set(SOURCES
db/blob/blob_file_addition.cc
db/blob/blob_file_builder.cc
db/blob/blob_file_cache.cc
db/blob/blob_file_partition_manager.cc
db/blob/blob_file_garbage.cc
db/blob/blob_file_meta.cc
db/blob/blob_file_reader.cc
@@ -698,6 +741,7 @@ set(SOURCES
db/blob/blob_log_sequential_reader.cc
db/blob/blob_log_writer.cc
db/blob/blob_source.cc
db/blob/blob_write_batch_transformer.cc
db/blob/prefetch_buffer_collection.cc
db/builder.cc
db/c.cc
@@ -766,8 +810,10 @@ set(SOURCES
db/version_edit.cc
db/version_edit_handler.cc
db/version_set.cc
db/version_util.cc
db/wal_edit.cc
db/wal_manager.cc
db/wide/read_path_blob_resolver.cc
db/wide/wide_column_serialization.cc
db/wide/wide_columns.cc
db/wide/wide_columns_helper.cc
@@ -842,6 +888,7 @@ set(SOURCES
table/block_based/block_based_table_builder.cc
table/block_based/block_based_table_factory.cc
table/block_based/block_based_table_iterator.cc
table/block_based/multi_scan_index_iterator.cc
table/block_based/block_based_table_reader.cc
table/block_based/block_builder.cc
table/block_based/block_cache.cc
@@ -948,6 +995,7 @@ set(SOURCES
utilities/cassandra/merge_operator.cc
utilities/checkpoint/checkpoint_impl.cc
utilities/compaction_filters.cc
utilities/sorted_run_builder/sorted_run_builder.cc
utilities/compaction_filters/remove_emptyvalue_compactionfilter.cc
utilities/counted_fs.cc
utilities/debug.cc
@@ -999,6 +1047,9 @@ set(SOURCES
utilities/transactions/write_prepared_txn_db.cc
utilities/transactions/write_unprepared_txn.cc
utilities/transactions/write_unprepared_txn_db.cc
utilities/trie_index/bitvector.cc
utilities/trie_index/louds_trie.cc
utilities/trie_index/trie_index_factory.cc
utilities/types_util.cc
utilities/ttl/db_ttl_impl.cc
utilities/wal_filter.cc
@@ -1114,8 +1165,23 @@ if(USE_FOLLY_LITE)
FMT_INCLUDE_DIR)
include_directories(${FMT_INCLUDE_DIR})
exec_program(python3 ${PROJECT_SOURCE_DIR}/third-party/folly ARGS
build/fbcode_builder/getdeps.py show-inst-dir glog OUTPUT_VARIABLE
GLOG_INST_PATH)
if(EXISTS ${GLOG_INST_PATH}/lib64)
set(GLOG_LIB_DIR ${GLOG_INST_PATH}/lib64)
else()
set(GLOG_LIB_DIR ${GLOG_INST_PATH}/lib)
endif()
add_definitions(-DUSE_FOLLY -DFOLLY_NO_CONFIG)
list(APPEND THIRDPARTY_LIBS glog)
find_library(GLOG_LIBRARY NAMES glog PATHS ${GLOG_LIB_DIR} NO_DEFAULT_PATH)
if(NOT GLOG_LIBRARY)
find_library(GLOG_LIBRARY NAMES glog)
endif()
if(GLOG_LIBRARY)
list(APPEND THIRDPARTY_LIBS ${GLOG_LIBRARY})
endif()
endif()
set(ROCKSDB_STATIC_LIB rocksdb${ARTIFACT_SUFFIX})
@@ -1361,6 +1427,7 @@ if(WITH_TESTS)
db/blob/blob_garbage_meter_test.cc
db/blob/blob_source_test.cc
db/blob/db_blob_basic_test.cc
db/blob/db_blob_direct_write_test.cc
db/blob/db_blob_compaction_test.cc
db/blob/db_blob_corruption_test.cc
db/blob/db_blob_index_test.cc
@@ -1398,6 +1465,7 @@ if(WITH_TESTS)
db/db_memtable_test.cc
db/db_merge_operator_test.cc
db/db_merge_operand_test.cc
db/db_open_with_config_test.cc
db/db_options_test.cc
db/db_properties_test.cc
db/db_range_del_test.cc
@@ -1450,6 +1518,7 @@ if(WITH_TESTS)
db/wal_manager_test.cc
db/wal_edit_test.cc
db/wide/db_wide_basic_test.cc
db/wide/db_wide_blob_direct_write_test.cc
db/wide/wide_column_serialization_test.cc
db/wide/wide_columns_helper_test.cc
db/write_batch_test.cc
@@ -1530,7 +1599,9 @@ if(WITH_TESTS)
utilities/cassandra/cassandra_row_merge_test.cc
utilities/cassandra/cassandra_serialize_test.cc
utilities/checkpoint/checkpoint_test.cc
utilities/sorted_run_builder/sorted_run_builder_test.cc
utilities/env_timed_test.cc
utilities/fault_injection_fs_test.cc
utilities/memory/memory_test.cc
utilities/merge_operators/string_append/stringappend_test.cc
utilities/object_registry_test.cc
@@ -1548,9 +1619,12 @@ if(WITH_TESTS)
utilities/transactions/lock/point/point_lock_manager_stress_test.cc
utilities/transactions/write_committed_transaction_ts_test.cc
utilities/transactions/write_prepared_transaction_test.cc
utilities/transactions/write_prepared_transaction_seqno_test.cc
utilities/transactions/write_unprepared_transaction_test.cc
utilities/transactions/lock/range/range_locking_test.cc
utilities/transactions/timestamped_snapshot_test.cc
utilities/trie_index/trie_index_db_test.cc
utilities/trie_index/trie_index_test.cc
utilities/ttl/ttl_test.cc
utilities/types_util_test.cc
utilities/util_merge_operators_test.cc
+64
View File
@@ -1,6 +1,70 @@
# Rocksdb Change Log
> NOTE: Entries for next release do not go here. Follow instructions in `unreleased_history/README.txt`
## 11.2.0 (04/18/2026)
### New Features
* Added experimental `DBOptions::fast_sst_open` option. When enabled, RocksDB retrieves opaque file system metadata for SST files after flush, compaction, and external file ingestion, persists it in the MANIFEST, and passes it back to the file system on subsequent file opens to accelerate DB open time.
* Added new option `min_tombstones_for_range_conversion` in `AdvancedColumnFamilyOptions`. When set to a non-zero value N, forward or reverse iteration will convert N or more contiguous point tombstones into a range tombstone in the mutable memtable. Future read operations will then be able to benefit from range tombstone optimizations. There are some limitations when it comes to table_filters, prefix_filters, and UDTs. See header comments for more details.
* Added read-triggered compaction: a new column family option `read_triggered_compaction_threshold` (default 0, disabled) that marks SST files for compaction when their read frequency (`num_collapsible_entry_reads_sampled / file_size`) exceeds the threshold. This helps reduce read amplification for frequently-read ("hot") keys. A new DB option `max_compaction_trigger_wakeup_seconds` (default 43200s / 12 hours) controls the maximum interval for periodic compaction score re-evaluation, which is necessary for this feature to work on quiet (no-write) databases.
### Public API Changes
* Added new `WideColumnBlobResolver` interface and `CompactionFilter::FilterV4()` method, including `ResolveColumn()` / `ResolveColumns()` helpers for lazy loading blob column values in wide-column entities during compaction. This allows compaction filters to resolve blob values on-demand, avoiding unnecessary I/O for blob columns they don't need to access.
* Changed experimental feature `ExternalTableReader::Get` and `ExternalTableReader::MultiGet` to use `PinnableSlice` instead of `std::string` for output values, enabling zero-copy pinning. This will break existing implementations.
* Added `SstFileReader::Get` and `SstFileReader::MultiGet` overloads that accept `PinnableSlice`/`std::vector<PinnableSlice>*`, enabling zero-copy reads when the underlying `TableReader` supports pinning.
### Behavior Changes
* Prefix filter changes - when seeking to a key that is out of domain, and total_order_seek is false, total_order_seek is treated as if it were true. When prefix_same_as_start = true, now iterating past a key that is out of domain invalidates the iterator. The existing behavior does not check for InDomain in the DBIter, so Transform() can produce undefined behavior (e.g. key of size 3 on FixedPrefixTransform(4)).
* Wide-column entities with blob-backed columns now use a new V2 on-disk encoding; older RocksDB versions that do not support wide-column blob separation will reject DBs or SSTs containing those entities.
### Bug Fixes
* Fix blob garbage accounting for blob direct-write flushes so flush-time filtering and overwrite elision correctly register obsolete blob bytes in blob metadata.
* Fix a memory accounting leak in IODispatcher where ReadIndex() moved block values out of ReadSet without releasing the associated prefetch memory, causing subsequent prefetches to be blocked when max_prefetch_memory_bytes was set.
* Fix MultiScan to fall back to synchronous coalesced reads when async I/O is unsupported at runtime.
## 11.1.0 (03/23/2026)
### New Features
* Add a new option `open_files_async`. The existing behavior is on DB open, we open all sst files and do basic validations. For very large DBs on remote filesystems with many ssts, this may take very long. This option performs these validations instead in the background. Open errors found by this async background task are surfaced as a new background error kAsyncFileOpen.
* Added `BlockBasedTableOptions::kAuto` index block search type that automatically selects between binary and interpolation search on a per-index-block basis. During SST construction, each index block's key distribution uniformity is analyzed using the coefficient of variation of key gaps, and index blocks with uniform keys use interpolation search while others fall back to binary search. The uniformity threshold is configurable via `BlockBasedTableOptions::uniform_cv_threshold` (default: 0.2).
* Introduced enforce_write_buffer_manager_during_recovery option to allow WriteBufferManager to be enforced during WAL recovery. (Default: true)
* Add `memtable_batch_lookup_optimization` option to use batch lookup optimization for memtable MultiGet. For skip list memtables, after each key lookup, the search path is cached and reused for the next key, reducing per-key cost from O(log N) to O(log d) where d is the distance between consecutive keys. Benchmarks show ~7% improvement in memtable-resident MultiGet throughput.
* Added `BlockBasedTableOptions::PrepopulateBlockCache::kFlushAndCompaction` to prepopulate the block cache during both flush and compaction. Compaction-warmed blocks are inserted at `BOTTOM` priority (vs `LOW` for flush) so they are evicted first under cache pressure. Recommended only for use cases where most or all of the database is expected to reside in cache (e.g., tiered or remote storage where the working set fits in cache).
* Added new mutable DB option `verify_manifest_content_on_close` (default: false). When enabled, on DB close the MANIFEST file is read back and all records are validated (CRC checksums and logical content). If corruption is detected, a fresh MANIFEST is written from in-memory state.
### Behavior Changes
* num_reads_sampled now factors in re-seeks and next/prev() on file iterators for files in L1+. next/prev() is discounted by 64x compared to seek due to being a much cheaper call.
* Remote compaction workers now skip WAL recovery when opening the secondary DB instance, since only the LSM state from MANIFEST is needed for compaction. This reduces I/O and speeds up remote compaction startup.
### Bug Fixes
* Fix a bug in round-robin compaction that missed selecting input files that are needed to guarantee data correctness and cause crashing in debug builds or silent data corruption in release builds for Get().
## 11.0.0 (02/23/2026)
### New Features
* Added support for storing wide-column entity column values in blob files. When `min_blob_size` is configured, large column values in wide-column entities will be stored in blob files, reducing SST file size and improving read performance.
* Added `CompactionOptionsFIFO::max_data_files_size` to support FIFO compaction trimming based on combined SST and blob file sizes. Added `CompactionOptionsFIFO::use_kv_ratio_compaction` to enable a capacity-derived intra-L0 compaction strategy optimized for BlobDB workloads, producing uniform-sized compacted files for predictable FIFO trimming.
* Include interpolation search as an alternative to binary search, which typically performs better when keys are uniformly distributed. This is exposed as a new table option `index_block_search_type`. The default is `binary_search`.
### Public API Changes
* Added new virtual methods `AbortAllCompactions()` and `ResumeAllCompactions()` to the `DB` class. Added new `Status::SubCode::kCompactionAborted` to indicate a compaction was aborted. Added `Status::IsCompactionAborted()` helper method to check if a status represents an aborted compaction.
* Drop support for reading (and writing) SST files using `BlockBasedTableOptions.format_version` < 2, which hasn't been the default format for about 10 years. An upgrade path is still possible with full compaction using a RocksDB version >= 4.6.0 and < 11.0.0 and then using the newer version.
* Remove deprecated raw `DB*` variants of `DB::Open` and related functions. Some other minor public APIs were updated as a result
* Remove deprecated `DB::MaxMemCompactionLevel()`
* Remove useless option `CompressedSecondaryCacheOptions::compress_format_version`
* Remove deprecated DB option `skip_checking_sst_file_sizes_on_db_open`. The option was deprecated in 10.5.0 and has been a no-op since then. File size validation is now always performed in parallel during DB open.
* Remove deprecated `SliceTransform::InRange()` virtual method and the `in_range` callback parameter from `rocksdb_slicetransform_create()` in the C API. `InRange()` was never called by RocksDB and existed only for backward compatibility.
* Remove deprecated, unused APIs and options: `ReadOptions::managed` and `ColumnFamilyOptions::snap_refresh_nanos`. Corresponding C and Java APIs are also removed.
* Remove deprecated `SstFileWriter::Add()` method (use `Put()` instead) and the deprecated `skip_filters` parameter from `SstFileWriter` constructors (use `BlockBasedTableOptions::filter_policy` set to `nullptr` to skip filter generation instead).
### Behavior Changes
* Change the default value of `CompactionOptionsUniversal::reduce_file_locking` from `false` to `true` to improve write stall and reduce read regression
### Bug Fixes
* Fix longstanding failures that can arise from reading and/or compacting old DB dirs with range deletions (likely from version < 5.19.0) in many newer versions.
* Fix a bug where WritePrepared/WriteUnprepared TransactionDB with two_write_queues=true could experience "sequence number going backwards" corruption during recovery from a background error, due to allocated-but-not-published sequence numbers not being synced before creating new WAL files.
### Performance Improvements
* Add a new table option `separate_key_value_in_data_block`. When set to true keys and values will be stored separately in the data block, which can result in higher cpu cache hit rate and better compression. Works best with data blocks with sufficient restart intervals and large values. Previous versions of RocksDB will reject files written using this option.
## 10.11.0 (01/23/2026)
### Public API Changes
* New SetOptions API that allows setting options for multiple CFs, avoiding the need to reserialize OPTIONS file for each CF
+135 -23
View File
@@ -450,7 +450,7 @@ ifndef USE_FOLLY
endif
ifndef GTEST_THROW_ON_FAILURE
export GTEST_THROW_ON_FAILURE=1
export GTEST_THROW_ON_FAILURE=0
endif
ifndef GTEST_HAS_EXCEPTIONS
export GTEST_HAS_EXCEPTIONS=1
@@ -621,25 +621,24 @@ endif
ROCKSDBTESTS_SUBSET ?= $(TESTS)
# c_test - doesn't use gtest
# env_test - suspicious use of test::TmpDir
# deletefile_test - serial because it generates giant temporary files in
# its various tests. Parallel can fill up your /dev/shm
# db_bloom_filter_test - serial because excessive space usage by instances
# of DBFilterConstructionReserveMemoryTestWithParam can fill up /dev/shm
# c_test - doesn't use gtest, can't be sharded
# Other tests previously listed here (backup_engine_test, db_bloom_filter_test,
# perf_context_test, etc.) were NON_PARALLEL due to /dev/shm memory concerns
# when the old per-test-case sharding spawned thousands of processes. With the
# new gtest-based sharding (GTEST_SHARD_SIZE=10, max NCORES*8 shards), at most
# ~4 shards run concurrently on CI (4 cores), so peak memory is manageable
# (4 * 1GB = 4GB << 16GB shm).
NON_PARALLEL_TEST = \
c_test \
env_test \
deletefile_test \
db_bloom_filter_test \
$(PLUGIN_TESTS) \
PARALLEL_TEST = $(filter-out $(NON_PARALLEL_TEST), $(TESTS))
PARALLEL_TEST = $(filter-out $(NON_PARALLEL_TEST), $(ROCKSDBTESTS_SUBSET))
# Not necessarily well thought out or up-to-date, but matches old list
TESTS_PLATFORM_DEPENDENT := \
db_basic_test \
db_blob_basic_test \
db_blob_direct_write_test \
db_encryption_test \
external_sst_file_basic_test \
auto_roll_logger_test \
@@ -647,6 +646,7 @@ TESTS_PLATFORM_DEPENDENT := \
dynamic_bloom_test \
c_test \
checkpoint_test \
sorted_run_builder_test \
crc32c_test \
coding_test \
inlineskiplist_test \
@@ -805,9 +805,20 @@ endif # PLATFORM_SHARED_EXT
.PHONY: check clean coverage ldb_tests package dbg gen-pc build_size \
release tags tags0 valgrind_check format static_lib shared_lib all \
rocksdbjavastatic rocksdbjava install install-static install-shared \
uninstall analyze tools tools_lib check-headers checkout_folly
uninstall analyze tools tools_lib check-headers checkout_folly clang-tidy
all: $(LIBRARY) $(BENCHMARKS) tools tools_lib test_libs $(TESTS)
# Auto-configure git hooks on first build so developers do not need to run
# "make install-hooks" manually. This is a no-op if already set.
setup-hooks:
@if [ -d .git ] && [ -d githooks ]; then \
cur=$$(git config core.hooksPath 2>/dev/null); \
if [ "$$cur" != "githooks" ]; then \
git config core.hooksPath githooks; \
echo "git hooks: configured core.hooksPath = githooks"; \
fi; \
fi
all: setup-hooks $(LIBRARY) $(BENCHMARKS) tools tools_lib test_libs $(TESTS)
all_but_some_tests: $(LIBRARY) $(BENCHMARKS) tools tools_lib test_libs $(ROCKSDBTESTS_SUBSET)
@@ -871,21 +882,42 @@ coverage: clean
parallel_tests = $(patsubst %,parallel_%,$(PARALLEL_TEST))
.PHONY: gen_parallel_tests $(parallel_tests)
# Shard size controls how many test cases run per process. The actual number
# of shards per binary is: min(ceil(test_count / GTEST_SHARD_SIZE), NCORES * 8)
# This adapts to machine size: many small shards on beefy machines, fewer
# larger shards on CI (typically 2-4 cores).
GTEST_SHARD_SIZE ?= 10
NCORES ?= $(shell nproc 2>/dev/null || sysctl -n hw.ncpu 2>/dev/null || echo 4)
$(parallel_tests):
$(AM_V_at)TEST_BINARY=$(patsubst parallel_%,%,$@); \
TEST_NAMES=` \
(./$$TEST_BINARY --gtest_list_tests || echo " $${TEST_BINARY}__list_tests_failure") \
| awk '/^[^ ]/ { prefix = $$1 } /^[ ]/ { print prefix $$1 }'`; \
echo " Generating parallel test scripts for $$TEST_BINARY"; \
for TEST_NAME in $$TEST_NAMES; do \
TEST_SCRIPT=t/run-$$TEST_BINARY-$${TEST_NAME//\//-}; \
TEST_COUNT=` \
(./$$TEST_BINARY --gtest_list_tests 2>/dev/null || echo " list_failure") \
| grep -c '^ '`; \
if [ "$$TEST_COUNT" -le 0 ]; then TEST_COUNT=1; fi; \
MAX_SHARDS=$$(( $(NCORES) * 8 )); \
NUM_SHARDS=$$(( (TEST_COUNT + $(GTEST_SHARD_SIZE) - 1) / $(GTEST_SHARD_SIZE) )); \
if [ "$$NUM_SHARDS" -gt "$$MAX_SHARDS" ]; then NUM_SHARDS=$$MAX_SHARDS; fi; \
if [ "$$NUM_SHARDS" -le 0 ]; then NUM_SHARDS=1; fi; \
echo " Generating $$NUM_SHARDS shards for $$TEST_BINARY ($$TEST_COUNT tests)"; \
SHARD_IDX=0; \
while [ "$$SHARD_IDX" -lt "$$NUM_SHARDS" ]; do \
if [ -n "$(CI_TOTAL_SHARDS)" ] && [ $$(( $$SHARD_IDX % $(CI_TOTAL_SHARDS) )) -ne $(CI_SHARD_INDEX) ]; then \
SHARD_IDX=$$((SHARD_IDX + 1)); \
continue; \
fi; \
TEST_SCRIPT=t/run-$$TEST_BINARY-shard-$$SHARD_IDX; \
printf '%s\n' \
'#!/bin/sh' \
"d=\$(TEST_TMPDIR)$$TEST_SCRIPT" \
'mkdir -p $$d' \
"TEST_TMPDIR=\$$d $(DRIVER) ./$$TEST_BINARY --gtest_filter=$$TEST_NAME" \
"TEST_TMPDIR=\$$d GTEST_TOTAL_SHARDS=$$NUM_SHARDS GTEST_SHARD_INDEX=$$SHARD_IDX $(DRIVER) ./$$TEST_BINARY" \
'test_retcode=$$?' \
'[ $$test_retcode -eq 0 ] && rm -rf $$d' \
'exit $$test_retcode' \
> $$TEST_SCRIPT; \
chmod a=rx $$TEST_SCRIPT; \
SHARD_IDX=$$((SHARD_IDX + 1)); \
done
gen_parallel_tests:
@@ -909,8 +941,10 @@ gen_parallel_tests:
# 152.120 PASS t/DBTest.FileCreationRandomFailure
# 107.816 PASS t/DBTest.EncodeDecompressedBlockSizeTest
#
# With sharded test execution, prioritize binaries known to be slow.
# These generate many shards and should start early for good load balancing.
slow_test_regexp = \
^.*MySQLStyleTransactionTest.*$$|^.*SnapshotConcurrentAccessTest.*$$|^.*SeqAdvanceConcurrentTest.*$$|^t/run-table_test-HarnessTest.Randomized$$|^t/run-db_test-.*(?:FileCreationRandomFailure|EncodeDecompressedBlockSizeTest)$$|^.*RecoverFromCorruptedWALWithoutFlush$$
^.*block_based_table_reader_test.*$$|^.*table_test.*$$|^.*block_test.*$$|^.*write_prepared_transaction_test.*$$|^.*transaction_test.*$$|^.*external_sst_file_test.*$$|^.*db_wal_test.*$$|^.*db_with_timestamp_basic_test.*$$|^.*db_test-.*$$
prioritize_long_running_tests = \
perl -pe 's,($(slow_test_regexp)),100 $$1,' \
| sort -k1,1gr \
@@ -945,7 +979,13 @@ check_0:
'To monitor subtest <duration,pass/fail,name>,' \
' run "make watch-log" in a separate window' ''; \
{ \
printf './%s\n' $(filter-out $(PARALLEL_TEST),$(TESTS)); \
NON_PARALLEL_LIST="$(filter-out $(PARALLEL_TEST),$(ROCKSDBTESTS_SUBSET))"; \
if [ -n "$$NON_PARALLEL_LIST" ]; then \
printf './%s\n' $$NON_PARALLEL_LIST \
| if [ -n "$(CI_TOTAL_SHARDS)" ]; then \
awk -v s=$(CI_SHARD_INDEX) -v n=$(CI_TOTAL_SHARDS) '(NR-1)%n==s'; \
else cat; fi; \
fi; \
find t -name 'run-*' -print; \
} \
| $(prioritize_long_running_tests) \
@@ -1001,6 +1041,11 @@ check-progress:
# If J != 1 and GNU parallel is installed, run the tests in parallel,
# via the check_0 rule above. Otherwise, run them sequentially.
check: all
$(AM_V_at)echo "Cleaning up stale test directories older than 3 hours..."; \
test_tmpdir_parent=$$(dirname $(TEST_TMPDIR)); \
find $$test_tmpdir_parent -maxdepth 1 -name 'rocksdb.*' -type d \
-mmin +180 -exec rm -rf {} + 2>/dev/null; \
true
$(MAKE) gen_parallel_tests
$(AM_V_GEN)if test "$(J)" != 1 \
&& (build_tools/gnu_parallel --gnu --help 2>/dev/null) | \
@@ -1016,6 +1061,7 @@ ifneq ($(PLATFORM), OS_AIX)
$(PYTHON) tools/check_all_python.py
ifndef ASSERT_STATUS_CHECKED # not yet working with these tests
$(PYTHON) tools/ldb_test.py
$(PYTHON) tools/db_crashtest_test.py
sh tools/rocksdb_dump_test.sh
endif
endif
@@ -1023,6 +1069,7 @@ ifndef SKIP_FORMAT_BUCK_CHECKS
$(MAKE) check-format
$(MAKE) check-buck-targets
$(MAKE) check-sources
$(MAKE) check-workflow-yaml
endif
# TODO add ldb_tests
@@ -1033,6 +1080,10 @@ check_some: $(ROCKSDBTESTS_SUBSET)
ldb_tests: ldb
$(PYTHON) tools/ldb_test.py
.PHONY: db_crashtest_tests
db_crashtest_tests:
$(PYTHON) tools/db_crashtest_test.py
include crash_test.mk
asan_check: clean
@@ -1216,12 +1267,49 @@ format-auto:
check-format:
build_tools/format-diff.sh -c
install-hooks:
@echo "Installing git hooks from githooks/..."
@if [ -d githooks ]; then \
for hook in githooks/*; do \
hook_name=$$(basename "$$hook"); \
cp "$$hook" .git/hooks/"$$hook_name"; \
chmod +x .git/hooks/"$$hook_name"; \
echo " Installed $$hook_name"; \
done; \
echo "Done. Hooks installed to .git/hooks/"; \
else \
echo "Error: githooks/ directory not found"; \
exit 1; \
fi
uninstall-hooks:
@echo "Removing installed git hooks..."
@for hook in githooks/*; do \
hook_name=$$(basename "$$hook"); \
rm -f .git/hooks/"$$hook_name"; \
echo " Removed $$hook_name"; \
done
@echo "Done."
check-buck-targets:
buckifier/check_buck_targets.sh
check-sources:
build_tools/check-sources.sh
check-workflow-yaml:
build_tools/check-workflow-yaml.sh
# Run clang-tidy on locally changed files, filtered to changed lines only.
# Requires compile_commands.json (generate with cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=ON).
# Override CLANG_TIDY_BINARY and CLANG_TIDY_JOBS as needed:
# make clang-tidy CLANG_TIDY_BINARY=/usr/bin/clang-tidy CLANG_TIDY_JOBS=8
CLANG_TIDY_BINARY ?= /opt/homebrew/opt/llvm/bin/clang-tidy
CLANG_TIDY_JOBS ?= $(shell nproc 2>/dev/null || sysctl -n hw.ncpu 2>/dev/null || echo 4)
clang-tidy:
python3 tools/run_clang_tidy.py --clang-tidy-binary $(CLANG_TIDY_BINARY) -j $(CLANG_TIDY_JOBS)
package:
bash build_tools/make_package.sh $(SHARED_MAJOR).$(SHARED_MINOR)
@@ -1400,6 +1488,9 @@ db_basic_test: $(OBJ_DIR)/db/db_basic_test.o $(TEST_LIBRARY) $(LIBRARY)
db_blob_basic_test: $(OBJ_DIR)/db/blob/db_blob_basic_test.o $(TEST_LIBRARY) $(LIBRARY)
$(AM_LINK)
db_blob_direct_write_test: $(OBJ_DIR)/db/blob/db_blob_direct_write_test.o $(TEST_LIBRARY) $(LIBRARY)
$(AM_LINK)
db_blob_compaction_test: $(OBJ_DIR)/db/blob/db_blob_compaction_test.o $(TEST_LIBRARY) $(LIBRARY)
$(AM_LINK)
@@ -1409,6 +1500,9 @@ db_readonly_with_timestamp_test: $(OBJ_DIR)/db/db_readonly_with_timestamp_test.o
db_wide_basic_test: $(OBJ_DIR)/db/wide/db_wide_basic_test.o $(TEST_LIBRARY) $(LIBRARY)
$(AM_LINK)
db_wide_blob_direct_write_test: $(OBJ_DIR)/db/wide/db_wide_blob_direct_write_test.o $(TEST_LIBRARY) $(LIBRARY)
$(AM_LINK)
db_with_timestamp_basic_test: $(OBJ_DIR)/db/db_with_timestamp_basic_test.o $(TEST_LIBRARY) $(LIBRARY)
$(AM_LINK)
@@ -1418,6 +1512,9 @@ db_with_timestamp_compaction_test: db/db_with_timestamp_compaction_test.o $(TEST
db_encryption_test: $(OBJ_DIR)/db/db_encryption_test.o $(TEST_LIBRARY) $(LIBRARY)
$(AM_LINK)
db_open_with_config_test: $(OBJ_DIR)/db/db_open_with_config_test.o $(TEST_LIBRARY) $(LIBRARY)
$(AM_LINK)
db_test: $(OBJ_DIR)/db/db_test.o $(TEST_LIBRARY) $(LIBRARY)
$(AM_LINK)
@@ -1562,6 +1659,9 @@ backup_engine_test: $(OBJ_DIR)/utilities/backup/backup_engine_test.o $(TEST_LIBR
checkpoint_test: $(OBJ_DIR)/utilities/checkpoint/checkpoint_test.o $(TEST_LIBRARY) $(LIBRARY)
$(AM_LINK)
sorted_run_builder_test: $(OBJ_DIR)/utilities/sorted_run_builder/sorted_run_builder_test.o $(TEST_LIBRARY) $(LIBRARY)
$(AM_LINK)
cache_simulator_test: $(OBJ_DIR)/utilities/simulator_cache/cache_simulator_test.o $(TEST_LIBRARY) $(LIBRARY)
$(AM_LINK)
@@ -1580,6 +1680,12 @@ object_registry_test: $(OBJ_DIR)/utilities/object_registry_test.o $(TEST_LIBRARY
ttl_test: $(OBJ_DIR)/utilities/ttl/ttl_test.o $(TEST_LIBRARY) $(LIBRARY)
$(AM_LINK)
trie_index_db_test: $(OBJ_DIR)/utilities/trie_index/trie_index_db_test.o $(TEST_LIBRARY) $(LIBRARY)
$(AM_LINK)
trie_index_test: $(OBJ_DIR)/utilities/trie_index/trie_index_test.o $(TEST_LIBRARY) $(LIBRARY)
$(AM_LINK)
types_util_test: $(OBJ_DIR)/utilities/types_util_test.o $(TEST_LIBRARY) $(LIBRARY)
$(AM_LINK)
@@ -1631,6 +1737,9 @@ io_posix_test: $(OBJ_DIR)/env/io_posix_test.o $(TEST_LIBRARY) $(LIBRARY)
fault_injection_test: $(OBJ_DIR)/db/fault_injection_test.o $(TEST_LIBRARY) $(LIBRARY)
$(AM_LINK)
fault_injection_fs_test: $(OBJ_DIR)/utilities/fault_injection_fs_test.o $(TEST_LIBRARY) $(LIBRARY)
$(AM_LINK)
rate_limiter_test: $(OBJ_DIR)/util/rate_limiter_test.o $(TEST_LIBRARY) $(LIBRARY)
$(AM_LINK)
@@ -1829,6 +1938,9 @@ write_committed_transaction_ts_test: $(OBJ_DIR)/utilities/transactions/write_com
write_prepared_transaction_test: $(OBJ_DIR)/utilities/transactions/write_prepared_transaction_test.o $(TEST_LIBRARY) $(LIBRARY)
$(AM_LINK)
write_prepared_transaction_seqno_test: $(OBJ_DIR)/utilities/transactions/write_prepared_transaction_seqno_test.o $(TEST_LIBRARY) $(LIBRARY)
$(AM_LINK)
write_unprepared_transaction_test: $(OBJ_DIR)/utilities/transactions/write_unprepared_transaction_test.o $(TEST_LIBRARY) $(LIBRARY)
$(AM_LINK)
@@ -2549,7 +2661,7 @@ list_all_tests:
# Remove the rules for which dependencies should not be generated and see if any are left.
#If so, include the dependencies; if not, do not include the dependency files
ROCKS_DEP_RULES=$(filter-out clean format check-format check-buck-targets check-headers check-sources jclean jtest package analyze tags rocksdbjavastatic% unity.% unity_test checkout_folly, $(MAKECMDGOALS))
ROCKS_DEP_RULES=$(filter-out clean format check-format check-buck-targets check-headers check-sources check-workflow-yaml clang-tidy jclean jtest package analyze tags rocksdbjavastatic% unity.% unity_test checkout_folly, $(MAKECMDGOALS))
ifneq ("$(ROCKS_DEP_RULES)", "")
-include $(DEPFILES)
endif
+4 -4
View File
@@ -146,10 +146,10 @@ def generate_buck(repo_path, deps_map):
src_mk["RANGE_TREE_SOURCES"] + src_mk["TOOL_LIB_SOURCES"],
deps=[
"//folly/container:f14_hash",
"//folly/experimental/coro:blocking_wait",
"//folly/experimental/coro:collect",
"//folly/experimental/coro:coroutine",
"//folly/experimental/coro:task",
"//folly/coro:blocking_wait",
"//folly/coro:collect",
"//folly/coro:coroutine",
"//folly/coro:task",
"//folly/synchronization:distributed_mutex",
],
headers=LiteralValue("glob([\"**/*.h\"])")
+18
View File
@@ -148,6 +148,24 @@ PLATFORM_SHARED_LDFLAGS="-Wl,--no-as-needed -shared -Wl,-soname -Wl,"
PLATFORM_SHARED_CFLAGS="-fPIC"
PLATFORM_SHARED_VERSIONED=true
# Prefer lld linker when available on Linux. lld is typically 5-10x faster
# than the default ld.bfd for large C++ projects. macOS uses ld64 (or
# ld-prime) which is already fast, so we skip lld detection there.
# Set ROCKSDB_NO_FAST_LINKER=1 to disable this auto-detection.
if [ -z "$ROCKSDB_NO_FAST_LINKER" ] && [ "$TARGET_OS" = "Linux" ]; then
if $CXX -fuse-ld=lld -L/usr/local/lib -x c++ - -o /dev/null 2>/dev/null <<EOF
int main() { return 0; }
EOF
then
PLATFORM_LDFLAGS="$PLATFORM_LDFLAGS -fuse-ld=lld"
# Ensure lld can find libraries in /usr/local/lib (lld does not
# search there by default, unlike ld.bfd)
if [ -d /usr/local/lib ]; then
PLATFORM_LDFLAGS="$PLATFORM_LDFLAGS -L/usr/local/lib"
fi
fi
fi
# generic port files (working on all platform by #ifdef) go directly in /port
GENERIC_PORT_FILES=`cd "$ROCKSDB_ROOT"; find port -name '*.cc' | tr "\n" " "`
+35
View File
@@ -0,0 +1,35 @@
#!/usr/bin/env bash
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved.
#
# Validate GitHub Actions workflow YAML before it reaches CI runtime.
set -euo pipefail
if ! command -v ruby >/dev/null 2>&1; then
echo "ruby is required to validate GitHub Actions workflow YAML"
exit 1
fi
ruby <<'RUBY'
require "psych"
bad = false
workflow_files = Dir[".github/workflows/*.{yml,yaml}"].sort
if workflow_files.empty?
warn "No workflow YAML files found under .github/workflows"
exit 1
end
workflow_files.each do |path|
begin
Psych.parse_file(path)
puts "OK #{path}"
rescue Psych::Exception => e
warn "Invalid YAML in #{path}: #{e.message}"
bad = true
end
end
exit(bad ? 1 : 0)
RUBY
+1 -1
View File
@@ -1913,7 +1913,7 @@ sub drain_job_queue {
}
$last_progress_time = time();
$ps_reported = 0;
} elsif (not $ps_reported and (time() - $last_progress_time) >= 60) {
} elsif (not $ps_reported and (time() - $last_progress_time) >= 300) {
# No progress in at least 60 seconds: run ps
print $Global::original_stderr "\n";
my $script_dir = ::dirname($0);
+5 -3
View File
@@ -5,6 +5,8 @@
# the docker and docker-registry groups, and logging out and back in to pick
# those up.)
#
# Meta employees: see https://fburl.com/rocksdb-docker to build this on a devvm.
#
# Follow https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry#authenticating-with-a-personal-access-token-classic
# to login with your GitHub credentials, as in
#
@@ -15,8 +17,8 @@
# Then in the build_tools/ubuntu22_image directory, (bump minor version for
# random docker file updates, major version tracks Ubuntu release)
#
# $ docker build -t ghcr.io/facebook/rocksdb_ubuntu:22.0
# $ docker push ghcr.io/facebook/rocksdb_ubuntu:22.0
# $ docker build -t ghcr.io/facebook/rocksdb_ubuntu:22.2
# $ docker push ghcr.io/facebook/rocksdb_ubuntu:22.2
#
# Might need to change visibility to public through
# https://github.com/orgs/facebook/packages/container/rocksdb_ubuntu/settings
@@ -28,7 +30,7 @@ FROM ubuntu:22.04
RUN apt-get update
RUN apt-get upgrade -y
# install basic tools
RUN apt-get install -y vim wget curl
RUN apt-get install -y vim wget curl ccache
# install tzdata noninteractive
RUN DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC apt-get -y install tzdata
# install git and default compilers
+9 -3
View File
@@ -5,6 +5,8 @@
# the docker and docker-registry groups, and logging out and back in to pick
# those up.)
#
# Meta employees: see https://fburl.com/rocksdb-docker to build this on a devvm.
#
# Follow https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry#authenticating-with-a-personal-access-token-classic
# to login with your GitHub credentials, as in
#
@@ -15,8 +17,8 @@
# Then in the build_tools/ubuntu24_image directory, (bump minor version for
# random docker file updates, major version tracks Ubuntu release)
#
# $ docker build -t ghcr.io/facebook/rocksdb_ubuntu:24.0
# $ docker push ghcr.io/facebook/rocksdb_ubuntu:24.0
# $ docker build -t ghcr.io/facebook/rocksdb_ubuntu:24.1
# $ docker push ghcr.io/facebook/rocksdb_ubuntu:24.1
#
# Might need to change visibility to public through
# https://github.com/orgs/facebook/packages/container/rocksdb_ubuntu/settings
@@ -28,11 +30,15 @@ FROM ubuntu:24.04
RUN apt-get update
RUN apt-get upgrade -y
# install basic tools
RUN apt-get install -y vim wget curl
RUN apt-get install -y vim wget curl ccache
# install tzdata noninteractive
RUN DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC apt-get -y install tzdata
# install git and default compilers
RUN apt-get install -y git gcc g++ clang clang-tools
# install clang-21 from LLVM snapshot repo
RUN curl -fsSL https://apt.llvm.org/llvm-snapshot.gpg.key | gpg --dearmor -o /etc/apt/trusted.gpg.d/llvm.gpg && \
echo "deb https://apt.llvm.org/noble/ llvm-toolchain-noble-21 main" > /etc/apt/sources.list.d/llvm-21.list && \
apt-get update && apt-get install -y clang-21
# install basic package
RUN apt-get install -y lsb-release software-properties-common gnupg
# install gflags, tbb
-5
View File
@@ -54,11 +54,6 @@ static std::unordered_map<std::string, OptionTypeInfo>
{offsetof(struct CompressedSecondaryCacheOptions, compression_type),
OptionType::kCompressionType, OptionVerificationType::kNormal,
OptionTypeFlags::kMutable}},
{"compress_format_version",
{offsetof(struct CompressedSecondaryCacheOptions,
compress_format_version),
OptionType::kUInt32T, OptionVerificationType::kNormal,
OptionTypeFlags::kMutable}},
{"enable_custom_split_merge",
{offsetof(struct CompressedSecondaryCacheOptions,
enable_custom_split_merge),
+9 -9
View File
@@ -101,23 +101,23 @@ class CacheEntryStatsCollector {
}
// Gets saved stats, regardless of age
void GetStats(Stats *stats) {
void GetStats(Stats* stats) {
std::lock_guard<std::mutex> lock(saved_mutex_);
*stats = saved_stats_;
}
Cache *GetCache() const { return cache_; }
Cache* GetCache() const { return cache_; }
// Gets or creates a shared instance of CacheEntryStatsCollector in the
// cache itself, and saves into `ptr`. This shared_ptr will hold the
// entry in cache until all refs are destroyed.
static Status GetShared(Cache *raw_cache, SystemClock *clock,
std::shared_ptr<CacheEntryStatsCollector> *ptr) {
static Status GetShared(Cache* raw_cache, SystemClock* clock,
std::shared_ptr<CacheEntryStatsCollector>* ptr) {
assert(raw_cache);
BasicTypedCacheInterface<CacheEntryStatsCollector, CacheEntryRole::kMisc>
cache{raw_cache};
const Slice &cache_key = GetCacheKey();
const Slice& cache_key = GetCacheKey();
auto h = cache.Lookup(cache_key);
if (h == nullptr) {
// Not yet in cache, but Cache doesn't provide a built-in way to
@@ -152,7 +152,7 @@ class CacheEntryStatsCollector {
}
private:
explicit CacheEntryStatsCollector(Cache *cache, SystemClock *clock)
explicit CacheEntryStatsCollector(Cache* cache, SystemClock* clock)
: saved_stats_(),
working_stats_(),
last_start_time_micros_(0),
@@ -160,7 +160,7 @@ class CacheEntryStatsCollector {
cache_(cache),
clock_(clock) {}
static const Slice &GetCacheKey() {
static const Slice& GetCacheKey() {
// For each template instantiation
static CacheKey ckey = CacheKey::CreateUniqueForProcessLifetime();
static Slice ckey_slice = ckey.AsSlice();
@@ -175,8 +175,8 @@ class CacheEntryStatsCollector {
uint64_t last_start_time_micros_;
uint64_t last_end_time_micros_;
Cache *const cache_;
SystemClock *const clock_;
Cache* const cache_;
SystemClock* const clock_;
};
} // namespace ROCKSDB_NAMESPACE
+3 -3
View File
@@ -24,7 +24,7 @@ namespace ROCKSDB_NAMESPACE {
// 0 | >= 1<<63 | CreateUniqueForProcessLifetime
// > 0 | any | OffsetableCacheKey.WithOffset
CacheKey CacheKey::CreateUniqueForCacheLifetime(Cache *cache) {
CacheKey CacheKey::CreateUniqueForCacheLifetime(Cache* cache) {
// +1 so that we can reserve all zeros for "unset" cache key
uint64_t id = cache->NewId() + 1;
// Ensure we don't collide with CreateUniqueForProcessLifetime
@@ -297,8 +297,8 @@ CacheKey CacheKey::CreateUniqueForProcessLifetime() {
//
// TODO: Nevertheless / regardless, an efficient way to detect (and thus
// quantify) block cache corruptions, including collisions, should be added.
OffsetableCacheKey::OffsetableCacheKey(const std::string &db_id,
const std::string &db_session_id,
OffsetableCacheKey::OffsetableCacheKey(const std::string& db_id,
const std::string& db_session_id,
uint64_t file_number) {
UniqueId64x2 internal_id;
Status s = GetSstInternalUniqueId(db_id, db_session_id, file_number,
+5 -5
View File
@@ -44,13 +44,13 @@ class CacheKey {
inline Slice AsSlice() const {
static_assert(sizeof(*this) == 16, "Standardized on 16-byte cache key");
assert(!IsEmpty());
return Slice(reinterpret_cast<const char *>(this), sizeof(*this));
return Slice(reinterpret_cast<const char*>(this), sizeof(*this));
}
// Create a CacheKey that is unique among others associated with this Cache
// instance. Depends on Cache::NewId. This is useful for block cache
// "reservations".
static CacheKey CreateUniqueForCacheLifetime(Cache *cache);
static CacheKey CreateUniqueForCacheLifetime(Cache* cache);
// Create a CacheKey that is unique among others for the lifetime of this
// process. This is useful for saving in a static data member so that
@@ -87,7 +87,7 @@ class OffsetableCacheKey : private CacheKey {
// Constructs an OffsetableCacheKey with the given information about a file.
// This constructor never generates an "empty" base key.
OffsetableCacheKey(const std::string &db_id, const std::string &db_session_id,
OffsetableCacheKey(const std::string& db_id, const std::string& db_session_id,
uint64_t file_number);
// Creates an OffsetableCacheKey from an SST unique ID, so that cache keys
@@ -134,9 +134,9 @@ class OffsetableCacheKey : private CacheKey {
static_assert(sizeof(file_num_etc64_) == kCommonPrefixSize,
"8 byte common prefix expected");
assert(!IsEmpty());
assert(&this->file_num_etc64_ == static_cast<const void *>(this));
assert(&this->file_num_etc64_ == static_cast<const void*>(this));
return Slice(reinterpret_cast<const char *>(this), kCommonPrefixSize);
return Slice(reinterpret_cast<const char*>(this), kCommonPrefixSize);
}
};
+16 -16
View File
@@ -44,8 +44,8 @@ class CacheReservationManager {
bool increase) = 0;
virtual Status MakeCacheReservation(
std::size_t incremental_memory_used,
std::unique_ptr<CacheReservationManager::CacheReservationHandle>
*handle) = 0;
std::unique_ptr<CacheReservationManager::CacheReservationHandle>*
handle) = 0;
virtual std::size_t GetTotalReservedCacheSize() = 0;
virtual std::size_t GetTotalMemoryUsed() = 0;
};
@@ -90,11 +90,11 @@ class CacheReservationManagerImpl
bool delayed_decrease = false);
// no copy constructor, copy assignment, move constructor, move assignment
CacheReservationManagerImpl(const CacheReservationManagerImpl &) = delete;
CacheReservationManagerImpl &operator=(const CacheReservationManagerImpl &) =
CacheReservationManagerImpl(const CacheReservationManagerImpl&) = delete;
CacheReservationManagerImpl& operator=(const CacheReservationManagerImpl&) =
delete;
CacheReservationManagerImpl(CacheReservationManagerImpl &&) = delete;
CacheReservationManagerImpl &operator=(CacheReservationManagerImpl &&) =
CacheReservationManagerImpl(CacheReservationManagerImpl&&) = delete;
CacheReservationManagerImpl& operator=(CacheReservationManagerImpl&&) =
delete;
~CacheReservationManagerImpl() override;
@@ -178,7 +178,7 @@ class CacheReservationManagerImpl
// REQUIRES: handle != nullptr
Status MakeCacheReservation(
std::size_t incremental_memory_used,
std::unique_ptr<CacheReservationManager::CacheReservationHandle> *handle)
std::unique_ptr<CacheReservationManager::CacheReservationHandle>* handle)
override;
// Return the size of the cache (which is a multiple of kSizeDummyEntry)
@@ -200,7 +200,7 @@ class CacheReservationManagerImpl
// For testing only - it is to help ensure the CacheItemHelperForRole<R>
// accessed from CacheReservationManagerImpl and the one accessed from the
// test are from the same translation units
static const Cache::CacheItemHelper *TEST_GetCacheItemHelperForRole();
static const Cache::CacheItemHelper* TEST_GetCacheItemHelperForRole();
private:
static constexpr std::size_t kSizeDummyEntry = 256 * 1024;
@@ -216,7 +216,7 @@ class CacheReservationManagerImpl
bool delayed_decrease_;
std::atomic<std::size_t> cache_allocated_size_;
std::size_t memory_used_;
std::vector<Cache::Handle *> dummy_handles_;
std::vector<Cache::Handle*> dummy_handles_;
CacheKey cache_key_;
};
@@ -251,14 +251,14 @@ class ConcurrentCacheReservationManager
std::shared_ptr<CacheReservationManager> cache_res_mgr) {
cache_res_mgr_ = std::move(cache_res_mgr);
}
ConcurrentCacheReservationManager(const ConcurrentCacheReservationManager &) =
ConcurrentCacheReservationManager(const ConcurrentCacheReservationManager&) =
delete;
ConcurrentCacheReservationManager &operator=(
const ConcurrentCacheReservationManager &) = delete;
ConcurrentCacheReservationManager(ConcurrentCacheReservationManager &&) =
ConcurrentCacheReservationManager& operator=(
const ConcurrentCacheReservationManager&) = delete;
ConcurrentCacheReservationManager(ConcurrentCacheReservationManager&&) =
delete;
ConcurrentCacheReservationManager &operator=(
ConcurrentCacheReservationManager &&) = delete;
ConcurrentCacheReservationManager& operator=(
ConcurrentCacheReservationManager&&) = delete;
~ConcurrentCacheReservationManager() override {}
@@ -286,7 +286,7 @@ class ConcurrentCacheReservationManager
inline Status MakeCacheReservation(
std::size_t incremental_memory_used,
std::unique_ptr<CacheReservationManager::CacheReservationHandle> *handle)
std::unique_ptr<CacheReservationManager::CacheReservationHandle>* handle)
override {
std::unique_ptr<CacheReservationManager::CacheReservationHandle>
wrapped_handle;
+1 -5
View File
@@ -50,8 +50,7 @@ CompressedSecondaryCache::CompressedSecondaryCache(
std::make_shared<CacheReservationManagerImpl<CacheEntryRole::kMisc>>(
cache_))),
disable_cache_(opts.capacity == 0) {
auto mgr =
GetBuiltinCompressionManager(cache_options_.compress_format_version);
auto mgr = GetBuiltinV2CompressionManager();
compressor_ = mgr->GetCompressor(cache_options_.compression_opts,
cache_options_.compression_type);
decompressor_ =
@@ -356,9 +355,6 @@ std::string CompressedSecondaryCache::GetPrintableOptions() const {
const_cast<CompressionOptions&>(cache_options_.compression_opts))
.c_str());
ret.append(buffer);
snprintf(buffer, kBufferSize, " compress_format_version : %d\n",
cache_options_.compress_format_version);
ret.append(buffer);
return ret;
}
+2 -3
View File
@@ -856,8 +856,7 @@ TEST_P(CompressedSecondaryCacheTestWithCompressionParam, BasicTestFromString) {
if (LZ4_Supported()) {
sec_cache_uri =
"compressed_secondary_cache://"
"capacity=2048;num_shard_bits=0;compression_type=kLZ4Compression;"
"compress_format_version=2";
"capacity=2048;num_shard_bits=0;compression_type=kLZ4Compression";
} else {
ROCKSDB_GTEST_SKIP("This test requires LZ4 support.");
sec_cache_uri =
@@ -888,7 +887,7 @@ TEST_P(CompressedSecondaryCacheTestWithCompressionParam,
sec_cache_uri =
"compressed_secondary_cache://"
"capacity=2048;num_shard_bits=0;compression_type=kLZ4Compression;"
"compress_format_version=2;enable_custom_split_merge=true";
"enable_custom_split_merge=true";
} else {
ROCKSDB_GTEST_SKIP("This test requires LZ4 support.");
sec_cache_uri =
+11 -11
View File
@@ -2179,7 +2179,7 @@ TEST_P(DBSecondaryCacheTest, LRUCacheDumpLoadBasic) {
&cache_dumper);
ASSERT_OK(s);
std::vector<DB*> db_list;
db_list.push_back(db_);
db_list.push_back(db_.get());
s = cache_dumper->SetDumpFilter(db_list);
ASSERT_OK(s);
s = cache_dumper->DumpCacheEntriesToWriter();
@@ -2263,11 +2263,11 @@ TEST_P(DBSecondaryCacheTest, LRUCacheDumpLoadWithFilter) {
options.env = fault_env_.get();
std::string dbname1 = test::PerThreadDBPath("db_1");
ASSERT_OK(DestroyDB(dbname1, options));
DB* db1 = nullptr;
std::unique_ptr<DB> db1;
ASSERT_OK(DB::Open(options, dbname1, &db1));
std::string dbname2 = test::PerThreadDBPath("db_2");
ASSERT_OK(DestroyDB(dbname2, options));
DB* db2 = nullptr;
std::unique_ptr<DB> db2;
ASSERT_OK(DB::Open(options, dbname2, &db2));
fault_fs_->SetFailGetUniqueId(true);
@@ -2335,7 +2335,7 @@ TEST_P(DBSecondaryCacheTest, LRUCacheDumpLoadWithFilter) {
&cache_dumper);
ASSERT_OK(s);
std::vector<DB*> db_list;
db_list.push_back(db1);
db_list.push_back(db1.get());
s = cache_dumper->SetDumpFilter(db_list);
ASSERT_OK(s);
s = cache_dumper->DumpCacheEntriesToWriter();
@@ -2377,7 +2377,7 @@ TEST_P(DBSecondaryCacheTest, LRUCacheDumpLoadWithFilter) {
ASSERT_OK(s);
ASSERT_OK(db1->Close());
delete db1;
db1.reset();
ASSERT_OK(DB::Open(options, dbname1, &db1));
// After load, we do the Get again. To validate the cache, we do not allow any
@@ -2406,8 +2406,8 @@ TEST_P(DBSecondaryCacheTest, LRUCacheDumpLoadWithFilter) {
ASSERT_EQ(256, static_cast<int>(block_lookup));
fault_fs_->SetFailGetUniqueId(false);
fault_fs_->SetFilesystemActive(true);
delete db1;
delete db2;
db1.reset();
db2.reset();
ASSERT_OK(DestroyDB(dbname1, options));
ASSERT_OK(DestroyDB(dbname2, options));
}
@@ -2619,11 +2619,11 @@ TEST_P(DBSecondaryCacheTest, TestSecondaryCacheOptionTwoDB) {
options.paranoid_file_checks = true;
std::string dbname1 = test::PerThreadDBPath("db_t_1");
ASSERT_OK(DestroyDB(dbname1, options));
DB* db1 = nullptr;
std::unique_ptr<DB> db1;
ASSERT_OK(DB::Open(options, dbname1, &db1));
std::string dbname2 = test::PerThreadDBPath("db_t_2");
ASSERT_OK(DestroyDB(dbname2, options));
DB* db2 = nullptr;
std::unique_ptr<DB> db2;
Options options2 = options;
options2.lowest_used_cache_tier = CacheTier::kVolatileTier;
ASSERT_OK(DB::Open(options2, dbname2, &db2));
@@ -2700,8 +2700,8 @@ TEST_P(DBSecondaryCacheTest, TestSecondaryCacheOptionTwoDB) {
fault_fs_->SetFailGetUniqueId(false);
fault_fs_->SetFilesystemActive(true);
delete db1;
delete db2;
db1.reset();
db2.reset();
ASSERT_OK(DestroyDB(dbname1, options));
ASSERT_OK(DestroyDB(dbname2, options));
}
+3
View File
@@ -0,0 +1,3 @@
You are an expert C++ engineer for RocksDB.
Read CLAUDE.md in the repo root and claude_md/ files for project context.
Answer thoroughly with exact file paths and line numbers.
+14
View File
@@ -0,0 +1,14 @@
Read review-findings.md. This file contains partial code
review findings from a review session that ran out of turns before
finishing. Format them into a clean final review comment using the
output format defined in claude_md/code_review.md (Summary, Issues
Found, Cross-Component Analysis, Positive Observations).
At the top of the review, add this notice:
> **Partial review** -- the analysis was interrupted before all
> perspectives were completed. The findings below cover the
> perspectives that were finished. You can request a full re-review
> with `/claude-review`.
Write ONLY the formatted review as your final response -- no
commentary or explanation outside the review itself.
+606
View File
@@ -0,0 +1,606 @@
# Code Review Workflow — CI Prompt
## Purpose
Thorough multi-agent code review for RocksDB commits following CLAUDE.md guidelines,
with structured codebase exploration, inter-agent debate, and verification.
IMPORTANT — Incremental output: After completing EACH major phase (Setup,
Codebase Context, Initial Review, Debate, Synthesis), append your findings
to review-findings.md using the Write tool. This ensures partial results are
saved even if the review is interrupted by the turn limit. After the final
synthesis, write the complete review to review-findings.md, replacing
previous content. Also output the final review as your response text.
## Workflow Steps
### 1. Setup Phase
- Read CLAUDE.md to understand review guidelines
- Identify the commit/diff/PR to review
- The PR diff is provided below in this prompt. The repository is already
checked out — use local file search tools (Grep, Glob, Read) to explore.
- Parse the changed files and categorize them (API, core logic, tests, etc.)
- Read all changed files to build full context
### 2. Codebase Context Phase (CRITICAL — Do Not Skip)
**Why this matters**: Review agents that only see the diff miss systemic bugs.
For example, a change to an iterator's return value might look correct in
isolation but break multi-SST iteration 5 layers up the call stack. A change
to error handling might silently drop errors that a caller 3 levels up relies
on for recovery. This phase builds the context that prevents those blind spots.
Go deep. Surface-level reading leads to reviews that miss caching layers,
existing helper functions, or concurrency requirements. The most expensive
failure mode is findings that look correct in isolation but miss how the
change breaks the surrounding system.
The team lead spawns one or more research agents to perform the following
analyses using local file search tools (Grep, Glob, Read). The research
must be written to `context.md` BEFORE any review agent is spawned.
#### 2a. Subsystem Deep-Read
Read the changed files AND their surrounding subsystem **in depth** — not just
signatures, but actual logic, edge cases, data flows, locking protocols,
concurrency patterns, and interactions between components.
For each changed file, also read:
- The header that declares its interface (if the change is in a .cc file)
- The sibling implementation that does the same thing the "standard" way
(e.g., for a UDI iterator change, read how `IndexBlockIter` handles the
same scenario — it's the reference implementation)
- The wrapper/adapter layer that sits between this code and its callers
- Any existing tests to understand what behaviors are currently guaranteed
#### 2b. Caller-Chain Analysis (upstream impact)
For each changed function/method, trace the call chain UPWARD 3-5 levels to
understand who consumes the changed behavior and what invariants they rely on.
**Focus on control flow decisions** — if/else branches, while loop conditions,
early returns — that depend on the changed return values or state.
**How to do this**:
- Search for callers of each changed function (search for the function name, the
class method, virtual overrides)
- For each caller, search for ITS callers, repeating 3-5 levels up
- At each level, read the actual code to understand the control flow decision
- Stop when you reach a "terminal" caller (e.g., user-facing API, top-level
iterator) or when the propagation is clearly safe
#### 2c. Callee-Chain Analysis (downstream dependencies AND side effects)
For each changed function, trace what it calls and whether those calls have
new preconditions or changed semantics. **Critically, trace SIDE EFFECTS —
not just return values.** Many bugs hide in side effects on shared state
(counters, sequence numbers, flags) that are invisible if you only check
return values.
**For each callee, ask:**
1. What does it RETURN? (the obvious check)
2. What shared state does it MUTATE? (the non-obvious check)
3. Under what PRECONDITIONS does it operate correctly?
4. Are those preconditions met in the new calling context?
#### 2d. Sibling Implementation Comparison
Identify the "reference" or "standard" implementation that does the same thing
the changed code does, and compare their behavior. This is critical for plugin/
extension APIs where the implementation must match an implicit contract.
#### 2e. Invariant Documentation
Document the key invariants that the changed code must maintain. Read existing
comments, assertions, and test expectations to discover these invariants.
#### 2f. Related Functionality and Existing Conventions
Identify related functionality that already exists in the codebase:
- Are there existing helper functions the change should use (or is duplicating)?
- Are there existing patterns for the same kind of change (e.g., how other
Customizable subclasses handle deprecation)?
- Are there existing tests that test similar scenarios (e.g., multi-SST
iteration tests that could be extended)?
#### 2g. Cross-Component Data Consumer Analysis (CRITICAL — common blind spot)
**Why this is separate from caller-chain analysis:** Sections 2b-2c trace
who CALLS the changed functions. This section traces who CONSUMES the data
that the changed code PRODUCES. These are often completely different
subsystems.
When the change writes data to a shared structure (memtable, SST block, cache
entry, statistics counter), ask: **"Who reads this data, under what rules,
and do those rules match the writer's assumptions?"**
**How to do this:**
1. Identify every piece of data the change WRITES (e.g., a range tombstone
entry, a counter update, a metadata field)
2. For each, search the codebase for ALL readers of that data structure
3. For each reader, check:
- Does the reader apply visibility rules (seqno, read_callback, snapshot)?
- Are those rules compatible with the writer's assumptions?
- Does the reader assume invariants about the data (e.g., seqno ordering,
monotonicity) that the writer might violate?
#### 2h. Alternative Execution Contexts
The same code path may run in different contexts with different assumptions.
**Enumerate all contexts and check each one.** This is where "works in the
common case but breaks in edge configurations" bugs hide.
For RocksDB, always check whether the changed code interacts differently with:
| Context | Key difference | Common failure mode |
|---------|---------------|-------------------|
| WritePreparedTxnDB / WriteUnpreparedTxnDB | `read_callback_` controls visibility, not just seqno | Visibility bypass |
| ReadOnly DB / SecondaryInstance | No mutable memtable, writes not allowed | Null pointer or no-op needed |
| CompactionService / Remote compaction | Different process, serialized state | Serialization mismatch |
| User-defined timestamps | Extra dimension in key comparison and visibility | Wrong ordering |
| MemPurge | Memtable-to-memtable, not memtable-to-SST | Missing data |
| Column family with BlobDB | Values may be in blob files, not inline | Missing dereference |
| Snapshots (old, held long-term) | Snapshot seqno may be far behind current state | Metadata corruption |
| Concurrent writers (allow_concurrent_memtable_write) | Lock-free paths vs locked paths | Lost updates |
| FIFO / Universal compaction | Different compaction invariants than Level | Wrong assumptions |
| Prefix seek / total order seek | Different iterator behavior | Wrong iteration results |
**For each context, ask:**
1. Does the code execute in this context? (Check constructor, feature flags)
2. If yes, do the assumptions still hold?
3. If not, should the feature be disabled in this context?
#### 2i. Assumption Stress-Testing (the "logically X" audit)
When the change claims a property (e.g., "logically redundant," "no-op in
this case," "safe because X"), **systematically break it:**
1. **State the claim precisely.** E.g., "The inserted range tombstone is
logically redundant — it covers only keys already deleted by point
tombstones."
2. **List the preconditions** that must be true for the claim to hold.
3. **For each precondition, construct a counterexample** — a concrete scenario
where the precondition is violated.
4. **Verify each counterexample against the code.** Does the code guard
against it? If not, it's a bug.
**Anti-pattern: treating asserts as proofs.** When you see an assert, do NOT
conclude "the invariant holds." Instead ask: "Can I construct an input where
this assert fires?" If yes, it's a bug (the assert catches it in debug but
release builds silently corrupt). If you cannot construct a counterexample
after trying, document WHY it's impossible.
#### 2j. Write Context Document
Write the complete analysis to `context.md` in the review folder. This document
is provided to ALL review agents as part of their prompt.
**Context document template**:
```markdown
# Codebase Context for Review
## How the Relevant Subsystem Works Today
[Detailed description of the subsystem architecture, data flows,
and component interactions. Not just "what" but "how" and "why."]
## Changed Functions and Their Call Chains
### function_name() (file:line)
**What changed**: [brief description of the behavioral change]
**Upstream callers** (who depends on this):
1. CallerA::method() (file:line) — uses return value to decide X
2. CallerB::method() (file:line) — passes result to CallerC
3. CallerC::method() (file:line) — THE CRITICAL DECISION POINT: ...
**Downstream callees** (what this depends on):
1. calleeA() — behavior unchanged
2. calleeB() — NEW dependency, requires X
**Sibling implementation** (how the "standard" version handles this):
- StandardImpl::method() does Y at file:line, which ensures invariant Z
- The changed code must match or document any deviation
**Key invariants**:
- Must never return X when Y is true because CallerC will...
- Must always set Z before returning because CallerB assumes...
## System Architecture Context
[How the changed components fit into the overall system]
## Known Invariants That Must Be Preserved
1. ...
2. ...
## Existing Conventions and Related Code
- [helper functions, patterns, existing tests that are relevant]
## Cross-Component Data Consumers
For each piece of data the change WRITES to a shared structure:
### [data item] written to [structure]
**Writer assumptions**: [what the writer assumes about how this data is consumed]
**All readers**:
1. ReaderA::method() (file:line) — reads under [visibility rules]
- Compatible with writer? YES/NO. If NO, explain the mismatch.
2. ReaderB::method() (file:line) — reads under [different rules]
- Compatible with writer? YES/NO.
## Alternative Execution Contexts
| Context | Does code execute? | Assumptions hold? | Action needed? |
|---------|-------------------|-------------------|----------------|
| WritePreparedTxnDB | YES/NO | YES/NO | [disable/guard/safe] |
| Old snapshots | YES/NO | YES/NO | ... |
| User-defined timestamps | YES/NO | YES/NO | ... |
| ReadOnly DB | YES/NO | YES/NO | ... |
| ... | ... | ... | ... |
## Assumption Stress Test
### Claim: "[the design claim, e.g., logically redundant]"
**Preconditions for claim to hold:**
1. [precondition] — counterexample: [scenario]. Guarded? YES/NO.
2. [precondition] — counterexample: [scenario]. Guarded? YES/NO.
## Potential Pitfalls
- [specific scenarios where the change could interact badly with
the surrounding system, identified from the caller-chain analysis]
```
### 3. Initial Review Phase (Parallel)
Create a team and spawn 5 review agents in parallel. **Include the context
document from Phase 2 in each agent's prompt.** Each agent writes findings
to its own file and sends a summary message to the team lead.
Each agent's prompt should include:
```
## Codebase Context (READ THIS FIRST)
[paste or reference the context.md file]
You MUST consider how your findings interact with the upstream callers
and system invariants documented above. A finding that looks correct
in isolation may be a critical bug when you consider the full call chain.
```
#### Agent: Design & Approach Reviewer
- Read the commit message deeply to understand the problem being solved
- Analyze whether this is the right approach to the problem
- Consider alternative designs and trade-offs:
- Are there simpler solutions that achieve the same goal?
- Does the approach introduce unnecessary complexity?
- Would a different data structure or algorithm be more appropriate?
- Evaluate whether the change follows existing architectural patterns or
introduces new patterns that may be inconsistent
- Assess the scope: is the change too large? Should it be split into
smaller, independently reviewable pieces?
- Check if the approach has precedent in the codebase or in the academic
literature it references (e.g., SuRF paper)
#### Agent: Correctness Reviewer
- Thread safety and concurrency analysis
- Error handling and propagation via Status type
- Edge cases (empty inputs, overflow, boundary conditions)
- Data corruption scenarios
- Logic correctness of new algorithms
- **Behavioral contract changes**: Do return value semantics match what
upstream callers expect? (Use the caller-chain analysis from context.md)
- **Callee side effects**: Does the change call functions that mutate shared
state (counters, seqnos, metadata)? Are the mutations correct for the new
calling context? (Use the callee-chain analysis from context.md)
#### Agent: Cross-Component & Adversarial Reviewer
This agent exists because the most critical bugs hide at component boundaries,
not within a single component. It uses a fundamentally different methodology
than the correctness reviewer: instead of verifying "does the code do what it
intends?", it asks "what breaks when we change the assumptions?"
**Data-flow analysis:** Perform the cross-component data consumer analysis
described in section 2g. For every piece of data the change WRITES to a shared
structure, trace ALL READERS and verify their visibility rules are compatible
with the writer's assumptions. This is a DATA-FLOW question, not a
CONTROL-FLOW question — readers may be in completely different subsystems.
**Alternative execution contexts:** Use the canonical table from section 2h.
Enumerate all contexts where the changed code executes and verify assumptions
hold in each.
**Assumption stress-testing and assert-breaking:** Follow the methodology from
section 2i. Identify every design claim, enumerate preconditions, construct
counterexamples. Treat asserts as hypotheses to break, not proofs.
**Red flag words**: "logically redundant", "safe because", "no-op",
"always true", "cannot happen", "invariant holds"
**Callee side-effect audit:** Perform the callee side-effect audit described
in section 2c. For every callee, ask what it RETURNS and what it MUTATES.
Write findings to `findings-cross-component.md`.
#### Agent: Invariant Adversary
This agent does NOT read the diff in detail. It takes the feature summary and
systematically tries to break it. While other agents verify "does the code do
what it says?", this agent asks "what existing system invariants does this
feature violate?" and "under what inputs do the design claims fail?"
**Steps 1-4: Assumption stress-testing.** Follow the methodology from section
2i: extract design claims, enumerate preconditions, construct counterexamples,
and attempt to break every assert. Be exhaustive about input parameter ranges,
shared state configurations, and concurrent interleaving.
**Step 5: Callee side-effect audit.** Perform the callee side-effect audit
described in section 2c. For every function the changed code calls, list ALL
mutations to shared state (atomic CAS loops, counter increments, flag/metadata
updates, cache invalidation). A function returning `Status::OK()` does NOT
mean its side effects are correct.
Write findings to `findings-invariant-adversary.md`.
#### Agent: Caller-Context Auditor
This agent traces BACKWARD from every entry point the changed code hooks into.
While other agents read the changed code forward ("what does it do?"), this
agent enumerates WHO invokes it and WITH WHAT parameter ranges.
The key insight: a function that is correct for all *typical* inputs may be
catastrophically wrong for *atypical-but-reachable* inputs. This agent's job
is to find the atypical inputs.
**Step 1: Identify entry points.** List every function/constructor/method in
the changed code that receives external input (parameters, config values,
pointers to shared state). Include:
- Constructor parameters (e.g., `active_mem`, `read_callback`, `sequence`)
- Virtual method overrides that callers invoke polymorphically
- Functions called from multiple subsystems
**Step 2: Enumerate all callers (3-5 levels up).** For each entry point,
search the codebase for ALL callers. Do NOT stop at the first caller — trace
the full call chain. Use `Grep` and `Glob` to find:
- Direct callers of the function
- Callers of wrapper/adapter functions that forward to it
- Factory functions that construct the object
- Virtual dispatch sites (search for the base class method name)
**Step 3: Parameter range analysis.** For each caller, determine:
- What values does it pass for each parameter?
- Under what conditions does it call this function?
- Are there callers that pass unusual values? (e.g., `kMaxSequenceNumber`,
`nullptr`, old snapshot seqnos, non-null `read_callback_`)
Build a parameter range table for each entry point, listing all callers
and the values they pass for each parameter.
**Step 4: Configuration matrix.** Enumerate option/config combinations that
affect the changed code path:
- Which options enable/disable the feature?
- Which options change the execution context? (e.g.,
`allow_concurrent_memtable_write`, `use_trie_index`)
- Are there option combinations that create contradictions?
**Step 5: Cross-reference with context.md.** Compare your caller analysis
with the invariants and execution contexts documented in context.md. Flag
any caller that violates an assumed precondition.
Write findings to `findings-caller-audit.md`.
#### Agent: Performance Reviewer
- Memory allocation on hot paths
- Unnecessary copies (string, buffer)
- Cache efficiency and data locality
- Loop optimization opportunities
- Branch prediction (LIKELY/UNLIKELY)
- Zero-overhead design verification
#### Agent: API & Compatibility Reviewer
- Public API backwards compatibility
- API consistency with existing patterns
- Documentation completeness
- Deprecation policy compliance
- Forward compatibility (struct extensibility)
- **Behavioral compatibility**: Do changed semantics (e.g., kOutOfBound vs
kUnknown) break implicit contracts with callers?
#### Agent: Serialization & Deserialization Reviewer
- Format correctness and versioning
- Alignment requirements
- Integer overflow in size computations
- Bounds checking on untrusted data
- Crafted input attack vectors
#### Agent: Test Coverage Reviewer
- Edge case coverage
- Failure mode testing (corruption, truncation)
- Round-trip testing (build → serialize → deserialize → verify)
- Integration test coverage
- **System-level test coverage**: Are the caller-chain interactions tested?
(e.g., multi-SST iteration with UDI, level compaction with new behavior)
- Test quality (no flaky patterns, good assertions)
### 4. Round-Robin Debate Phase
After all 5 agents complete their initial review, the team lead orchestrates
a structured debate:
#### Step 4a: Share findings
- Team lead sends each agent a summary of ALL other agents' findings
(or instructs each agent to read the other agents' findings files)
#### Step 4b: Critique round (2 rounds)
Each agent reviews the findings from the other agents and sends messages to
challenge, support, or refine them:
- **Challenge**: "I disagree with [agent] [finding] because [evidence]."
- **Support**: "I independently found the same issue — this confirms it."
- **Refine**: "[Finding A] is actually a consequence of [Finding B].
The root cause is the same."
The debate assignment follows a round-robin pattern:
```
correctness-reviewer → critiques → invariant-adversary, serialization
cross-component-reviewer → critiques → correctness, caller-audit
invariant-adversary → critiques → correctness, cross-component
caller-audit → critiques → invariant-adversary, cross-component
performance-reviewer → critiques → api, cross-component
api-reviewer → critiques → correctness, performance
serialization-reviewer → critiques → correctness, invariant-adversary
test-reviewer → critiques → serialization, caller-audit
design-reviewer → critiques → cross-component, invariant-adversary
```
**Note:** The invariant-adversary and caller-audit agents are deliberately
cross-linked with correctness and cross-component because their findings
often reveal the same underlying bug from different angles (invariant
violation vs reachable bad input vs data-flow mismatch).
Each agent should:
1. Read the assigned agents' findings files
2. Send a critique message to each assigned agent
3. Respond to critiques received from other agents
4. Update their own findings file with any revisions (upgraded/downgraded severity,
withdrawn findings, new findings inspired by others)
#### Step 4c: Team lead collects debate results
- Read all updated findings files
- Review the debate messages
- Build consensus: findings supported by 2+ agents → HIGH confidence
- Write consensus document with final severity classifications
### 5. Consensus Phase
Team lead synthesizes the debate into a consensus document:
- Cross-reference overlapping findings across agents
- Count agreement (majority = 2+ agents independently flagging or supporting)
- Note disagreements and how they were resolved
- Classify findings as HIGH/MEDIUM/LOW severity
- Write consensus document
### 6. Summary Phase
- Compile all validated findings ordered by severity
- Include detailed bug analysis (root cause, vulnerable code paths)
- Include suggested fixes
- Note which findings were debated and the outcome
- Write the complete review to review-findings.md and output as response
**Final report quality rules:**
- The final report must be CLEAN and POLISHED. No stream-of-consciousness,
no "Wait, actually...", no "on closer re-examination". Those belong in
the working notes, not the output.
- If a finding was raised during review but later disproven during debate
or deeper analysis, REMOVE IT entirely from the final report. Do not
include it with a retraction — just drop it.
- If a finding was downgraded (e.g., Critical → Suggestion), present it
at the final severity only. Do not narrate the severity change.
- Each finding in the final report should read as a confident, verified
conclusion — not a record of the analysis process.
- The reader should never see the reviewer arguing with itself.
## Review Checklist
### Context Phase (must be completed before agents spawn)
- [ ] Subsystem deep-read — read changed files AND surrounding subsystem
in depth (logic, edge cases, data flows, locking, concurrency)
- [ ] Caller chain for every changed public/virtual method (3-5 levels up)
- [ ] Critical decision points where callers branch on changed behavior
- [ ] Callee chain with SIDE EFFECTS — downstream dependencies, new
preconditions, AND mutations to shared state (section 2c)
- [ ] Sibling implementations — how does the standard version handle same scenarios?
- [ ] Invariants that callers rely on
- [ ] Related functionality — existing helpers, patterns, conventions
- [ ] Cross-component data consumers — for every data WRITTEN, find ALL readers
and verify visibility rules match (section 2g)
- [ ] Alternative execution contexts verified (section 2h table)
- [ ] Assumption stress-test — for every design claim, list preconditions
and construct counterexamples (section 2i)
- [ ] Multi-component interactions (compaction, recovery, snapshots, iterators)
- [ ] Configuration dependencies and unexpected option combinations
- [ ] Potential pitfalls from caller-chain analysis
### Review Phase (verified by agents)
- [ ] Database semantics preserved (snapshot isolation, key ordering)
- [ ] All error cases handled
- [ ] Thread-safe with correct synchronization
- [ ] No data races or deadlocks
- [ ] Appropriate test coverage (edge cases, failure modes, system-level)
- [ ] No unnecessary allocations or copies in hot paths
- [ ] Backwards compatible
- [ ] New APIs consistent with existing patterns and documented
- [ ] Code follows RocksDB style conventions
- [ ] Behavioral contracts with upstream callers preserved
- [ ] All callers enumerated with parameter range table
## Output Structure
Write all review artifacts to the working directory root:
- `review-findings.md` — Incremental findings (appended after each phase),
then replaced with the final synthesized review at the end
- `context.md` — Codebase context (call chains, invariants)
- `findings-*.md` — Per-agent findings (design, correctness, cross-component,
invariant-adversary, caller-audit, performance, api, serialization, tests)
- `consensus.md` — Cross-review consensus (post-debate)
## Team Structure
```
Team Lead (you)
├── Phase 2: Codebase Context (team lead or dedicated research agent)
│ └── context-researcher (general-purpose agent)
│ ├── Trace caller chains (3-5 levels up)
│ ├── Trace callee chains (dependencies AND side effects)
│ ├── Trace data consumers (who reads what the change writes?)
│ └── Document invariants
├── Phase 3: Initial Review (parallel, run_in_background)
│ │ (all agents receive context.md in their prompt)
│ ├── design-reviewer (general-purpose agent)
│ ├── correctness-reviewer (general-purpose agent)
│ ├── cross-component-reviewer (general-purpose agent)
│ ├── invariant-adversary (general-purpose agent) ← NEW
│ ├── caller-audit (general-purpose agent) ← NEW
│ ├── performance-reviewer (general-purpose agent)
│ ├── api-reviewer (general-purpose agent)
│ ├── serialization-reviewer (general-purpose agent)
│ └── test-reviewer (general-purpose agent)
├── Phase 4: Debate (agents message each other)
│ ├── correctness ↔ invariant-adversary, serialization
│ ├── cross-component ↔ correctness, caller-audit
│ ├── invariant-adversary ↔ correctness, cross-component
│ ├── caller-audit ↔ invariant-adversary, cross-component
│ ├── performance ↔ api, cross-component
│ ├── api ↔ correctness, performance
│ ├── serialization ↔ correctness, invariant-adversary
│ ├── test-coverage ↔ serialization, caller-audit
│ └── design ↔ cross-component, invariant-adversary
```
## Agent Communication Protocol
### During Initial Review (Phase 3)
- Each agent writes findings to its own file
- Each agent sends a summary message to team lead when done
- Agents do NOT communicate with each other yet
### During Debate (Phase 4)
- Team lead sends each agent a message with instructions to:
1. Read the assigned agents' findings files
2. Send critique messages to those agents
3. Respond to incoming critiques
4. Update their own findings file with revisions
- Each critique message should include:
- Which finding they're addressing (e.g., "Correctness F1")
- Whether they AGREE, DISAGREE, or want to REFINE
- Their reasoning with code evidence
- Suggested severity adjustment (if any)
### Message format example
```
To: correctness-reviewer
Re: Your Finding F1
AGREE/DISAGREE/REFINE - [reasoning with code evidence].
[Suggested severity adjustment if any.]
```
## Review Anti-Patterns
These recurring failure modes lead to missed bugs. Each is detailed in the
referenced section; this table is a quick-reference checklist.
| Anti-Pattern | Fix | Reference |
|---|---|---|
| Return-Value Tunnel Vision | Trace callee MUTATIONS, not just returns | Section 2c |
| Default-Configuration Bias | Enumerate all execution contexts | Section 2h |
| Assert-as-Proof | Try to BREAK every assert | Section 2i |
| Write-Path-Only Analysis | Trace data readers, not just writers | Section 2g |
| Confirmation-Seeking Research | Use adversarial prompts ("find where X fails") | Invariant Adversary agent |
| Data-Flow vs Control-Flow Confusion | Separate who CALLS from who READS the data | Section 2g |
+104
View File
@@ -0,0 +1,104 @@
# Code Review Skill
This document defines the methodology for performing thorough, multi-perspective
code reviews on RocksDB changes. It is used both by CI (GitHub Actions) and
local review workflows.
## Prerequisites
Before starting a review:
- Read `CLAUDE.md` in the repository root for project-specific guidelines
- Read any files in `claude_md/` for architecture context
- Identify the commit/diff/PR to review and parse the changed files
## Review Methodology
Conduct the review from multiple perspectives, then synthesize findings into a
single report. Each perspective catches different classes of bugs.
### Perspective 1: Codebase Context & Call-Chain Analysis
Before reviewing the diff itself, build deep context:
- For each changed function/method, trace the call chain UPWARD 3-5 levels.
Who consumes the changed behavior? What invariants do callers rely on?
- Trace DOWNWARD into callees. What side effects do they have? What shared
state do they mutate (counters, sequence numbers, flags, metadata)?
- Identify the "sibling" or "reference" implementation that handles the same
scenario the standard way. Compare behaviors.
- For data written to shared structures (memtable, SST block, cache entry),
find ALL readers and verify their visibility rules match the writer.
### Perspective 2: Correctness & Edge Cases
- Thread safety and concurrency (lock ordering, data races)
- Error handling and Status propagation
- Edge cases: empty inputs, overflow, boundary conditions
- Data corruption scenarios
- Behavioral contract changes: do return value semantics match callers?
### Perspective 3: Cross-Component & Adversarial Analysis
Check the change in ALL execution contexts:
| Context | Key difference |
|---------|---------------|
| WritePreparedTxnDB | read_callback_ controls visibility |
| ReadOnly DB / SecondaryInstance | No mutable memtable |
| CompactionService / Remote compaction | Different process |
| User-defined timestamps | Extra key comparison dimension |
| MemPurge | Memtable-to-memtable path |
| BlobDB | Values in blob files, not inline |
| Old snapshots | Seqno far behind current |
| Concurrent writers | Lock-free vs locked paths |
| FIFO / Universal compaction | Different invariants |
| Prefix seek / total order seek | Different iterator behavior |
When the change claims a property ("logically redundant", "safe because X"),
systematically try to break it: list preconditions, construct counterexamples.
### Perspective 4: Performance
RocksDB is a high-performance storage engine.
- Memory allocation on hot paths
- Unnecessary copies (prefer move semantics, Slice, string_view)
- Cache efficiency and data locality
- Loop optimization, branch prediction (LIKELY/UNLIKELY)
### Perspective 5: API, Compatibility & Testing
- Public API backwards compatibility
- Serialization format correctness and versioning
- Test coverage for edge cases, failure modes, and system-level interactions
## How to Explore the Codebase
Use available tools to explore deeply — do NOT just review the diff in isolation.
The most critical bugs hide at component boundaries.
- Read the header files for changed implementations
- Search for callers of changed functions (3-5 levels up)
- Read existing tests to understand guaranteed behaviors
- Check related implementations for conventions and patterns
## Output Format
### Summary
Brief overall assessment (1-2 sentences).
### Issues Found
Categorize by severity:
- :red_circle: **Critical**: Must fix (correctness, data corruption, security)
- :yellow_circle: **Suggestion**: Should consider (performance, edge cases)
- :green_circle: **Nitpick**: Minor style issues
For each issue include:
- File and line reference
- Which review perspective found it
- Description with root cause analysis
- Suggested fix
### Cross-Component Analysis
Execution context checks and assumption stress-test results.
### Positive Observations
Good patterns, clever optimizations, or improvements.
+7
View File
@@ -0,0 +1,7 @@
# Removing Deprecated Options from RocksDB
### Keep in these files:
- [ ] **KEEP** entry in type_info (`options/cf_options.cc` or `options/db_options.cc`) with `OptionVerificationType::kDeprecated` for loading old option files
- [ ] **KEEP** or add test in `options/options_test.cc` `OptionsOldApiTest::GetOptionsFromMapTest` for loading old option files
### Documentation:
- [ ] Add release note to `HISTORY.md` and `unreleased_history/`
+4
View File
@@ -34,6 +34,7 @@ CRASHTEST_PY=$(PYTHON) -u tools/db_crashtest.py --stress_cmd=$(DB_STRESS_CMD) --
whitebox_crash_test_with_txn whitebox_crash_test_with_ts \
whitebox_crash_test_with_optimistic_txn \
whitebox_crash_test_with_tiered_storage \
crash_test_db_cleanup \
crash_test: $(DB_STRESS_CMD)
# Do not parallelize
@@ -161,6 +162,9 @@ whitebox_crash_test_with_optimistic_txn: $(DB_STRESS_CMD)
$(CRASHTEST_PY) --optimistic_txn whitebox --random_kill_odd \
$(CRASH_TEST_KILL_ODD) $(CRASH_TEST_EXT_ARGS)
crash_test_db_cleanup: $(DB_STRESS_CMD)
$(DB_STRESS_CMD) --delete_dir_and_exit=$(TEST_TMPDIR)
# Old names DEPRECATED
crash_test_with_txn: crash_test_with_wc_txn
whitebox_crash_test_with_txn: whitebox_crash_test_with_wc_txn
+18 -7
View File
@@ -5,6 +5,8 @@
#include "db/blob/blob_fetcher.h"
#include "db/blob/blob_file_partition_manager.h"
#include "db/blob/blob_index.h"
#include "db/version_set.h"
namespace ROCKSDB_NAMESPACE {
@@ -14,10 +16,13 @@ Status BlobFetcher::FetchBlob(const Slice& user_key,
FilePrefetchBuffer* prefetch_buffer,
PinnableSlice* blob_value,
uint64_t* bytes_read) const {
assert(version_);
return version_->GetBlob(read_options_, user_key, blob_index_slice,
prefetch_buffer, blob_value, bytes_read);
BlobIndex blob_index;
Status status = blob_index.DecodeFrom(blob_index_slice);
if (status.ok()) {
status = FetchBlob(user_key, blob_index, prefetch_buffer, blob_value,
bytes_read);
}
return status;
}
Status BlobFetcher::FetchBlob(const Slice& user_key,
@@ -25,10 +30,16 @@ Status BlobFetcher::FetchBlob(const Slice& user_key,
FilePrefetchBuffer* prefetch_buffer,
PinnableSlice* blob_value,
uint64_t* bytes_read) const {
assert(version_);
if (!allow_write_path_fallback_) {
assert(version_);
return version_->GetBlob(read_options_, user_key, blob_index, prefetch_buffer,
blob_value, bytes_read);
return version_->GetBlob(read_options_, user_key, blob_index,
prefetch_buffer, blob_value, bytes_read);
}
return BlobFilePartitionManager::ResolveBlobDirectWriteIndex(
read_options_, user_key, blob_index, version_, blob_file_cache_,
prefetch_buffer, blob_value, bytes_read);
}
} // namespace ROCKSDB_NAMESPACE
+13 -3
View File
@@ -10,17 +10,25 @@
namespace ROCKSDB_NAMESPACE {
class BlobFileCache;
class Version;
class Slice;
class FilePrefetchBuffer;
class PinnableSlice;
class BlobIndex;
// A thin wrapper around the blob retrieval functionality of Version.
// A thin wrapper around blob retrieval. By default it reads through Version,
// and it can optionally fall back to direct-write blob files that are not yet
// manifest-visible.
class BlobFetcher {
public:
BlobFetcher(const Version* version, const ReadOptions& read_options)
: version_(version), read_options_(read_options) {}
BlobFetcher(const Version* version, const ReadOptions& read_options,
BlobFileCache* blob_file_cache = nullptr,
bool allow_write_path_fallback = false)
: version_(version),
read_options_(read_options),
blob_file_cache_(blob_file_cache),
allow_write_path_fallback_(allow_write_path_fallback) {}
Status FetchBlob(const Slice& user_key, const Slice& blob_index_slice,
FilePrefetchBuffer* prefetch_buffer,
@@ -33,5 +41,7 @@ class BlobFetcher {
private:
const Version* version_;
ReadOptions read_options_;
BlobFileCache* blob_file_cache_;
bool allow_write_path_fallback_;
};
} // namespace ROCKSDB_NAMESPACE
+2 -4
View File
@@ -67,13 +67,11 @@ BlobFileBuilder::BlobFileBuilder(
min_blob_size_(mutable_cf_options->min_blob_size),
blob_file_size_(mutable_cf_options->blob_file_size),
blob_compression_type_(mutable_cf_options->blob_compression_type),
// TODO: support most CompressionOptions with a new CF option
// blob_compression_opts
// TODO with schema change: support custom compression manager and options
// such as max_compressed_bytes_per_kb
// NOTE: returns nullptr for kNoCompression
blob_compressor_(GetBuiltinV2CompressionManager()->GetCompressor(
CompressionOptions{}, blob_compression_type_)),
mutable_cf_options->blob_compression_opts, blob_compression_type_)),
blob_compressor_wa_(blob_compressor_
? blob_compressor_->ObtainWorkingArea()
: Compressor::ManagedWorkingArea{}),
@@ -111,7 +109,7 @@ Status BlobFileBuilder::Add(const Slice& key, const Slice& value,
assert(blob_index);
assert(blob_index->empty());
if (value.size() < min_blob_size_) {
if (value.empty() || value.size() < min_blob_size_) {
return Status::OK();
}
+7 -10
View File
@@ -403,22 +403,19 @@ TEST_F(BlobFileBuilderTest, Compression) {
ASSERT_EQ(blob_file_addition.GetBlobFileNumber(), blob_file_number);
ASSERT_EQ(blob_file_addition.GetTotalBlobCount(), 1);
CompressionOptions opts;
CompressionContext context(kSnappyCompression, opts);
CompressionInfo info(opts, context, CompressionDict::GetEmptyDict(),
kSnappyCompression);
std::string compressed_value;
ASSERT_TRUE(Snappy_Compress(info, uncompressed_value.data(),
uncompressed_value.size(), &compressed_value));
auto compressor =
GetBuiltinV2CompressionManager()->GetCompressor({}, kSnappyCompression);
GrowableBuffer compressed_value;
ASSERT_OK(LegacyForceBuiltinCompression(*compressor, /*working_area=*/nullptr,
uncompressed_value,
&compressed_value));
ASSERT_EQ(blob_file_addition.GetTotalBlobBytes(),
BlobLogRecord::kHeaderSize + key_size + compressed_value.size());
// Verify the contents of the new blob file as well as the blob reference
std::vector<std::pair<std::string, std::string>> expected_key_value_pairs{
{key, compressed_value}};
{key, compressed_value.AsSlice().ToString()}};
std::vector<std::string> blob_indexes{blob_index};
VerifyBlobFile(blob_file_number, blob_file_path, column_family_id,
+125 -3
View File
@@ -9,6 +9,7 @@
#include <memory>
#include "db/blob/blob_file_reader.h"
#include "logging/logging.h"
#include "options/cf_options.h"
#include "rocksdb/cache.h"
#include "rocksdb/slice.h"
@@ -38,7 +39,8 @@ BlobFileCache::BlobFileCache(Cache* cache,
Status BlobFileCache::GetBlobFileReader(
const ReadOptions& read_options, uint64_t blob_file_number,
CacheHandleGuard<BlobFileReader>* blob_file_reader) {
CacheHandleGuard<BlobFileReader>* blob_file_reader,
bool allow_footer_skip_retry) {
assert(blob_file_reader);
assert(blob_file_reader->IsEmpty());
@@ -73,10 +75,22 @@ Status BlobFileCache::GetBlobFileReader(
{
assert(file_options_);
const Status s = BlobFileReader::Create(
Status s = BlobFileReader::Create(
*immutable_options_, read_options, *file_options_, column_family_id_,
blob_file_read_hist_, blob_file_number, io_tracer_, &reader);
blob_file_read_hist_, blob_file_number, io_tracer_,
/*skip_footer_validation=*/false, &reader);
if (!s.ok() && s.IsCorruption() && allow_footer_skip_retry) {
reader.reset();
s = BlobFileReader::Create(
*immutable_options_, read_options, *file_options_, column_family_id_,
blob_file_read_hist_, blob_file_number, io_tracer_,
/*skip_footer_validation=*/true, &reader);
}
if (!s.ok()) {
ROCKS_LOG_WARN(immutable_options_->logger,
"BlobFileCache open failed for blob file %" PRIu64
" in CF %u: %s",
blob_file_number, column_family_id_, s.ToString().c_str());
RecordTick(statistics, NO_FILE_ERRORS);
return s;
}
@@ -99,12 +113,120 @@ Status BlobFileCache::GetBlobFileReader(
return Status::OK();
}
Status BlobFileCache::OpenBlobFileReaderUncached(
const ReadOptions& read_options, uint64_t blob_file_number,
std::unique_ptr<BlobFileReader>* blob_file_reader,
bool allow_footer_skip_retry) {
assert(blob_file_reader);
assert(!*blob_file_reader);
Statistics* const statistics = immutable_options_->stats;
RecordTick(statistics, NO_FILE_OPENS);
Status s = BlobFileReader::Create(
*immutable_options_, read_options, *file_options_, column_family_id_,
blob_file_read_hist_, blob_file_number, io_tracer_,
/*skip_footer_validation=*/false, blob_file_reader);
if (!s.ok() && s.IsCorruption() && allow_footer_skip_retry) {
blob_file_reader->reset();
s = BlobFileReader::Create(
*immutable_options_, read_options, *file_options_, column_family_id_,
blob_file_read_hist_, blob_file_number, io_tracer_,
/*skip_footer_validation=*/true, blob_file_reader);
}
if (!s.ok()) {
RecordTick(statistics, NO_FILE_ERRORS);
}
return s;
}
Status BlobFileCache::InsertBlobFileReader(
uint64_t blob_file_number,
std::unique_ptr<BlobFileReader>* blob_file_reader,
CacheHandleGuard<BlobFileReader>* cached_blob_file_reader) {
assert(blob_file_reader);
assert(*blob_file_reader);
assert(cached_blob_file_reader);
assert(cached_blob_file_reader->IsEmpty());
const Slice key = GetSliceForKey(&blob_file_number);
// Serialize refreshes for the same blob file. The cache API does not expose
// a conditional replace, so refresh is intentionally modeled as
// Lookup/optional Erase/Insert under this per-key mutex.
MutexLock lock(&mutex_.Get(key));
TypedHandle* handle = cache_.Lookup(key);
if (handle) {
*cached_blob_file_reader = cache_.Guard(handle);
blob_file_reader->reset();
return Status::OK();
}
constexpr size_t charge = 1;
Status s = cache_.Insert(key, blob_file_reader->get(), charge, &handle);
if (!s.ok()) {
RecordTick(immutable_options_->stats, NO_FILE_ERRORS);
return s;
}
// Ownership transferred to the cache.
[[maybe_unused]] BlobFileReader* released_reader =
blob_file_reader->release();
*cached_blob_file_reader = cache_.Guard(handle);
return Status::OK();
}
Status BlobFileCache::RefreshBlobFileReader(
uint64_t blob_file_number,
std::unique_ptr<BlobFileReader>* blob_file_reader,
CacheHandleGuard<BlobFileReader>* cached_blob_file_reader) {
assert(blob_file_reader);
assert(*blob_file_reader);
assert(cached_blob_file_reader);
assert(cached_blob_file_reader->IsEmpty());
const Slice key = GetSliceForKey(&blob_file_number);
MutexLock lock(&mutex_.Get(key));
TypedHandle* handle = cache_.Lookup(key);
if (handle) {
BlobFileReader* const cached_reader = cache_.Value(handle);
assert(cached_reader != nullptr);
// Active direct-write blob files can grow between refresh attempts. Keep
// whichever reader observed the larger on-disk size so an older refresh
// cannot overwrite a newer one that another thread already installed.
if (cached_reader->GetFileSize() >= (*blob_file_reader)->GetFileSize()) {
*cached_blob_file_reader = cache_.Guard(handle);
blob_file_reader->reset();
return Status::OK();
}
cache_.Release(handle);
cache_.get()->Erase(key);
}
constexpr size_t charge = 1;
Status s = cache_.Insert(key, blob_file_reader->get(), charge, &handle);
if (!s.ok()) {
RecordTick(immutable_options_->stats, NO_FILE_ERRORS);
return s;
}
// Ownership transferred to the cache.
[[maybe_unused]] BlobFileReader* released_reader =
blob_file_reader->release();
*cached_blob_file_reader = cache_.Guard(handle);
return Status::OK();
}
void BlobFileCache::Evict(uint64_t blob_file_number) {
// NOTE: sharing same Cache with table_cache
const Slice key = GetSliceForKey(&blob_file_number);
assert(cache_);
MutexLock lock(&mutex_.Get(key));
cache_.get()->Erase(key);
}
+27 -1
View File
@@ -32,9 +32,35 @@ class BlobFileCache {
BlobFileCache(const BlobFileCache&) = delete;
BlobFileCache& operator=(const BlobFileCache&) = delete;
// Returns a cached reader for `blob_file_number`, opening and caching it on
// miss. If `allow_footer_skip_retry` is true, a footer-validation corruption
// retries once without requiring a footer.
Status GetBlobFileReader(const ReadOptions& read_options,
uint64_t blob_file_number,
CacheHandleGuard<BlobFileReader>* blob_file_reader);
CacheHandleGuard<BlobFileReader>* blob_file_reader,
bool allow_footer_skip_retry = false);
// Opens a blob file reader without inserting it into the cache.
Status OpenBlobFileReaderUncached(
const ReadOptions& read_options, uint64_t blob_file_number,
std::unique_ptr<BlobFileReader>* blob_file_reader,
bool allow_footer_skip_retry = false);
// Inserts a freshly opened uncached reader unless another thread already
// cached the same blob file.
Status InsertBlobFileReader(
uint64_t blob_file_number,
std::unique_ptr<BlobFileReader>* blob_file_reader,
CacheHandleGuard<BlobFileReader>* cached_blob_file_reader);
// Installs a refresh-opened reader into the cache. If another thread has
// already cached a reader opened on at least as large a file size, keep that
// reader instead so a racing refresh cannot reintroduce an older active-file
// view after the blob file grows.
Status RefreshBlobFileReader(
uint64_t blob_file_number,
std::unique_ptr<BlobFileReader>* blob_file_reader,
CacheHandleGuard<BlobFileReader>* cached_blob_file_reader);
// Called when a blob file is obsolete to ensure it is removed from the cache
// to avoid effectively leaking the open file and assicated memory
+96
View File
@@ -192,6 +192,102 @@ TEST_F(BlobFileCacheTest, GetBlobFileReader_Race) {
SyncPoint::GetInstance()->ClearAllCallBacks();
}
TEST_F(BlobFileCacheTest, RefreshBlobFileReaderPrefersLargestObservedFileSize) {
Options options;
options.env = mock_env_.get();
options.statistics = CreateDBStatistics();
options.cf_paths.emplace_back(
test::PerThreadDBPath(
mock_env_.get(),
"BlobFileCacheTest_"
"RefreshBlobFileReaderPrefersLargestObservedFileSize"),
0);
options.enable_blob_files = true;
constexpr uint32_t column_family_id = 1;
ImmutableOptions immutable_options(options);
constexpr uint64_t blob_file_number = 123;
const std::string blob_file_path =
BlobFileName(immutable_options.cf_paths.front().path, blob_file_number);
std::unique_ptr<FSWritableFile> file;
ASSERT_OK(NewWritableFile(immutable_options.fs.get(), blob_file_path, &file,
FileOptions()));
std::unique_ptr<WritableFileWriter> file_writer(new WritableFileWriter(
std::move(file), blob_file_path, FileOptions(), immutable_options.clock));
constexpr Statistics* statistics = nullptr;
constexpr bool use_fsync = false;
constexpr bool do_flush = false;
BlobLogWriter blob_log_writer(std::move(file_writer), immutable_options.clock,
statistics, blob_file_number, use_fsync,
do_flush);
constexpr bool has_ttl = false;
constexpr ExpirationRange expiration_range;
BlobLogHeader header(column_family_id, kNoCompression, has_ttl,
expiration_range);
ASSERT_OK(blob_log_writer.WriteHeader(WriteOptions(), header));
uint64_t key_offset = 0;
uint64_t blob_offset = 0;
ASSERT_OK(blob_log_writer.AddRecord(WriteOptions(), "key0", "blob0",
&key_offset, &blob_offset));
ASSERT_OK(blob_log_writer.Sync(WriteOptions()));
constexpr size_t capacity = 10;
std::shared_ptr<Cache> backing_cache = NewLRUCache(capacity);
FileOptions file_options;
constexpr HistogramImpl* blob_file_read_hist = nullptr;
BlobFileCache blob_file_cache(backing_cache.get(), &immutable_options,
&file_options, column_family_id,
blob_file_read_hist, nullptr /*IOTracer*/);
const ReadOptions read_options;
std::unique_ptr<BlobFileReader> stale_reader;
ASSERT_OK(blob_file_cache.OpenBlobFileReaderUncached(
read_options, blob_file_number, &stale_reader,
/*allow_footer_skip_retry=*/true));
std::unique_ptr<BlobFileReader> initial_cached_reader;
ASSERT_OK(blob_file_cache.OpenBlobFileReaderUncached(
read_options, blob_file_number, &initial_cached_reader,
/*allow_footer_skip_retry=*/true));
CacheHandleGuard<BlobFileReader> cached_reader;
ASSERT_OK(blob_file_cache.RefreshBlobFileReader(
blob_file_number, &initial_cached_reader, &cached_reader));
ASSERT_NE(cached_reader.GetValue(), nullptr);
const uint64_t initial_file_size = cached_reader.GetValue()->GetFileSize();
ASSERT_OK(blob_log_writer.AddRecord(WriteOptions(), "key1", "blob1",
&key_offset, &blob_offset));
ASSERT_OK(blob_log_writer.Sync(WriteOptions()));
std::unique_ptr<BlobFileReader> fresh_reader;
ASSERT_OK(blob_file_cache.OpenBlobFileReaderUncached(
read_options, blob_file_number, &fresh_reader,
/*allow_footer_skip_retry=*/true));
ASSERT_GT(fresh_reader->GetFileSize(), initial_file_size);
CacheHandleGuard<BlobFileReader> refreshed_reader;
ASSERT_OK(blob_file_cache.RefreshBlobFileReader(
blob_file_number, &fresh_reader, &refreshed_reader));
ASSERT_NE(refreshed_reader.GetValue(), nullptr);
ASSERT_GT(refreshed_reader.GetValue()->GetFileSize(), initial_file_size);
BlobFileReader* const largest_reader = refreshed_reader.GetValue();
CacheHandleGuard<BlobFileReader> preserved_reader;
ASSERT_OK(blob_file_cache.RefreshBlobFileReader(
blob_file_number, &stale_reader, &preserved_reader));
ASSERT_NE(preserved_reader.GetValue(), nullptr);
ASSERT_EQ(preserved_reader.GetValue(), largest_reader);
ASSERT_EQ(stale_reader.get(), nullptr);
}
TEST_F(BlobFileCacheTest, GetBlobFileReader_IOError) {
Options options;
options.env = mock_env_.get();
+840
View File
@@ -0,0 +1,840 @@
// Copyright (c) Meta Platforms, Inc. and affiliates.
// This source code is licensed under both the GPLv2 (found in the
// COPYING file in the root directory) and Apache 2.0 License
// (found in the LICENSE.Apache file in the root directory).
#include "db/blob/blob_file_partition_manager.h"
#include <array>
#include <atomic>
#include <cinttypes>
#include <memory>
#include <utility>
#include "cache/cache_key.h"
#include "cache/typed_cache.h"
#include "db/blob/blob_contents.h"
#include "db/blob/blob_file_cache.h"
#include "db/blob/blob_file_completion_callback.h"
#include "db/blob/blob_file_reader.h"
#include "db/blob/blob_index.h"
#include "db/blob/blob_log_writer.h"
#include "db/version_set.h"
#include "file/filename.h"
#include "file/read_write_util.h"
#include "file/writable_file_writer.h"
#include "logging/logging.h"
#include "util/aligned_buffer.h"
#include "util/compression.h"
#include "util/mutexlock.h"
namespace ROCKSDB_NAMESPACE {
namespace {
class RoundRobinBlobFilePartitionStrategy : public BlobFilePartitionStrategy {
public:
using BlobFilePartitionStrategy::SelectPartition;
const char* Name() const override {
return "RoundRobinBlobFilePartitionStrategy";
}
uint32_t SelectPartition(uint32_t num_partitions,
uint32_t /*column_family_id*/, const Slice& /*key*/,
const Slice& /*value*/) override {
assert(num_partitions > 0);
return static_cast<uint32_t>(
next_partition_.fetch_add(1, std::memory_order_relaxed) %
static_cast<uint64_t>(num_partitions));
}
private:
std::atomic<uint64_t> next_partition_{0};
};
struct DirectWriteCompressionState {
// `working_area` must be released before its owning compressor.
std::unique_ptr<Compressor> compressor;
Compressor::ManagedWorkingArea working_area;
};
DirectWriteCompressionState& GetDirectWriteCompressionState(
CompressionType compression) {
assert(compression <= kLastBuiltinCompression);
static thread_local std::array<DirectWriteCompressionState,
static_cast<size_t>(kLastBuiltinCompression) +
1>
compression_states;
auto& compression_state =
compression_states[static_cast<size_t>(compression)];
if (compression != kNoCompression &&
compression_state.compressor == nullptr) {
// BDW v1 mirrors BlobFileBuilder by using the built-in compressor with the
// default CompressionOptions only. Cache it per thread so repeated writes
// do not allocate a new compressor and working area every time.
compression_state.compressor =
GetBuiltinV2CompressionManager()->GetCompressor(CompressionOptions{},
compression);
if (compression_state.compressor != nullptr) {
compression_state.working_area =
compression_state.compressor->ObtainWorkingArea();
}
}
return compression_state;
}
} // namespace
BlobFilePartitionManager::BlobFilePartitionManager(
uint32_t num_partitions,
std::shared_ptr<BlobFilePartitionStrategy> strategy,
FileNumberAllocator file_number_allocator, FileSystem* fs,
SystemClock* clock, Statistics* statistics, const FileOptions& file_options,
std::string db_path, std::string column_family_name,
uint64_t blob_file_size, bool use_fsync, BlobFileCache* blob_file_cache,
BlobFileCompletionCallback* blob_callback,
const std::vector<std::shared_ptr<EventListener>>& listeners,
FileChecksumGenFactory* file_checksum_gen_factory,
const FileTypeSet& checksum_handoff_file_types,
const std::shared_ptr<IOTracer>& io_tracer, std::string db_id,
std::string db_session_id, Logger* info_log)
: num_partitions_(num_partitions == 0 ? 1 : num_partitions),
strategy_(strategy != nullptr
? std::move(strategy)
: std::make_shared<RoundRobinBlobFilePartitionStrategy>()),
file_number_allocator_(std::move(file_number_allocator)),
fs_(fs),
clock_(clock),
statistics_(statistics),
file_options_(file_options),
db_path_(std::move(db_path)),
column_family_name_(std::move(column_family_name)),
blob_file_size_(blob_file_size),
use_fsync_(use_fsync),
blob_file_cache_(blob_file_cache),
blob_callback_(blob_callback),
listeners_(listeners),
file_checksum_gen_factory_(file_checksum_gen_factory),
checksum_handoff_file_types_(checksum_handoff_file_types),
io_tracer_(io_tracer),
db_id_(std::move(db_id)),
db_session_id_(std::move(db_session_id)),
info_log_(info_log) {
partitions_.reserve(num_partitions_);
for (uint32_t i = 0; i < num_partitions_; ++i) {
partitions_.emplace_back(new Partition());
}
}
BlobFilePartitionManager::~BlobFilePartitionManager() {
if (blob_file_cache_ != nullptr) {
std::vector<uint64_t> tracked_file_numbers;
{
ReadLock lock(&file_partition_mutex_);
tracked_file_numbers.reserve(file_to_partition_.size());
for (const auto& entry : file_to_partition_) {
tracked_file_numbers.push_back(entry.first);
}
}
for (uint64_t file_number : tracked_file_numbers) {
// Dropping the CF or closing the DB can destroy the manager while active
// direct-write readers are still cached. Evict them before the CF-owned
// blob cache goes away so Close() does not retain obsolete footer-less
// readers for files that are no longer live.
blob_file_cache_->Evict(file_number);
}
}
}
void BlobFilePartitionManager::ResetPartitionState(Partition* partition) {
partition->writer.reset();
partition->file_number = 0;
partition->file_size = 0;
partition->blob_count = 0;
partition->total_blob_bytes = 0;
partition->garbage_blob_count = 0;
partition->garbage_blob_bytes = 0;
partition->column_family_id = 0;
partition->compression = kNoCompression;
partition->sync_required = false;
}
void BlobFilePartitionManager::AddFilePartitionMapping(uint64_t file_number,
uint32_t partition_idx) {
WriteLock lock(&file_partition_mutex_);
file_to_partition_[file_number] = partition_idx;
}
void BlobFilePartitionManager::RemoveFilePartitionMapping(
uint64_t file_number) {
WriteLock lock(&file_partition_mutex_);
file_to_partition_.erase(file_number);
}
Status BlobFilePartitionManager::OpenNewBlobFile(Partition* partition,
uint32_t column_family_id,
CompressionType compression,
uint32_t partition_idx) {
assert(partition != nullptr);
assert(!partition->writer);
const uint64_t blob_file_number = file_number_allocator_();
const std::string blob_file_path = BlobFileName(db_path_, blob_file_number);
AddFilePartitionMapping(blob_file_number, partition_idx);
if (blob_callback_ != nullptr) {
blob_callback_->OnBlobFileCreationStarted(
blob_file_path, column_family_name_,
/*job_id=*/0, BlobFileCreationReason::kFlush);
}
std::unique_ptr<FSWritableFile> file;
Status s = NewWritableFile(fs_, blob_file_path, &file, file_options_);
if (!s.ok()) {
RemoveFilePartitionMapping(blob_file_number);
return s;
}
const bool perform_data_verification =
checksum_handoff_file_types_.Contains(FileType::kBlobFile);
auto file_writer = std::make_unique<WritableFileWriter>(
std::move(file), blob_file_path, file_options_, clock_, io_tracer_,
statistics_, Histograms::BLOB_DB_BLOB_FILE_WRITE_MICROS, listeners_,
file_checksum_gen_factory_, perform_data_verification);
// This only drains WritableFileWriter's buffered bytes so readers can see
// each appended record promptly. Durability still comes from SyncAllOpenFiles
// or AppendFooter(), both of which call Sync().
constexpr bool kDoFlushEachRecord = true;
auto blob_log_writer = std::make_unique<BlobLogWriter>(
std::move(file_writer), clock_, statistics_, blob_file_number, use_fsync_,
kDoFlushEachRecord);
constexpr bool has_ttl = false;
constexpr ExpirationRange expiration_range{};
BlobLogHeader header(column_family_id, compression, has_ttl,
expiration_range);
s = blob_log_writer->WriteHeader(WriteOptions(), header);
if (!s.ok()) {
RemoveFilePartitionMapping(blob_file_number);
return s;
}
partition->writer = std::move(blob_log_writer);
partition->file_number = blob_file_number;
partition->file_size = BlobLogHeader::kSize;
partition->blob_count = 0;
partition->total_blob_bytes = 0;
partition->garbage_blob_count = 0;
partition->garbage_blob_bytes = 0;
partition->column_family_id = column_family_id;
partition->compression = compression;
partition->sync_required = false;
return Status::OK();
}
Status BlobFilePartitionManager::SealActiveBlobFile(
const WriteOptions& write_options, Partition* partition,
SealedFile* sealed_file) {
assert(partition != nullptr);
assert(partition->writer);
assert(sealed_file != nullptr);
const uint64_t file_number = partition->file_number;
Status s =
FinalizeBlobFile(write_options, partition->writer.get(), file_number,
partition->blob_count, partition->total_blob_bytes,
&sealed_file->addition);
if (!s.ok()) {
RemoveFilePartitionMapping(file_number);
ResetPartitionState(partition);
return s;
}
sealed_file->garbage_blob_count = partition->garbage_blob_count;
sealed_file->garbage_blob_bytes = partition->garbage_blob_bytes;
ResetPartitionState(partition);
return Status::OK();
}
Status BlobFilePartitionManager::SealDeferredFile(
const WriteOptions& write_options, DeferredFile* deferred,
SealedFile* sealed_file) {
assert(deferred != nullptr);
assert(deferred->writer);
assert(sealed_file != nullptr);
Status s = FinalizeBlobFile(
write_options, deferred->writer.get(), deferred->file_number,
deferred->blob_count, deferred->total_blob_bytes, &sealed_file->addition);
if (!s.ok()) {
RemoveFilePartitionMapping(deferred->file_number);
return s;
}
sealed_file->garbage_blob_count = deferred->garbage_blob_count;
sealed_file->garbage_blob_bytes = deferred->garbage_blob_bytes;
deferred->writer.reset();
return Status::OK();
}
Status BlobFilePartitionManager::FinalizeBlobFile(
const WriteOptions& write_options, BlobLogWriter* writer,
uint64_t file_number, uint64_t blob_count, uint64_t total_blob_bytes,
BlobFileAddition* addition) {
assert(writer != nullptr);
assert(addition != nullptr);
BlobLogFooter footer;
footer.blob_count = blob_count;
std::string checksum_method;
std::string checksum_value;
Status s = writer->AppendFooter(write_options, footer, &checksum_method,
&checksum_value);
if (!s.ok()) {
return s;
}
if (blob_callback_ != nullptr) {
const std::string blob_file_path = BlobFileName(db_path_, file_number);
s = blob_callback_->OnBlobFileCompleted(
blob_file_path, column_family_name_, /*job_id=*/0, file_number,
BlobFileCreationReason::kFlush, s, checksum_value, checksum_method,
blob_count, total_blob_bytes);
if (!s.ok()) {
return s;
}
}
*addition =
BlobFileAddition(file_number, blob_count, total_blob_bytes,
std::move(checksum_method), std::move(checksum_value));
return Status::OK();
}
void BlobFilePartitionManager::AddSealedFileGarbage(
const SealedFile& sealed_file, std::vector<BlobFileGarbage>* garbages) {
assert(garbages != nullptr);
if (sealed_file.garbage_blob_count == 0) {
assert(sealed_file.garbage_blob_bytes == 0);
return;
}
garbages->emplace_back(sealed_file.addition.GetBlobFileNumber(),
sealed_file.garbage_blob_count,
sealed_file.garbage_blob_bytes);
}
bool BlobFilePartitionManager::MarkPartitionGarbage(Partition* partition,
uint64_t file_number,
uint64_t blob_count,
uint64_t blob_bytes) {
assert(partition != nullptr);
if (!partition->writer || partition->file_number != file_number) {
return false;
}
partition->garbage_blob_count += blob_count;
partition->garbage_blob_bytes += blob_bytes;
assert(partition->garbage_blob_count <= partition->blob_count);
assert(partition->garbage_blob_bytes <= partition->total_blob_bytes);
return true;
}
bool BlobFilePartitionManager::MarkDeferredFileGarbage(DeferredFile* deferred,
uint64_t file_number,
uint64_t blob_count,
uint64_t blob_bytes) {
assert(deferred != nullptr);
if (deferred->file_number != file_number) {
return false;
}
deferred->garbage_blob_count += blob_count;
deferred->garbage_blob_bytes += blob_bytes;
assert(deferred->garbage_blob_count <= deferred->blob_count);
assert(deferred->garbage_blob_bytes <= deferred->total_blob_bytes);
return true;
}
bool BlobFilePartitionManager::MarkSealedFileGarbage(
std::vector<SealedFile>* sealed_files, uint64_t file_number,
uint64_t blob_count, uint64_t blob_bytes) {
assert(sealed_files != nullptr);
for (auto& sealed_file : *sealed_files) {
if (sealed_file.addition.GetBlobFileNumber() != file_number) {
continue;
}
sealed_file.garbage_blob_count += blob_count;
sealed_file.garbage_blob_bytes += blob_bytes;
assert(sealed_file.garbage_blob_count <=
sealed_file.addition.GetTotalBlobCount());
assert(sealed_file.garbage_blob_bytes <=
sealed_file.addition.GetTotalBlobBytes());
return true;
}
return false;
}
Status BlobFilePartitionManager::MaybePrepopulateBlobCache(
const BlobDirectWriteSettings* settings, const Slice& original_value,
uint64_t blob_file_number, uint64_t blob_offset) {
if (settings == nullptr || settings->blob_cache == nullptr ||
settings->prepopulate_blob_cache != PrepopulateBlobCache::kFlushOnly) {
return Status::OK();
}
FullTypedCacheInterface<BlobContents, BlobContentsCreator> blob_cache{
settings->blob_cache};
const OffsetableCacheKey base_cache_key(db_id_, db_session_id_,
blob_file_number);
const CacheKey cache_key = base_cache_key.WithOffset(blob_offset);
return blob_cache.InsertSaved(cache_key.AsSlice(), original_value, nullptr,
Cache::Priority::BOTTOM,
CacheTier::kVolatileTier);
}
uint32_t BlobFilePartitionManager::SelectWideColumnPartition(
uint32_t column_family_id, const Slice& key,
const WideColumns& columns) const {
return strategy_->SelectPartition(num_partitions_, column_family_id, key,
columns) %
num_partitions_;
}
Status BlobFilePartitionManager::WriteBlob(
const WriteOptions& write_options, uint32_t column_family_id,
CompressionType compression, const Slice& key, const Slice& value,
uint64_t* blob_file_number, uint64_t* blob_offset, uint64_t* blob_size,
const BlobDirectWriteSettings* settings, const uint32_t* partition_idx) {
assert(blob_file_number != nullptr);
assert(blob_offset != nullptr);
assert(blob_size != nullptr);
// Do compression before taking the partition mutex so large-value CPU work
// does not serialize writers. A concurrent file rollover can still cause
// this compressed buffer to be discarded below, which is acceptable on this
// non-hot failure/configuration-change path.
GrowableBuffer compressed_value;
Slice write_value = value;
if (compression != kNoCompression) {
auto& compression_state = GetDirectWriteCompressionState(compression);
if (compression_state.compressor == nullptr) {
return Status::NotSupported(
"Blob direct write compression type not supported");
}
Status s = LegacyForceBuiltinCompression(*compression_state.compressor,
&compression_state.working_area,
value, &compressed_value);
if (!s.ok()) {
return s;
}
write_value = Slice(compressed_value);
}
// Partition selection is based on the logical write inputs. In particular,
// strategies that inspect value contents or size see the original
// uncompressed value rather than `write_value`. The modulo here is
// intentional so custom strategies can return arbitrary hashed or sentinel
// values without violating the partition bounds.
const uint32_t selected_partition_idx =
partition_idx != nullptr
? (*partition_idx % num_partitions_)
: (strategy_->SelectPartition(num_partitions_, column_family_id, key,
value) %
num_partitions_);
{
MutexLock lock(&mutex_);
Partition* partition = partitions_[selected_partition_idx].get();
auto seal_current_file = [&]() -> Status {
if (!partition->writer) {
return Status::OK();
}
SealedFile sealed_file;
Status s = SealActiveBlobFile(write_options, partition, &sealed_file);
if (s.ok()) {
current_generation_sealed_files_.push_back(std::move(sealed_file));
}
return s;
};
const uint64_t record_size =
BlobLogRecord::kHeaderSize + key.size() + write_value.size();
const uint64_t future_file_size =
partition->file_size + record_size + BlobLogFooter::kSize;
if (partition->writer &&
(partition->column_family_id != column_family_id ||
partition->compression != compression ||
(partition->blob_count > 0 && future_file_size > blob_file_size_))) {
Status s = seal_current_file();
if (!s.ok()) {
return s;
}
}
if (!partition->writer) {
Status s = OpenNewBlobFile(partition, column_family_id, compression,
selected_partition_idx);
if (!s.ok()) {
return s;
}
}
uint64_t key_offset = 0;
Status s = partition->writer->AddRecord(write_options, key, write_value,
&key_offset, blob_offset);
if (!s.ok()) {
return s;
}
partition->sync_required = true;
partition->blob_count += 1;
partition->total_blob_bytes += record_size;
partition->file_size = BlobLogHeader::kSize + partition->total_blob_bytes;
*blob_file_number = partition->file_number;
*blob_size = write_value.size();
}
Status prepopulate_s = MaybePrepopulateBlobCache(
settings, value, *blob_file_number, *blob_offset);
if (!prepopulate_s.ok() && info_log_ != nullptr) {
ROCKS_LOG_WARN(info_log_,
"Failed to pre-populate direct-write blob cache entry: %s",
prepopulate_s.ToString().c_str());
}
return Status::OK();
}
void BlobFilePartitionManager::RotateCurrentGeneration() {
MutexLock lock(&mutex_);
GenerationBatch batch;
batch.sealed_files = std::move(current_generation_sealed_files_);
current_generation_sealed_files_.clear();
for (auto& partition : partitions_) {
if (!partition->writer) {
continue;
}
DeferredFile deferred;
deferred.writer = std::move(partition->writer);
deferred.file_number = partition->file_number;
deferred.blob_count = partition->blob_count;
deferred.total_blob_bytes = partition->total_blob_bytes;
deferred.garbage_blob_count = partition->garbage_blob_count;
deferred.garbage_blob_bytes = partition->garbage_blob_bytes;
ResetPartitionState(partition.get());
batch.deferred_files.emplace_back(std::move(deferred));
}
pending_generations_.emplace_back(std::move(batch));
}
Status BlobFilePartitionManager::PrepareFlushAdditions(
const WriteOptions& write_options, size_t num_generations,
std::vector<BlobFileAddition>* additions,
std::vector<BlobFileGarbage>* garbages,
std::vector<std::vector<uint64_t>>* generation_blob_file_numbers) {
assert(additions != nullptr);
assert(garbages != nullptr);
additions->clear();
garbages->clear();
if (generation_blob_file_numbers != nullptr) {
generation_blob_file_numbers->clear();
}
MutexLock lock(&mutex_);
if (num_generations > pending_generations_.size()) {
return Status::Corruption(
"Missing blob direct write generation metadata for flush");
}
for (size_t i = 0; i < num_generations; ++i) {
GenerationBatch& batch = pending_generations_[i];
while (!batch.deferred_files.empty()) {
DeferredFile deferred = std::move(batch.deferred_files.front());
batch.deferred_files.pop_front();
SealedFile sealed_file;
Status s = SealDeferredFile(write_options, &deferred, &sealed_file);
if (!s.ok()) {
return s;
}
// Keep each successfully sealed file attached to the generation
// immediately. If a later seal or the flush job itself fails, retry must
// reuse these exact on-disk files instead of finalizing replacements.
batch.sealed_files.push_back(std::move(sealed_file));
}
std::vector<uint64_t>* generation_file_numbers = nullptr;
if (generation_blob_file_numbers != nullptr) {
generation_blob_file_numbers->emplace_back();
generation_file_numbers = &generation_blob_file_numbers->back();
generation_file_numbers->reserve(batch.sealed_files.size());
}
for (const auto& sealed_file : batch.sealed_files) {
additions->push_back(sealed_file.addition);
AddSealedFileGarbage(sealed_file, garbages);
if (generation_file_numbers != nullptr) {
generation_file_numbers->push_back(
sealed_file.addition.GetBlobFileNumber());
}
}
}
return Status::OK();
}
Status BlobFilePartitionManager::MarkBlobWriteAsGarbage(uint64_t file_number,
uint64_t blob_count,
uint64_t blob_bytes) {
if (blob_count == 0) {
assert(blob_bytes == 0);
return Status::OK();
}
MutexLock lock(&mutex_);
for (auto& partition : partitions_) {
if (MarkPartitionGarbage(partition.get(), file_number, blob_count,
blob_bytes)) {
return Status::OK();
}
}
if (MarkSealedFileGarbage(&current_generation_sealed_files_, file_number,
blob_count, blob_bytes)) {
return Status::OK();
}
for (auto& batch : pending_generations_) {
for (auto& deferred : batch.deferred_files) {
if (MarkDeferredFileGarbage(&deferred, file_number, blob_count,
blob_bytes)) {
return Status::OK();
}
}
if (MarkSealedFileGarbage(&batch.sealed_files, file_number, blob_count,
blob_bytes)) {
return Status::OK();
}
}
const std::string message =
"Could not match failed blob direct-write rollback for file #" +
std::to_string(file_number);
if (info_log_ != nullptr) {
ROCKS_LOG_ERROR(info_log_, "%s", message.c_str());
}
return Status::Corruption(message);
}
void BlobFilePartitionManager::CommitPreparedGenerations(
size_t num_generations) {
MutexLock lock(&mutex_);
while (num_generations-- > 0 && !pending_generations_.empty()) {
pending_generations_.pop_front();
}
}
Status BlobFilePartitionManager::SyncAllOpenFiles(
const WriteOptions& write_options) {
MutexLock lock(&mutex_);
for (const auto& partition : partitions_) {
if (!partition->writer || !partition->sync_required) {
continue;
}
Status s = partition->writer->Sync(write_options);
if (!s.ok()) {
return s;
}
partition->sync_required = false;
}
return Status::OK();
}
void BlobFilePartitionManager::GetActiveBlobFileNumbers(
UnorderedSet<uint64_t>* file_numbers) const {
assert(file_numbers != nullptr);
ReadLock lock(&file_partition_mutex_);
for (const auto& entry : file_to_partition_) {
file_numbers->insert(entry.first);
}
}
void BlobFilePartitionManager::GetProtectedBlobFileNumbers(
UnorderedSet<uint64_t>* file_numbers) const {
assert(file_numbers != nullptr);
ReadLock lock(&file_partition_mutex_);
for (const auto& entry : protected_blob_file_refs_) {
file_numbers->insert(entry.first);
}
}
bool BlobFilePartitionManager::IsTrackedBlobFileNumber(
uint64_t file_number) const {
ReadLock lock(&file_partition_mutex_);
return file_to_partition_.find(file_number) != file_to_partition_.end() ||
protected_blob_file_refs_.find(file_number) !=
protected_blob_file_refs_.end();
}
void BlobFilePartitionManager::ProtectSealedBlobFileNumbers(
const std::vector<uint64_t>& file_numbers) {
if (file_numbers.empty()) {
return;
}
WriteLock lock(&file_partition_mutex_);
for (uint64_t file_number : file_numbers) {
++protected_blob_file_refs_[file_number];
}
}
void BlobFilePartitionManager::UnprotectSealedBlobFileNumbers(
const std::vector<uint64_t>& file_numbers) {
if (file_numbers.empty()) {
return;
}
WriteLock lock(&file_partition_mutex_);
for (uint64_t file_number : file_numbers) {
auto it = protected_blob_file_refs_.find(file_number);
if (it == protected_blob_file_refs_.end()) {
if (info_log_ != nullptr) {
ROCKS_LOG_ERROR(info_log_,
"Memtable blob protection underflow for file #%" PRIu64,
file_number);
}
continue;
}
assert(it->second > 0);
--it->second;
if (it->second == 0) {
protected_blob_file_refs_.erase(it);
if (blob_file_cache_ != nullptr) {
// Once the last memtable-backed reference goes away, any cached reader
// for this file is either obsolete or can be reopened through the
// manifest-visible path. Evict it here so delayed protection does not
// leave an obsolete blob reader behind until DB close.
blob_file_cache_->Evict(file_number);
}
}
}
}
void BlobFilePartitionManager::RemoveFilePartitionMappings(
const std::vector<uint64_t>& file_numbers) {
if (file_numbers.empty()) {
return;
}
WriteLock lock(&file_partition_mutex_);
for (uint64_t file_number : file_numbers) {
file_to_partition_.erase(file_number);
if (blob_file_cache_ != nullptr) {
// In-flight direct-write reads may have cached a footer-less reader for
// this file before it was sealed. Drop it now so future manifest-visible
// reads reopen against the finalized on-disk size and footer state.
blob_file_cache_->Evict(file_number);
}
}
}
Status BlobFilePartitionManager::ResolveBlobDirectWriteIndex(
const ReadOptions& read_options, const Slice& user_key,
const BlobIndex& blob_idx, const Version* version,
BlobFileCache* blob_file_cache, FilePrefetchBuffer* prefetch_buffer,
PinnableSlice* blob_value, uint64_t* bytes_read) {
assert(blob_value != nullptr);
if (version != nullptr) {
// Only fall back when the blob file is still owned exclusively by the
// write path and therefore absent from Version metadata. Once Version
// knows the file number, every result from Version::GetBlob(), including
// corruption, must propagate directly.
if (blob_idx.HasTTL() || blob_idx.IsInlined() ||
version->storage_info()->GetBlobFileMetaData(blob_idx.file_number()) !=
nullptr) {
return version->GetBlob(read_options, user_key, blob_idx, prefetch_buffer,
blob_value, bytes_read);
}
}
Status s = version != nullptr ? Status::Corruption("Invalid blob file number")
: Status::NotFound();
if (blob_file_cache == nullptr) {
return s;
}
CacheHandleGuard<BlobFileReader> reader;
s = blob_file_cache->GetBlobFileReader(read_options, blob_idx.file_number(),
&reader,
/*allow_footer_skip_retry=*/true);
if (!s.ok()) {
return s;
}
std::unique_ptr<BlobContents> blob_contents;
s = reader.GetValue()->GetBlob(read_options, user_key, blob_idx.offset(),
blob_idx.size(), blob_idx.compression(),
prefetch_buffer, nullptr, &blob_contents,
bytes_read);
if (s.ok()) {
blob_value->PinSelf(blob_contents->data());
return s;
}
if (!s.IsCorruption()) {
return s;
}
reader.Reset();
blob_file_cache->Evict(blob_idx.file_number());
std::unique_ptr<BlobFileReader> fresh_reader;
s = blob_file_cache->OpenBlobFileReaderUncached(
read_options, blob_idx.file_number(), &fresh_reader,
/*allow_footer_skip_retry=*/true);
if (!s.ok()) {
return s;
}
std::unique_ptr<BlobContents> fresh_contents;
s = fresh_reader->GetBlob(read_options, user_key, blob_idx.offset(),
blob_idx.size(), blob_idx.compression(),
prefetch_buffer, nullptr, &fresh_contents,
bytes_read);
if (s.ok()) {
blob_value->PinSelf(fresh_contents->data());
CacheHandleGuard<BlobFileReader> ignored;
blob_file_cache
->RefreshBlobFileReader(blob_idx.file_number(), &fresh_reader, &ignored)
.PermitUncheckedError();
}
return s;
}
} // namespace ROCKSDB_NAMESPACE
+294
View File
@@ -0,0 +1,294 @@
// Copyright (c) Meta Platforms, Inc. and affiliates.
// This source code is licensed under both the GPLv2 (found in the
// COPYING file in the root directory) and Apache 2.0 License
// (found in the LICENSE.Apache file in the root directory).
#pragma once
#include <cstdint>
#include <deque>
#include <functional>
#include <memory>
#include <string>
#include <unordered_map>
#include <vector>
#include "db/blob/blob_file_addition.h"
#include "db/blob/blob_file_garbage.h"
#include "db/blob/blob_log_format.h"
#include "db/blob/blob_write_batch_transformer.h"
#include "port/port.h"
#include "rocksdb/compression_type.h"
#include "rocksdb/env.h"
#include "rocksdb/file_system.h"
#include "rocksdb/options.h"
#include "rocksdb/rocksdb_namespace.h"
#include "rocksdb/slice.h"
#include "rocksdb/status.h"
#include "util/hash_containers.h"
namespace ROCKSDB_NAMESPACE {
class BlobFileCache;
class BlobFileCompletionCallback;
class BlobIndex;
class BlobLogWriter;
class FilePrefetchBuffer;
class IOTracer;
class Logger;
class PinnableSlice;
class Version;
class WritableFileWriter;
struct FileOptions;
struct ReadOptions;
// Manages per-partition blob files for write-path blob direct write.
//
// The v1 design keeps each memtable switch as one FIFO generation batch.
// Direct-write files for that memtable are sealed and registered when the
// matching flush commits. The current implementation still serializes blob
// appends through one manager mutex; follow-up PRs can add independent
// per-partition locks to allow concurrent blob writes without changing the
// generation/flush contract. Follow-up PRs can also broaden compatibility as
// the feature matures.
class BlobFilePartitionManager {
public:
using FileNumberAllocator = std::function<uint64_t()>;
// Creates the per-column-family manager for write-path blob files.
BlobFilePartitionManager(
uint32_t num_partitions,
std::shared_ptr<BlobFilePartitionStrategy> strategy,
FileNumberAllocator file_number_allocator, FileSystem* fs,
SystemClock* clock, Statistics* statistics,
const FileOptions& file_options, std::string db_path,
std::string column_family_name, uint64_t blob_file_size, bool use_fsync,
BlobFileCache* blob_file_cache, BlobFileCompletionCallback* blob_callback,
const std::vector<std::shared_ptr<EventListener>>& listeners,
FileChecksumGenFactory* file_checksum_gen_factory,
const FileTypeSet& checksum_handoff_file_types,
const std::shared_ptr<IOTracer>& io_tracer, std::string db_id,
std::string db_session_id, Logger* info_log);
// Evicts cached readers for manager-owned files during shutdown.
~BlobFilePartitionManager();
// Appends one blob record to a partition file and returns the resulting
// BlobIndex location metadata.
Status WriteBlob(const WriteOptions& write_options, uint32_t column_family_id,
CompressionType compression, const Slice& key,
const Slice& value, uint64_t* blob_file_number,
uint64_t* blob_offset, uint64_t* blob_size,
const BlobDirectWriteSettings* settings = nullptr,
const uint32_t* partition_idx = nullptr);
// Selects the partition to use for all blob-backed columns of one PutEntity
// operation. The return value is already normalized to [0, num_partitions_).
uint32_t SelectWideColumnPartition(uint32_t column_family_id,
const Slice& key,
const WideColumns& columns) const;
// Move the current active partition files into the next immutable
// memtable-generation batch. Called from SwitchMemtable() while DB mutex is
// held. No I/O is performed here.
void RotateCurrentGeneration();
// Seal the first `num_generations` queued immutable generations and return
// all blob additions and initial-garbage updates that must be registered
// with the matching flush. Generations stay queued until
// CommitPreparedGenerations() is called. If `generation_blob_file_numbers`
// is non-null, it receives the sealed blob file numbers for each prepared
// generation in the same FIFO order.
Status PrepareFlushAdditions(const WriteOptions& write_options,
size_t num_generations,
std::vector<BlobFileAddition>* additions,
std::vector<BlobFileGarbage>* garbages,
std::vector<std::vector<uint64_t>>*
generation_blob_file_numbers = nullptr);
// Remove the first `num_generations` prepared immutable generations after
// their blob-file additions and garbage edits were committed to MANIFEST.
void CommitPreparedGenerations(size_t num_generations);
// In v1 flush-on-write mode each AddRecord flushes to the OS immediately, so
// there is nothing extra to do for the non-sync write path.
Status FlushAllOpenFiles(const WriteOptions& /*write_options*/) {
return Status::OK();
}
// Sync the active open blob files before a sync WAL write.
Status SyncAllOpenFiles(const WriteOptions& write_options);
// Returns blob files still owned by the manager and therefore not yet safe
// to consider obsolete.
void GetActiveBlobFileNumbers(UnorderedSet<uint64_t>* file_numbers) const;
// Returns sealed blob files that are still reachable through live memtables
// or old SuperVersions and therefore must not be purged yet.
void GetProtectedBlobFileNumbers(UnorderedSet<uint64_t>* file_numbers) const;
// Returns true when the blob file is still owned by the write path or
// protected by a live memtable / old SuperVersion.
bool IsTrackedBlobFileNumber(uint64_t file_number) const;
// Increments / decrements memtable-held protection on sealed blob files.
void ProtectSealedBlobFileNumbers(const std::vector<uint64_t>& file_numbers);
void UnprotectSealedBlobFileNumbers(
const std::vector<uint64_t>& file_numbers);
// Stops protecting file numbers that are now registered in MANIFEST.
void RemoveFilePartitionMappings(const std::vector<uint64_t>& file_numbers);
// Marks blob records from a failed transformed write as initial garbage for
// the target blob file while keeping the physical bytes in place.
// Returns Corruption if the manager can no longer match the blob file.
Status MarkBlobWriteAsGarbage(uint64_t file_number, uint64_t blob_count,
uint64_t blob_bytes);
// Resolves a direct-write BlobIndex by consulting manifest-visible state
// first, then falling back to direct blob-file reads only when the target
// blob file is still write-path-owned and therefore not yet tracked by
// Version. Existing manifest-visible read results, including I/O failures,
// are returned directly rather than masked by fallback logic.
static Status ResolveBlobDirectWriteIndex(
const ReadOptions& read_options, const Slice& user_key,
const BlobIndex& blob_idx, const Version* version,
BlobFileCache* blob_file_cache, FilePrefetchBuffer* prefetch_buffer,
PinnableSlice* blob_value, uint64_t* bytes_read);
private:
struct Partition {
// Active writer for this partition, or null when the partition is idle.
std::unique_ptr<BlobLogWriter> writer;
// Metadata tracked while the active file remains open.
uint64_t file_number = 0;
uint64_t file_size = 0;
uint64_t blob_count = 0;
uint64_t total_blob_bytes = 0;
// Failed transformed writes already appended to this physical file.
uint64_t garbage_blob_count = 0;
uint64_t garbage_blob_bytes = 0;
uint32_t column_family_id = 0;
CompressionType compression = kNoCompression;
bool sync_required = false;
};
struct DeferredFile {
// Blob writer moved out of an active partition at memtable rotation.
std::unique_ptr<BlobLogWriter> writer;
// Metadata preserved until flush preparation seals the file.
uint64_t file_number = 0;
uint64_t blob_count = 0;
uint64_t total_blob_bytes = 0;
// Failed transformed writes already appended before the file is sealed.
uint64_t garbage_blob_count = 0;
uint64_t garbage_blob_bytes = 0;
};
struct SealedFile {
// Blob-file metadata ready to be added to MANIFEST.
BlobFileAddition addition;
// Initial unreachable bytes that should be registered as garbage
// alongside the file addition.
uint64_t garbage_blob_count = 0;
uint64_t garbage_blob_bytes = 0;
};
struct GenerationBatch {
// Files that were still open when the memtable became immutable.
std::deque<DeferredFile> deferred_files;
// Files already sealed for this memtable generation.
std::vector<SealedFile> sealed_files;
};
// Clears all active-file bookkeeping from a partition after sealing.
void ResetPartitionState(Partition* partition);
// Marks a blob file as manager-owned until it is committed or abandoned.
void AddFilePartitionMapping(uint64_t file_number, uint32_t partition_idx);
// Stops tracking a blob file after open/seal failure.
void RemoveFilePartitionMapping(uint64_t file_number);
// Opens a new active blob file for one partition.
Status OpenNewBlobFile(Partition* partition, uint32_t column_family_id,
CompressionType compression, uint32_t partition_idx);
// Finalizes one active partition file and returns the MANIFEST addition.
Status SealActiveBlobFile(const WriteOptions& write_options,
Partition* partition, SealedFile* sealed_file);
// Finalizes one deferred file that was carried over to flush preparation.
Status SealDeferredFile(const WriteOptions& write_options,
DeferredFile* deferred, SealedFile* sealed_file);
// Appends the footer, reports file completion, and builds the resulting
// BlobFileAddition for one sealed blob file.
Status FinalizeBlobFile(const WriteOptions& write_options,
BlobLogWriter* writer, uint64_t file_number,
uint64_t blob_count, uint64_t total_blob_bytes,
BlobFileAddition* addition);
// Appends one initial-garbage record to the flush output if any failed
// transformed writes targeted the sealed blob file.
static void AddSealedFileGarbage(const SealedFile& sealed_file,
std::vector<BlobFileGarbage>* garbages);
// Returns true if the active open file matches `file_number` and consumes
// the failed-write garbage accounting.
static bool MarkPartitionGarbage(Partition* partition, uint64_t file_number,
uint64_t blob_count, uint64_t blob_bytes);
// Returns true if the deferred flush-preparation file matches `file_number`
// and consumes the failed-write garbage accounting.
static bool MarkDeferredFileGarbage(DeferredFile* deferred,
uint64_t file_number, uint64_t blob_count,
uint64_t blob_bytes);
// Returns true if one already-sealed file matches `file_number` and
// consumes the failed-write garbage accounting.
static bool MarkSealedFileGarbage(std::vector<SealedFile>* sealed_files,
uint64_t file_number, uint64_t blob_count,
uint64_t blob_bytes);
// Optionally seeds the blob cache with the original uncompressed value.
Status MaybePrepopulateBlobCache(const BlobDirectWriteSettings* settings,
const Slice& original_value,
uint64_t blob_file_number,
uint64_t blob_offset);
// Configured partition fanout and write-selection strategy.
const uint32_t num_partitions_;
std::shared_ptr<BlobFilePartitionStrategy> strategy_;
// Shared services needed to create, track, and cache blob files.
FileNumberAllocator file_number_allocator_;
FileSystem* fs_;
SystemClock* clock_;
Statistics* statistics_;
FileOptions file_options_;
// Column-family context used for file naming and callbacks.
std::string db_path_;
std::string column_family_name_;
// File creation policy and shared blob-file infrastructure.
uint64_t blob_file_size_;
bool use_fsync_;
BlobFileCache* blob_file_cache_;
BlobFileCompletionCallback* blob_callback_;
std::vector<std::shared_ptr<EventListener>> listeners_;
FileChecksumGenFactory* file_checksum_gen_factory_;
FileTypeSet checksum_handoff_file_types_;
std::shared_ptr<IOTracer> io_tracer_;
// DB identity used when constructing cache keys.
std::string db_id_;
std::string db_session_id_;
Logger* info_log_;
// One active writer slot per configured partition.
std::vector<std::unique_ptr<Partition>> partitions_;
// Protects partition state and generation queues.
mutable port::Mutex mutex_;
// Files already sealed while the current mutable memtable was active.
std::vector<SealedFile> current_generation_sealed_files_;
// FIFO immutable memtable generations waiting to be flushed.
std::deque<GenerationBatch> pending_generations_;
// Tracks blob files still owned by active or deferred write-path state until
// MANIFEST commit publishes them.
std::unordered_map<uint64_t, uint32_t> file_to_partition_;
// Sealed direct-write files that remain reachable from live memtables, such
// as old SuperVersions serving lazy iterator reads after flush commit.
std::unordered_map<uint64_t, uint32_t> protected_blob_file_refs_;
// Protects file_to_partition_ and protected_blob_file_refs_.
mutable port::RWMutex file_partition_mutex_;
};
} // namespace ROCKSDB_NAMESPACE
+36 -14
View File
@@ -29,7 +29,7 @@ Status BlobFileReader::Create(
const ImmutableOptions& immutable_options, const ReadOptions& read_options,
const FileOptions& file_options, uint32_t column_family_id,
HistogramImpl* blob_file_read_hist, uint64_t blob_file_number,
const std::shared_ptr<IOTracer>& io_tracer,
const std::shared_ptr<IOTracer>& io_tracer, bool skip_footer_validation,
std::unique_ptr<BlobFileReader>* blob_file_reader) {
assert(blob_file_reader);
assert(!*blob_file_reader);
@@ -40,7 +40,8 @@ Status BlobFileReader::Create(
{
const Status s =
OpenFile(immutable_options, file_options, blob_file_read_hist,
blob_file_number, io_tracer, &file_size, &file_reader);
blob_file_number, io_tracer, &file_size, &file_reader,
/*skip_footer_size_check=*/skip_footer_validation);
if (!s.ok()) {
return s;
}
@@ -61,7 +62,7 @@ Status BlobFileReader::Create(
}
}
{
if (!skip_footer_validation) {
const Status s =
ReadFooter(file_reader.get(), read_options, file_size, statistics);
if (!s.ok()) {
@@ -76,9 +77,10 @@ Status BlobFileReader::Create(
compression_type);
}
blob_file_reader->reset(new BlobFileReader(
std::move(file_reader), file_size, compression_type,
std::move(decompressor), immutable_options.clock, statistics));
blob_file_reader->reset(
new BlobFileReader(std::move(file_reader), file_size, compression_type,
std::move(decompressor), immutable_options.clock,
statistics, !skip_footer_validation));
return Status::OK();
}
@@ -87,7 +89,8 @@ Status BlobFileReader::OpenFile(
const ImmutableOptions& immutable_options, const FileOptions& file_opts,
HistogramImpl* blob_file_read_hist, uint64_t blob_file_number,
const std::shared_ptr<IOTracer>& io_tracer, uint64_t* file_size,
std::unique_ptr<RandomAccessFileReader>* file_reader) {
std::unique_ptr<RandomAccessFileReader>* file_reader,
bool skip_footer_size_check) {
assert(file_size);
assert(file_reader);
@@ -112,17 +115,26 @@ Status BlobFileReader::OpenFile(
}
}
if (*file_size < BlobLogHeader::kSize + BlobLogFooter::kSize) {
if (!skip_footer_size_check &&
*file_size < BlobLogHeader::kSize + BlobLogFooter::kSize) {
return Status::Corruption("Malformed blob file");
}
if (skip_footer_size_check && *file_size < BlobLogHeader::kSize) {
return Status::Corruption("Malformed blob file");
}
std::unique_ptr<FSRandomAccessFile> file;
FileOptions reader_file_opts = file_opts;
if (skip_footer_size_check && reader_file_opts.use_direct_reads) {
reader_file_opts.use_direct_reads = false;
}
{
TEST_SYNC_POINT("BlobFileReader::OpenFile:NewRandomAccessFile");
const Status s =
fs->NewRandomAccessFile(blob_file_path, file_opts, &file, dbg);
fs->NewRandomAccessFile(blob_file_path, reader_file_opts, &file, dbg);
if (!s.ok()) {
return s;
}
@@ -291,13 +303,14 @@ BlobFileReader::BlobFileReader(
std::unique_ptr<RandomAccessFileReader>&& file_reader, uint64_t file_size,
CompressionType compression_type,
std::shared_ptr<Decompressor> decompressor, SystemClock* clock,
Statistics* statistics)
Statistics* statistics, bool has_footer)
: file_reader_(std::move(file_reader)),
file_size_(file_size),
compression_type_(compression_type),
decompressor_(std::move(decompressor)),
clock_(clock),
statistics_(statistics) {
statistics_(statistics),
has_footer_(has_footer) {
assert(file_reader_);
}
@@ -312,7 +325,8 @@ Status BlobFileReader::GetBlob(
const uint64_t key_size = user_key.size();
if (!IsValidBlobOffset(offset, key_size, value_size, file_size_)) {
if (!IsValidBlobOffset(offset, key_size, value_size, file_size_,
has_footer_)) {
return Status::Corruption("Invalid blob offset");
}
@@ -428,7 +442,8 @@ void BlobFileReader::MultiGetBlob(
const uint64_t offset = req->offset;
const uint64_t value_size = req->len;
if (!IsValidBlobOffset(offset, key_size, value_size, file_size_)) {
if (!IsValidBlobOffset(offset, key_size, value_size, file_size_,
has_footer_)) {
*req->status = Status::Corruption("Invalid blob offset");
continue;
}
@@ -454,6 +469,13 @@ void BlobFileReader::MultiGetBlob(
RecordTick(statistics_, BLOB_DB_BLOB_FILE_BYTES_READ, total_len);
if (read_reqs.empty()) {
if (bytes_read) {
*bytes_read = 0;
}
return;
}
Buffer buf;
AlignedBuf aligned_buf;
@@ -533,7 +555,7 @@ void BlobFileReader::MultiGetBlob(
}
// Uncompress blob if needed
Slice value_slice(record_slice.data() + adjustments[i], req->len);
Slice value_slice(record_slice.data() + adjustments[j - 1], req->len);
*req->status = UncompressBlobIfNeeded(
value_slice, compression_type_, decompressor_.get(), allocator, clock_,
statistics_, &blob_reqs[i].second);
+22 -3
View File
@@ -36,7 +36,20 @@ class BlobFileReader {
HistogramImpl* blob_file_read_hist,
uint64_t blob_file_number,
const std::shared_ptr<IOTracer>& io_tracer,
std::unique_ptr<BlobFileReader>* reader);
std::unique_ptr<BlobFileReader>* reader) {
return Create(immutable_options, read_options, file_options,
column_family_id, blob_file_read_hist, blob_file_number,
io_tracer, /*skip_footer_validation=*/false, reader);
}
// Allows opening in-flight direct-write blob files by optionally skipping
// footer validation.
static Status Create(
const ImmutableOptions& immutable_options,
const ReadOptions& read_options, const FileOptions& file_options,
uint32_t column_family_id, HistogramImpl* blob_file_read_hist,
uint64_t blob_file_number, const std::shared_ptr<IOTracer>& io_tracer,
bool skip_footer_validation, std::unique_ptr<BlobFileReader>* reader);
BlobFileReader(const BlobFileReader&) = delete;
BlobFileReader& operator=(const BlobFileReader&) = delete;
@@ -63,18 +76,22 @@ class BlobFileReader {
uint64_t GetFileSize() const { return file_size_; }
private:
// `has_footer` tracks whether offset validation should reserve footer space.
BlobFileReader(std::unique_ptr<RandomAccessFileReader>&& file_reader,
uint64_t file_size, CompressionType compression_type,
std::shared_ptr<Decompressor> decompressor, SystemClock* clock,
Statistics* statistics);
Statistics* statistics, bool has_footer);
// `skip_footer_size_check` is used for direct-write files that are still
// missing their footer at open time.
static Status OpenFile(const ImmutableOptions& immutable_options,
const FileOptions& file_opts,
HistogramImpl* blob_file_read_hist,
uint64_t blob_file_number,
const std::shared_ptr<IOTracer>& io_tracer,
uint64_t* file_size,
std::unique_ptr<RandomAccessFileReader>* file_reader);
std::unique_ptr<RandomAccessFileReader>* file_reader,
bool skip_footer_size_check);
static Status ReadHeader(const RandomAccessFileReader* file_reader,
const ReadOptions& read_options,
@@ -110,6 +127,8 @@ class BlobFileReader {
std::shared_ptr<Decompressor> decompressor_;
SystemClock* clock_;
Statistics* statistics_;
// False when the reader was opened before the blob file footer was written.
bool has_footer_;
};
} // namespace ROCKSDB_NAMESPACE
+86
View File
@@ -1011,6 +1011,92 @@ TEST_P(BlobFileReaderDecodingErrorTest, DecodingError) {
SyncPoint::GetInstance()->ClearAllCallBacks();
}
TEST_F(BlobFileReaderTest, MultiGetBlobWithFailedValidation) {
// Test that MultiGetBlob correctly handles the case where some requests
// fail validation. The adjustments vector is only populated for requests
// that pass validation, so the indices must track correctly.
Options options;
options.env = mock_env_.get();
options.cf_paths.emplace_back(
test::PerThreadDBPath(
mock_env_.get(),
"BlobFileReaderTest_MultiGetBlobWithFailedValidation"),
0);
options.enable_blob_files = true;
ImmutableOptions immutable_options(options);
constexpr uint32_t column_family_id = 1;
constexpr bool has_ttl = false;
constexpr ExpirationRange expiration_range;
constexpr uint64_t blob_file_number = 1;
constexpr size_t num_blobs = 3;
const std::vector<std::string> key_strs = {"key1", "key2", "key3"};
const std::vector<std::string> blob_strs = {"blob1", "blob2", "blob3"};
const std::vector<Slice> keys = {key_strs[0], key_strs[1], key_strs[2]};
const std::vector<Slice> blobs = {blob_strs[0], blob_strs[1], blob_strs[2]};
std::vector<uint64_t> blob_offsets(keys.size());
std::vector<uint64_t> blob_sizes(keys.size());
WriteBlobFile(immutable_options, column_family_id, has_ttl, expiration_range,
expiration_range, blob_file_number, keys, blobs, kNoCompression,
blob_offsets, blob_sizes);
constexpr HistogramImpl* blob_file_read_hist = nullptr;
std::unique_ptr<BlobFileReader> reader;
ReadOptions read_options;
ASSERT_OK(BlobFileReader::Create(
immutable_options, read_options, FileOptions(), column_family_id,
blob_file_read_hist, blob_file_number, nullptr /*IOTracer*/, &reader));
// Enable checksum verification so adjustments are non-zero
read_options.verify_checksums = true;
constexpr MemoryAllocator* allocator = nullptr;
// Fail the first request by giving it an invalid offset (too small).
// This causes the first request to be skipped in the adjustments vector,
// so adjustments has only 2 entries (for blobs 1 and 2), while blob_reqs
// has 3 entries. Without the fix, adjustments[i] would use the wrong
// index for the remaining valid requests.
std::array<Status, num_blobs> statuses_buf;
std::array<BlobReadRequest, num_blobs> requests_buf;
autovector<std::pair<BlobReadRequest*, std::unique_ptr<BlobContents>>>
blob_reqs;
// First request: invalid offset (too small, will fail validation)
requests_buf[0] = BlobReadRequest(keys[0], /*offset=*/0, blob_sizes[0],
kNoCompression, nullptr, &statuses_buf[0]);
// Second and third requests: valid
requests_buf[1] = BlobReadRequest(keys[1], blob_offsets[1], blob_sizes[1],
kNoCompression, nullptr, &statuses_buf[1]);
requests_buf[2] = BlobReadRequest(keys[2], blob_offsets[2], blob_sizes[2],
kNoCompression, nullptr, &statuses_buf[2]);
for (size_t i = 0; i < num_blobs; ++i) {
blob_reqs.emplace_back(&requests_buf[i], std::unique_ptr<BlobContents>());
}
uint64_t bytes_read = 0;
reader->MultiGetBlob(read_options, allocator, blob_reqs, &bytes_read);
// First request should fail validation
ASSERT_TRUE(statuses_buf[0].IsCorruption());
ASSERT_EQ(blob_reqs[0].second, nullptr);
// Second and third requests should succeed with correct data
ASSERT_OK(statuses_buf[1]);
ASSERT_NE(blob_reqs[1].second, nullptr);
ASSERT_EQ(blob_reqs[1].second->data(), blobs[1]);
ASSERT_OK(statuses_buf[2]);
ASSERT_NE(blob_reqs[2].second, nullptr);
ASSERT_EQ(blob_reqs[2].second->data(), blobs[2]);
}
} // namespace ROCKSDB_NAMESPACE
int main(int argc, char** argv) {
+88 -55
View File
@@ -8,70 +8,48 @@
#include "db/blob/blob_index.h"
#include "db/blob/blob_log_format.h"
#include "db/dbformat.h"
#include "db/wide/wide_column_serialization.h"
namespace ROCKSDB_NAMESPACE {
Status BlobGarbageMeter::ProcessInFlow(const Slice& key, const Slice& value) {
uint64_t blob_file_number = kInvalidBlobFileNumber;
uint64_t bytes = 0;
const Status s = Parse(key, value, &blob_file_number, &bytes);
if (!s.ok()) {
return s;
}
if (blob_file_number == kInvalidBlobFileNumber) {
return Status::OK();
}
flows_[blob_file_number].AddInFlow(bytes);
return Status::OK();
return ProcessFlow(key, value, /*is_inflow=*/true);
}
Status BlobGarbageMeter::ProcessOutFlow(const Slice& key, const Slice& value) {
uint64_t blob_file_number = kInvalidBlobFileNumber;
uint64_t bytes = 0;
const Status s = Parse(key, value, &blob_file_number, &bytes);
if (!s.ok()) {
return s;
}
if (blob_file_number == kInvalidBlobFileNumber) {
return Status::OK();
}
// Note: in order to measure the amount of additional garbage, we only need to
// track the outflow for preexisting files, i.e. those that also had inflow.
// (Newly written files would only have outflow.)
auto it = flows_.find(blob_file_number);
if (it == flows_.end()) {
return Status::OK();
}
it->second.AddOutFlow(bytes);
return Status::OK();
return ProcessFlow(key, value, /*is_inflow=*/false);
}
Status BlobGarbageMeter::Parse(const Slice& key, const Slice& value,
uint64_t* blob_file_number, uint64_t* bytes) {
Status BlobGarbageMeter::GetBlobReferenceDetails(const ParsedInternalKey& ikey,
const BlobIndex& blob_index,
uint64_t* blob_file_number,
uint64_t* bytes) {
assert(blob_file_number);
assert(*blob_file_number == kInvalidBlobFileNumber);
assert(bytes);
assert(*bytes == 0);
ParsedInternalKey ikey;
{
constexpr bool log_err_key = false;
const Status s = ParseInternalKey(key, &ikey, log_err_key);
if (!s.ok()) {
return s;
}
if (blob_index.IsInlined() || blob_index.HasTTL()) {
return Status::Corruption("Unexpected TTL/inlined blob index");
}
*blob_file_number = blob_index.file_number();
*bytes =
blob_index.size() +
BlobLogRecord::CalculateAdjustmentForRecordHeader(ikey.user_key.size());
return Status::OK();
}
Status BlobGarbageMeter::ParseBlobIndexReference(const ParsedInternalKey& ikey,
const Slice& value,
uint64_t* blob_file_number,
uint64_t* bytes) {
assert(blob_file_number);
assert(*blob_file_number == kInvalidBlobFileNumber);
assert(bytes);
assert(*bytes == 0);
if (ikey.type != kTypeBlobIndex) {
return Status::OK();
}
@@ -85,16 +63,71 @@ Status BlobGarbageMeter::Parse(const Slice& key, const Slice& value,
}
}
if (blob_index.IsInlined() || blob_index.HasTTL()) {
return Status::Corruption("Unexpected TTL/inlined blob index");
return GetBlobReferenceDetails(ikey, blob_index, blob_file_number, bytes);
}
void BlobGarbageMeter::AddFlow(uint64_t blob_file_number, uint64_t bytes,
bool is_inflow) {
if (is_inflow) {
flows_[blob_file_number].AddInFlow(bytes);
return;
}
*blob_file_number = blob_index.file_number();
*bytes =
blob_index.size() +
BlobLogRecord::CalculateAdjustmentForRecordHeader(ikey.user_key.size());
// Note: in order to measure the amount of additional garbage, we only need to
// track the outflow for preexisting files, i.e. those that also had inflow.
// (Newly written files would only have outflow.)
auto it = flows_.find(blob_file_number);
if (it != flows_.end()) {
it->second.AddOutFlow(bytes);
}
}
return Status::OK();
Status BlobGarbageMeter::ProcessFlow(const Slice& key, const Slice& value,
bool is_inflow) {
ParsedInternalKey ikey;
{
constexpr bool log_err_key = false;
const Status s = ParseInternalKey(key, &ikey, log_err_key);
if (!s.ok()) {
return s;
}
}
uint64_t blob_file_number = kInvalidBlobFileNumber;
uint64_t bytes = 0;
if (Status s =
ParseBlobIndexReference(ikey, value, &blob_file_number, &bytes);
!s.ok()) {
return s;
}
if (blob_file_number != kInvalidBlobFileNumber) {
AddFlow(blob_file_number, bytes, is_inflow);
return Status::OK();
}
return ProcessEntityBlobReferences(ikey, value, is_inflow);
}
Status BlobGarbageMeter::ProcessEntityBlobReferences(
const ParsedInternalKey& ikey, const Slice& value, bool is_inflow) {
if (ikey.type != kTypeWideColumnEntity) {
return Status::OK();
}
return WideColumnSerialization::ForEachBlobFileNumber(
value, [&](const BlobIndex& blob_index) -> Status {
uint64_t file_number = kInvalidBlobFileNumber;
uint64_t blob_bytes = 0;
if (const Status s = GetBlobReferenceDetails(ikey, blob_index,
&file_number, &blob_bytes);
!s.ok()) {
return s;
}
AddFlow(file_number, blob_bytes, is_inflow);
return Status::OK();
});
}
} // namespace ROCKSDB_NAMESPACE
+15 -2
View File
@@ -15,6 +15,8 @@
namespace ROCKSDB_NAMESPACE {
class BlobIndex;
struct ParsedInternalKey;
class Slice;
// A class that can be used to compute the amount of additional garbage
@@ -93,8 +95,19 @@ class BlobGarbageMeter {
}
private:
static Status Parse(const Slice& key, const Slice& value,
uint64_t* blob_file_number, uint64_t* bytes);
static Status GetBlobReferenceDetails(const ParsedInternalKey& ikey,
const BlobIndex& blob_index,
uint64_t* blob_file_number,
uint64_t* bytes);
static Status ParseBlobIndexReference(const ParsedInternalKey& ikey,
const Slice& value,
uint64_t* blob_file_number,
uint64_t* bytes);
void AddFlow(uint64_t blob_file_number, uint64_t bytes, bool is_inflow);
Status ProcessFlow(const Slice& key, const Slice& value, bool is_inflow);
Status ProcessEntityBlobReferences(const ParsedInternalKey& ikey,
const Slice& value, bool is_inflow);
std::unordered_map<uint64_t, BlobInOutFlow> flows_;
};
+74
View File
@@ -11,10 +11,23 @@
#include "db/blob/blob_index.h"
#include "db/blob/blob_log_format.h"
#include "db/dbformat.h"
#include "db/wide/wide_column_serialization.h"
#include "test_util/testharness.h"
namespace ROCKSDB_NAMESPACE {
namespace {
BlobIndex MakeBlobIndex(uint64_t file_number, uint64_t offset, uint64_t size) {
std::string encoded;
BlobIndex::EncodeBlob(&encoded, file_number, offset, size, kNoCompression);
BlobIndex blob_index;
EXPECT_OK(blob_index.DecodeFrom(encoded));
return blob_index;
}
} // namespace
TEST(BlobGarbageMeterTest, MeasureGarbage) {
BlobGarbageMeter blob_garbage_meter;
@@ -188,6 +201,67 @@ TEST(BlobGarbageMeterTest, InlinedTTLBlobIndex) {
ASSERT_NOK(blob_garbage_meter.ProcessOutFlow(key_slice, value_slice));
}
TEST(BlobGarbageMeterTest, WideColumnEntity) {
constexpr char user_key[] = "entity_key";
constexpr SequenceNumber seq = 123;
const InternalKey key(user_key, seq, kTypeWideColumnEntity);
const Slice key_slice = key.Encode();
const std::vector<std::pair<std::string, std::string>> columns = {
{"", "default_inline"},
{"blob_attr", "blob_inline"},
{"ttl", "00000001"},
};
const BlobIndex default_blob = MakeBlobIndex(4, 123, 555);
const BlobIndex attr_blob = MakeBlobIndex(5, 456, 777);
std::string inflow_value;
ASSERT_OK(WideColumnSerialization::SerializeV2(
columns, {{0, default_blob}, {1, attr_blob}}, inflow_value));
std::string outflow_value;
ASSERT_OK(WideColumnSerialization::SerializeV2(columns, {{0, default_blob}},
outflow_value));
BlobGarbageMeter blob_garbage_meter;
ASSERT_OK(blob_garbage_meter.ProcessInFlow(key_slice, inflow_value));
ASSERT_OK(blob_garbage_meter.ProcessOutFlow(key_slice, outflow_value));
const auto& flows = blob_garbage_meter.flows();
ASSERT_EQ(flows.size(), 2);
constexpr size_t kUserKeySize = sizeof(user_key) - 1;
const uint64_t default_bytes =
default_blob.size() +
BlobLogRecord::CalculateAdjustmentForRecordHeader(kUserKeySize);
const uint64_t attr_bytes =
attr_blob.size() +
BlobLogRecord::CalculateAdjustmentForRecordHeader(kUserKeySize);
{
const auto it = flows.find(default_blob.file_number());
ASSERT_NE(it, flows.end());
ASSERT_EQ(it->second.GetInFlow().GetCount(), 1);
ASSERT_EQ(it->second.GetInFlow().GetBytes(), default_bytes);
ASSERT_EQ(it->second.GetOutFlow().GetCount(), 1);
ASSERT_EQ(it->second.GetOutFlow().GetBytes(), default_bytes);
ASSERT_FALSE(it->second.HasGarbage());
}
{
const auto it = flows.find(attr_blob.file_number());
ASSERT_NE(it, flows.end());
ASSERT_EQ(it->second.GetInFlow().GetCount(), 1);
ASSERT_EQ(it->second.GetInFlow().GetBytes(), attr_bytes);
ASSERT_EQ(it->second.GetOutFlow().GetCount(), 0);
ASSERT_EQ(it->second.GetOutFlow().GetBytes(), 0);
ASSERT_TRUE(it->second.HasGarbage());
ASSERT_EQ(it->second.GetGarbageCount(), 1);
ASSERT_EQ(it->second.GetGarbageBytes(), attr_bytes);
}
}
} // namespace ROCKSDB_NAMESPACE
int main(int argc, char** argv) {
+12
View File
@@ -137,6 +137,18 @@ class BlobIndex {
return oss.str();
}
// Encode this blob index into dst based on its type.
void EncodeTo(std::string* dst) const {
if (IsInlined()) {
EncodeInlinedTTL(dst, expiration_, value_);
} else if (HasTTL()) {
EncodeBlobTTL(dst, expiration_, file_number_, offset_, size_,
compression_);
} else {
EncodeBlob(dst, file_number_, offset_, size_, compression_);
}
}
static void EncodeInlinedTTL(std::string* dst, uint64_t expiration,
const Slice& value) {
assert(dst != nullptr);
+17 -4
View File
@@ -148,13 +148,26 @@ struct BlobLogRecord {
// Checks whether a blob offset is potentially valid or not.
inline bool IsValidBlobOffset(uint64_t value_offset, uint64_t key_size,
uint64_t value_size, uint64_t file_size) {
if (value_offset <
BlobLogHeader::kSize + BlobLogRecord::kHeaderSize + key_size) {
uint64_t value_size, uint64_t file_size,
bool has_footer = true) {
constexpr uint64_t kMinPrefix =
BlobLogHeader::kSize + BlobLogRecord::kHeaderSize;
if (value_offset < kMinPrefix) {
return false;
}
if (value_offset - kMinPrefix < key_size) {
return false;
}
if (value_offset + value_size + BlobLogFooter::kSize > file_size) {
const uint64_t footer_size = has_footer ? BlobLogFooter::kSize : 0;
if (file_size < footer_size) {
return false;
}
const uint64_t max_value_end = file_size - footer_size;
if (value_size > max_value_end) {
return false;
}
if (value_offset > max_value_end - value_size) {
return false;
}
+130
View File
@@ -20,6 +20,21 @@
namespace ROCKSDB_NAMESPACE {
namespace {
Status AppendBlobRefreshRetryFailure(const Status& stale_status,
const Status& retry_status) {
assert(stale_status.IsCorruption());
assert(!retry_status.ok());
if (retry_status.IsCorruption()) {
return retry_status;
}
return Status::CopyAppendMessage(
stale_status, "; refresh retry failed: ", retry_status.ToString());
}
} // namespace
BlobSource::BlobSource(const ImmutableOptions& immutable_options,
const MutableCFOptions& mutable_cf_options,
const std::string& db_id,
@@ -231,6 +246,38 @@ Status BlobSource::GetBlob(const ReadOptions& read_options,
s = blob_file_reader.GetValue()->GetBlob(
read_options, user_key, offset, value_size, compression_type,
prefetch_buffer, allocator, &blob_contents, &read_size);
if (s.IsCorruption()) {
const Status stale_status = s;
blob_file_reader.Reset();
blob_file_cache_->Evict(file_number);
std::unique_ptr<BlobFileReader> fresh_reader;
s = blob_file_cache_->OpenBlobFileReaderUncached(
read_options, file_number, &fresh_reader,
/*allow_footer_skip_retry=*/false);
if (!s.ok()) {
return AppendBlobRefreshRetryFailure(stale_status, s);
}
if (compression_type != fresh_reader->GetCompressionType()) {
return Status::Corruption(
"Compression type mismatch when reading blob");
}
blob_contents.reset();
read_size = 0;
s = fresh_reader->GetBlob(read_options, user_key, offset, value_size,
compression_type, prefetch_buffer, allocator,
&blob_contents, &read_size);
if (!s.ok()) {
s = AppendBlobRefreshRetryFailure(stale_status, s);
} else {
CacheHandleGuard<BlobFileReader> ignored_reader;
blob_file_cache_
->RefreshBlobFileReader(file_number, &fresh_reader, &ignored_reader)
.PermitUncheckedError();
}
}
if (!s.ok()) {
return s;
}
@@ -397,6 +444,89 @@ void BlobSource::MultiGetBlobFromOneFile(const ReadOptions& read_options,
blob_file_reader.GetValue()->MultiGetBlob(read_options, allocator,
_blob_reqs, &_bytes_read);
bool needs_reader_refresh = false;
for (const auto& blob_req : _blob_reqs) {
BlobReadRequest* const req = blob_req.first;
assert(req != nullptr);
assert(req->status != nullptr);
if (req->status->IsCorruption()) {
needs_reader_refresh = true;
break;
}
}
if (needs_reader_refresh) {
blob_file_reader.Reset();
blob_file_cache_->Evict(file_number);
std::unique_ptr<BlobFileReader> fresh_reader;
s = blob_file_cache_->OpenBlobFileReaderUncached(
read_options, file_number, &fresh_reader,
/*allow_footer_skip_retry=*/false);
if (!s.ok()) {
for (const auto& blob_req : _blob_reqs) {
BlobReadRequest* const req = blob_req.first;
assert(req != nullptr);
assert(req->status != nullptr);
if (req->status->IsCorruption()) {
*req->status = AppendBlobRefreshRetryFailure(*req->status, s);
}
}
return;
}
autovector<std::pair<BlobReadRequest*, std::unique_ptr<BlobContents>>>
retry_blob_reqs;
autovector<Status> stale_statuses;
for (auto& blob_req : _blob_reqs) {
BlobReadRequest* const req = blob_req.first;
assert(req != nullptr);
assert(req->status != nullptr);
if (!req->status->IsCorruption()) {
continue;
}
stale_statuses.emplace_back(*req->status);
*req->status = Status::OK();
blob_req.second.reset();
retry_blob_reqs.emplace_back(req, std::unique_ptr<BlobContents>());
}
uint64_t refreshed_bytes_read = 0;
fresh_reader->MultiGetBlob(read_options, allocator, retry_blob_reqs,
&refreshed_bytes_read);
_bytes_read += refreshed_bytes_read;
bool install_fresh_reader = false;
for (size_t i = 0; i < retry_blob_reqs.size(); ++i) {
auto& retried_blob_req = retry_blob_reqs[i];
BlobReadRequest* const retried_req = retried_blob_req.first;
assert(retried_req != nullptr);
if (retried_req->status->ok()) {
install_fresh_reader = true;
} else {
*retried_req->status = AppendBlobRefreshRetryFailure(
stale_statuses[i], *retried_req->status);
}
for (auto& blob_req : _blob_reqs) {
if (blob_req.first != retried_req) {
continue;
}
blob_req.second = std::move(retried_blob_req.second);
break;
}
}
if (install_fresh_reader) {
CacheHandleGuard<BlobFileReader> ignored_reader;
blob_file_cache_
->RefreshBlobFileReader(file_number, &fresh_reader, &ignored_reader)
.PermitUncheckedError();
}
}
if (blob_cache_ && read_options.fill_cache) {
// If filling cache is allowed and a cache is configured, try to put
// the blob(s) to the cache.
+150
View File
@@ -1044,6 +1044,156 @@ TEST_F(BlobSourceTest, MultiGetBlobsFromCache) {
}
}
TEST_F(BlobSourceTest, GetBlobPreservesCorruptionDetailsWhenRefreshOpenFails) {
options_.cf_paths.emplace_back(
test::PerThreadDBPath(env_,
"BlobSourceTest_GetBlobPreservesCorruptionDetails"),
0);
DestroyAndReopen(options_);
ImmutableOptions immutable_options(options_);
MutableCFOptions mutable_cf_options(options_);
constexpr uint32_t column_family_id = 1;
constexpr bool has_ttl = false;
constexpr ExpirationRange expiration_range;
constexpr uint64_t blob_file_number = 1;
const std::string key_str = "key0";
const std::string blob_str = "blob0";
std::vector<Slice> keys{Slice(key_str)};
std::vector<Slice> blobs{Slice(blob_str)};
std::vector<uint64_t> blob_offsets(1);
std::vector<uint64_t> blob_sizes(1);
const uint64_t file_size = BlobLogHeader::kSize + BlobLogRecord::kHeaderSize +
key_str.size() + blob_str.size() +
BlobLogFooter::kSize;
WriteBlobFile(immutable_options, column_family_id, has_ttl, expiration_range,
expiration_range, blob_file_number, keys, blobs, kNoCompression,
blob_offsets, blob_sizes);
constexpr size_t capacity = 1024;
std::shared_ptr<Cache> backing_cache = NewLRUCache(capacity);
FileOptions file_options;
std::unique_ptr<BlobFileCache> blob_file_cache =
std::make_unique<BlobFileCache>(
backing_cache.get(), &immutable_options, &file_options,
column_family_id, nullptr /*HistogramImpl*/, nullptr /*IOTracer*/);
BlobSource blob_source(immutable_options, mutable_cf_options, db_id_,
db_session_id_, blob_file_cache.get());
ReadOptions read_options;
read_options.verify_checksums = true;
constexpr FilePrefetchBuffer* prefetch_buffer = nullptr;
PinnableSlice value;
uint64_t bytes_read = 0;
ASSERT_OK(blob_source.GetBlob(
read_options, keys[0], blob_file_number, blob_offsets[0], file_size,
blob_sizes[0], kNoCompression, prefetch_buffer, &value, &bytes_read));
ASSERT_EQ(value, blobs[0]);
const std::string blob_file_path =
BlobFileName(immutable_options.cf_paths.front().path, blob_file_number);
ASSERT_OK(env_->DeleteFile(blob_file_path));
PinnableSlice invalid_value;
bytes_read = 0;
Status s =
blob_source.GetBlob(read_options, keys[0], blob_file_number, file_size,
file_size, blob_sizes[0], kNoCompression,
prefetch_buffer, &invalid_value, &bytes_read);
ASSERT_TRUE(s.IsCorruption());
ASSERT_EQ(bytes_read, 0);
const std::string status_str = s.ToString();
ASSERT_NE(status_str.find("Invalid blob offset"), std::string::npos);
ASSERT_NE(status_str.find("refresh retry failed"), std::string::npos);
ASSERT_NE(status_str.find("IO error"), std::string::npos);
}
TEST_F(BlobSourceTest,
MultiGetBlobPreservesCorruptionDetailsWhenRefreshOpenFails) {
options_.cf_paths.emplace_back(
test::PerThreadDBPath(
env_, "BlobSourceTest_MultiGetBlobPreservesCorruptionDetails"),
0);
DestroyAndReopen(options_);
ImmutableOptions immutable_options(options_);
MutableCFOptions mutable_cf_options(options_);
constexpr uint32_t column_family_id = 1;
constexpr bool has_ttl = false;
constexpr ExpirationRange expiration_range;
constexpr uint64_t blob_file_number = 1;
const std::string key_str = "key0";
const std::string blob_str = "blob0";
std::vector<Slice> keys{Slice(key_str)};
std::vector<Slice> blobs{Slice(blob_str)};
std::vector<uint64_t> blob_offsets(1);
std::vector<uint64_t> blob_sizes(1);
const uint64_t file_size = BlobLogHeader::kSize + BlobLogRecord::kHeaderSize +
key_str.size() + blob_str.size() +
BlobLogFooter::kSize;
WriteBlobFile(immutable_options, column_family_id, has_ttl, expiration_range,
expiration_range, blob_file_number, keys, blobs, kNoCompression,
blob_offsets, blob_sizes);
constexpr size_t capacity = 1024;
std::shared_ptr<Cache> backing_cache = NewLRUCache(capacity);
FileOptions file_options;
std::unique_ptr<BlobFileCache> blob_file_cache =
std::make_unique<BlobFileCache>(
backing_cache.get(), &immutable_options, &file_options,
column_family_id, nullptr /*HistogramImpl*/, nullptr /*IOTracer*/);
BlobSource blob_source(immutable_options, mutable_cf_options, db_id_,
db_session_id_, blob_file_cache.get());
ReadOptions read_options;
read_options.verify_checksums = true;
constexpr FilePrefetchBuffer* prefetch_buffer = nullptr;
PinnableSlice value;
uint64_t bytes_read = 0;
ASSERT_OK(blob_source.GetBlob(
read_options, keys[0], blob_file_number, blob_offsets[0], file_size,
blob_sizes[0], kNoCompression, prefetch_buffer, &value, &bytes_read));
ASSERT_EQ(value, blobs[0]);
const std::string blob_file_path =
BlobFileName(immutable_options.cf_paths.front().path, blob_file_number);
ASSERT_OK(env_->DeleteFile(blob_file_path));
std::array<Status, 1> statuses;
std::array<PinnableSlice, 1> values;
autovector<BlobReadRequest> blob_reqs;
blob_reqs.emplace_back(keys[0], file_size, blob_sizes[0], kNoCompression,
&values[0], &statuses[0]);
bytes_read = 0;
blob_source.MultiGetBlobFromOneFile(read_options, blob_file_number, file_size,
blob_reqs, &bytes_read);
ASSERT_TRUE(statuses[0].IsCorruption());
ASSERT_EQ(bytes_read, 0);
const std::string status_str = statuses[0].ToString();
ASSERT_NE(status_str.find("Invalid blob offset"), std::string::npos);
ASSERT_NE(status_str.find("refresh retry failed"), std::string::npos);
ASSERT_NE(status_str.find("IO error"), std::string::npos);
}
class BlobSecondaryCacheTest : public DBTestBase {
protected:
public:
+316
View File
@@ -0,0 +1,316 @@
// Copyright (c) Meta Platforms, Inc. and affiliates.
// This source code is licensed under both the GPLv2 (found in the
// COPYING file in the root directory) and Apache 2.0 License
// (found in the LICENSE.Apache file in the root directory).
// Copyright (c) 2011-present, Facebook, Inc. All rights reserved.
// This source code is licensed under both the GPLv2 (found in the
// COPYING file in the root directory) and Apache 2.0 License
// (found in the LICENSE.Apache file in the root directory).
#include "db/blob/blob_write_batch_transformer.h"
#include "db/blob/blob_file_partition_manager.h"
#include "db/blob/blob_index.h"
#include "db/blob/blob_log_format.h"
#include "db/wide/wide_column_serialization.h"
#include "db/write_batch_internal.h"
#include "port/likely.h"
namespace ROCKSDB_NAMESPACE {
BlobWriteBatchTransformer::BlobWriteBatchTransformer(
const BlobPartitionManagerProvider& partition_mgr_provider,
WriteBatch* output_batch,
const BlobDirectWriteSettingsProvider& settings_provider,
const WriteOptions& write_options)
: partition_mgr_provider_(partition_mgr_provider),
output_batch_(output_batch),
settings_provider_(settings_provider),
write_options_(write_options) {
assert(partition_mgr_provider_);
assert(output_batch_);
assert(settings_provider_);
}
Status BlobWriteBatchTransformer::TransformBatch(
const WriteOptions& write_options, WriteBatch* input_batch,
WriteBatch* output_batch,
const BlobPartitionManagerProvider& partition_mgr_provider,
const BlobDirectWriteSettingsProvider& settings_provider, bool* transformed,
std::vector<BlobFilePartitionManager*>* used_managers,
std::vector<RollbackInfo>* rollback_infos) {
assert(input_batch);
assert(output_batch);
assert(transformed);
output_batch->Clear();
*transformed = false;
BlobWriteBatchTransformer transformer(partition_mgr_provider, output_batch,
settings_provider, write_options);
Status s = input_batch->Iterate(&transformer);
*transformed = transformer.HasTransformed();
if (used_managers) {
used_managers->assign(transformer.used_managers_.begin(),
transformer.used_managers_.end());
}
if (rollback_infos) {
*rollback_infos = std::move(transformer.rollback_infos_);
}
return s;
}
Status BlobWriteBatchTransformer::MaybePreprocessWideColumns(
const WriteOptions& write_options, uint32_t column_family_id,
const Slice& key, const WideColumns& columns,
BlobFilePartitionManager* partition_mgr,
const BlobDirectWriteSettings& settings, bool serialize_inline_entity,
std::string* entity, bool* transformed,
std::vector<RollbackInfo>* rollback_infos) {
assert(entity != nullptr);
assert(transformed != nullptr);
*transformed = false;
if (partition_mgr == nullptr || !settings.enable_blob_direct_write) {
if (!serialize_inline_entity) {
return Status::OK();
}
entity->clear();
return WideColumnSerialization::Serialize(columns, *entity);
}
std::vector<std::pair<size_t, BlobIndex>> new_blob_columns;
new_blob_columns.reserve(columns.size());
std::string blob_index_buf;
bool has_entity_partition = false;
uint32_t entity_partition = 0;
for (size_t column_idx = 0; column_idx < columns.size(); ++column_idx) {
const Slice& column_value = columns[column_idx].value();
if (column_value.size() < settings.min_blob_size) {
continue;
}
if (!has_entity_partition) {
entity_partition = partition_mgr->SelectWideColumnPartition(
column_family_id, key, columns);
has_entity_partition = true;
}
uint64_t blob_file_number = 0;
uint64_t blob_offset = 0;
uint64_t blob_size = 0;
Status s = partition_mgr->WriteBlob(
write_options, column_family_id, settings.compression_type, key,
column_value, &blob_file_number, &blob_offset, &blob_size, &settings,
&entity_partition);
if (!s.ok()) {
return s;
}
if (rollback_infos != nullptr) {
const uint64_t record_bytes =
BlobLogRecord::kHeaderSize + key.size() + blob_size;
rollback_infos->push_back(
{partition_mgr, blob_file_number, /*count=*/1, record_bytes});
}
BlobIndex blob_index;
BlobIndex::EncodeBlob(&blob_index_buf, blob_file_number, blob_offset,
blob_size, settings.compression_type);
s = blob_index.DecodeFrom(blob_index_buf);
if (!s.ok()) {
return s;
}
new_blob_columns.emplace_back(column_idx, blob_index);
}
if (new_blob_columns.empty()) {
if (!serialize_inline_entity) {
return Status::OK();
}
entity->clear();
return WideColumnSerialization::Serialize(columns, *entity);
}
entity->clear();
Status s =
WideColumnSerialization::SerializeV2(columns, new_blob_columns, *entity);
if (s.ok()) {
*transformed = true;
}
return s;
}
Status BlobWriteBatchTransformer::PutCF(uint32_t column_family_id,
const Slice& key, const Slice& value) {
// Use cached settings/manager for the same CF to avoid per-entry lookup.
if (column_family_id != cached_cf_id_) {
cached_settings_ = settings_provider_(column_family_id);
cached_partition_mgr_ = partition_mgr_provider_(column_family_id);
cached_cf_id_ = column_family_id;
}
const auto& settings = cached_settings_;
if (!cached_partition_mgr_ || !settings.enable_blob_direct_write ||
value.size() < settings.min_blob_size) {
return WriteBatchInternal::Put(output_batch_, column_family_id, key, value);
}
uint64_t blob_file_number = 0;
uint64_t blob_offset = 0;
uint64_t blob_size = 0;
Status s = cached_partition_mgr_->WriteBlob(
write_options_, column_family_id, settings.compression_type, key, value,
&blob_file_number, &blob_offset, &blob_size, &settings);
if (!s.ok()) {
return s;
}
used_managers_.insert(cached_partition_mgr_);
// Track the exact file so stale transformed attempts can rollback
// per-file rather than smearing bytes across all partitions at seal time.
const uint64_t record_bytes =
BlobLogRecord::kHeaderSize + key.size() + blob_size;
rollback_infos_.push_back(
{cached_partition_mgr_, blob_file_number, /*count=*/1, record_bytes});
BlobIndex::EncodeBlob(&blob_index_buf_, blob_file_number, blob_offset,
blob_size, settings.compression_type);
has_transformed_ = true;
return WriteBatchInternal::PutBlobIndex(output_batch_, column_family_id, key,
blob_index_buf_);
}
Status BlobWriteBatchTransformer::TimedPutCF(uint32_t column_family_id,
const Slice& key,
const Slice& value,
uint64_t write_time) {
// TimedPut: pass through without blob separation for now.
return WriteBatchInternal::TimedPut(output_batch_, column_family_id, key,
value, write_time);
}
Status BlobWriteBatchTransformer::PutEntityCF(uint32_t column_family_id,
const Slice& key,
const Slice& entity) {
if (column_family_id != cached_cf_id_) {
cached_settings_ = settings_provider_(column_family_id);
cached_partition_mgr_ = partition_mgr_provider_(column_family_id);
cached_cf_id_ = column_family_id;
}
const auto& settings = cached_settings_;
if (!cached_partition_mgr_ || !settings.enable_blob_direct_write) {
return WriteBatchInternal::PutEntitySerialized(
output_batch_, column_family_id, key, entity);
}
Slice entity_ref = entity;
std::vector<WideColumn> columns;
std::vector<std::pair<size_t, BlobIndex>> existing_blob_columns;
Status status = WideColumnSerialization::DeserializeV2(entity_ref, columns,
existing_blob_columns);
if (status.ok()) {
// Reject pre-serialized entities that already contain blob references.
// Passing them through would preserve references to blob files outside this
// batch's direct-write lifetime tracking, which can later turn into stale
// reads after blob GC.
if (UNLIKELY(!existing_blob_columns.empty())) {
return Status::NotSupported(
"Blob direct write does not support pre-serialized wide entities "
"with blob references");
}
std::string rewritten_entity;
bool transformed = false;
status = MaybePreprocessWideColumns(
write_options_, column_family_id, key, columns, cached_partition_mgr_,
settings, /*serialize_inline_entity=*/false, &rewritten_entity,
&transformed, &rollback_infos_);
if (status.ok()) {
if (!transformed) {
return WriteBatchInternal::PutEntitySerialized(
output_batch_, column_family_id, key, entity);
}
has_transformed_ = true;
used_managers_.insert(cached_partition_mgr_);
return WriteBatchInternal::PutEntitySerialized(
output_batch_, column_family_id, key, rewritten_entity);
}
}
return status;
}
Status BlobWriteBatchTransformer::DeleteCF(uint32_t column_family_id,
const Slice& key) {
return WriteBatchInternal::Delete(output_batch_, column_family_id, key);
}
Status BlobWriteBatchTransformer::SingleDeleteCF(uint32_t column_family_id,
const Slice& key) {
return WriteBatchInternal::SingleDelete(output_batch_, column_family_id, key);
}
Status BlobWriteBatchTransformer::DeleteRangeCF(uint32_t column_family_id,
const Slice& begin_key,
const Slice& end_key) {
return WriteBatchInternal::DeleteRange(output_batch_, column_family_id,
begin_key, end_key);
}
Status BlobWriteBatchTransformer::MergeCF(uint32_t column_family_id,
const Slice& key,
const Slice& value) {
return WriteBatchInternal::Merge(output_batch_, column_family_id, key, value);
}
Status BlobWriteBatchTransformer::PutBlobIndexCF(uint32_t column_family_id,
const Slice& key,
const Slice& value) {
// Already a blob index — pass through unchanged.
return WriteBatchInternal::PutBlobIndex(output_batch_, column_family_id, key,
value);
}
void BlobWriteBatchTransformer::LogData(const Slice& blob) {
output_batch_->PutLogData(blob).PermitUncheckedError();
}
Status BlobWriteBatchTransformer::MarkBeginPrepare(bool unprepared) {
return WriteBatchInternal::InsertBeginPrepare(
output_batch_, !unprepared /* write_after_commit */, unprepared);
}
Status BlobWriteBatchTransformer::MarkEndPrepare(const Slice& xid) {
return WriteBatchInternal::InsertEndPrepare(output_batch_, xid);
}
Status BlobWriteBatchTransformer::MarkCommit(const Slice& xid) {
return WriteBatchInternal::MarkCommit(output_batch_, xid);
}
Status BlobWriteBatchTransformer::MarkCommitWithTimestamp(const Slice& xid,
const Slice& ts) {
return WriteBatchInternal::MarkCommitWithTimestamp(output_batch_, xid, ts);
}
Status BlobWriteBatchTransformer::MarkRollback(const Slice& xid) {
return WriteBatchInternal::MarkRollback(output_batch_, xid);
}
Status BlobWriteBatchTransformer::MarkNoop(bool /*empty_batch*/) {
return WriteBatchInternal::InsertNoop(output_batch_);
}
} // namespace ROCKSDB_NAMESPACE
+174
View File
@@ -0,0 +1,174 @@
// Copyright (c) Meta Platforms, Inc. and affiliates.
// This source code is licensed under both the GPLv2 (found in the
// COPYING file in the root directory) and Apache 2.0 License
// (found in the LICENSE.Apache file in the root directory).
#pragma once
#include <cstdint>
#include <functional>
#include <memory>
#include <string>
#include <unordered_map>
#include <vector>
#include "rocksdb/advanced_options.h"
#include "rocksdb/compression_type.h"
#include "rocksdb/options.h"
#include "rocksdb/rocksdb_namespace.h"
#include "rocksdb/slice.h"
#include "rocksdb/status.h"
#include "rocksdb/write_batch.h"
#include "util/hash_containers.h"
namespace ROCKSDB_NAMESPACE {
class BlobFilePartitionManager;
class Cache;
// Callback to look up per-CF blob settings.
struct BlobDirectWriteSettings {
// Whether this column family should attempt direct-write blob separation.
bool enable_blob_direct_write = false;
// Minimum value size eligible for direct-write blob separation.
uint64_t min_blob_size = 0;
// Compression to use for newly written blob records.
CompressionType compression_type = kNoCompression;
// Raw pointer — the Cache is owned by ColumnFamilyOptions and outlives all
// settings snapshots. Using raw avoids 2 atomic ref-count ops per Put().
Cache* blob_cache = nullptr;
// Blob-cache prepopulation policy for direct-write records.
PrepopulateBlobCache prepopulate_blob_cache = PrepopulateBlobCache::kDisable;
};
using BlobDirectWriteSettingsProvider =
std::function<BlobDirectWriteSettings(uint32_t column_family_id)>;
// Callback to look up per-CF partition manager.
using BlobPartitionManagerProvider =
std::function<BlobFilePartitionManager*(uint32_t column_family_id)>;
// Transforms a WriteBatch by writing large values directly to blob files
// and replacing them with BlobIndex entries. Non-qualifying entries
// (small values, deletes, merges, etc.) are passed through unchanged.
class BlobWriteBatchTransformer : public WriteBatch::Handler {
public:
struct RollbackInfo {
// Manager that owns the file updated during transform.
BlobFilePartitionManager* partition_mgr = nullptr;
// Blob file number written during transform.
uint64_t file_number = 0;
// Number of blob records appended to that file.
uint64_t count = 0;
// Number of bytes appended to that file.
uint64_t bytes = 0;
};
// Builds the final serialized entity for one PutEntity write.
//
// `columns` must already be sorted by column name. When direct-write blob
// separation is enabled and one or more columns meet `min_blob_size`, this
// writes those columns to blob files and returns a V2 entity containing
// BlobIndex references. Otherwise, if `serialize_inline_entity` is true, it
// returns a normal V1 entity with all columns inline.
//
// `rollback_infos`, when provided, is appended with the exact blob writes
// performed before the method returns, even if it later fails, so callers
// can account those bytes as initial garbage.
static Status MaybePreprocessWideColumns(
const WriteOptions& write_options, uint32_t column_family_id,
const Slice& key, const WideColumns& columns,
BlobFilePartitionManager* partition_mgr,
const BlobDirectWriteSettings& settings, bool serialize_inline_entity,
std::string* entity, bool* transformed,
std::vector<RollbackInfo>* rollback_infos = nullptr);
// Constructs a per-batch transformer. Most callers should use
// TransformBatch() instead of driving the handler directly.
BlobWriteBatchTransformer(
const BlobPartitionManagerProvider& partition_mgr_provider,
WriteBatch* output_batch,
const BlobDirectWriteSettingsProvider& settings_provider,
const WriteOptions& write_options);
// Transform a WriteBatch. If no values qualify for blob separation,
// output_batch will be empty and the caller should use the original batch.
// If any values are separated, output_batch contains the full transformed
// batch. used_managers (if non-null) receives the set of partition managers
// that had data written to them, so the caller can flush/sync them.
// rollback_infos (if non-null) receives the exact file/count/byte writes so
// a failed transformed attempt can be accounted as initial garbage. Partial
// results are returned even if this method fails after some blob writes.
static Status TransformBatch(
const WriteOptions& write_options, WriteBatch* input_batch,
WriteBatch* output_batch,
const BlobPartitionManagerProvider& partition_mgr_provider,
const BlobDirectWriteSettingsProvider& settings_provider,
bool* transformed,
std::vector<BlobFilePartitionManager*>* used_managers = nullptr,
std::vector<RollbackInfo>* rollback_infos = nullptr);
// WriteBatch::Handler overrides
Status PutCF(uint32_t column_family_id, const Slice& key,
const Slice& value) override;
Status TimedPutCF(uint32_t column_family_id, const Slice& key,
const Slice& value, uint64_t write_time) override;
Status PutEntityCF(uint32_t column_family_id, const Slice& key,
const Slice& entity) override;
Status DeleteCF(uint32_t column_family_id, const Slice& key) override;
Status SingleDeleteCF(uint32_t column_family_id, const Slice& key) override;
Status DeleteRangeCF(uint32_t column_family_id, const Slice& begin_key,
const Slice& end_key) override;
Status MergeCF(uint32_t column_family_id, const Slice& key,
const Slice& value) override;
Status PutBlobIndexCF(uint32_t column_family_id, const Slice& key,
const Slice& value) override;
void LogData(const Slice& blob) override;
Status MarkBeginPrepare(bool unprepared = false) override;
Status MarkEndPrepare(const Slice& xid) override;
Status MarkCommit(const Slice& xid) override;
Status MarkCommitWithTimestamp(const Slice& xid, const Slice& ts) override;
Status MarkRollback(const Slice& xid) override;
Status MarkNoop(bool empty_batch) override;
// Returns true once at least one value has been rewritten to a BlobIndex.
bool HasTransformed() const { return has_transformed_; }
private:
// Callback to look up the partition manager for a given column family ID.
BlobPartitionManagerProvider partition_mgr_provider_;
// Output batch that receives transformed entries (BlobIndex for qualifying
// values, passthrough for everything else).
WriteBatch* output_batch_;
// Callback to look up blob direct write settings for a given CF ID.
BlobDirectWriteSettingsProvider settings_provider_;
// Write options from the caller, forwarded to WriteBlob calls.
const WriteOptions& write_options_;
// True once at least one value has been separated into a blob file.
bool has_transformed_ = false;
// Reusable buffer for encoding BlobIndex entries (avoids per-Put alloc).
std::string blob_index_buf_;
// Per-batch cache of the last CF's settings and manager, avoiding
// redundant provider lookups when consecutive entries share the same CF.
uint32_t cached_cf_id_ = UINT32_MAX;
// Cached settings for `cached_cf_id_`.
BlobDirectWriteSettings cached_settings_;
// Cached partition manager for `cached_cf_id_`.
BlobFilePartitionManager* cached_partition_mgr_ = nullptr;
// Set of partition managers that received data during this batch,
// returned to the caller so it can flush/sync them.
UnorderedSet<BlobFilePartitionManager*> used_managers_;
// Exact blob writes performed during this batch. We only aggregate these
// entries if rollback is needed so the normal path keeps minimal overhead.
std::vector<RollbackInfo> rollback_infos_;
};
} // namespace ROCKSDB_NAMESPACE
+181
View File
@@ -3,17 +3,29 @@
// COPYING file in the root directory) and Apache 2.0 License
// (found in the LICENSE.Apache file in the root directory).
#include <algorithm>
#include <array>
#include <set>
#include <sstream>
#include <string>
#include "cache/compressed_secondary_cache.h"
#include "db/blob/blob_file_partition_manager.h"
#include "db/blob/blob_index.h"
#include "db/blob/blob_log_format.h"
#include "db/blob/blob_log_sequential_reader.h"
#include "db/column_family.h"
#include "db/db_test_util.h"
#include "db/db_with_timestamp_test_util.h"
#include "file/filename.h"
#include "file/random_access_file_reader.h"
#include "port/stack_trace.h"
#include "rocksdb/convenience.h"
#include "rocksdb/trace_reader_writer.h"
#include "rocksdb/trace_record.h"
#include "rocksdb/utilities/replayer.h"
#include "test_util/sync_point.h"
#include "util/compression.h"
#include "utilities/fault_injection_env.h"
namespace ROCKSDB_NAMESPACE {
@@ -51,6 +63,43 @@ TEST_F(DBBlobBasicTest, GetBlob) {
.IsIncomplete());
}
TEST_F(DBBlobBasicTest, EmptyValueNotStoredAsBlob) {
// Regression test for crash when empty blob value is evicted to
// CompressedSecondaryCache (T261142690). Empty values should always be
// stored inline in the SST, never as blobs, even with min_blob_size=0.
// A BlobIndex for an empty value is strictly larger than the value itself,
// so storing it as a blob is pure overhead.
Options options = GetDefaultOptions();
options.enable_blob_files = true;
options.min_blob_size = 0;
options.disable_auto_compactions = true;
Reopen(options);
// Write an empty value and a non-empty value.
ASSERT_OK(Put("empty_key", ""));
constexpr char blob_value[] = "blob_value";
ASSERT_OK(Put("nonempty_key", blob_value));
ASSERT_OK(Flush());
// Both values should be readable.
ASSERT_EQ(Get("empty_key"), "");
ASSERT_EQ(Get("nonempty_key"), blob_value);
// The empty value should be stored inline (readable from block cache
// without blob file I/O), while the non-empty value requires blob I/O.
ReadOptions ro;
ro.read_tier = kBlockCacheTier;
PinnableSlice result;
ASSERT_OK(db_->Get(ro, db_->DefaultColumnFamily(), "empty_key", &result));
ASSERT_EQ(result, "");
result.Reset();
ASSERT_TRUE(db_->Get(ro, db_->DefaultColumnFamily(), "nonempty_key", &result)
.IsIncomplete());
}
TEST_F(DBBlobBasicTest, GetBlobFromCache) {
Options options = GetDefaultOptions();
@@ -2459,6 +2508,138 @@ TEST_F(DBBlobWithTimestampTest, IterateBlobs) {
}
}
TEST_F(DBBlobBasicTest, GetApproximateSizesIncludingBlobFiles) {
Options options = GetDefaultOptions();
options.enable_blob_files = true;
options.min_blob_size = 0;
Reopen(options);
// Write some key-value pairs with blob values and flush to create blob files.
constexpr int kNumKeys = 1000;
constexpr int kValueSize = 1024;
Random rnd(301);
for (int i = 0; i < kNumKeys; ++i) {
ASSERT_OK(Put(Key(i), rnd.RandomString(kValueSize)));
}
ASSERT_OK(Flush());
// Verify blob files exist.
std::vector<std::string> files;
ASSERT_OK(env_->GetChildren(dbname_, &files));
bool has_blob_files = false;
for (const auto& f : files) {
if (f.size() > 5 && f.substr(f.size() - 5) == ".blob") {
has_blob_files = true;
break;
}
}
ASSERT_TRUE(has_blob_files);
// Query the full range - all keys are covered.
std::string start = Key(0);
std::string end = Key(kNumKeys);
Range r(start, end);
// Without include_blob_files (default behavior): should not include blob
// file sizes.
uint64_t size_without_blobs = 0;
{
SizeApproximationOptions size_approx_options;
size_approx_options.include_files = true;
size_approx_options.include_blob_files = false;
ASSERT_OK(db_->GetApproximateSizes(size_approx_options,
db_->DefaultColumnFamily(), &r, 1,
&size_without_blobs));
ASSERT_GT(size_without_blobs, 0);
}
// With include_blob_files: should be strictly larger.
{
SizeApproximationOptions size_approx_options;
size_approx_options.include_files = true;
size_approx_options.include_blob_files = true;
uint64_t size_with_blobs = 0;
ASSERT_OK(db_->GetApproximateSizes(size_approx_options,
db_->DefaultColumnFamily(), &r, 1,
&size_with_blobs));
ASSERT_GT(size_with_blobs, size_without_blobs);
}
// Range that doesn't overlap any data should return 0.
{
std::string no_start = Key(kNumKeys + 100);
std::string no_end = Key(kNumKeys + 200);
Range no_r(no_start, no_end);
SizeApproximationOptions size_approx_options;
size_approx_options.include_files = true;
size_approx_options.include_blob_files = true;
uint64_t no_size = 0;
ASSERT_OK(db_->GetApproximateSizes(
size_approx_options, db_->DefaultColumnFamily(), &no_r, 1, &no_size));
ASSERT_EQ(no_size, 0);
}
// Partial range should return proportionally less blob size than full range.
{
SizeApproximationOptions size_approx_options;
size_approx_options.include_files = true;
size_approx_options.include_blob_files = true;
uint64_t full_size = 0;
ASSERT_OK(db_->GetApproximateSizes(
size_approx_options, db_->DefaultColumnFamily(), &r, 1, &full_size));
// Query roughly the first half of keys.
std::string half_end = Key(kNumKeys / 2);
Range half_r(start, half_end);
uint64_t half_size = 0;
ASSERT_OK(db_->GetApproximateSizes(size_approx_options,
db_->DefaultColumnFamily(), &half_r, 1,
&half_size));
ASSERT_GT(half_size, 0);
ASSERT_LT(half_size, full_size);
}
// Via SizeApproximationFlags API.
{
uint64_t size_flags = 0;
ASSERT_OK(db_->GetApproximateSizes(
db_->DefaultColumnFamily(), &r, 1, &size_flags,
DB::SizeApproximationFlags::INCLUDE_FILES |
DB::SizeApproximationFlags::INCLUDE_BLOB_FILES));
ASSERT_GT(size_flags, size_without_blobs);
}
// Multi-range query: two non-overlapping sub-ranges should sum to
// approximately the full-range result.
{
SizeApproximationOptions size_approx_options;
size_approx_options.include_files = true;
size_approx_options.include_blob_files = true;
std::string mid = Key(kNumKeys / 2);
std::string r1_start = Key(0);
std::string r1_end = mid;
std::string r2_start = mid;
std::string r2_end = Key(kNumKeys);
Range ranges[2] = {Range(r1_start, r1_end), Range(r2_start, r2_end)};
uint64_t sizes[2] = {0, 0};
ASSERT_OK(db_->GetApproximateSizes(
size_approx_options, db_->DefaultColumnFamily(), ranges, 2, sizes));
// Each sub-range should return a positive size.
ASSERT_GT(sizes[0], 0);
ASSERT_GT(sizes[1], 0);
// Sum of sub-ranges should be close to the full-range result.
uint64_t full_size = 0;
ASSERT_OK(db_->GetApproximateSizes(
size_approx_options, db_->DefaultColumnFamily(), &r, 1, &full_size));
ASSERT_NEAR(static_cast<double>(sizes[0] + sizes[1]),
static_cast<double>(full_size), full_size * 0.1);
}
}
} // namespace ROCKSDB_NAMESPACE
int main(int argc, char** argv) {
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+74 -8
View File
@@ -13,7 +13,10 @@
#include <deque>
#include <vector>
#include "db/blob/blob_counting_iterator.h"
#include "db/blob/blob_file_builder.h"
#include "db/blob/blob_garbage_meter.h"
#include "db/column_family.h"
#include "db/compaction/compaction_iterator.h"
#include "db/dbformat.h"
#include "db/event_helpers.h"
@@ -87,7 +90,8 @@ Status BuildTable(
Env::WriteLifeTimeHint write_hint, const std::string* full_history_ts_low,
BlobFileCompletionCallback* blob_callback, Version* version,
uint64_t* memtable_payload_bytes, uint64_t* memtable_garbage_bytes,
InternalStats::CompactionStats* flush_stats) {
InternalStats::CompactionStats* flush_stats,
std::vector<BlobFileGarbage>* blob_file_garbages, bool fast_sst_open) {
assert((tboptions.column_family_id ==
TablePropertiesCollectorFactory::Context::kUnknownColumnFamily) ==
tboptions.column_family_name.empty());
@@ -99,7 +103,24 @@ Status BuildTable(
/*enable_hash=*/paranoid_file_checks);
Status s;
meta->fd.file_size = 0;
iter->SeekToFirst();
if (blob_file_garbages != nullptr) {
blob_file_garbages->clear();
}
InternalIterator* input = iter;
std::unique_ptr<BlobGarbageMeter> blob_garbage_meter;
std::unique_ptr<BlobCountingIterator> blob_counting_iter;
if (blob_file_garbages != nullptr) {
// Flush can drop blob-backed records via overwrite elision and flush-time
// compaction filters. Track input and surviving output refs just like the
// compaction path so the manifest learns about the new garbage.
blob_garbage_meter.reset(new BlobGarbageMeter());
blob_counting_iter.reset(
new BlobCountingIterator(iter, blob_garbage_meter.get()));
input = blob_counting_iter.get();
}
input->SeekToFirst();
std::unique_ptr<CompactionRangeDelAggregator> range_del_agg(
new CompactionRangeDelAggregator(&tboptions.internal_comparator,
snapshots, full_history_ts_low));
@@ -128,7 +149,7 @@ Status BuildTable(
TableProperties tp;
bool table_file_created = false;
if (iter->Valid() || !range_del_agg->IsEmpty()) {
if (input->Valid() || !range_del_agg->IsEmpty()) {
std::unique_ptr<CompactionFilter> compaction_filter;
if (ioptions.compaction_filter_factory != nullptr &&
ioptions.compaction_filter_factory->ShouldFilterTableFileCreation(
@@ -211,7 +232,7 @@ Status BuildTable(
const std::atomic<bool> kManualCompactionCanceledFalse{false};
CompactionIterator c_iter(
iter, ucmp, &merge, kMaxSequenceNumber, &snapshots, earliest_snapshot,
input, ucmp, &merge, kMaxSequenceNumber, &snapshots, earliest_snapshot,
earliest_write_conflict_snapshot, job_snapshot, snapshot_checker, env,
ShouldReportDetailedTime(env, ioptions.stats), range_del_agg.get(),
blob_file_builder.get(), ioptions.allow_data_in_errors,
@@ -219,7 +240,17 @@ Status BuildTable(
/*manual_compaction_canceled=*/kManualCompactionCanceledFalse,
true /* must_count_input_entries */,
/*compaction=*/nullptr, compaction_filter.get(),
/*shutting_down=*/nullptr, db_options.info_log, full_history_ts_low);
/*shutting_down=*/nullptr, db_options.info_log, full_history_ts_low,
std::nullopt, version, tboptions.read_options.io_activity);
if (version != nullptr) {
ColumnFamilyData* const cfd = version->cfd();
c_iter.SetBlobFetcher(
version, cfd != nullptr ? cfd->blob_file_cache() : nullptr,
tboptions.read_options.io_activity,
tboptions.reason == TableFileCreationReason::kFlush ||
tboptions.reason == TableFileCreationReason::kRecovery);
}
SequenceNumber smallest_preferred_seqno = kMaxSequenceNumber;
std::string key_after_flush_buf;
@@ -268,6 +299,14 @@ Status BuildTable(
flush_stats->num_output_records++;
}
if (blob_garbage_meter != nullptr) {
s = blob_garbage_meter->ProcessOutFlow(key_after_flush,
value_after_flush);
if (!s.ok()) {
break;
}
}
s = meta->UpdateBoundaries(key_after_flush, value_after_flush,
ikey.sequence, ikey.type);
if (!s.ok()) {
@@ -322,6 +361,8 @@ Status BuildTable(
}
TEST_SYNC_POINT("BuildTable:BeforeFinishBuildTable");
TEST_SYNC_POINT_CALLBACK("BuildTable:BeforeCheckEmpty",
static_cast<TableBuilder*>(builder));
const bool empty = builder->IsEmpty();
if (flush_stats) {
assert(c_iter.HasNumInputEntryScanned());
@@ -330,6 +371,14 @@ Status BuildTable(
}
if (!s.ok() || empty) {
builder->Abandon();
// Propagate the builder's error when the builder is empty due to an
// internal error (e.g., write fault injection causing all Add() calls to
// return early). Without this, `s` would remain OK and the downstream
// key count validation in flush_job.cc would produce a misleading
// Corruption error instead of the actual builder error.
if (s.ok() && !builder->status().ok()) {
s = builder->status();
}
} else {
SeqnoToTimeMapping relevant_mapping;
if (seqno_to_time_mapping) {
@@ -447,6 +496,8 @@ Status BuildTable(
// here because this is a special case after we finish the table building.
// No matter whether use_direct_io_for_flush_and_compaction is true,
// the goal is to cache it here for further user reads.
std::string* file_open_metadata_ptr =
fast_sst_open ? &meta->file_open_metadata : nullptr;
std::unique_ptr<InternalIterator> it(table_cache->NewIterator(
tboptions.read_options, file_options, tboptions.internal_comparator,
*meta, nullptr /* range_del_agg */, mutable_cf_options, nullptr,
@@ -457,7 +508,10 @@ Status BuildTable(
MaxFileSizeForL0MetaPin(mutable_cf_options),
/*smallest_compaction_key=*/nullptr,
/*largest_compaction_key*/ nullptr,
/*allow_unprepared_value*/ false));
/*allow_unprepared_value*/ false,
/*range_del_read_seqno=*/nullptr,
/*range_del_iter=*/nullptr,
/*maybe_pin_table_handle=*/false, file_open_metadata_ptr));
s = it->status();
if (s.ok() && paranoid_file_checks) {
OutputValidator file_validator(tboptions.internal_comparator,
@@ -475,8 +529,20 @@ Status BuildTable(
}
// Check for input iterator errors
if (!iter->status().ok()) {
s = iter->status();
if (!input->status().ok()) {
s = input->status();
}
if (s.ok() && blob_file_garbages != nullptr &&
blob_garbage_meter != nullptr) {
for (const auto& pair : blob_garbage_meter->flows()) {
const uint64_t blob_file_number = pair.first;
const BlobGarbageMeter::BlobInOutFlow& flow = pair.second;
if (flow.HasGarbage()) {
blob_file_garbages->emplace_back(
blob_file_number, flow.GetGarbageCount(), flow.GetGarbageBytes());
}
}
}
if (!s.ok() || meta->fd.GetFileSize() == 0) {
+4 -1
View File
@@ -31,6 +31,7 @@ struct FileMetaData;
class VersionSet;
class BlobFileAddition;
class BlobFileGarbage;
class SnapshotChecker;
class TableCache;
class TableBuilder;
@@ -79,6 +80,8 @@ Status BuildTable(
BlobFileCompletionCallback* blob_callback = nullptr,
Version* version = nullptr, uint64_t* memtable_payload_bytes = nullptr,
uint64_t* memtable_garbage_bytes = nullptr,
InternalStats::CompactionStats* flush_stats = nullptr);
InternalStats::CompactionStats* flush_stats = nullptr,
std::vector<BlobFileGarbage>* blob_file_garbages = nullptr,
bool fast_sst_open = false);
} // namespace ROCKSDB_NAMESPACE
+132 -57
View File
@@ -11,6 +11,7 @@
#include <cstdlib>
#include <map>
#include <memory>
#include <unordered_set>
#include <vector>
@@ -595,7 +596,6 @@ struct rocksdb_slicetransform_t : public SliceTransform {
char* (*transform_)(void*, const char* key, size_t length,
size_t* dst_length);
unsigned char (*in_domain_)(void*, const char* key, size_t length);
unsigned char (*in_range_)(void*, const char* key, size_t length);
~rocksdb_slicetransform_t() override { (*destructor_)(state_); }
@@ -610,10 +610,6 @@ struct rocksdb_slicetransform_t : public SliceTransform {
bool InDomain(const Slice& src) const override {
return (*in_domain_)(state_, src.data(), src.size());
}
bool InRange(const Slice& src) const override {
return (*in_range_)(state_, src.data(), src.size());
}
};
struct rocksdb_universal_compaction_options_t {
@@ -1222,12 +1218,12 @@ char* rocksdb_open_and_compact_with_options(
rocksdb_t* rocksdb_open(const rocksdb_options_t* options, const char* name,
char** errptr) {
DB* db;
if (SaveError(errptr, DB::Open(options->rep, std::string(name), &db))) {
std::unique_ptr<DB> dbptr;
if (SaveError(errptr, DB::Open(options->rep, std::string(name), &dbptr))) {
return nullptr;
}
rocksdb_t* result = new rocksdb_t;
result->rep = db;
result->rep = dbptr.release();
return result;
}
@@ -1247,13 +1243,14 @@ rocksdb_t* rocksdb_open_for_read_only(const rocksdb_options_t* options,
const char* name,
unsigned char error_if_wal_file_exists,
char** errptr) {
DB* db;
if (SaveError(errptr, DB::OpenForReadOnly(options->rep, std::string(name),
&db, error_if_wal_file_exists))) {
std::unique_ptr<DB> dbptr;
if (SaveError(errptr,
DB::OpenForReadOnly(options->rep, std::string(name), &dbptr,
error_if_wal_file_exists))) {
return nullptr;
}
rocksdb_t* result = new rocksdb_t;
result->rep = db;
result->rep = dbptr.release();
return result;
}
@@ -1261,14 +1258,14 @@ rocksdb_t* rocksdb_open_as_secondary(const rocksdb_options_t* options,
const char* name,
const char* secondary_path,
char** errptr) {
DB* db;
std::unique_ptr<DB> dbptr;
if (SaveError(errptr,
DB::OpenAsSecondary(options->rep, std::string(name),
std::string(secondary_path), &db))) {
std::string(secondary_path), &dbptr))) {
return nullptr;
}
rocksdb_t* result = new rocksdb_t;
result->rep = db;
result->rep = dbptr.release();
return result;
}
@@ -1582,11 +1579,11 @@ rocksdb_t* rocksdb_open_and_trim_history(
std::string trim_ts_(trim_ts, trim_tslen);
DB* db;
std::unique_ptr<DB> dbptr;
std::vector<ColumnFamilyHandle*> handles;
if (SaveError(errptr, DB::OpenAndTrimHistory(
DBOptions(db_options->rep), std::string(name),
column_families, &handles, &db, trim_ts_))) {
column_families, &handles, &dbptr, trim_ts_))) {
return nullptr;
}
@@ -1598,7 +1595,7 @@ rocksdb_t* rocksdb_open_and_trim_history(
column_family_handles[i] = c_handle;
}
rocksdb_t* result = new rocksdb_t;
result->rep = db;
result->rep = dbptr.release();
return result;
}
@@ -1614,10 +1611,10 @@ rocksdb_t* rocksdb_open_column_families(
ColumnFamilyOptions(column_family_options[i]->rep));
}
DB* db;
std::unique_ptr<DB> dbptr;
std::vector<ColumnFamilyHandle*> handles;
if (SaveError(errptr, DB::Open(DBOptions(db_options->rep), std::string(name),
column_families, &handles, &db))) {
column_families, &handles, &dbptr))) {
return nullptr;
}
@@ -1629,7 +1626,7 @@ rocksdb_t* rocksdb_open_column_families(
column_family_handles[i] = c_handle;
}
rocksdb_t* result = new rocksdb_t;
result->rep = db;
result->rep = dbptr.release();
return result;
}
@@ -1682,12 +1679,12 @@ rocksdb_t* rocksdb_open_for_read_only_column_families(
ColumnFamilyOptions(column_family_options[i]->rep));
}
DB* db;
std::unique_ptr<DB> dbptr;
std::vector<ColumnFamilyHandle*> handles;
if (SaveError(errptr,
DB::OpenForReadOnly(DBOptions(db_options->rep),
std::string(name), column_families,
&handles, &db, error_if_wal_file_exists))) {
if (SaveError(errptr, DB::OpenForReadOnly(DBOptions(db_options->rep),
std::string(name), column_families,
&handles, &dbptr,
error_if_wal_file_exists))) {
return nullptr;
}
@@ -1699,7 +1696,7 @@ rocksdb_t* rocksdb_open_for_read_only_column_families(
column_family_handles[i] = c_handle;
}
rocksdb_t* result = new rocksdb_t;
result->rep = db;
result->rep = dbptr.release();
return result;
}
@@ -1715,12 +1712,12 @@ rocksdb_t* rocksdb_open_as_secondary_column_families(
std::string(column_family_names[i]),
ColumnFamilyOptions(column_family_options[i]->rep));
}
DB* db;
std::unique_ptr<DB> dbptr;
std::vector<ColumnFamilyHandle*> handles;
if (SaveError(errptr, DB::OpenAsSecondary(DBOptions(db_options->rep),
std::string(name),
std::string(secondary_path),
column_families, &handles, &db))) {
if (SaveError(errptr, DB::OpenAsSecondary(
DBOptions(db_options->rep), std::string(name),
std::string(secondary_path), column_families,
&handles, &dbptr))) {
return nullptr;
}
for (size_t i = 0; i != handles.size(); ++i) {
@@ -1731,7 +1728,7 @@ rocksdb_t* rocksdb_open_as_secondary_column_families(
column_family_handles[i] = c_handle;
}
rocksdb_t* result = new rocksdb_t;
result->rep = db;
result->rep = dbptr.release();
return result;
}
@@ -1762,9 +1759,13 @@ rocksdb_column_family_handle_t* rocksdb_create_column_family(
rocksdb_t* db, const rocksdb_options_t* column_family_options,
const char* column_family_name, char** errptr) {
rocksdb_column_family_handle_t* handle = new rocksdb_column_family_handle_t;
SaveError(errptr, db->rep->CreateColumnFamily(
ColumnFamilyOptions(column_family_options->rep),
std::string(column_family_name), &(handle->rep)));
handle->rep = nullptr;
if (SaveError(errptr, db->rep->CreateColumnFamily(
ColumnFamilyOptions(column_family_options->rep),
std::string(column_family_name), &(handle->rep)))) {
delete handle;
return nullptr;
}
handle->immortal = false;
return handle;
}
@@ -3055,6 +3056,7 @@ class H : public WriteBatch::Handler {
(*log_data_)(state_, blob.data(), blob.size());
}
}
Status MarkNoop(bool /* empty_batch */) override { return Status::OK(); }
};
class HCF : public WriteBatch::Handler {
@@ -3088,6 +3090,7 @@ class HCF : public WriteBatch::Handler {
(*log_data_)(state_, blob.data(), blob.size());
}
}
Status MarkNoop(bool /* empty_batch */) override { return Status::OK(); }
};
void rocksdb_writebatch_iterate(rocksdb_writebatch_t* b, void* state,
@@ -3746,6 +3749,11 @@ void rocksdb_block_based_options_set_format_version(
options->rep.format_version = v;
}
void rocksdb_block_based_options_set_separate_key_value_in_data_block(
rocksdb_block_based_table_options_t* options, unsigned char v) {
options->rep.separate_key_value_in_data_block = v;
}
void rocksdb_block_based_options_set_index_type(
rocksdb_block_based_table_options_t* options, int v) {
options->rep.index_type = static_cast<BlockBasedTableOptions::IndexType>(v);
@@ -3757,6 +3765,12 @@ void rocksdb_block_based_options_set_data_block_index_type(
static_cast<BlockBasedTableOptions::DataBlockIndexType>(v);
}
void rocksdb_block_based_options_set_index_block_search_type(
rocksdb_block_based_table_options_t* options, int v) {
options->rep.index_block_search_type =
static_cast<BlockBasedTableOptions::BlockSearchType>(v);
}
void rocksdb_block_based_options_set_data_block_hash_ratio(
rocksdb_block_based_table_options_t* options, double v) {
options->rep.data_block_hash_table_util_ratio = v;
@@ -4343,6 +4357,15 @@ unsigned char rocksdb_options_get_paranoid_checks(rocksdb_options_t* opt) {
return opt->rep.paranoid_checks;
}
void rocksdb_options_set_open_files_async(rocksdb_options_t* opt,
unsigned char v) {
opt->rep.open_files_async = v;
}
unsigned char rocksdb_options_get_open_files_async(rocksdb_options_t* opt) {
return opt->rep.open_files_async;
}
void rocksdb_options_set_db_paths(rocksdb_options_t* opt,
const rocksdb_dbpath_t** dbpath_values,
size_t num_paths) {
@@ -4627,6 +4650,16 @@ uint32_t rocksdb_options_get_memtable_avg_op_scan_flush_trigger(
return opt->rep.memtable_avg_op_scan_flush_trigger;
}
void rocksdb_options_set_min_tombstones_for_range_conversion(
rocksdb_options_t* opt, uint32_t n) {
opt->rep.min_tombstones_for_range_conversion = n;
}
uint32_t rocksdb_options_get_min_tombstones_for_range_conversion(
rocksdb_options_t* opt) {
return opt->rep.min_tombstones_for_range_conversion;
}
void rocksdb_options_enable_statistics(rocksdb_options_t* opt) {
opt->rep.statistics = ROCKSDB_NAMESPACE::CreateDBStatistics();
}
@@ -4664,16 +4697,6 @@ unsigned char rocksdb_options_get_skip_stats_update_on_db_open(
return opt->rep.skip_stats_update_on_db_open;
}
void rocksdb_options_set_skip_checking_sst_file_sizes_on_db_open(
rocksdb_options_t* opt, unsigned char val) {
opt->rep.skip_checking_sst_file_sizes_on_db_open = val;
}
unsigned char rocksdb_options_get_skip_checking_sst_file_sizes_on_db_open(
rocksdb_options_t* opt) {
return opt->rep.skip_checking_sst_file_sizes_on_db_open;
}
/* Blob Options Settings */
void rocksdb_options_set_enable_blob_files(rocksdb_options_t* opt,
unsigned char val) {
@@ -4736,6 +4759,26 @@ double rocksdb_options_get_blob_gc_force_threshold(rocksdb_options_t* opt) {
return opt->rep.blob_garbage_collection_force_threshold;
}
void rocksdb_options_set_read_triggered_compaction_threshold(
rocksdb_options_t* opt, double val) {
opt->rep.read_triggered_compaction_threshold = val;
}
double rocksdb_options_get_read_triggered_compaction_threshold(
rocksdb_options_t* opt) {
return opt->rep.read_triggered_compaction_threshold;
}
void rocksdb_options_set_max_compaction_trigger_wakeup_seconds(
rocksdb_options_t* opt, uint64_t val) {
opt->rep.max_compaction_trigger_wakeup_seconds = val;
}
uint64_t rocksdb_options_get_max_compaction_trigger_wakeup_seconds(
rocksdb_options_t* opt) {
return opt->rep.max_compaction_trigger_wakeup_seconds;
}
void rocksdb_options_set_blob_compaction_readahead_size(rocksdb_options_t* opt,
uint64_t val) {
opt->rep.blob_compaction_readahead_size = val;
@@ -5791,6 +5834,16 @@ uint64_t rocksdb_perfcontext_metric(rocksdb_perfcontext_t* context,
return rep->internal_range_del_reseek_count;
case rocksdb_block_read_cpu_time:
return rep->block_read_cpu_time;
case rocksdb_data_block_read_byte:
return rep->data_block_read_byte;
case rocksdb_index_block_read_byte:
return rep->index_block_read_byte;
case rocksdb_filter_block_read_byte:
return rep->filter_block_read_byte;
case rocksdb_compression_dict_block_read_byte:
return rep->compression_dict_block_read_byte;
case rocksdb_metadata_block_read_byte:
return rep->metadata_block_read_byte;
default:
break;
}
@@ -6089,11 +6142,6 @@ unsigned char rocksdb_readoptions_get_tailing(rocksdb_readoptions_t* opt) {
return opt->rep.tailing;
}
void rocksdb_readoptions_set_managed(rocksdb_readoptions_t* opt,
unsigned char v) {
opt->rep.managed = v;
}
void rocksdb_readoptions_set_readahead_size(rocksdb_readoptions_t* opt,
size_t v) {
opt->rep.readahead_size = v;
@@ -6894,14 +6942,12 @@ rocksdb_slicetransform_t* rocksdb_slicetransform_create(
char* (*transform)(void*, const char* key, size_t length,
size_t* dst_length),
unsigned char (*in_domain)(void*, const char* key, size_t length),
unsigned char (*in_range)(void*, const char* key, size_t length),
const char* (*name)(void*)) {
rocksdb_slicetransform_t* result = new rocksdb_slicetransform_t;
result->state_ = state;
result->destructor_ = destructor;
result->transform_ = transform;
result->in_domain_ = in_domain;
result->in_range_ = in_range;
result->name_ = name;
return result;
}
@@ -6917,7 +6963,6 @@ struct SliceTransformWrapper : public rocksdb_slicetransform_t {
return rep_->Transform(src);
}
bool InDomain(const Slice& src) const override { return rep_->InDomain(src); }
bool InRange(const Slice& src) const override { return rep_->InRange(src); }
static void DoNothing(void*) {}
};
@@ -7041,6 +7086,27 @@ uint64_t rocksdb_fifo_compaction_options_get_max_table_files_size(
return fifo_opts->rep.max_table_files_size;
}
void rocksdb_fifo_compaction_options_set_max_data_files_size(
rocksdb_fifo_compaction_options_t* fifo_opts, uint64_t size) {
fifo_opts->rep.max_data_files_size = size;
}
uint64_t rocksdb_fifo_compaction_options_get_max_data_files_size(
rocksdb_fifo_compaction_options_t* fifo_opts) {
return fifo_opts->rep.max_data_files_size;
}
void rocksdb_fifo_compaction_options_set_use_kv_ratio_compaction(
rocksdb_fifo_compaction_options_t* fifo_opts,
unsigned char use_kv_ratio_compaction) {
fifo_opts->rep.use_kv_ratio_compaction = use_kv_ratio_compaction;
}
unsigned char rocksdb_fifo_compaction_options_get_use_kv_ratio_compaction(
rocksdb_fifo_compaction_options_t* fifo_opts) {
return fifo_opts->rep.use_kv_ratio_compaction;
}
void rocksdb_fifo_compaction_options_destroy(
rocksdb_fifo_compaction_options_t* fifo_opts) {
delete fifo_opts;
@@ -7514,9 +7580,13 @@ rocksdb_column_family_handle_t* rocksdb_transactiondb_create_column_family(
const rocksdb_options_t* column_family_options,
const char* column_family_name, char** errptr) {
rocksdb_column_family_handle_t* handle = new rocksdb_column_family_handle_t;
SaveError(errptr, txn_db->rep->CreateColumnFamily(
ColumnFamilyOptions(column_family_options->rep),
std::string(column_family_name), &(handle->rep)));
handle->rep = nullptr;
if (SaveError(errptr, txn_db->rep->CreateColumnFamily(
ColumnFamilyOptions(column_family_options->rep),
std::string(column_family_name), &(handle->rep)))) {
delete handle;
return nullptr;
}
handle->immortal = false;
return handle;
}
@@ -8246,6 +8316,11 @@ void rocksdb_transaction_delete_cf(
SaveError(errptr, txn->rep->Delete(column_family->rep, Slice(key, klen)));
}
void rocksdb_transaction_put_log_data(rocksdb_transaction_t* txn,
const char* blob, size_t len) {
txn->rep->PutLogData(Slice(blob, len));
}
// Delete a key outside a transaction
void rocksdb_transactiondb_delete(rocksdb_transactiondb_t* txn_db,
const rocksdb_writeoptions_t* options,
+87
View File
@@ -205,6 +205,27 @@ static void CheckDel(void* ptr, const char* k, size_t klen) {
(*state)++;
}
static void NopPut(void* ptr, const char* k, size_t klen, const char* v,
size_t vlen) {
(void)ptr;
(void)k;
(void)klen;
(void)v;
(void)vlen;
}
static void NopDel(void* ptr, const char* k, size_t klen) {
(void)ptr;
(void)k;
(void)klen;
}
static void CheckLogData(void* ptr, const char* blob, size_t blen) {
CheckEqual("log_blob", blob, blen);
int* found = (int*)ptr;
*found = 1;
}
// Callback from rocksdb_writebatch_iterate_cf()
static void CheckPutCF(void* ptr, uint32_t cfid, const char* k, size_t klen,
const char* v, size_t vlen) {
@@ -2643,6 +2664,10 @@ int main(int argc, char** argv) {
CheckCondition(150 ==
rocksdb_options_get_memtable_avg_op_scan_flush_trigger(o));
rocksdb_options_set_min_tombstones_for_range_conversion(o, 200);
CheckCondition(200 ==
rocksdb_options_get_min_tombstones_for_range_conversion(o));
rocksdb_options_set_ttl(o, 5000);
CheckCondition(5000 == rocksdb_options_get_ttl(o));
@@ -2867,6 +2892,14 @@ int main(int argc, char** argv) {
rocksdb_options_set_blob_gc_force_threshold(o, 0.75);
CheckCondition(0.75 == rocksdb_options_get_blob_gc_force_threshold(o));
rocksdb_options_set_read_triggered_compaction_threshold(o, 0.001);
CheckCondition(0.001 ==
rocksdb_options_get_read_triggered_compaction_threshold(o));
rocksdb_options_set_max_compaction_trigger_wakeup_seconds(o, 3600);
CheckCondition(
3600 == rocksdb_options_get_max_compaction_trigger_wakeup_seconds(o));
rocksdb_options_set_blob_compaction_readahead_size(o, 262144);
CheckCondition(262144 ==
rocksdb_options_get_blob_compaction_readahead_size(o));
@@ -3086,6 +3119,12 @@ int main(int argc, char** argv) {
CheckCondition(150 ==
rocksdb_options_get_memtable_avg_op_scan_flush_trigger(o));
rocksdb_options_set_min_tombstones_for_range_conversion(copy, 1000);
CheckCondition(
1000 == rocksdb_options_get_min_tombstones_for_range_conversion(copy));
CheckCondition(200 ==
rocksdb_options_get_min_tombstones_for_range_conversion(o));
rocksdb_options_set_ttl(copy, 8000);
CheckCondition(8000 == rocksdb_options_get_ttl(copy));
CheckCondition(5000 == rocksdb_options_get_ttl(o));
@@ -3596,6 +3635,14 @@ int main(int argc, char** argv) {
100000 ==
rocksdb_fifo_compaction_options_get_max_table_files_size(fco));
rocksdb_fifo_compaction_options_set_max_data_files_size(fco, 200000);
CheckCondition(
200000 == rocksdb_fifo_compaction_options_get_max_data_files_size(fco));
rocksdb_fifo_compaction_options_set_use_kv_ratio_compaction(fco, 1);
CheckCondition(
1 == rocksdb_fifo_compaction_options_get_use_kv_ratio_compaction(fco));
rocksdb_fifo_compaction_options_destroy(fco);
}
@@ -3842,10 +3889,35 @@ int main(int argc, char** argv) {
CheckMultiGetValues(3, vals, vals_sizes, errs, expected);
}
rocksdb_transaction_put_log_data(txn, "log_blob", 8);
// record sequence number before commit so we can scan the WAL after
rocksdb_t* base_db_ld = rocksdb_transactiondb_get_base_db(txn_db);
uint64_t seq_before = rocksdb_get_latest_sequence_number(base_db_ld);
// commit
rocksdb_transaction_commit(txn, &err);
CheckNoError(err);
// verify log data was written to WAL by scanning batches since seq_before
{
rocksdb_wal_iterator_t* wal_iter =
rocksdb_get_updates_since(base_db_ld, seq_before, NULL, &err);
CheckNoError(err);
int log_found = 0;
for (; rocksdb_wal_iter_valid(wal_iter) && !log_found;
rocksdb_wal_iter_next(wal_iter)) {
uint64_t seq;
rocksdb_writebatch_t* wal_wb =
rocksdb_wal_iter_get_batch(wal_iter, &seq);
rocksdb_writebatch_iterate_ld(wal_wb, &log_found, NopPut, NopDel,
CheckLogData);
rocksdb_writebatch_destroy(wal_wb);
}
CheckCondition(log_found == 1);
rocksdb_wal_iter_destroy(wal_iter);
}
rocksdb_transactiondb_close_base_db(base_db_ld);
// read from outside transaction, after commit
CheckTxnDBGet(txn_db, roptions, "foo", "hello");
CheckTxnDBPinGet(txn_db, roptions, "foo", "hello");
@@ -4872,6 +4944,21 @@ int main(int argc, char** argv) {
rocksdb_sst_file_manager_destroy(sst_file_manager);
}
StartPhase("create_column_family_error_returns_null");
{
// Creating a column family with a name that already exists should fail
// and return NULL. Without the fix, the handle is leaked and a non-NULL
// pointer with an indeterminate rep field is returned.
char* cf_err = NULL;
rocksdb_column_family_handle_t* cf_handle =
rocksdb_create_column_family(db, options, "default", &cf_err);
// Should have an error since "default" already exists
CheckCondition(cf_err != NULL);
// The handle should be NULL on error (this is the bug fix)
CheckCondition(cf_handle == NULL);
free(cf_err);
}
StartPhase("cancel_all_background_work");
rocksdb_cancel_all_background_work(db, 1);
+12 -2
View File
@@ -12,6 +12,7 @@ namespace ROCKSDB_NAMESPACE {
void CoalescingIterator::Coalesce(
const autovector<MultiCfIteratorInfo>& items) {
assert(wide_columns_.empty());
assert(owned_columns_.empty());
MinHeap heap;
for (const auto& item : items) {
assert(item.iterator);
@@ -22,13 +23,22 @@ void CoalescingIterator::Coalesce(
if (heap.empty()) {
return;
}
// Own the coalesced bytes so the result stays valid even if a child iterator
// refreshes or reuses its internal buffers.
owned_columns_.reserve(heap.size());
wide_columns_.reserve(heap.size());
auto add_column = [this](const WideColumn& column) {
owned_columns_.emplace_back(column.name().ToString(),
column.value().ToString());
const auto& owned = owned_columns_.back();
wide_columns_.emplace_back(owned.first, owned.second);
};
auto current = heap.top();
heap.pop();
while (!heap.empty()) {
int comparison = current.column->name().compare(heap.top().column->name());
if (comparison < 0) {
wide_columns_.push_back(*current.column);
add_column(*current.column);
} else if (comparison > 0) {
// Shouldn't reach here.
// Current item in the heap is greater than the top item in the min heap
@@ -37,7 +47,7 @@ void CoalescingIterator::Coalesce(
current = heap.top();
heap.pop();
}
wide_columns_.push_back(*current.column);
add_column(*current.column);
if (WideColumnsHelper::HasDefaultColumn(wide_columns_)) {
value_ = WideColumnsHelper::GetDefaultColumn(wide_columns_);
+6
View File
@@ -5,6 +5,10 @@
#pragma once
#include <string>
#include <utility>
#include <vector>
#include "db/multi_cf_iterator_impl.h"
namespace ROCKSDB_NAMESPACE {
@@ -45,6 +49,7 @@ class CoalescingIterator : public Iterator {
void Reset() {
value_.clear();
wide_columns_.clear();
owned_columns_.clear();
}
bool PrepareValue() override { return impl_.PrepareValue(); }
@@ -79,6 +84,7 @@ class CoalescingIterator : public Iterator {
MultiCfIteratorImpl<ResetFunc, PopulateFunc> impl_;
Slice value_;
WideColumns wide_columns_;
std::vector<std::pair<std::string, std::string>> owned_columns_;
struct WideColumnWithOrder {
const WideColumn* column;
+79 -1
View File
@@ -11,12 +11,14 @@
#include <algorithm>
#include <cinttypes>
#include <cmath>
#include <limits>
#include <sstream>
#include <string>
#include <vector>
#include "db/blob/blob_file_cache.h"
#include "db/blob/blob_file_partition_manager.h"
#include "db/blob/blob_source.h"
#include "db/compaction/compaction_picker.h"
#include "db/compaction/compaction_picker_fifo.h"
@@ -401,7 +403,13 @@ ColumnFamilyOptions SanitizeCfOptions(const ImmutableDBOptions& db_options,
}
if (result.max_compaction_bytes == 0) {
result.max_compaction_bytes = result.target_file_size_base * 25;
// For FIFO with use_kv_ratio_compaction, leave max_compaction_bytes as 0
// to signal "auto-calculate target from capacity and SST/blob ratio."
// When explicitly set by the user, it overrides the auto-calculated target.
if (result.compaction_style != kCompactionStyleFIFO ||
!result.compaction_options_fifo.use_kv_ratio_compaction) {
result.max_compaction_bytes = result.target_file_size_base * 25;
}
}
bool is_block_based_table = (result.table_factory->IsInstanceOf(
@@ -489,6 +497,15 @@ ColumnFamilyOptions SanitizeCfOptions(const ImmutableDBOptions& db_options,
result.memtable_avg_op_scan_flush_trigger = 0;
}
}
if (result.enable_blob_direct_write && !result.enable_blob_files) {
ROCKS_LOG_WARN(db_options.info_log.get(),
"enable_blob_direct_write requires enable_blob_files=true. "
"Disabling blob direct write.");
result.enable_blob_direct_write = false;
}
if (result.blob_direct_write_partitions == 0) {
result.blob_direct_write_partitions = 1;
}
return result;
}
@@ -776,6 +793,11 @@ ColumnFamilyData::~ColumnFamilyData() {
}
}
void ColumnFamilyData::SetBlobPartitionManager(
std::shared_ptr<BlobFilePartitionManager> mgr) {
blob_partition_manager_ = std::move(mgr);
}
bool ColumnFamilyData::UnrefAndTryDelete() {
int old_refs = refs_.fetch_sub(1);
assert(old_refs > 0);
@@ -1517,6 +1539,32 @@ Status ColumnFamilyData::ValidateOptions(
const auto* ucmp = cf_options.comparator;
assert(ucmp);
if (cf_options.enable_blob_direct_write) {
if (db_options.enable_pipelined_write) {
return Status::NotSupported(
"Blob direct write v1 does not support pipelined writes.");
}
if (db_options.allow_concurrent_memtable_write) {
return Status::NotSupported(
"Blob direct write v1 does not support concurrent memtable writes.");
}
if (db_options.unordered_write) {
return Status::NotSupported(
"Blob direct write v1 does not support unordered writes.");
}
if (db_options.two_write_queues) {
return Status::NotSupported(
"Blob direct write v1 does not support two write queues.");
}
if (cf_options.experimental_mempurge_threshold > 0.0) {
return Status::NotSupported(
"Blob direct write does not support MemPurge.");
}
if (ucmp->timestamp_size() > 0) {
return Status::NotSupported(
"Blob direct write does not support user-defined timestamps.");
}
}
if (ucmp->timestamp_size() > 0 &&
!cf_options.persist_user_defined_timestamps) {
if (db_options.atomic_flush) {
@@ -1557,12 +1605,42 @@ Status ColumnFamilyData::ValidateOptions(
}
}
if (cf_options.read_triggered_compaction_threshold < 0.0 ||
std::isnan(cf_options.read_triggered_compaction_threshold) ||
std::isinf(cf_options.read_triggered_compaction_threshold)) {
return Status::InvalidArgument(
"read_triggered_compaction_threshold must be >= 0.0 and finite. "
"Use 0.0 to disable.");
}
if (cf_options.compaction_style == kCompactionStyleFIFO &&
db_options.max_open_files != -1 && cf_options.ttl > 0) {
return Status::NotSupported(
"FIFO compaction only supported with max_open_files = -1.");
}
if (db_options.open_files_async) {
// FIFO TTL picker relies on reading the files table properties inside a
// DB mutex. This can be slow if files are opened asynchronously, so we
// disable it.
//
// TODO: consider blocking fifo compaction until async file open task
// completes.
if (cf_options.compaction_style == kCompactionStyleFIFO) {
return Status::NotSupported(
"FIFO compaction is not supported with open_files_async = true.");
}
// Open files async is not useful if skip_stats_update_on_db_open=true
// because DB open will still block on IO.
//
// TODO: consider moving stats update inside async file open background
// task.
if (!db_options.skip_stats_update_on_db_open) {
return Status::InvalidArgument(
"open_files_async requires skip_stats_update_on_db_open = true.");
}
}
std::vector<uint32_t> supported{0, 1, 2, 4, 8};
if (std::find(supported.begin(), supported.end(),
cf_options.memtable_protection_bytes_per_key) ==
+16
View File
@@ -49,6 +49,7 @@ class InstrumentedMutex;
class InstrumentedMutexLock;
struct SuperVersionContext;
class BlobFileCache;
class BlobFilePartitionManager;
class BlobSource;
extern const double kIncSlowdownRatio;
@@ -415,6 +416,19 @@ class ColumnFamilyData {
TableCache* table_cache() const { return table_cache_.get(); }
BlobFileCache* blob_file_cache() const { return blob_file_cache_.get(); }
BlobSource* blob_source() const { return blob_source_.get(); }
// Returns the write-path blob partition manager for this CF, or null if BDW
// is disabled.
BlobFilePartitionManager* blob_partition_manager() const {
return blob_partition_manager_.get();
}
// Returns a shared ownership handle for cold paths that must keep the
// manager alive beyond the lifetime of this ColumnFamilyData.
std::shared_ptr<BlobFilePartitionManager> blob_partition_manager_handle()
const {
return blob_partition_manager_;
}
// Installs the write-path blob partition manager for this CF.
void SetBlobPartitionManager(std::shared_ptr<BlobFilePartitionManager> mgr);
// See documentation in compaction_picker.h
// REQUIRES: DB mutex held
@@ -648,6 +662,8 @@ class ColumnFamilyData {
std::unique_ptr<TableCache> table_cache_;
std::unique_ptr<BlobFileCache> blob_file_cache_;
std::unique_ptr<BlobSource> blob_source_;
// Per-CF manager for write-path blob direct-write files.
std::shared_ptr<BlobFilePartitionManager> blob_partition_manager_;
std::unique_ptr<InternalStats> internal_stats_;
+10 -13
View File
@@ -118,8 +118,7 @@ class ColumnFamilyTestBase : public testing::Test {
for (int i = 0; i < n; i++) {
if (flush_every != 0 && i != 0 && i % flush_every == 0) {
DBImpl* dbi = static_cast_with_check<DBImpl>(db_);
dbi->TEST_FlushMemTable();
dbfull()->TEST_FlushMemTable();
}
int keyi = base + i;
@@ -177,8 +176,7 @@ class ColumnFamilyTestBase : public testing::Test {
}
handles_.clear();
names_.clear();
delete db_;
db_ = nullptr;
db_.reset();
}
Status TryOpen(std::vector<std::string> cf,
@@ -218,7 +216,7 @@ class ColumnFamilyTestBase : public testing::Test {
void Open() { Open({"default"}); }
DBImpl* dbfull() { return static_cast_with_check<DBImpl>(db_); }
DBImpl* dbfull() { return static_cast_with_check<DBImpl>(db_.get()); }
int GetProperty(int cf, std::string property) {
std::string value;
@@ -500,7 +498,7 @@ class ColumnFamilyTestBase : public testing::Test {
ColumnFamilyOptions column_family_options_;
DBOptions db_options_;
std::string dbname_;
DB* db_ = nullptr;
std::unique_ptr<DB> db_;
EnvCounter* env_;
std::shared_ptr<Env> env_guard_;
Random rnd_;
@@ -517,7 +515,7 @@ class ColumnFamilyTest
INSTANTIATE_TEST_CASE_P(FormatDef, ColumnFamilyTest,
testing::Values(test::kDefaultFormatVersion));
INSTANTIATE_TEST_CASE_P(FormatLatest, ColumnFamilyTest,
testing::Values(kLatestFormatVersion));
testing::Values(kLatestBbtFormatVersion));
TEST_P(ColumnFamilyTest, DontReuseColumnFamilyID) {
for (int iter = 0; iter < 3; ++iter) {
@@ -707,8 +705,8 @@ INSTANTIATE_TEST_CASE_P(
std::make_tuple(test::kDefaultFormatVersion, false)));
INSTANTIATE_TEST_CASE_P(
FormatLatest, FlushEmptyCFTestWithParam,
testing::Values(std::make_tuple(kLatestFormatVersion, true),
std::make_tuple(kLatestFormatVersion, false)));
testing::Values(std::make_tuple(kLatestBbtFormatVersion, true),
std::make_tuple(kLatestBbtFormatVersion, false)));
TEST_P(ColumnFamilyTest, AddDrop) {
Open();
@@ -3542,11 +3540,10 @@ TEST_P(ColumnFamilyTest, MultipleCFPathsTest) {
// Re-open and verify the keys.
Reopen({ColumnFamilyOptions(), cf_opt1, cf_opt2});
DBImpl* dbi = static_cast_with_check<DBImpl>(db_);
for (int cf = 1; cf != 3; ++cf) {
ReadOptions read_options;
read_options.readahead_size = 0;
auto it = dbi->NewIterator(read_options, handles_[cf]);
auto it = db_->NewIterator(read_options, handles_[cf]);
for (it->SeekToFirst(); it->Valid(); it->Next()) {
ASSERT_OK(it->status());
Slice key(it->key());
@@ -3636,7 +3633,7 @@ TEST(ColumnFamilyTest, ValidateMemtableKVChecksumOption) {
// the behavior of manual flush is that it skips retaining UDTs.
class ColumnFamilyRetainUDTTest : public ColumnFamilyTestBase {
public:
ColumnFamilyRetainUDTTest() : ColumnFamilyTestBase(kLatestFormatVersion) {}
ColumnFamilyRetainUDTTest() : ColumnFamilyTestBase(kLatestBbtFormatVersion) {}
void SetUp() override {
db_options_.allow_concurrent_memtable_write = false;
@@ -3886,7 +3883,7 @@ TEST_F(ManualFlushSkipRetainUDTTest, FlushRemovesStaleEntries) {
static_cast_with_check<ColumnFamilyHandleImpl>(cfh)->cfd();
for (int version = 0; version < 100; version++) {
if (version == 50) {
ASSERT_OK(static_cast_with_check<DBImpl>(db_)->TEST_SwitchMemtable(cfd));
ASSERT_OK(dbfull()->TEST_SwitchMemtable(cfd));
}
ASSERT_OK(
Put(0, "foo", EncodeAsUint64(version), "v" + std::to_string(version)));
+32 -46
View File
@@ -75,10 +75,9 @@ TEST_F(CompactFilesTest, L0ConflictsFiles) {
options.level0_file_num_compaction_trigger = kLevel0Trigger;
options.compression = kNoCompression;
DB* db = nullptr;
std::unique_ptr<DB> db;
ASSERT_OK(DestroyDB(db_name_, options));
Status s = DB::Open(options, db_name_, &db);
assert(s.ok());
ASSERT_OK(DB::Open(options, db_name_, &db));
assert(db);
ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->LoadDependency({
@@ -114,7 +113,6 @@ TEST_F(CompactFilesTest, L0ConflictsFiles) {
}
}
ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->DisableProcessing();
delete db;
}
TEST_F(CompactFilesTest, MultipleLevel) {
@@ -128,11 +126,11 @@ TEST_F(CompactFilesTest, MultipleLevel) {
FlushedFileCollector* collector = new FlushedFileCollector();
options.listeners.emplace_back(collector);
DB* db = nullptr;
std::unique_ptr<DB> db;
ASSERT_OK(DestroyDB(db_name_, options));
Status s = DB::Open(options, db_name_, &db);
ASSERT_OK(s);
ASSERT_NE(db, nullptr);
ASSERT_NE(db.get(), nullptr);
// create couple files in L0, L3, L4 and L5
for (int i = 5; i > 2; --i) {
@@ -141,7 +139,8 @@ TEST_F(CompactFilesTest, MultipleLevel) {
ASSERT_OK(db->Flush(FlushOptions()));
// Ensure background work is fully finished including listener callbacks
// before accessing listener state.
ASSERT_OK(static_cast_with_check<DBImpl>(db)->TEST_WaitForBackgroundWork());
ASSERT_OK(
static_cast_with_check<DBImpl>(db.get())->TEST_WaitForBackgroundWork());
auto l0_files = collector->GetFlushedFiles();
ASSERT_OK(db->CompactFiles(CompactionOptions(), l0_files, i));
@@ -191,8 +190,6 @@ TEST_F(CompactFilesTest, MultipleLevel) {
ASSERT_OK(db->CompactFiles(CompactionOptions(), files, 5));
SyncPoint::GetInstance()->DisableProcessing();
thread.join();
delete db;
}
TEST_F(CompactFilesTest, ObsoleteFiles) {
@@ -212,11 +209,11 @@ TEST_F(CompactFilesTest, ObsoleteFiles) {
FlushedFileCollector* collector = new FlushedFileCollector();
options.listeners.emplace_back(collector);
DB* db = nullptr;
std::unique_ptr<DB> db;
ASSERT_OK(DestroyDB(db_name_, options));
Status s = DB::Open(options, db_name_, &db);
ASSERT_OK(s);
ASSERT_NE(db, nullptr);
ASSERT_NE(db.get(), nullptr);
// create couple files
for (int i = 1000; i < 2000; ++i) {
@@ -226,13 +223,12 @@ TEST_F(CompactFilesTest, ObsoleteFiles) {
auto l0_files = collector->GetFlushedFiles();
ASSERT_OK(db->CompactFiles(CompactionOptions(), l0_files, 1));
ASSERT_OK(static_cast_with_check<DBImpl>(db)->TEST_WaitForCompact());
ASSERT_OK(static_cast_with_check<DBImpl>(db.get())->TEST_WaitForCompact());
// verify all compaction input files are deleted
for (const auto& fname : l0_files) {
ASSERT_EQ(Status::NotFound(), env_->FileExists(fname));
}
delete db;
}
TEST_F(CompactFilesTest, NotCutOutputOnLevel0) {
@@ -251,10 +247,9 @@ TEST_F(CompactFilesTest, NotCutOutputOnLevel0) {
FlushedFileCollector* collector = new FlushedFileCollector();
options.listeners.emplace_back(collector);
DB* db = nullptr;
std::unique_ptr<DB> db;
ASSERT_OK(DestroyDB(db_name_, options));
Status s = DB::Open(options, db_name_, &db);
assert(s.ok());
ASSERT_OK(DB::Open(options, db_name_, &db));
assert(db);
// create couple files
@@ -262,19 +257,20 @@ TEST_F(CompactFilesTest, NotCutOutputOnLevel0) {
ASSERT_OK(db->Put(WriteOptions(), std::to_string(i),
std::string(1000, 'a' + (i % 26))));
}
ASSERT_OK(static_cast_with_check<DBImpl>(db)->TEST_WaitForFlushMemTable());
ASSERT_OK(
static_cast_with_check<DBImpl>(db.get())->TEST_WaitForFlushMemTable());
auto l0_files_1 = collector->GetFlushedFiles();
collector->ClearFlushedFiles();
for (int i = 0; i < 500; ++i) {
ASSERT_OK(db->Put(WriteOptions(), std::to_string(i),
std::string(1000, 'a' + (i % 26))));
}
ASSERT_OK(static_cast_with_check<DBImpl>(db)->TEST_WaitForFlushMemTable());
ASSERT_OK(
static_cast_with_check<DBImpl>(db.get())->TEST_WaitForFlushMemTable());
auto l0_files_2 = collector->GetFlushedFiles();
ASSERT_OK(db->CompactFiles(CompactionOptions(), l0_files_1, 0));
ASSERT_OK(db->CompactFiles(CompactionOptions(), l0_files_2, 0));
// no assertion failure
delete db;
}
TEST_F(CompactFilesTest, CapturingPendingFiles) {
@@ -289,7 +285,7 @@ TEST_F(CompactFilesTest, CapturingPendingFiles) {
FlushedFileCollector* collector = new FlushedFileCollector();
options.listeners.emplace_back(collector);
DB* db = nullptr;
std::unique_ptr<DB> db;
ASSERT_OK(DestroyDB(db_name_, options));
Status s = DB::Open(options, db_name_, &db);
ASSERT_OK(s);
@@ -303,7 +299,8 @@ TEST_F(CompactFilesTest, CapturingPendingFiles) {
// Ensure background work is fully finished including listener callbacks
// before accessing listener state.
ASSERT_OK(static_cast_with_check<DBImpl>(db)->TEST_WaitForBackgroundWork());
ASSERT_OK(
static_cast_with_check<DBImpl>(db.get())->TEST_WaitForBackgroundWork());
auto l0_files = collector->GetFlushedFiles();
EXPECT_EQ(5, l0_files.size());
@@ -327,13 +324,12 @@ TEST_F(CompactFilesTest, CapturingPendingFiles) {
ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->DisableProcessing();
delete db;
db.reset();
// Make sure we can reopen the DB.
s = DB::Open(options, db_name_, &db);
ASSERT_OK(s);
assert(db);
delete db;
}
TEST_F(CompactFilesTest, CompactionFilterWithGetSv) {
@@ -365,12 +361,12 @@ TEST_F(CompactFilesTest, CompactionFilterWithGetSv) {
options.create_if_missing = true;
options.compaction_filter = cf.get();
DB* db = nullptr;
std::unique_ptr<DB> db;
ASSERT_OK(DestroyDB(db_name_, options));
Status s = DB::Open(options, db_name_, &db);
ASSERT_OK(s);
cf->SetDB(db);
cf->SetDB(db.get());
// Write one L0 file
ASSERT_OK(db->Put(WriteOptions(), "K1", "V1"));
@@ -384,8 +380,6 @@ TEST_F(CompactFilesTest, CompactionFilterWithGetSv) {
ASSERT_OK(
db->CompactFiles(ROCKSDB_NAMESPACE::CompactionOptions(), {fname}, 0));
}
delete db;
}
TEST_F(CompactFilesTest, SentinelCompressionType) {
@@ -413,7 +407,7 @@ TEST_F(CompactFilesTest, SentinelCompressionType) {
options.create_if_missing = true;
FlushedFileCollector* collector = new FlushedFileCollector();
options.listeners.emplace_back(collector);
DB* db = nullptr;
std::unique_ptr<DB> db;
ASSERT_OK(DB::Open(options, db_name_, &db));
ASSERT_OK(db->Put(WriteOptions(), "key", "val"));
@@ -421,7 +415,8 @@ TEST_F(CompactFilesTest, SentinelCompressionType) {
// Ensure background work is fully finished including listener callbacks
// before accessing listener state.
ASSERT_OK(static_cast_with_check<DBImpl>(db)->TEST_WaitForBackgroundWork());
ASSERT_OK(
static_cast_with_check<DBImpl>(db.get())->TEST_WaitForBackgroundWork());
auto l0_files = collector->GetFlushedFiles();
ASSERT_EQ(1, l0_files.size());
@@ -437,7 +432,6 @@ TEST_F(CompactFilesTest, SentinelCompressionType) {
// compression_name property
ASSERT_EQ("BuiltinV2;02;", name_and_table_props.second->compression_name);
}
delete db;
}
}
@@ -462,11 +456,7 @@ TEST_F(CompactFilesTest, CompressionWithBlockAlign) {
}
std::unique_ptr<DB> db;
{
DB* _db = nullptr;
ASSERT_OK(DB::Open(options, db_name_, &_db));
db.reset(_db);
}
ASSERT_OK(DB::Open(options, db_name_, &db));
ASSERT_OK(db->Put(WriteOptions(), "key", "val"));
ASSERT_OK(db->Flush(FlushOptions()));
@@ -505,7 +495,7 @@ TEST_F(CompactFilesTest, GetCompactionJobInfo) {
FlushedFileCollector* collector = new FlushedFileCollector();
options.listeners.emplace_back(collector);
DB* db = nullptr;
std::unique_ptr<DB> db;
ASSERT_OK(DestroyDB(db_name_, options));
Status s = DB::Open(options, db_name_, &db);
ASSERT_OK(s);
@@ -516,7 +506,8 @@ TEST_F(CompactFilesTest, GetCompactionJobInfo) {
ASSERT_OK(db->Put(WriteOptions(), std::to_string(i),
std::string(1000, 'a' + (i % 26))));
}
ASSERT_OK(static_cast_with_check<DBImpl>(db)->TEST_WaitForFlushMemTable());
ASSERT_OK(
static_cast_with_check<DBImpl>(db.get())->TEST_WaitForFlushMemTable());
auto l0_files_1 = collector->GetFlushedFiles();
CompactionOptions co;
co.compression = CompressionType::kLZ4Compression;
@@ -532,7 +523,6 @@ TEST_F(CompactFilesTest, GetCompactionJobInfo) {
ASSERT_EQ(compaction_job_info.output_level, 0);
ASSERT_OK(compaction_job_info.status);
// no assertion failure
delete db;
}
// Helper function to generate zero-padded keys
@@ -548,11 +538,11 @@ TEST_F(CompactFilesTest, TrivialMoveNonOverlappingFiles) {
options.compression = kNoCompression;
options.level_compaction_dynamic_level_bytes = false;
DB* db = nullptr;
std::unique_ptr<DB> db;
ASSERT_OK(DestroyDB(db_name_, options));
Status s = DB::Open(options, db_name_, &db);
ASSERT_OK(s);
ASSERT_NE(db, nullptr);
ASSERT_NE(db.get(), nullptr);
// Create 3 non-overlapping files in L0
// File 1: keys [a00-a99]
@@ -665,8 +655,6 @@ TEST_F(CompactFilesTest, TrivialMoveNonOverlappingFiles) {
ASSERT_OK(db->Get(ReadOptions(), key, &value));
ASSERT_EQ(value, "value_" + key);
}
delete db;
}
TEST_F(CompactFilesTest, TrivialMoveBlockedByOverlap) {
@@ -677,11 +665,11 @@ TEST_F(CompactFilesTest, TrivialMoveBlockedByOverlap) {
options.level_compaction_dynamic_level_bytes = false;
options.num_levels = 7;
DB* db = nullptr;
std::unique_ptr<DB> db;
ASSERT_OK(DestroyDB(db_name_, options));
Status s = DB::Open(options, db_name_, &db);
ASSERT_OK(s);
ASSERT_NE(db, nullptr);
ASSERT_NE(db.get(), nullptr);
// Create a file in L6 with keys [m00-m99] (wide range)
for (int i = 0; i < 100; i++) {
@@ -757,8 +745,6 @@ TEST_F(CompactFilesTest, TrivialMoveBlockedByOverlap) {
ASSERT_OK(db->Get(ReadOptions(), key, &value));
ASSERT_EQ(value, "updated_value_" + key);
}
delete db;
}
} // namespace ROCKSDB_NAMESPACE
+664 -34
View File
@@ -7,12 +7,14 @@
#include <iterator>
#include <limits>
#include <memory>
#include "db/blob/blob_fetcher.h"
#include "db/blob/blob_file_builder.h"
#include "db/blob/blob_index.h"
#include "db/blob/prefetch_buffer_collection.h"
#include "db/snapshot_checker.h"
#include "db/wide/blob_column_resolver_util.h"
#include "db/wide/wide_column_serialization.h"
#include "db/wide/wide_columns_helper.h"
#include "logging/logging.h"
@@ -20,8 +22,106 @@
#include "rocksdb/listener.h"
#include "table/internal_iterator.h"
#include "test_util/sync_point.h"
#include "util/autovector.h"
namespace ROCKSDB_NAMESPACE {
// CompactionBlobResolver implementation
void CompactionBlobResolver::Init(BlobFetcher* blob_fetcher,
PrefetchBufferCollection* prefetch_buffers,
CompactionIterationStats* iter_stats) {
blob_fetcher_ = blob_fetcher;
prefetch_buffers_ = prefetch_buffers;
iter_stats_ = iter_stats;
}
void CompactionBlobResolver::Reset(
const Slice& user_key, const std::vector<WideColumn>* columns,
const std::vector<std::pair<size_t, BlobIndex>>* blob_columns,
bool track_resolve_error) {
user_key_ = user_key;
columns_ = columns;
blob_columns_ = blob_columns;
track_resolve_error_ = track_resolve_error;
resolved_cache_.clear();
resolve_error_.reset();
}
Status CompactionBlobResolver::ResolveColumn(size_t column_index,
Slice* resolved_value) {
Status status = Status::OK();
if (columns_ == nullptr || column_index >= columns_->size()) {
status = Status::InvalidArgument("Column index out of bounds");
} else {
const BlobIndex* blob_index_ptr =
blob_resolver_util::FindBlobColumn(blob_columns_, column_index);
if (blob_index_ptr == nullptr) {
// Not a blob column - return the inline value directly
*resolved_value = (*columns_)[column_index].value();
} else {
// Check if we've already resolved this column
PinnableSlice* cached =
blob_resolver_util::FindInCache(resolved_cache_, column_index);
if (cached != nullptr) {
*resolved_value = Slice(*cached);
} else if (blob_fetcher_ == nullptr) {
status = Status::NotSupported("Blob fetcher not available");
} else {
const BlobIndex& blob_index = *blob_index_ptr;
// Handle inlined blobs (e.g., from legacy stacked BlobDB with TTL)
if (blob_index.IsInlined()) {
*resolved_value = blob_resolver_util::CacheInlinedBlob(
resolved_cache_, column_index, blob_index);
} else {
FilePrefetchBuffer* prefetch_buffer =
prefetch_buffers_ ? prefetch_buffers_->GetOrCreatePrefetchBuffer(
blob_index.file_number())
: nullptr;
uint64_t bytes_read = 0;
resolved_cache_.emplace_back(column_index,
std::make_unique<PinnableSlice>());
auto& new_entry = resolved_cache_.back();
status =
blob_fetcher_->FetchBlob(user_key_, blob_index, prefetch_buffer,
new_entry.second.get(), &bytes_read);
if (!status.ok()) {
resolved_cache_.pop_back();
} else {
if (iter_stats_ != nullptr) {
++iter_stats_->num_blobs_read;
iter_stats_->total_blob_bytes_read += bytes_read;
}
*resolved_value = Slice(*new_entry.second);
}
}
}
}
}
if (!status.ok() && track_resolve_error_ && !resolve_error_.has_value()) {
resolve_error_.emplace(status);
}
return status;
}
bool CompactionBlobResolver::IsBlobColumn(size_t column_index) const {
if (columns_ == nullptr || column_index >= columns_->size()) {
return false;
}
return blob_resolver_util::IsBlobColumnIndex(blob_columns_, column_index);
}
size_t CompactionBlobResolver::NumColumns() const {
if (columns_ == nullptr) {
return 0;
}
return columns_->size();
}
CompactionIterator::CompactionIterator(
InternalIterator* input, const Comparator* cmp, MergeHelper* merge_helper,
SequenceNumber last_sequence, std::vector<SequenceNumber>* snapshots,
@@ -38,7 +138,8 @@ CompactionIterator::CompactionIterator(
const std::atomic<bool>* shutting_down,
const std::shared_ptr<Logger> info_log,
const std::string* full_history_ts_low,
std::optional<SequenceNumber> preserve_seqno_min)
std::optional<SequenceNumber> preserve_seqno_min,
const Version* input_version, Env::IOActivity blob_read_io_activity)
: CompactionIterator(
input, cmp, merge_helper, last_sequence, snapshots, earliest_snapshot,
earliest_write_conflict_snapshot, job_snapshot, snapshot_checker, env,
@@ -47,7 +148,8 @@ CompactionIterator::CompactionIterator(
manual_compaction_canceled,
compaction ? std::make_unique<RealCompaction>(compaction) : nullptr,
must_count_input_entries, compaction_filter, shutting_down, info_log,
full_history_ts_low, preserve_seqno_min) {}
full_history_ts_low, preserve_seqno_min, input_version,
blob_read_io_activity) {}
CompactionIterator::CompactionIterator(
InternalIterator* input, const Comparator* cmp, MergeHelper* merge_helper,
@@ -65,7 +167,8 @@ CompactionIterator::CompactionIterator(
const std::atomic<bool>* shutting_down,
const std::shared_ptr<Logger> info_log,
const std::string* full_history_ts_low,
std::optional<SequenceNumber> preserve_seqno_min)
std::optional<SequenceNumber> preserve_seqno_min,
const Version* input_version, Env::IOActivity blob_read_io_activity)
: input_(input, cmp, must_count_input_entries),
cmp_(cmp),
merge_helper_(merge_helper),
@@ -80,6 +183,8 @@ CompactionIterator::CompactionIterator(
blob_file_builder_(blob_file_builder),
compaction_(std::move(compaction)),
compaction_filter_(compaction_filter),
filter_supports_v4_(
compaction_filter_ ? compaction_filter_->SupportsFilterV4() : false),
shutting_down_(shutting_down),
manual_compaction_canceled_(manual_compaction_canceled),
bottommost_level_(compaction_ && compaction_->bottommost_level() &&
@@ -98,7 +203,8 @@ CompactionIterator::CompactionIterator(
merge_out_iter_(merge_helper_),
blob_garbage_collection_cutoff_file_number_(
ComputeBlobGarbageCollectionCutoffFileNumber(compaction_.get())),
blob_fetcher_(CreateBlobFetcherIfNeeded(compaction_.get())),
blob_fetcher_(CreateBlobFetcherIfNeeded(compaction_.get(), input_version,
blob_read_io_activity)),
prefetch_buffers_(
CreatePrefetchBufferCollectionIfNeeded(compaction_.get())),
current_key_committed_(false),
@@ -120,6 +226,8 @@ CompactionIterator::CompactionIterator(
timestamp_size_ == full_history_ts_low_->size());
#endif
input_.SetPinnedItersMgr(&pinned_iters_mgr_);
blob_resolver_.Init(blob_fetcher_.get(), prefetch_buffers_.get(),
&iter_stats_);
// The default `merge_until_status_` does not need to be checked since it is
// overwritten as soon as `MergeUntil()` is called
merge_until_status_.PermitUncheckedError();
@@ -131,6 +239,24 @@ CompactionIterator::~CompactionIterator() {
input_.SetPinnedItersMgr(nullptr);
}
void CompactionIterator::SetBlobFetcher(const Version* version,
BlobFileCache* blob_file_cache,
Env::IOActivity io_activity,
bool allow_write_path_fallback) {
if (blob_fetcher_ != nullptr || version == nullptr) {
return;
}
ReadOptions read_options;
read_options.io_activity = io_activity;
read_options.fill_cache = false;
blob_fetcher_ = std::make_unique<BlobFetcher>(
version, read_options, blob_file_cache, allow_write_path_fallback);
blob_resolver_.Init(blob_fetcher_.get(), prefetch_buffers_.get(),
&iter_stats_);
}
void CompactionIterator::ResetRecordCounts() {
iter_stats_.num_record_drop_user = 0;
iter_stats_.num_record_drop_hidden = 0;
@@ -258,9 +384,10 @@ bool CompactionIterator::InvokeFilterIfNeeded(bool* need_skip,
compaction_filter_skip_until_.rep());
if (decision == CompactionFilter::Decision::kUndetermined &&
!compaction_filter_->IsStackedBlobDbInternalCompactionFilter()) {
if (!compaction_) {
status_ =
Status::Corruption("Unexpected blob index outside of compaction");
if (!blob_fetcher_) {
status_ = Status::NotSupported(
"Blob-backed value filtering requires blob read support in the "
"current table-file creation context");
validity_info_.Invalidate();
return false;
}
@@ -287,8 +414,6 @@ bool CompactionIterator::InvokeFilterIfNeeded(bool* need_skip,
uint64_t bytes_read = 0;
assert(blob_fetcher_);
s = blob_fetcher_->FetchBlob(ikey_.user_key, blob_index,
prefetch_buffer, &blob_value_,
&bytes_read);
@@ -309,7 +434,19 @@ bool CompactionIterator::InvokeFilterIfNeeded(bool* need_skip,
const Slice* existing_val = nullptr;
const WideColumns* existing_col = nullptr;
WideColumns existing_columns;
// Reuse member variable to avoid per-key heap allocation.
filter_existing_columns_.clear();
WideColumnBlobResolver* blob_resolver_ptr = nullptr;
// Use member variables for entity column storage to avoid per-key
// heap allocations. Cleared at start of each use.
entity_columns_.clear();
entity_blob_columns_.clear();
// Storage for eagerly resolved blob values used by the FilterV3
// compatibility path. Keep this in the same scope as the FilterV4()
// invocation below because filter_existing_columns_ stores Slices into
// these buffers.
autovector<PinnableSlice, 4> resolved_blobs_storage;
if (ikey_.type != kTypeWideColumnEntity) {
if (!blob_value_.empty()) {
@@ -318,23 +455,125 @@ bool CompactionIterator::InvokeFilterIfNeeded(bool* need_skip,
existing_val = &value_;
}
} else {
Slice value_copy = value_;
const Status s =
WideColumnSerialization::Deserialize(value_copy, existing_columns);
// Check if entity has blob columns that need resolution
bool has_blob_columns = false;
{
Status s_hbc =
WideColumnSerialization::HasBlobColumns(value_, has_blob_columns);
if (!s_hbc.ok()) {
status_ = s_hbc;
validity_info_.Invalidate();
return false;
}
}
if (UNLIKELY(has_blob_columns)) {
// Entity has blob columns. DeserializeV2() populates
// entity_columns_ with the full column set; blob-backed entries still
// carry the serialized BlobIndex bytes in value(). The companion
// entity_blob_columns_ side list identifies which column indexes are
// blob references and provides the decoded BlobIndex objects.
Slice input_copy = value_;
Status s = WideColumnSerialization::DeserializeV2(
input_copy, entity_columns_, entity_blob_columns_);
if (!s.ok()) {
status_ = s;
validity_info_.Invalidate();
return false;
}
// Mark as deserialized so PrepareOutput can skip re-deserialization
// if the filter returns kKeep.
entity_deserialized_ = true;
if (!s.ok()) {
status_ = s;
// Build filter_existing_columns_ with raw values (blob columns
// have blob index as value, not the actual blob data)
for (size_t i = 0; i < entity_columns_.size(); ++i) {
filter_existing_columns_.emplace_back(entity_columns_[i].name(),
entity_columns_[i].value());
}
if (filter_supports_v4_) {
// Reset member blob resolver for this entity - filter can call
// resolver->ResolveColumn() to fetch blob values on-demand
blob_resolver_.Reset(ikey_.user_key, &entity_columns_,
&entity_blob_columns_,
/*track_resolve_error=*/true);
blob_resolver_ptr = &blob_resolver_;
} else {
// FilterV3 compatibility: eagerly resolve all blob columns so
// the filter sees actual values (not raw BlobIndex bytes).
// resolved_blobs_storage is in the outer scope so the data
// outlives filter_existing_columns_ (which holds Slices into
// it).
resolved_blobs_storage.resize(entity_blob_columns_.size());
for (size_t bi = 0; bi < entity_blob_columns_.size(); ++bi) {
const size_t col_idx = entity_blob_columns_[bi].first;
const BlobIndex& blob_idx = entity_blob_columns_[bi].second;
if (blob_idx.IsInlined()) {
resolved_blobs_storage[bi].PinSelf(blob_idx.value());
} else {
FilePrefetchBuffer* prefetch_buffer =
prefetch_buffers_
? prefetch_buffers_->GetOrCreatePrefetchBuffer(
blob_idx.file_number())
: nullptr;
uint64_t bytes_read = 0;
assert(blob_fetcher_);
Status s_fetch = blob_fetcher_->FetchBlob(
ikey_.user_key, blob_idx, prefetch_buffer,
&resolved_blobs_storage[bi], &bytes_read);
if (!s_fetch.ok()) {
status_ = s_fetch;
validity_info_.Invalidate();
return false;
}
++iter_stats_.num_blobs_read;
iter_stats_.total_blob_bytes_read += bytes_read;
}
// Replace the blob index value in filter_existing_columns_
// with the resolved blob value
filter_existing_columns_[col_idx] =
WideColumn(entity_columns_[col_idx].name(),
Slice(resolved_blobs_storage[bi]));
}
}
} else {
// No blob columns, use fast path
Slice value_copy = value_;
const Status s = WideColumnSerialization::Deserialize(
value_copy, filter_existing_columns_);
if (!s.ok()) {
status_ = s;
validity_info_.Invalidate();
return false;
}
}
existing_col = &filter_existing_columns_;
}
// existing_val / existing_col point into value_, blob_value_,
// entity_columns_, and resolved_blobs_storage, all of which stay alive
// until this call returns.
decision = compaction_filter_->FilterV4(
level_, filter_key, value_type, existing_val, existing_col,
&compaction_filter_value_, &new_columns,
compaction_filter_skip_until_.rep(), blob_resolver_ptr);
if (blob_resolver_ptr != nullptr) {
Status resolve_status = blob_resolver_.resolve_status();
if (!resolve_status.ok()) {
// Keep lazy FilterV4 failure semantics aligned with the eager
// FilterV3 compatibility path: if blob resolution fails while the
// filter is inspecting the entry, fail compaction even if the
// filter returned kKeep after noticing the error.
status_ = std::move(resolve_status);
validity_info_.Invalidate();
return false;
}
existing_col = &existing_columns;
}
decision = compaction_filter_->FilterV3(
level_, filter_key, value_type, existing_val, existing_col,
&compaction_filter_value_, &new_columns,
compaction_filter_skip_until_.rep());
}
iter_stats_.total_filter_time +=
@@ -342,10 +581,10 @@ bool CompactionIterator::InvokeFilterIfNeeded(bool* need_skip,
}
if (decision == CompactionFilter::Decision::kUndetermined) {
// Should not reach here, since FilterV2/FilterV3 should never return
// Should not reach here, since FilterV2/V3/V4 should never return
// kUndetermined.
status_ = Status::NotSupported(
"FilterV2/FilterV3 should never return kUndetermined");
"FilterV2/V3/V4 should never return kUndetermined");
validity_info_.Invalidate();
return false;
}
@@ -354,7 +593,7 @@ bool CompactionIterator::InvokeFilterIfNeeded(bool* need_skip,
cmp_->Compare(*compaction_filter_skip_until_.rep(), ikey_.user_key) <=
0) {
// Can't skip to a key smaller than the current one.
// Keep the key as per FilterV2/FilterV3 documentation.
// Keep the key as per FilterV2/V3/V4 documentation.
decision = CompactionFilter::Decision::kKeep;
}
@@ -442,6 +681,16 @@ bool CompactionIterator::InvokeFilterIfNeeded(bool* need_skip,
}
value_ = compaction_filter_value_;
// Value changed, force re-deserialization in PrepareOutput
entity_deserialized_ = false;
}
if (UNLIKELY(compaction_filter_value_.size() >
std::numeric_limits<uint32_t>::max())) {
status_ = Status::Corruption(
"CompactionFilter result value size exceeds 4GB limit");
validity_info_.Invalidate();
return false;
}
return true;
@@ -450,9 +699,14 @@ bool CompactionIterator::InvokeFilterIfNeeded(bool* need_skip,
void CompactionIterator::NextFromInput() {
at_next_ = false;
validity_info_.Invalidate();
entity_deserialized_ = false;
while (!Valid() && input_.Valid() && !IsPausingManualCompaction() &&
!IsShuttingDown()) {
// A filtered-out key can advance directly to the next input record without
// returning to PrepareOutput(), so clear the per-record deserialization
// cache at the start of each loop iteration.
entity_deserialized_ = false;
key_ = input_.key();
value_ = input_.value();
blob_value_.Reset();
@@ -1266,6 +1520,340 @@ void CompactionIterator::GarbageCollectBlobIfNeeded() {
}
}
// Wide-column entity compaction flow after InvokeFilterIfNeeded():
// 1. PrepareOutput() is the only caller of the helper block below.
// 2. PrepareOutput() ensures entity_columns_/entity_blob_columns_ describe
// the current entity, either by reusing the filter-time deserialization or
// by deserializing once itself.
// 3. PrepareOutput() then picks exactly one follow-up path:
// - inline-only entity:
// ExtractLargeColumnValuesIfNeeded()
// This is the flush/compaction extraction path. It may rewrite large
// inline columns into blob references.
// - entity already containing blob references:
// GarbageCollectEntityBlobsIfNeeded()
// This is the blob-GC path. It only fetches / relocates references
// that fall below the current GC cutoff.
// Both helpers leave value_ unchanged when no rewrite is needed.
void CompactionIterator::ExtractLargeColumnValuesIfNeeded() {
assert(ikey_.type == kTypeWideColumnEntity);
// Check if blob extraction is enabled
if (!blob_file_builder_) {
return;
}
// Deserialize using member variables (already populated by PrepareOutput
// for the combined deserialization path). If not yet populated, deserialize
// here.
if (entity_columns_.empty()) {
Slice entity_slice = value_;
entity_columns_.clear();
entity_blob_columns_.clear();
Status s = WideColumnSerialization::DeserializeV2(
entity_slice, entity_columns_, entity_blob_columns_);
if (!s.ok()) {
status_ = s;
validity_info_.Invalidate();
return;
}
}
// IMPORTANT: If there are already blob columns, we skip extraction entirely.
// This means new large inline columns added to such entities (e.g., via
// PutEntity after a previous PutEntity that already had blob columns) will
// NOT be extracted to blob files until the entity undergoes a full rewrite
// (e.g., via compaction that GCs all existing blob refs).
//
// This is a deliberate simplification: supporting partial extraction
// (extracting only new large inline columns while preserving existing
// blob refs) would require merging old and new blob column sets during
// serialization, and the benefit is marginal since full rewrites happen
// naturally during blob GC.
if (UNLIKELY(!entity_blob_columns_.empty())) {
return;
}
// Try to extract large column values to blob files
// Track which columns were extracted and their blob indices
std::vector<std::pair<size_t, BlobIndex>> new_blob_columns;
std::string temp_blob_index;
for (size_t i = 0; i < entity_columns_.size(); ++i) {
const Slice& col_value = entity_columns_[i].value();
// Clear the temporary buffer for each column
temp_blob_index.clear();
// Try to add the column value to blob file
// blob_file_builder_->Add() will check min_blob_size internally
// and only write to blob file if the value is large enough
const Status add_status =
blob_file_builder_->Add(user_key(), col_value, &temp_blob_index);
if (!add_status.ok()) {
status_ = add_status;
validity_info_.Invalidate();
return;
}
// If blob_index is not empty, the value was extracted to a blob file
if (!temp_blob_index.empty()) {
BlobIndex blob_idx;
const Status decode_status = blob_idx.DecodeFrom(temp_blob_index);
if (!decode_status.ok()) {
status_ = decode_status;
validity_info_.Invalidate();
return;
}
new_blob_columns.emplace_back(i, blob_idx);
}
}
// If no columns were extracted, nothing to do
if (new_blob_columns.empty()) {
return;
}
// Re-serialize the entity with blob indices using the Slice-based overload
// to avoid copying column names and values to strings.
entity_wide_columns_.clear();
entity_wide_columns_.reserve(entity_columns_.size());
for (const auto& col : entity_columns_) {
entity_wide_columns_.emplace_back(col.name(), col.value());
}
rewritten_entity_.clear();
const Status serialize_status = WideColumnSerialization::SerializeV2(
entity_wide_columns_, new_blob_columns, rewritten_entity_);
if (!serialize_status.ok()) {
status_ = serialize_status;
validity_info_.Invalidate();
return;
}
// Update value_ to point to the rewritten entity
value_ = rewritten_entity_;
}
bool CompactionIterator::FetchBlobsNeedingGC(
const std::vector<WideColumn>& /*columns*/,
const std::vector<std::pair<size_t, BlobIndex>>& blob_columns,
std::vector<std::pair<size_t, PinnableSlice>>* fetched_blob_values) {
assert(fetched_blob_values != nullptr);
for (const auto& blob_col : blob_columns) {
const size_t col_idx = blob_col.first;
const BlobIndex& blob_index = blob_col.second;
// Skip inlined blobs - they don't need GC
if (blob_index.IsInlined()) {
continue;
}
// Check if this blob file needs garbage collection
if (blob_index.file_number() >=
blob_garbage_collection_cutoff_file_number_) {
continue;
}
// This blob needs to be relocated - fetch its value
FilePrefetchBuffer* prefetch_buffer =
prefetch_buffers_ ? prefetch_buffers_->GetOrCreatePrefetchBuffer(
blob_index.file_number())
: nullptr;
uint64_t bytes_read = 0;
assert(blob_fetcher_);
PinnableSlice blob_value;
Status s = blob_fetcher_->FetchBlob(user_key(), blob_index, prefetch_buffer,
&blob_value, &bytes_read);
if (!s.ok()) {
status_ = s;
validity_info_.Invalidate();
return false;
}
++iter_stats_.num_blobs_read;
iter_stats_.total_blob_bytes_read += bytes_read;
fetched_blob_values->emplace_back(col_idx, std::move(blob_value));
}
return !fetched_blob_values->empty();
}
std::vector<std::pair<size_t, BlobIndex>>
CompactionIterator::RelocateBlobValues(
const std::vector<std::pair<size_t, PinnableSlice>>& fetched_blob_values) {
std::vector<std::pair<size_t, BlobIndex>> new_blob_columns;
if (blob_file_builder_) {
for (const auto& fv : fetched_blob_values) {
const size_t col_idx = fv.first;
const Slice col_value(fv.second);
std::string temp_blob_index;
Status status =
blob_file_builder_->Add(user_key(), col_value, &temp_blob_index);
if (!status.ok()) {
status_ = status;
validity_info_.Invalidate();
new_blob_columns.clear();
break;
}
if (!temp_blob_index.empty()) {
BlobIndex blob_idx;
status = blob_idx.DecodeFrom(temp_blob_index);
if (!status.ok()) {
status_ = status;
validity_info_.Invalidate();
new_blob_columns.clear();
break;
}
new_blob_columns.emplace_back(col_idx, blob_idx);
// Track relocation stats here (after successful relocation)
++iter_stats_.num_blobs_relocated;
iter_stats_.total_blob_bytes_relocated += col_value.size();
}
// If temp_blob_index is empty, the value will be inlined
}
}
return new_blob_columns;
}
void CompactionIterator::SerializeEntityAfterGC(
const std::vector<WideColumn>& columns,
const std::vector<std::pair<size_t, BlobIndex>>& original_blob_columns,
const std::vector<std::pair<size_t, PinnableSlice>>& fetched_blob_values,
const std::vector<std::pair<size_t, BlobIndex>>& new_blob_columns) {
// Collect all blob columns for final serialization: include non-GC'd
// originals and newly relocated. Use linear scans since these
// collections are typically small (<5 elements).
std::vector<std::pair<size_t, BlobIndex>> final_blob_columns;
for (const auto& blob_col : original_blob_columns) {
// Check if this column was fetched (GC'd) via linear scan
bool was_fetched = false;
for (const auto& fv : fetched_blob_values) {
if (fv.first == blob_col.first) {
was_fetched = true;
break;
}
}
if (!was_fetched) {
// This blob column wasn't GC'd - keep original
final_blob_columns.push_back(blob_col);
}
}
// Add newly relocated blobs
for (const auto& nb : new_blob_columns) {
final_blob_columns.push_back(nb);
}
// Build WideColumns for serialization, replacing fetched blob values
// with their resolved inline values.
WideColumns wide_columns;
wide_columns.reserve(columns.size());
for (size_t i = 0; i < columns.size(); ++i) {
// Linear scan to find if this column was fetched
const PinnableSlice* fetched_value = nullptr;
for (const auto& fv : fetched_blob_values) {
if (fv.first == i) {
fetched_value = &fv.second;
break;
}
}
if (fetched_value != nullptr) {
// This blob column was fetched for GC; use the resolved inline value.
wide_columns.emplace_back(columns[i].name(), Slice(*fetched_value));
} else {
// For inline columns, this is the actual value. For non-fetched blob
// columns, this contains the raw serialized blob index, which is fine
// because SerializeV2() re-encodes from the BlobIndex
// object in final_blob_columns, ignoring this value for blob-type
// columns.
wide_columns.emplace_back(columns[i].name(), columns[i].value());
}
}
// Serialize the entity
rewritten_entity_.clear();
Status s;
if (final_blob_columns.empty()) {
s = WideColumnSerialization::Serialize(wide_columns, rewritten_entity_);
} else {
s = WideColumnSerialization::SerializeV2(wide_columns, final_blob_columns,
rewritten_entity_);
}
if (!s.ok()) {
status_ = s;
validity_info_.Invalidate();
return;
}
value_ = rewritten_entity_;
}
void CompactionIterator::GarbageCollectEntityBlobsIfNeeded() {
assert(ikey_.type == kTypeWideColumnEntity);
if (!compaction_ || !compaction_->enable_blob_garbage_collection()) {
return;
}
// This is the second branch in the flow above: the entity already has V2
// blob references, so compaction considers only relocating the subset of
// referenced blob files that are old enough for blob GC.
// Use member variables already populated by PrepareOutput's combined
// deserialization path. If not yet populated, deserialize here.
if (entity_columns_.empty()) {
Slice entity_slice = value_;
entity_columns_.clear();
entity_blob_columns_.clear();
Status s = WideColumnSerialization::DeserializeV2(
entity_slice, entity_columns_, entity_blob_columns_);
if (!s.ok()) {
status_ = s;
validity_info_.Invalidate();
return;
}
}
// The caller (PrepareOutput) already verified blob columns exist.
assert(!entity_blob_columns_.empty());
if (entity_blob_columns_.empty()) {
return;
}
// Fetch blobs that need GC
std::vector<std::pair<size_t, PinnableSlice>> fetched_blob_values;
if (!FetchBlobsNeedingGC(entity_columns_, entity_blob_columns_,
&fetched_blob_values)) {
// Either no blobs needed GC (status_ is OK) or an error occurred
// (status_ is already set by FetchBlobsNeedingGC). In both cases,
// we return without modifying the entity.
return;
}
// Relocate fetched values to new blob files
std::vector<std::pair<size_t, BlobIndex>> new_blob_columns =
RelocateBlobValues(fetched_blob_values);
if (!status_.ok()) {
return; // Error occurred during relocation
}
// Serialize the final entity
SerializeEntityAfterGC(entity_columns_, entity_blob_columns_,
fetched_blob_values, new_blob_columns);
}
void CompactionIterator::PrepareOutput() {
if (Valid()) {
if (LIKELY(!is_range_del_)) {
@@ -1273,6 +1861,37 @@ void CompactionIterator::PrepareOutput() {
ExtractLargeValueIfNeeded();
} else if (ikey_.type == kTypeBlobIndex) {
GarbageCollectBlobIfNeeded();
} else if (ikey_.type == kTypeWideColumnEntity) {
// If entity was already deserialized (e.g., by InvokeFilterIfNeeded)
// and the filter returned kKeep, skip redundant re-deserialization.
if (!entity_deserialized_) {
TEST_SYNC_POINT(
"CompactionIterator::PrepareOutput:DeserializeEntity");
// Deserialize entity once into member variables, then decide between
// blob GC and extraction based on whether blob columns exist.
// This avoids the double parse of HasBlobColumns() + DeserializeV2().
entity_columns_.clear();
entity_blob_columns_.clear();
Slice entity_slice = value_;
{
Status s_deser = WideColumnSerialization::DeserializeV2(
entity_slice, entity_columns_, entity_blob_columns_);
if (!s_deser.ok()) {
status_ = s_deser;
validity_info_.Invalidate();
return;
}
}
} else {
TEST_SYNC_POINT(
"CompactionIterator::PrepareOutput:SkipDeserializeEntity");
}
entity_deserialized_ = false; // Reset for next iteration
if (UNLIKELY(!entity_blob_columns_.empty())) {
GarbageCollectEntityBlobsIfNeeded();
} else {
ExtractLargeColumnValuesIfNeeded();
}
}
}
@@ -1421,21 +2040,32 @@ uint64_t CompactionIterator::ComputeBlobGarbageCollectionCutoffFileNumber(
}
std::unique_ptr<BlobFetcher> CompactionIterator::CreateBlobFetcherIfNeeded(
const CompactionProxy* compaction) {
if (!compaction) {
return nullptr;
}
const Version* const version = compaction->input_version();
if (!version) {
const CompactionProxy* compaction, const Version* input_version,
Env::IOActivity blob_read_io_activity) {
const Version* const version =
compaction != nullptr ? compaction->input_version() : input_version;
if (version == nullptr) {
return nullptr;
}
ReadOptions read_options;
read_options.io_activity = Env::IOActivity::kCompaction;
read_options.io_activity = compaction != nullptr
? Env::IOActivity::kCompaction
: blob_read_io_activity;
read_options.fill_cache = false;
return std::unique_ptr<BlobFetcher>(new BlobFetcher(version, read_options));
BlobFileCache* blob_file_cache = nullptr;
bool allow_write_path_fallback = false;
if (compaction == nullptr) {
allow_write_path_fallback = true;
auto* cfd = version->cfd();
if (cfd != nullptr) {
blob_file_cache = cfd->blob_file_cache();
}
}
return std::unique_ptr<BlobFetcher>(new BlobFetcher(
version, read_options, blob_file_cache, allow_write_path_fallback));
}
std::unique_ptr<PrefetchBufferCollection>
+158 -23
View File
@@ -7,10 +7,12 @@
#include <algorithm>
#include <cinttypes>
#include <deque>
#include <memory>
#include <optional>
#include <string>
#include <unordered_set>
#include <vector>
#include "db/blob/blob_index.h"
#include "db/compaction/compaction.h"
#include "db/compaction/compaction_iteration_stats.h"
#include "db/merge_helper.h"
@@ -19,12 +21,69 @@
#include "db/snapshot_checker.h"
#include "options/cf_options.h"
#include "rocksdb/compaction_filter.h"
#include "rocksdb/slice.h"
namespace ROCKSDB_NAMESPACE {
class BlobFileBuilder;
class BlobFileCache;
class BlobFetcher;
class Version;
class PrefetchBufferCollection;
class FilePrefetchBuffer;
class Version;
// Internal implementation of WideColumnBlobResolver for compaction.
// This class enables lazy loading of blob column values during compaction
// filter processing.
class CompactionBlobResolver : public WideColumnBlobResolver {
public:
CompactionBlobResolver()
: columns_(nullptr),
blob_columns_(nullptr),
blob_fetcher_(nullptr),
prefetch_buffers_(nullptr),
iter_stats_(nullptr) {}
// Set the fixed context (blob fetcher, prefetch buffers, stats) once.
void Init(BlobFetcher* blob_fetcher,
PrefetchBufferCollection* prefetch_buffers,
CompactionIterationStats* iter_stats);
Status ResolveColumn(size_t column_index, Slice* resolved_value) override;
bool IsBlobColumn(size_t column_index) const override;
size_t NumColumns() const override;
Status resolve_status() const {
if (!resolve_error_.has_value()) {
return Status::OK();
}
return *resolve_error_;
}
// Reset the resolver for a new entity. Clears resolved cache.
void Reset(const Slice& user_key, const std::vector<WideColumn>* columns,
const std::vector<std::pair<size_t, BlobIndex>>* blob_columns,
bool track_resolve_error = false);
private:
Slice user_key_;
const std::vector<WideColumn>* columns_;
const std::vector<std::pair<size_t, BlobIndex>>* blob_columns_;
BlobFetcher* blob_fetcher_;
PrefetchBufferCollection* prefetch_buffers_;
CompactionIterationStats* iter_stats_;
bool track_resolve_error_ = false;
// Cache for resolved blob values to avoid re-fetching. Uses a vector of
// (column_index, PinnableSlice) pairs — typical entities have few blob
// columns (<5), making linear scan cheaper than hash map overhead.
std::vector<std::pair<size_t, std::unique_ptr<PinnableSlice>>>
resolved_cache_;
// Sticky resolver error for the current entity. This is enabled only for the
// lazy FilterV4 path so compaction can still fail even if the filter
// notices the error and returns kKeep.
std::optional<Status> resolve_error_;
};
// A wrapper of internal iterator whose purpose is to count how
// many entries there are in the iterator.
@@ -213,34 +272,43 @@ class CompactionIterator {
const std::atomic<bool>* shutting_down = nullptr,
const std::shared_ptr<Logger> info_log = nullptr,
const std::string* full_history_ts_low = nullptr,
std::optional<SequenceNumber> preserve_seqno_min = {});
std::optional<SequenceNumber> preserve_seqno_min = {},
const Version* input_version = nullptr,
Env::IOActivity blob_read_io_activity = Env::IOActivity::kCompaction);
// Constructor with custom CompactionProxy, used for tests.
CompactionIterator(InternalIterator* input, const Comparator* cmp,
MergeHelper* merge_helper, SequenceNumber last_sequence,
std::vector<SequenceNumber>* snapshots,
SequenceNumber earliest_snapshot,
SequenceNumber earliest_write_conflict_snapshot,
SequenceNumber job_snapshot,
const SnapshotChecker* snapshot_checker, Env* env,
bool report_detailed_time,
CompactionRangeDelAggregator* range_del_agg,
BlobFileBuilder* blob_file_builder,
bool allow_data_in_errors,
bool enforce_single_del_contracts,
const std::atomic<bool>& manual_compaction_canceled,
std::unique_ptr<CompactionProxy> compaction,
bool must_count_input_entries,
const CompactionFilter* compaction_filter = nullptr,
const std::atomic<bool>* shutting_down = nullptr,
const std::shared_ptr<Logger> info_log = nullptr,
const std::string* full_history_ts_low = nullptr,
std::optional<SequenceNumber> preserve_seqno_min = {});
CompactionIterator(
InternalIterator* input, const Comparator* cmp, MergeHelper* merge_helper,
SequenceNumber last_sequence, std::vector<SequenceNumber>* snapshots,
SequenceNumber earliest_snapshot,
SequenceNumber earliest_write_conflict_snapshot,
SequenceNumber job_snapshot, const SnapshotChecker* snapshot_checker,
Env* env, bool report_detailed_time,
CompactionRangeDelAggregator* range_del_agg,
BlobFileBuilder* blob_file_builder, bool allow_data_in_errors,
bool enforce_single_del_contracts,
const std::atomic<bool>& manual_compaction_canceled,
std::unique_ptr<CompactionProxy> compaction,
bool must_count_input_entries,
const CompactionFilter* compaction_filter = nullptr,
const std::atomic<bool>* shutting_down = nullptr,
const std::shared_ptr<Logger> info_log = nullptr,
const std::string* full_history_ts_low = nullptr,
std::optional<SequenceNumber> preserve_seqno_min = {},
const Version* input_version = nullptr,
Env::IOActivity blob_read_io_activity = Env::IOActivity::kCompaction);
~CompactionIterator();
void ResetRecordCounts();
// Non-compaction table-file creation (e.g., flush/recovery) can still need
// blob resolution when direct-write wide entities already contain blob
// references. Plumb a fetcher explicitly in those cases.
void SetBlobFetcher(const Version* version, BlobFileCache* blob_file_cache,
Env::IOActivity io_activity,
bool allow_write_path_fallback);
// Seek to the beginning of the compaction iterator output.
//
// REQUIRED: Call only once.
@@ -302,6 +370,12 @@ class CompactionIterator {
// for regular values (kTypeValue).
void ExtractLargeValueIfNeeded();
// Extracts large column values from wide column entities to blob files.
// For each column whose value size >= min_blob_size, the value is written
// to a blob file and replaced with a blob index reference. Should only be
// called for wide column entities (kTypeWideColumnEntity).
void ExtractLargeColumnValuesIfNeeded();
// Relocates valid blobs residing in the oldest blob files if garbage
// collection is enabled. Relocated blobs are written to new blob files or
// inlined in the LSM tree depending on the current settings (i.e.
@@ -312,6 +386,39 @@ class CompactionIterator {
// algorithm is also called from here.
void GarbageCollectBlobIfNeeded();
// Garbage collects blob references within wide column entities. For entities
// that contain blob column references, this method checks if any of the
// referenced blob files need garbage collection. If so, the blob values are
// fetched and either relocated to new blob files or inlined based on
// current settings. Should only be called for wide column entities
// (kTypeWideColumnEntity) that have blob columns.
void GarbageCollectEntityBlobsIfNeeded();
// Helper for GarbageCollectEntityBlobsIfNeeded: Fetches blob values that need
// garbage collection based on the cutoff file number.
// Returns true if any blobs were fetched, false otherwise.
// On error, sets status_ and returns false.
bool FetchBlobsNeedingGC(
const std::vector<WideColumn>& columns,
const std::vector<std::pair<size_t, BlobIndex>>& blob_columns,
std::vector<std::pair<size_t, PinnableSlice>>* fetched_blob_values);
// Helper for GarbageCollectEntityBlobsIfNeeded: Relocates fetched blob
// values to new blob files if blob_file_builder_ is available.
// Returns the new blob column indices. On error, sets status_ and returns
// empty.
std::vector<std::pair<size_t, BlobIndex>> RelocateBlobValues(
const std::vector<std::pair<size_t, PinnableSlice>>& fetched_blob_values);
// Helper for GarbageCollectEntityBlobsIfNeeded: Serializes the entity after
// garbage collection, combining original columns with fetched values and
// new blob indices. On error, sets status_.
void SerializeEntityAfterGC(
const std::vector<WideColumn>& columns,
const std::vector<std::pair<size_t, BlobIndex>>& original_blob_columns,
const std::vector<std::pair<size_t, PinnableSlice>>& fetched_blob_values,
const std::vector<std::pair<size_t, BlobIndex>>& new_blob_columns);
// Invoke compaction filter if needed.
// Return true on success, false on failures (e.g.: kIOError).
bool InvokeFilterIfNeeded(bool* need_skip, Slice* skip_until);
@@ -352,7 +459,8 @@ class CompactionIterator {
static uint64_t ComputeBlobGarbageCollectionCutoffFileNumber(
const CompactionProxy* compaction);
static std::unique_ptr<BlobFetcher> CreateBlobFetcherIfNeeded(
const CompactionProxy* compaction);
const CompactionProxy* compaction, const Version* input_version,
Env::IOActivity blob_read_io_activity);
static std::unique_ptr<PrefetchBufferCollection>
CreatePrefetchBufferCollectionIfNeeded(const CompactionProxy* compaction);
@@ -376,6 +484,7 @@ class CompactionIterator {
BlobFileBuilder* blob_file_builder_;
std::unique_ptr<CompactionProxy> compaction_;
const CompactionFilter* compaction_filter_;
const bool filter_supports_v4_;
const std::atomic<bool>* shutting_down_;
const std::atomic<bool>& manual_compaction_canceled_;
const bool bottommost_level_;
@@ -479,6 +588,32 @@ class CompactionIterator {
PinnableSlice blob_value_;
std::string compaction_filter_value_;
InternalKey compaction_filter_skip_until_;
// Buffer for storing rewritten entity with blob references after
// extracting large column values to blob files.
std::string rewritten_entity_;
// Cached deserialized columns for the current wide-column entity, reused
// across iterations to avoid per-key heap allocations. This is the full
// sorted column set. For blob-backed columns, value() initially contains the
// serialized BlobIndex bytes from the entity.
std::vector<WideColumn> entity_columns_;
// Side list for the blob-backed subset of entity_columns_. Each pair is
// (column_index_in_entity_columns_, decoded_blob_index), so callers can tell
// which entries in entity_columns_ are blob references without reparsing.
std::vector<std::pair<size_t, BlobIndex>> entity_blob_columns_;
// Temporary Slice-based view over entity_columns_ used when reserializing an
// entity after blob extraction, to avoid copying names/values into strings.
WideColumns entity_wide_columns_;
// Reusable storage for existing_columns in InvokeFilterIfNeeded,
// avoiding per-key heap allocations.
WideColumns filter_existing_columns_;
// Reusable blob resolver for compaction filter, avoiding per-key heap
// allocation. Init() is called once; Reset() is called per entity.
CompactionBlobResolver blob_resolver_;
// True when entity_columns_/entity_blob_columns_ already describe the
// current input record (e.g., because InvokeFilterIfNeeded() deserialized
// it) and PrepareOutput() can skip redundant work. Reset for each candidate
// input record and whenever the filter rewrites the entity value.
bool entity_deserialized_{false};
// "level_ptrs" holds indices that remember which file of an associated
// level we were last checking during the last call to compaction->
// KeyNotExistsBeyondOutputLevel(). This allows future calls to the function
File diff suppressed because it is too large Load Diff
+135 -22
View File
@@ -33,6 +33,7 @@
#include "db/range_del_aggregator.h"
#include "db/version_edit.h"
#include "db/version_set.h"
#include "file/file_util.h"
#include "file/filename.h"
#include "file/read_write_util.h"
#include "file/sst_file_manager_impl.h"
@@ -57,6 +58,7 @@
#include "table/table_builder.h"
#include "table/unique_id_impl.h"
#include "test_util/sync_point.h"
#include "util/hash_containers.h"
#include "util/stop_watch.h"
namespace ROCKSDB_NAMESPACE {
@@ -103,6 +105,8 @@ const char* GetCompactionReasonString(CompactionReason compaction_reason) {
return "RoundRobinTtl";
case CompactionReason::kRefitLevel:
return "RefitLevel";
case CompactionReason::kReadTriggered:
return "ReadTriggered";
case CompactionReason::kNumOfReasons:
// fall through
default:
@@ -895,6 +899,10 @@ Status CompactionJob::VerifyOutputFiles() {
}
auto verify_table = [&](SubcompactionState& subcompaction_state) {
// Collect file open metadata during verification when fast_sst_open
// is enabled, keyed by file number.
UnorderedMap<uint64_t, std::string> file_open_metadata_map;
for (const auto& output_file : subcompaction_state.GetOutputs()) {
// Verify that the table is usable
// We set for_compaction to false and don't
@@ -903,17 +911,21 @@ Status CompactionJob::VerifyOutputFiles() {
// use_direct_io_for_flush_and_compaction is true, we will regard this
// verification as user reads since the goal is to cache it here for
// further user reads
ReadOptions verify_table_read_options(Env::IOActivity::kCompaction);
verify_table_read_options.verify_checksums = true;
verify_table_read_options.readahead_size =
ReadOptions verification_read_options(Env::IOActivity::kCompaction);
verification_read_options.verify_checksums = true;
verification_read_options.readahead_size =
file_options_for_read_.compaction_readahead_size;
std::unique_ptr<TableReader> table_reader_guard;
TableReader* table_reader_ptr = table_reader_guard.get();
verify_table_read_options.rate_limiter_priority =
verification_read_options.rate_limiter_priority =
GetRateLimiterPriority();
std::string file_open_metadata;
std::string* file_open_metadata_ptr =
mutable_db_options_copy_.fast_sst_open ? &file_open_metadata
: nullptr;
InternalIterator* iter = cfd->table_cache()->NewIterator(
verify_table_read_options, file_options_, cfd->internal_comparator(),
verification_read_options, file_options_, cfd->internal_comparator(),
output_file.meta,
/*range_del_agg=*/nullptr, compact_->compaction->mutable_cf_options(),
/*table_reader_ptr=*/&table_reader_ptr,
@@ -924,7 +936,10 @@ Status CompactionJob::VerifyOutputFiles() {
MaxFileSizeForL0MetaPin(compact_->compaction->mutable_cf_options()),
/*smallest_compaction_key=*/nullptr,
/*largest_compaction_key=*/nullptr,
/*allow_unprepared_value=*/false);
/*allow_unprepared_value=*/false,
/*range_del_read_seqno=*/nullptr,
/*range_del_iter=*/nullptr,
/*maybe_pin_table_handle=*/false, file_open_metadata_ptr);
auto s = iter->status();
if (s.ok()) {
// Check for remote/local compaction and verify_output_flags flags
@@ -941,12 +956,17 @@ Status CompactionJob::VerifyOutputFiles() {
!!(verify_output_flags & VerifyOutputFlags::kVerifyBlockChecksum);
const bool should_verify_iteration =
!!(verify_output_flags & VerifyOutputFlags::kVerifyIteration);
const bool should_verify_file_checksum =
!!(verify_output_flags &
VerifyOutputFlags::kVerifyFileChecksum) &&
db_options_.file_checksum_gen_factory != nullptr &&
output_file.meta.file_checksum != kUnknownFileChecksum;
if (should_verify_block_checksum) {
assert(table_reader_ptr != nullptr);
// If verifying iteration as well, verify meta blocks here only to
// avoid redundant checks on data blocks
s = table_reader_ptr->VerifyChecksum(
verify_table_read_options, TableReaderCaller::kCompaction,
verification_read_options, TableReaderCaller::kCompaction,
/*meta_blocks_only=*/should_verify_iteration);
}
if (s.ok() && should_verify_iteration) {
@@ -967,6 +987,24 @@ Status CompactionJob::VerifyOutputFiles() {
"was computed when written");
}
}
if (s.ok() && should_verify_file_checksum) {
std::string file_checksum;
std::string file_checksum_func_name;
std::string fname =
GetTableFileName(output_file.meta.fd.GetNumber());
s = GenerateOneFileChecksum(
fs_.get(), fname, db_options_.file_checksum_gen_factory.get(),
output_file.meta.file_checksum_func_name, &file_checksum,
&file_checksum_func_name,
verification_read_options.readahead_size,
db_options_.allow_mmap_reads, io_tracer_,
db_options_.rate_limiter.get(), verification_read_options,
stats_, db_options_.clock, file_options_for_read_);
if (s.ok() && file_checksum != output_file.meta.file_checksum) {
s = Status::Corruption(
"File checksum mismatch for compaction output file " + fname);
}
}
}
}
@@ -976,6 +1014,27 @@ Status CompactionJob::VerifyOutputFiles() {
subcompaction_state.status = s;
break;
}
if (!file_open_metadata.empty()) {
file_open_metadata_map[output_file.meta.fd.GetNumber()] =
std::move(file_open_metadata);
}
}
// Apply collected file open metadata to mutable outputs
if (!file_open_metadata_map.empty()) {
auto apply_metadata =
[&file_open_metadata_map](
std::vector<CompactionOutputs::Output>& outputs) {
for (auto& output : outputs) {
auto it = file_open_metadata_map.find(output.meta.fd.GetNumber());
if (it != file_open_metadata_map.end()) {
output.meta.file_open_metadata = std::move(it->second);
}
}
};
apply_metadata(subcompaction_state.GetMutableCompactionOutputs());
apply_metadata(subcompaction_state.GetMutableProximalOutputs());
}
};
for (size_t i = 1; i < compact_->sub_compact_states.size(); i++) {
@@ -1092,6 +1151,9 @@ Status CompactionJob::Run() {
status = VerifyOutputFiles();
}
TEST_SYNC_POINT_CALLBACK("CompactionJob::Run():AfterVerifyOutputFiles",
&status);
if (status.ok()) {
SetOutputTableProperties();
}
@@ -1278,6 +1340,16 @@ Status CompactionJob::Install(bool* compaction_released) {
<< pl_stats.bytes_written_blob;
}
// Propagate Install failure to compact_->status so that
// CleanupCompaction() -> SubcompactionState::Cleanup() sees the failure and
// calls ReleaseObsolete on output files' table cache entries. Without this,
// if Run() succeeds but InstallCompactionResults() fails, Cleanup would see
// overall_status = OK and skip ReleaseObsolete, leaking entries for output
// files that were never installed into any Version.
if (!status.ok() && compact_->status.ok()) {
compact_->status = status;
}
CleanupCompaction();
return status;
}
@@ -2326,11 +2398,16 @@ Status CompactionJob::InstallCompactionResults(bool* compaction_released) {
*compaction_released = true;
};
return versions_->LogAndApply(compaction->column_family_data(), read_options,
write_options, edit, db_mutex_, db_directory_,
/*new_descriptor_log=*/false,
/*column_family_options=*/nullptr,
manifest_wcb);
Status s;
TEST_SYNC_POINT_CALLBACK(
"CompactionJob::InstallCompactionResults:BeforeLogAndApply", &s);
if (s.ok()) {
s = versions_->LogAndApply(compaction->column_family_data(), read_options,
write_options, edit, db_mutex_, db_directory_,
/*new_descriptor_log=*/false,
/*column_family_options=*/nullptr, manifest_wcb);
}
return s;
}
void CompactionJob::RecordCompactionIOStats() {
@@ -2470,8 +2547,16 @@ Status CompactionJob::OpenCompactionOutputFile(SubcompactionState* sub_compact,
return s;
}
// Enable hash computation if paranoid_file_checks is on or if
// verify_output_flags includes kVerifyIteration, so that
// VerifyOutputFiles() can compare the hash of the written data
// against a re-read of the output file.
bool enable_output_hash =
paranoid_file_checks_ ||
!!(sub_compact->compaction->mutable_cf_options().verify_output_flags &
VerifyOutputFlags::kVerifyIteration);
outputs.AddOutput(std::move(meta), cfd->internal_comparator(),
paranoid_file_checks_);
enable_output_hash);
}
writable_file->SetIOPriority(GetRateLimiterPriority());
@@ -2512,7 +2597,7 @@ Status CompactionJob::OpenCompactionOutputFile(SubcompactionState* sub_compact,
void CompactionJob::CleanupCompaction() {
for (SubcompactionState& sub_compact : compact_->sub_compact_states) {
sub_compact.Cleanup(table_cache_.get());
sub_compact.Cleanup(table_cache_.get(), compact_->status);
}
delete compact_;
compact_ = nullptr;
@@ -2537,6 +2622,32 @@ bool CompactionJob::UpdateInternalStatsFromInputFiles(
bool has_error = false;
const ReadOptions read_options(Env::IOActivity::kCompaction);
const auto& input_table_properties = compaction->GetInputTableProperties();
// Check all input files for old block-based SST format_version. Why? Old
// block-based SST files from roughly version 5.0 to 5.18 could produce
// inaccurate num_entries counts due to the evolution of its handling along
// with num_range_deletions. We have to disable some paranoid checks when
// compacting files from such an old release. However, we don't have great
// information to identify those files, so we heuristically over-approximate
// that set of files using
// (a) format_version < 5, which will be true for any files from RocksDB <
// 6.6.0 and should not be true for any recent production files
// (b) to avoid including non-block-based SST files (which still use older
// format_version markers, and do not support DeleteRange), we also require
// the presence of the user property "rocksdb.block.based.table.index.type",
// which was added in RocksDB 2.8 and is always present in block-based tables.
for (const auto& tp_pair : input_table_properties) {
if (tp_pair.second && tp_pair.second->format_version < 5) {
// Check for block-based table by looking for its index type property
const auto& user_props = tp_pair.second->user_collected_properties;
if (user_props.find(BlockBasedTablePropertyNames::kIndexType) !=
user_props.end()) {
job_stats_->has_accurate_num_input_records = false;
break;
}
}
}
for (int input_level = 0;
input_level < static_cast<int>(compaction->num_input_levels());
++input_level) {
@@ -2781,7 +2892,10 @@ Status CompactionJob::ReadTablePropertiesDirectly(
std::shared_ptr<const TableProperties>* tp) {
std::unique_ptr<FSRandomAccessFile> file;
std::string file_name = GetTableFileName(file_meta->fd.GetNumber());
Status s = ioptions.fs->NewRandomAccessFile(file_name, file_options_, &file,
FileOptions fopts = file_options_;
fopts.file_checksum = file_meta->file_checksum;
fopts.file_checksum_func_name = file_meta->file_checksum_func_name;
Status s = ioptions.fs->NewRandomAccessFile(file_name, fopts, &file,
nullptr /* dbg */);
if (!s.ok()) {
return s;
@@ -2875,12 +2989,17 @@ void CompactionJob::RestoreCompactionOutputs(
const auto& output_files = subcompaction_progress_per_level.GetOutputFiles();
const bool enable_output_hash =
paranoid_file_checks_ ||
!!(compact_->compaction->mutable_cf_options().verify_output_flags &
VerifyOutputFlags::kVerifyIteration);
for (size_t i = 0; i < output_files.size(); i++) {
FileMetaData file_copy = output_files[i];
outputs_to_restore->AddOutput(std::move(file_copy),
cfd->internal_comparator(),
paranoid_file_checks_, true /* finished */);
enable_output_hash, true /* finished */);
outputs_to_restore->UpdateTableProperties(
*output_files_table_properties[i]);
@@ -2903,12 +3022,6 @@ void CompactionJob::RestoreCompactionOutputs(
// - Status::NotFound(): No valid progress to resume from
// - Status::Corruption(): Resume key is invalid, beyond input range, or output
// restoration failed
// - Other non-OK status: Iterator errors or file system issues during
// restoration
//
// The caller must check for Status::IsIncomplete() to distinguish between
// "no resume needed" (proceed with `InternalIterator::SeekToFirst()`) vs
// "resume failed" scenarios.
Status CompactionJob::MaybeResumeSubcompactionProgressOnInputIterator(
SubcompactionState* sub_compact, InternalIterator* input_iter) {
const ReadOptions read_options(Env::IOActivity::kCompaction);
+4
View File
@@ -254,6 +254,10 @@ class CompactionJob {
// @param num_input_range_del if non-null, will be set to the number of range
// deletion entries in this compaction input.
//
// If any input file has potentially unreliable num_entries count (old SST
// files - details in implementation),
// job_stats_->has_accurate_num_input_records is set to false.
//
// Returns true iff internal_stats_.output_level_stats.num_input_records and
// num_input_range_del are calculated successfully.
//
+7 -7
View File
@@ -82,7 +82,7 @@ class CompactionJobStatsTest : public testing::Test,
std::string dbname_;
std::string alternative_wal_dir_;
Env* env_;
DB* db_;
std::unique_ptr<DB> db_;
std::vector<ColumnFamilyHandle*> handles_;
uint32_t max_subcompactions_;
@@ -123,7 +123,7 @@ class CompactionJobStatsTest : public testing::Test,
static void SetUpTestCase() {}
static void TearDownTestCase() {}
DBImpl* dbfull() { return static_cast_with_check<DBImpl>(db_); }
DBImpl* dbfull() { return static_cast_with_check<DBImpl>(db_.get()); }
void CreateColumnFamilies(const std::vector<std::string>& cfs,
const Options& options) {
@@ -162,7 +162,8 @@ class CompactionJobStatsTest : public testing::Test,
column_families.emplace_back(cfs[i], options[i]);
}
DBOptions db_opts = DBOptions(options[0]);
return DB::Open(db_opts, dbname_, column_families, &handles_, &db_);
auto s = DB::Open(db_opts, dbname_, column_families, &handles_, &db_);
return s;
}
Status TryReopenWithColumnFamilies(const std::vector<std::string>& cfs,
@@ -179,8 +180,7 @@ class CompactionJobStatsTest : public testing::Test,
delete h;
}
handles_.clear();
delete db_;
db_ = nullptr;
db_.reset();
}
void DestroyAndReopen(const Options& options) {
@@ -743,7 +743,7 @@ TEST_P(CompactionJobStatsTest, CompactionJobStatsTest) {
}
ASSERT_OK(Flush(1));
ASSERT_OK(static_cast_with_check<DBImpl>(db_)->TEST_WaitForCompact());
ASSERT_OK(dbfull()->TEST_WaitForCompact());
stats_checker->set_verify_next_comp_io_stats(true);
std::atomic<bool> first_prepare_write(true);
@@ -944,7 +944,7 @@ TEST_P(CompactionJobStatsTest, UniversalCompactionTest) {
start_key += key_base) {
MakeTableWithKeyValues(&rnd, start_key, start_key + key_base - 1, kKeySize,
kValueSize, key_interval, compression_ratio, 1);
ASSERT_OK(static_cast_with_check<DBImpl>(db_)->TEST_WaitForCompact());
ASSERT_OK(dbfull()->TEST_WaitForCompact());
}
ASSERT_EQ(stats_checker->NumberOfUnverifiedStats(), 0U);
}
-454
View File
@@ -2409,460 +2409,6 @@ TEST_F(CompactionJobIOPriorityTest, GetRateLimiterPriority) {
kMaxSequenceNumber, 1, false, {kInvalidBlobFileNumber}, true,
Env::IO_LOW, Env::IO_LOW);
}
class ResumableCompactionJobTest : public CompactionJobTestBase {
public:
ResumableCompactionJobTest()
: CompactionJobTestBase(
test::PerThreadDBPath("allow_resumption_job_test"),
BytewiseComparator(), [](uint64_t /*ts*/) { return ""; },
/*test_io_priority=*/false, TableTypeForTest::kBlockBasedTable) {}
protected:
static constexpr const char* kCancelBeforeThisKey = "cancel_before_this_key";
std::string progress_dir_;
bool enable_cancel_ = false;
std::atomic<int> stop_count_{0};
std::atomic<bool> cancel_{false};
SequenceNumber cancel_before_seqno = kMaxSequenceNumber;
void SetUp() override {
CompactionJobTestBase::SetUp();
SyncPoint::GetInstance()->SetCallBack(
"CompactionOutputs::ShouldStopBefore::manual_decision",
[this](void* p) {
auto* pair = static_cast<std::pair<bool*, const Slice>*>(p);
*(pair->first) = true;
// Cancel after outputting a specific key
if (enable_cancel_) {
ParsedInternalKey parsed_key;
if (ParseInternalKey(pair->second, &parsed_key, true).ok()) {
if (parsed_key.user_key == kCancelBeforeThisKey &&
(cancel_before_seqno == kMaxSequenceNumber ||
parsed_key.sequence == cancel_before_seqno)) {
cancel_.store(true);
}
}
}
});
SyncPoint::GetInstance()->EnableProcessing();
}
void TearDown() override {
SyncPoint::GetInstance()->DisableProcessing();
SyncPoint::GetInstance()->ClearAllCallBacks();
if (env_->FileExists(progress_dir_).ok()) {
std::vector<std::string> files;
EXPECT_OK(env_->GetChildren(progress_dir_, &files));
for (const auto& file : files) {
if (file != "." && file != "..") {
EXPECT_OK(env_->DeleteFile(progress_dir_ + "/" + file));
}
}
EXPECT_OK(env_->DeleteDir(progress_dir_));
}
CompactionJobTestBase::TearDown();
}
void NewDB() {
if (env_->FileExists(progress_dir_).ok()) {
std::vector<std::string> files;
EXPECT_OK(env_->GetChildren(progress_dir_, &files));
for (const auto& file : files) {
if (file != "." && file != "..") {
EXPECT_OK(env_->DeleteFile(progress_dir_ + "/" + file));
}
}
EXPECT_OK(env_->DeleteDir(progress_dir_));
}
CompactionJobTestBase::NewDB();
progress_dir_ = test::PerThreadDBPath("compaction_progress");
ASSERT_OK(env_->CreateDirIfMissing(progress_dir_));
}
void EnableCompactionCancel() { enable_cancel_ = true; }
void DisableCompactionCancel() {
enable_cancel_ = false;
cancel_.store(false);
}
std::unique_ptr<log::Writer> CreateCompactionProgressWriter(
const std::string& compaction_progress_file) {
std::unique_ptr<FSWritableFile> file;
EXPECT_OK(fs_->NewWritableFile(compaction_progress_file, FileOptions(),
&file, nullptr));
auto file_writer = std::make_unique<WritableFileWriter>(
std::move(file), compaction_progress_file, FileOptions());
auto compaction_progress_writer =
std::make_unique<log::Writer>(std::move(file_writer), 0, false);
return compaction_progress_writer;
}
Status RunCompactionWithProgressTracking(
const CompactionProgress& compaction_progress,
log::Writer* compaction_progress_writer,
std::vector<SequenceNumber> snapshots = {},
std::shared_ptr<Statistics> stats = nullptr) {
mutex_.Lock();
auto cfd = versions_->GetColumnFamilySet()->GetDefault();
auto files = cfd->current()->storage_info()->LevelFiles(0);
db_options_.statistics = stats;
db_options_.stats = db_options_.statistics.get();
std::vector<CompactionInputFiles> compaction_input_files;
CompactionInputFiles level;
level.level = 0;
level.files = files;
compaction_input_files.push_back(level);
Compaction compaction(
cfd->current()->storage_info(), cfd->ioptions(),
cfd->GetLatestMutableCFOptions(), mutable_db_options_,
compaction_input_files, 1, mutable_cf_options_.target_file_size_base,
mutable_cf_options_.max_compaction_bytes, 0, kNoCompression,
cfd->GetLatestMutableCFOptions().compression_opts,
Temperature::kUnknown, 0, {}, std::nullopt, nullptr,
CompactionReason::kManualCompaction);
compaction.FinalizeInputInfo(cfd->current());
LogBuffer log_buffer(InfoLogLevel::INFO_LEVEL, db_options_.info_log.get());
EventLogger event_logger(db_options_.info_log.get());
JobContext job_context(1, false);
job_context.InitSnapshotContext(nullptr, nullptr, kMaxSequenceNumber,
std::move(snapshots));
CompactionJobStats job_stats;
CompactionJob compaction_job(
0, &compaction, db_options_, mutable_db_options_, env_options_,
versions_.get(), &shutting_down_, &log_buffer, nullptr, nullptr,
nullptr, stats.get(), &mutex_, &error_handler_, &job_context,
table_cache_, &event_logger, false, false, dbname_, &job_stats,
Env::Priority::USER, nullptr, cancel_,
CompactionJob::kCompactionAbortedFalse, env_->GenerateUniqueId(),
DBImpl::GenerateDbSessionId(nullptr), "");
compaction_job.Prepare(std::nullopt, compaction_progress,
compaction_progress_writer);
mutex_.Unlock();
compaction_job.Run().PermitUncheckedError();
EXPECT_OK(compaction_job.io_status());
mutex_.Lock();
bool compaction_released = false;
Status s = compaction_job.Install(&compaction_released);
mutex_.Unlock();
if (!compaction_released) {
compaction.ReleaseCompactionFiles(s);
}
return s;
}
SubcompactionProgress ReadAndParseProgress(
const std::string& compaction_progress_file) {
std::unique_ptr<FSSequentialFile> seq_file;
EXPECT_OK(fs_->NewSequentialFile(compaction_progress_file, FileOptions(),
&seq_file, nullptr));
auto file_reader = std::make_unique<SequentialFileReader>(
std::move(seq_file), compaction_progress_file, 0, nullptr);
log::Reader reader(nullptr, std::move(file_reader), nullptr, true, 0);
SubcompactionProgressBuilder builder;
std::string record;
Slice slice;
while (reader.ReadRecord(&slice, &record)) {
VersionEdit edit;
if (!edit.DecodeFrom(slice).ok()) {
continue;
}
builder.ProcessVersionEdit(edit);
}
EXPECT_TRUE(builder.HasAccumulatedSubcompactionProgress());
return builder.GetAccumulatedSubcompactionProgress();
}
// Test utility function to verify that compaction progress was correctly
// persisted to the progress file after compaction interruption.
//
// VERIFIES:
// - Progress file exists and has expected size (empty if no progress
// expected)
// - Next internal key to compact matches expected user key with proper format
// - Number of processed input records matches position in ordered input keys
// - Number of processed output records equals number of processed input
// records (by test design to simplify verification)
// - Each output file contains exactly one user key (by test design to
// simplify verification)
void VerifyCompactionProgressPersisted(
const std::string& compaction_progress_file,
const std::string& next_user_key_to_compact,
const std::vector<std::string>& ordered_intput_keys) {
ASSERT_OK(env_->FileExists(compaction_progress_file));
uint64_t file_size;
ASSERT_OK(env_->GetFileSize(compaction_progress_file, &file_size));
if (next_user_key_to_compact.empty()) {
ASSERT_EQ(file_size, 0);
return;
}
const auto& subcompaction_progress =
ReadAndParseProgress(compaction_progress_file);
ASSERT_FALSE(subcompaction_progress.next_internal_key_to_compact.empty());
ParsedInternalKey parsed_next_key;
ASSERT_OK(
ParseInternalKey(subcompaction_progress.next_internal_key_to_compact,
&parsed_next_key, true /* log_err_key */));
ASSERT_EQ(parsed_next_key.user_key, next_user_key_to_compact);
ASSERT_EQ(parsed_next_key.sequence, kMaxSequenceNumber);
ASSERT_EQ(parsed_next_key.type, kValueTypeForSeek);
auto it = std::find(ordered_intput_keys.begin(), ordered_intput_keys.end(),
next_user_key_to_compact);
ASSERT_TRUE(it != ordered_intput_keys.end());
auto next_key_index = std::distance(ordered_intput_keys.begin(), it);
ASSERT_EQ(subcompaction_progress.num_processed_input_records,
next_key_index);
ASSERT_EQ(subcompaction_progress.output_level_progress
.GetNumProcessedOutputRecords(),
next_key_index);
ASSERT_EQ(
subcompaction_progress.output_level_progress.GetOutputFiles().size(),
next_key_index);
for (size_t i = 0;
i <
subcompaction_progress.output_level_progress.GetOutputFiles().size();
++i) {
const auto& output_file =
subcompaction_progress.output_level_progress.GetOutputFiles()[i];
ASSERT_EQ(output_file.smallest.user_key().ToString(),
output_file.largest.user_key().ToString());
ASSERT_EQ(output_file.largest.user_key().ToString(),
ordered_intput_keys[i]);
}
}
void RunCancelAndResumeTest(
const std::initializer_list<mock::KVPair>& input_file_1,
const std::initializer_list<mock::KVPair>& input_file_2,
uint64_t last_sequence, const std::vector<uint64_t>& snapshots,
const std::string& expected_next_key_to_compact,
const std::vector<std::string>& expected_input_keys,
bool cancelled_past_mid_point = false) {
std::shared_ptr<Statistics> stats = ROCKSDB_NAMESPACE::CreateDBStatistics();
auto file1 = mock::MakeMockFile(input_file_1);
AddMockFile(file1);
auto file2 = mock::MakeMockFile(input_file_2);
AddMockFile(file2);
SetLastSequence(last_sequence);
// First compaction (will be cancelled)
std::string compaction_progress_file =
CompactionProgressFileName(progress_dir_, 123);
std::unique_ptr<log::Writer> compaction_progress_writer =
CreateCompactionProgressWriter(compaction_progress_file);
ASSERT_OK(stats->Reset());
EnableCompactionCancel();
Status status = RunCompactionWithProgressTracking(
CompactionProgress{}, compaction_progress_writer.get(), snapshots,
stats);
ASSERT_TRUE(status.IsManualCompactionPaused());
DisableCompactionCancel();
HistogramData cancelled_compaction_stats;
stats->histogramData(FILE_WRITE_COMPACTION_MICROS,
&cancelled_compaction_stats);
VerifyCompactionProgressPersisted(compaction_progress_file,
expected_next_key_to_compact,
expected_input_keys);
// Resume compaction
CompactionProgress compaction_progress;
if (expected_next_key_to_compact != "") {
compaction_progress.push_back(
ReadAndParseProgress(compaction_progress_file));
}
std::string compaction_progress_file_2 =
CompactionProgressFileName(progress_dir_, 234);
std::unique_ptr<log::Writer> compaction_progress_writer_2 =
CreateCompactionProgressWriter(compaction_progress_file_2);
ASSERT_OK(stats->Reset());
status = RunCompactionWithProgressTracking(
compaction_progress, compaction_progress_writer_2.get(),
{} /* snapshots */, stats);
ASSERT_OK(status);
if (cancelled_past_mid_point) {
HistogramData resumed_compaction_stats;
stats->histogramData(FILE_WRITE_COMPACTION_MICROS,
&resumed_compaction_stats);
ASSERT_GT(cancelled_compaction_stats.count,
resumed_compaction_stats.count);
}
}
};
TEST_F(ResumableCompactionJobTest, BasicProgressPersistence) {
NewDB();
auto file1 = mock::MakeMockFile({
{KeyStr("a", 1U, kTypeValue), "val1"},
{KeyStr("b", 2U, kTypeValue), "val2"},
});
AddMockFile(file1);
auto file2 = mock::MakeMockFile({
{KeyStr("c", 3U, kTypeValue), "val3"},
{KeyStr("d", 4U, kTypeValue), "val4"},
});
AddMockFile(file2);
SetLastSequence(4U);
std::string compaction_progress_file =
CompactionProgressFileName(progress_dir_, 123);
std::unique_ptr<log::Writer> compaction_progress_writer =
CreateCompactionProgressWriter(compaction_progress_file);
Status status = RunCompactionWithProgressTracking(
CompactionProgress(), compaction_progress_writer.get());
ASSERT_OK(status);
VerifyCompactionProgressPersisted(
compaction_progress_file, "d" /* next_user_key_to_compact */,
{"a", "b", "c", "d"} /* ordered_intput_keys */);
}
TEST_F(ResumableCompactionJobTest, BasicProgressResume) {
NewDB();
RunCancelAndResumeTest(
{{KeyStr("a", 1U, kTypeValue), "val1"},
{KeyStr("b", 2U, kTypeValue), "val2"}} /* input_file_1 */,
{{KeyStr("bb", 3U, kTypeValue), "val3"},
{KeyStr(kCancelBeforeThisKey, 4U, kTypeValue),
"val4"}} /* input_file_2 */,
4U /* last_sequence */, {} /* snapshots */,
kCancelBeforeThisKey /* expected_next_key_to_compact */,
{"a", "b", "bb", kCancelBeforeThisKey} /* expected_input_keys */,
true /* cancelled_past_mid_point */);
}
TEST_F(ResumableCompactionJobTest, NoProgressResumeOnSameKey) {
NewDB();
// `cancel_before_seqno` is set to 0U to force cancellation after
// `kCancelBeforeThisKey@1` instead of `kCancelBeforeThisKey@2`.
// The seqno is 0 because `kCancelBeforeThisKey@1` will have its sequence
// number zeroed during compaction while `kCancelBeforeThisKey@2` won't be
cancel_before_seqno = 0U;
RunCancelAndResumeTest(
{{KeyStr(kCancelBeforeThisKey, 1U, kTypeValue),
"val1"}} /* input_file_1 */,
{{KeyStr(kCancelBeforeThisKey, 2U, kTypeValue), "val11"},
{KeyStr("d", 3U, kTypeValue), "val2"}} /* input_file_2 */,
3U /* last_sequence */, {1U} /* snapshots */,
"" /* expected_next_key_to_compact */,
{kCancelBeforeThisKey, kCancelBeforeThisKey,
"d"} /* expected_input_keys */);
}
TEST_F(ResumableCompactionJobTest, NoProgressResumeOnDeleteRange) {
NewDB();
RunCancelAndResumeTest(
{{KeyStr("a", 1U, kTypeValue), "val1"},
{KeyStr("b", 2U, kTypeValue), "val2"},
{KeyStr(kCancelBeforeThisKey, 3U, kTypeValue),
"val3"}} /* input_file_1 */,
{{KeyStr(kCancelBeforeThisKey, 4U, kTypeRangeDeletion),
"range_deletion_end_key"},
{KeyStr("d", 5U, kTypeValue), "val4"}} /* input_file_2 */,
5U /* last_sequence */, {3U} /* snapshots */,
"b" /* expected_next_key_to_compact */,
{"a", "b", kCancelBeforeThisKey, kCancelBeforeThisKey,
"d"} /* expected_input_keys */);
}
TEST_F(ResumableCompactionJobTest, NoProgressResumeOnMerge) {
merge_op_ = MergeOperators::CreateStringAppendOperator();
NewDB();
RunCancelAndResumeTest(
{{KeyStr("a", 1U, kTypeValue), "val1"},
{KeyStr("b", 2U, kTypeValue), "val2"}} /* input_file_1 */,
{{KeyStr("bb", 3U, kTypeValue), "val3"},
{KeyStr(kCancelBeforeThisKey, 4U, kTypeMerge),
"val4"}} /* input_file_2 */,
4U /* last_sequence */, {} /* snapshots */,
"bb" /* expected_next_key_to_compact */,
{"a", "b", "bb", kCancelBeforeThisKey} /* expected_input_keys */);
}
TEST_F(ResumableCompactionJobTest, NoProgressResumeOnSingleDelete) {
NewDB();
RunCancelAndResumeTest(
{{KeyStr("a", 1U, kTypeValue), "val1"},
{KeyStr("b", 2U, kTypeValue), "val2"},
{KeyStr(kCancelBeforeThisKey, 3U, kTypeValue),
"val3"}} /* input_file_1 */,
{{KeyStr(kCancelBeforeThisKey, 4U, kTypeSingleDeletion), ""},
{KeyStr("d", 5U, kTypeValue), "val4"}} /* input_file_2 */,
5U /* last_sequence */, {3U} /* snapshots */,
"b" /* expected_next_key_to_compact */,
{"a", "b", kCancelBeforeThisKey, kCancelBeforeThisKey,
"d"} /* expected_input_keys */);
}
TEST_F(ResumableCompactionJobTest, NoProgressResumeOnDeletionAtBottom) {
NewDB();
RunCancelAndResumeTest(
{{KeyStr("a", 1U, kTypeValue), "val1"},
{KeyStr("b", 2U, kTypeValue), "val2"},
{KeyStr(kCancelBeforeThisKey, 3U, kTypeValue),
"val3"}} /* input_file_1 */,
{{KeyStr(kCancelBeforeThisKey, 4U, kTypeDeletion), ""},
{KeyStr("d", 5U, kTypeValue), "val4"}} /* input_file_2 */,
5U /* last_sequence */, {3U} /* snapshots */,
"b" /* expected_next_key_to_compact */,
{"a", "b", kCancelBeforeThisKey, kCancelBeforeThisKey,
"d"} /* expected_input_keys */);
}
} // namespace ROCKSDB_NAMESPACE
int main(int argc, char** argv) {
+1
View File
@@ -60,6 +60,7 @@ class CompactionOutputs {
}
const std::vector<Output>& GetOutputs() const { return outputs_; }
std::vector<Output>& GetMutableOutputs() { return outputs_; }
// Set new table builder for the current output
void NewBuilder(const TableBuilderOptions& tboptions);
+66 -6
View File
@@ -27,12 +27,68 @@
namespace ROCKSDB_NAMESPACE {
bool FindIntraL0Compaction(const std::vector<FileMetaData*>& level_files,
size_t min_files_to_compact,
uint64_t max_compact_bytes_per_del_file,
uint64_t max_compaction_bytes,
CompactionInputFiles* comp_inputs) {
TEST_SYNC_POINT("FindIntraL0Compaction");
#ifndef NDEBUG
static void AssertCleanCut(const InternalKeyComparator* icmp,
VersionStorageInfo* vstorage,
CompactionInputFiles* inputs, int level,
Logger* logger) {
const std::vector<FileMetaData*>& level_files = vstorage->LevelFiles(level);
if (inputs->files.empty() || level_files.empty()) {
return;
}
const Comparator* ucmp = icmp->user_comparator();
// Find first and last input file indices in level
int first_input_idx = -1;
int last_input_idx = -1;
for (size_t i = 0; i < level_files.size(); i++) {
if (level_files[i] == inputs->files.front()) {
first_input_idx = static_cast<int>(i);
}
if (level_files[i] == inputs->files.back()) {
last_input_idx = static_cast<int>(i);
}
}
// Check file before first input
if (first_input_idx > 0) {
const FileMetaData* prev_file = level_files[first_input_idx - 1];
const FileMetaData* first_file = inputs->files.front();
int cmp = sstableKeyCompare(ucmp, prev_file->largest, first_file->smallest);
if (cmp == 0) {
ROCKS_LOG_ERROR(logger,
"Clean cut violated: L%d unselected file %" PRIu64
" adjacent to first selected file %" PRIu64,
level, prev_file->fd.GetNumber(),
first_file->fd.GetNumber());
assert(false);
}
}
// Check file after last input
if (last_input_idx >= 0 &&
static_cast<size_t>(last_input_idx) < level_files.size() - 1) {
const FileMetaData* last_file = inputs->files.back();
const FileMetaData* next_file = level_files[last_input_idx + 1];
int cmp = sstableKeyCompare(ucmp, last_file->largest, next_file->smallest);
if (cmp == 0) {
ROCKS_LOG_ERROR(logger,
"Clean cut violated: L%d unselected file %" PRIu64
" adjacent to last selected file %" PRIu64,
level, next_file->fd.GetNumber(),
last_file->fd.GetNumber());
assert(false);
}
}
}
#endif // NDEBUG
bool PickCostBasedIntraL0Compaction(
const std::vector<FileMetaData*>& level_files, size_t min_files_to_compact,
uint64_t max_compact_bytes_per_del_file, uint64_t max_compaction_bytes,
CompactionInputFiles* comp_inputs) {
TEST_SYNC_POINT("PickCostBasedIntraL0Compaction");
size_t start = 0;
@@ -250,6 +306,10 @@ bool CompactionPicker::ExpandInputsToCleanCut(const std::string& /*cf_name*/,
// inputs. thus, inputs should be non-empty here
assert(!inputs->empty());
#ifndef NDEBUG
AssertCleanCut(icmp_, vstorage, inputs, level, ioptions_.logger);
#endif // NDEBUG
// If, after the expansion, there are files that are already under
// compaction, then we must drop/cancel this compaction.
if (AreFilesInCompaction(inputs->files)) {
+4 -5
View File
@@ -328,11 +328,10 @@ class NullCompactionPicker : public CompactionPicker {
// files. Cannot be nullptr.
//
// @return true iff compaction was found.
bool FindIntraL0Compaction(const std::vector<FileMetaData*>& level_files,
size_t min_files_to_compact,
uint64_t max_compact_bytes_per_del_file,
uint64_t max_compaction_bytes,
CompactionInputFiles* comp_inputs);
bool PickCostBasedIntraL0Compaction(
const std::vector<FileMetaData*>& level_files, size_t min_files_to_compact,
uint64_t max_compact_bytes_per_del_file, uint64_t max_compaction_bytes,
CompactionInputFiles* comp_inputs);
CompressionType GetCompressionType(const VersionStorageInfo* vstorage,
const MutableCFOptions& mutable_cf_options,
+390 -76
View File
@@ -9,6 +9,7 @@
#include "db/compaction/compaction_picker_fifo.h"
#include <algorithm>
#include <cinttypes>
#include <string>
#include <vector>
@@ -31,6 +32,29 @@ uint64_t GetTotalFilesSize(const std::vector<FileMetaData*>& files) {
}
return total_size;
}
// Compute effective data size and capacity limit for FIFO compaction.
// When max_data_files_size > 0 (blob-aware mode), the effective size includes
// both SST and blob file sizes, and the limit is max_data_files_size.
// Otherwise, only SST sizes are used with max_table_files_size as the limit.
void GetEffectiveSizeAndLimit(const CompactionOptionsFIFO& fifo_opts,
uint64_t total_sst_size, uint64_t total_blob_size,
uint64_t* effective_size,
uint64_t* effective_max) {
*effective_size = total_sst_size;
*effective_max = fifo_opts.max_table_files_size;
if (fifo_opts.max_data_files_size > 0) {
*effective_size += total_blob_size;
*effective_max = fifo_opts.max_data_files_size;
}
}
// Return the effective capacity limit for FIFO compaction.
// Convenience wrapper when only the limit is needed (e.g., PickTTLCompaction).
uint64_t GetEffectiveMax(const CompactionOptionsFIFO& fifo_opts) {
return fifo_opts.max_data_files_size > 0 ? fifo_opts.max_data_files_size
: fifo_opts.max_table_files_size;
}
} // anonymous namespace
bool FIFOCompactionPicker::NeedsCompaction(
@@ -78,10 +102,10 @@ Compaction* FIFOCompactionPicker::PickTTLCompaction(
for (auto ritr = level_files.rbegin(); ritr != level_files.rend(); ++ritr) {
FileMetaData* f = *ritr;
assert(f);
if (f->fd.table_reader && f->fd.table_reader->GetTableProperties()) {
TableReader* reader = f->fd.pinned_reader.Get();
if (reader != nullptr && reader->GetTableProperties() != nullptr) {
uint64_t newest_key_time = f->TryGetNewestKeyTime();
uint64_t creation_time =
f->fd.table_reader->GetTableProperties()->creation_time;
uint64_t creation_time = reader->GetTableProperties()->creation_time;
uint64_t est_newest_key_time = newest_key_time == kUnknownNewestKeyTime
? creation_time
: newest_key_time;
@@ -98,10 +122,43 @@ Compaction* FIFOCompactionPicker::PickTTLCompaction(
// Return a nullptr and proceed to size-based FIFO compaction if:
// 1. there are no files older than ttl OR
// 2. there are a few files older than ttl, but deleting them will not bring
// the total size to be less than max_table_files_size threshold.
if (inputs[0].files.empty() ||
total_size >
mutable_cf_options.compaction_options_fifo.max_table_files_size) {
// the total size to be less than the size threshold.
uint64_t effective_max =
GetEffectiveMax(mutable_cf_options.compaction_options_fifo);
// Estimate the effective remaining data after dropping TTL-expired SSTs.
// Each dropped SST also frees a proportional share of blob data.
//
// In multi-level FIFO (migration), we must use total SST across ALL levels
// as the reference, because total_blob covers all levels. Using only L0
// SST would inflate the blob estimate.
uint64_t effective_remaining = total_size;
if (mutable_cf_options.compaction_options_fifo.max_data_files_size > 0) {
uint64_t total_blob = vstorage->GetBlobStats().total_file_size;
// Compute total SST across all levels so the reference scope matches
// total_blob's scope (all levels).
uint64_t total_sst_all_levels = GetTotalFilesSize(level_files);
for (int level = 1; level < vstorage->num_levels(); ++level) {
total_sst_all_levels += GetTotalFilesSize(vstorage->LevelFiles(level));
}
// remaining_sst_all = total_sst_all - dropped_l0_sst
// total_size is the remaining L0 SST after removing expired files;
// original L0 SST minus remaining L0 SST = dropped.
uint64_t original_l0_sst = GetTotalFilesSize(level_files);
uint64_t dropped_sst = original_l0_sst - total_size;
uint64_t remaining_sst_all = total_sst_all_levels - dropped_sst;
// Proportional blob estimate: each SST byte "owns" a proportional
// share of blob bytes. Both reference sizes must come from the same
// scope (all levels) to avoid inflated estimates.
if (total_sst_all_levels > 0 && total_blob > 0) {
effective_remaining =
remaining_sst_all +
static_cast<uint64_t>(static_cast<double>(remaining_sst_all) /
total_sst_all_levels * total_blob);
} else {
effective_remaining = remaining_sst_all;
}
}
if (inputs[0].files.empty() || effective_remaining > effective_max) {
return nullptr;
}
@@ -109,8 +166,9 @@ Compaction* FIFOCompactionPicker::PickTTLCompaction(
assert(f);
uint64_t newest_key_time = f->TryGetNewestKeyTime();
uint64_t creation_time = 0;
if (f->fd.table_reader && f->fd.table_reader->GetTableProperties()) {
creation_time = f->fd.table_reader->GetTableProperties()->creation_time;
TableReader* reader = f->fd.pinned_reader.Get();
if (reader != nullptr && reader->GetTableProperties() != nullptr) {
creation_time = reader->GetTableProperties()->creation_time;
}
uint64_t est_newest_key_time = newest_key_time == kUnknownNewestKeyTime
? creation_time
@@ -151,7 +209,9 @@ Compaction* FIFOCompactionPicker::PickSizeCompaction(
const std::string& cf_name, const MutableCFOptions& mutable_cf_options,
const MutableDBOptions& mutable_db_options, VersionStorageInfo* vstorage,
LogBuffer* log_buffer) {
// compute the total size and identify the last non-empty level
const auto& fifo_opts = mutable_cf_options.compaction_options_fifo;
// compute the total SST size and identify the last non-empty level
int last_level = 0;
uint64_t total_size = 0;
for (int level = 0; level < vstorage->num_levels(); ++level) {
@@ -164,52 +224,13 @@ Compaction* FIFOCompactionPicker::PickSizeCompaction(
const std::vector<FileMetaData*>& last_level_files =
vstorage->LevelFiles(last_level);
if (last_level == 0 &&
total_size <=
mutable_cf_options.compaction_options_fifo.max_table_files_size) {
// total size not exceeded, try to find intra level 0 compaction if enabled
const std::vector<FileMetaData*>& level0_files = vstorage->LevelFiles(0);
if (mutable_cf_options.compaction_options_fifo.allow_compaction &&
level0_files.size() > 0) {
CompactionInputFiles comp_inputs;
// try to prevent same files from being compacted multiple times, which
// could produce large files that may never TTL-expire. Achieve this by
// disallowing compactions with files larger than memtable (inflate its
// size by 10% to account for uncompressed L0 files that may have size
// slightly greater than memtable size limit).
size_t max_compact_bytes_per_del_file =
static_cast<size_t>(MultiplyCheckOverflow(
static_cast<uint64_t>(mutable_cf_options.write_buffer_size),
1.1));
if (FindIntraL0Compaction(
level0_files,
mutable_cf_options
.level0_file_num_compaction_trigger /* min_files_to_compact */
,
max_compact_bytes_per_del_file,
mutable_cf_options.max_compaction_bytes, &comp_inputs)) {
Compaction* c = new Compaction(
vstorage, ioptions_, mutable_cf_options, mutable_db_options,
{comp_inputs}, 0, 16 * 1024 * 1024 /* output file size limit */,
0 /* max compaction bytes, not applicable */,
0 /* output path ID */, mutable_cf_options.compression,
mutable_cf_options.compression_opts, Temperature::kUnknown,
0 /* max_subcompactions */, {},
/* earliest_snapshot */ std::nullopt,
/* snapshot_checker */ nullptr,
CompactionReason::kFIFOReduceNumFiles,
/* trim_ts */ "", vstorage->CompactionScore(0),
/* l0_files_might_overlap */ true);
return c;
}
}
// Compute effective size and limit for comparison.
uint64_t effective_size, effective_max;
GetEffectiveSizeAndLimit(fifo_opts, total_size,
vstorage->GetBlobStats().total_file_size,
&effective_size, &effective_max);
ROCKS_LOG_BUFFER(
log_buffer,
"[%s] FIFO compaction: nothing to do. Total size %" PRIu64
", max size %" PRIu64 "\n",
cf_name.c_str(), total_size,
mutable_cf_options.compaction_options_fifo.max_table_files_size);
if (last_level == 0 && effective_size <= effective_max) {
return nullptr;
}
@@ -227,11 +248,29 @@ Compaction* FIFOCompactionPicker::PickSizeCompaction(
inputs[0].level = last_level;
if (last_level == 0) {
// When using blob-aware sizing, use proportional estimation (same
// principle as EstimateTotalDataForSST): each SST "owns"
// effective_size / num_files of total data. This is an approximation
// — individual SSTs may reference different amounts of blob data,
// but uniform distribution is a reasonable estimate for FIFO dropping.
uint64_t remaining_size = effective_size;
const uint64_t num_files = last_level_files.size();
// Proportional estimate of data per file (SST + blob).
// Use max(1) to prevent stalling when effective_size < num_files.
const uint64_t data_per_file =
(fifo_opts.max_data_files_size > 0 && num_files > 0)
? std::max(effective_size / num_files, uint64_t{1})
: 0;
// In L0, right-most files are the oldest files.
for (auto ritr = last_level_files.rbegin(); ritr != last_level_files.rend();
++ritr) {
auto f = *ritr;
total_size -= f->fd.file_size;
if (fifo_opts.max_data_files_size > 0) {
remaining_size -= std::min(remaining_size, data_per_file);
} else {
remaining_size -= std::min(remaining_size, f->fd.file_size);
}
inputs[0].files.push_back(f);
char tmp_fsize[16];
AppendHumanBytes(f->fd.GetFileSize(), tmp_fsize, sizeof(tmp_fsize));
@@ -239,13 +278,11 @@ Compaction* FIFOCompactionPicker::PickSizeCompaction(
"[%s] FIFO compaction: picking file %" PRIu64
" with size %s for deletion",
cf_name.c_str(), f->fd.GetNumber(), tmp_fsize);
if (total_size <=
mutable_cf_options.compaction_options_fifo.max_table_files_size) {
if (remaining_size <= effective_max) {
break;
}
}
} else if (total_size >
mutable_cf_options.compaction_options_fifo.max_table_files_size) {
} else if (effective_size > effective_max) {
// If the last level is non-L0, we actually don't know which file is
// logically the oldest since the file creation time only represents
// when this file was compacted to this level, which is independent
@@ -255,34 +292,36 @@ Compaction* FIFOCompactionPicker::PickSizeCompaction(
// file with the smallest key will be deleted first. This design decision
// better serves a major type of FIFO use cases where smaller keys are
// associated with older data.
const uint64_t num_files = last_level_files.size();
// Proportional estimate of data per file (SST + blob), same as L0 path.
const uint64_t data_per_file =
(fifo_opts.max_data_files_size > 0 && num_files > 0)
? std::max(effective_size / num_files, uint64_t{1})
: 0;
for (const auto& f : last_level_files) {
if (f->being_compacted) {
continue;
}
total_size -= f->fd.file_size;
if (fifo_opts.max_data_files_size > 0) {
effective_size -= std::min(effective_size, data_per_file);
} else {
effective_size -= std::min(effective_size, f->fd.file_size);
}
inputs[0].files.push_back(f);
char tmp_fsize[16];
AppendHumanBytes(f->fd.GetFileSize(), tmp_fsize, sizeof(tmp_fsize));
ROCKS_LOG_BUFFER(
log_buffer,
"[%s] FIFO compaction: picking file %" PRIu64
" with size %s for deletion under total size %" PRIu64
" vs max table files size %" PRIu64,
cf_name.c_str(), f->fd.GetNumber(), tmp_fsize, total_size,
mutable_cf_options.compaction_options_fifo.max_table_files_size);
ROCKS_LOG_BUFFER(log_buffer,
"[%s] FIFO compaction: picking file %" PRIu64
" with size %s for deletion under total size %" PRIu64
" vs max size %" PRIu64,
cf_name.c_str(), f->fd.GetNumber(), tmp_fsize,
effective_size, effective_max);
if (total_size <=
mutable_cf_options.compaction_options_fifo.max_table_files_size) {
if (effective_size <= effective_max) {
break;
}
}
} else {
ROCKS_LOG_BUFFER(
log_buffer,
"[%s] FIFO compaction: nothing to do. Total size %" PRIu64
", max size %" PRIu64 "\n",
cf_name.c_str(), total_size,
mutable_cf_options.compaction_options_fifo.max_table_files_size);
return nullptr;
}
@@ -419,6 +458,269 @@ Compaction* FIFOCompactionPicker::PickTemperatureChangeCompaction(
return c;
}
Compaction* FIFOCompactionPicker::PickIntraL0Compaction(
const std::string& cf_name, const MutableCFOptions& mutable_cf_options,
const MutableDBOptions& mutable_db_options, VersionStorageInfo* vstorage,
LogBuffer* log_buffer) {
const auto& fifo_opts = mutable_cf_options.compaction_options_fifo;
if (!fifo_opts.allow_compaction) {
return nullptr;
}
const std::vector<FileMetaData*>& level0_files = vstorage->LevelFiles(0);
if (level0_files.empty()) {
return nullptr;
}
if (fifo_opts.use_kv_ratio_compaction) {
if (fifo_opts.max_data_files_size == 0) {
ROCKS_LOG_BUFFER(
log_buffer,
"[%s] FIFO kv-ratio compaction: skipping — "
"max_data_files_size is 0, cannot compute target file size. ",
cf_name.c_str());
} else if (fifo_opts.max_data_files_size < fifo_opts.max_table_files_size) {
ROCKS_LOG_BUFFER(log_buffer,
"[%s] FIFO kv-ratio compaction: skipping — "
"max_data_files_size (%" PRIu64
") < max_table_files_size "
"(%" PRIu64 ").",
cf_name.c_str(), fifo_opts.max_data_files_size,
fifo_opts.max_table_files_size);
} else {
return PickRatioBasedIntraL0Compaction(cf_name, mutable_cf_options,
mutable_db_options, vstorage,
log_buffer);
}
ROCKS_LOG_BUFFER(
log_buffer, "[%s] FIFO: falling back to cost-based intra-L0 compaction",
cf_name.c_str());
}
// Old intra-L0 path: merge small files using PickCostBasedIntraL0Compaction.
// Minimum files to compact follows level0_file_num_compaction_trigger.
// Try to prevent same files from being compacted multiple times, which
// could produce large files that may never TTL-expire. Achieve this by
// disallowing compactions with files larger than memtable (inflate its
// size by 10% to account for uncompressed L0 files that may have size
// slightly greater than memtable size limit).
CompactionInputFiles comp_inputs;
size_t max_compact_bytes_per_del_file =
static_cast<size_t>(MultiplyCheckOverflow(
static_cast<uint64_t>(mutable_cf_options.write_buffer_size), 1.1));
if (PickCostBasedIntraL0Compaction(
level0_files,
mutable_cf_options
.level0_file_num_compaction_trigger /* min_files_to_compact */,
max_compact_bytes_per_del_file,
mutable_cf_options.max_compaction_bytes, &comp_inputs)) {
Compaction* c = new Compaction(
vstorage, ioptions_, mutable_cf_options, mutable_db_options,
{comp_inputs}, 0, 16 * 1024 * 1024 /* output file size limit */,
0 /* max compaction bytes, not applicable */, 0 /* output path ID */,
mutable_cf_options.compression, mutable_cf_options.compression_opts,
Temperature::kUnknown, 0 /* max_subcompactions */, {},
/* earliest_snapshot */ std::nullopt,
/* snapshot_checker */ nullptr, CompactionReason::kFIFOReduceNumFiles,
/* trim_ts */ "", vstorage->CompactionScore(0),
/* l0_files_might_overlap */ true);
return c;
}
return nullptr;
}
Compaction* FIFOCompactionPicker::PickRatioBasedIntraL0Compaction(
const std::string& cf_name, const MutableCFOptions& mutable_cf_options,
const MutableDBOptions& mutable_db_options, VersionStorageInfo* vstorage,
LogBuffer* log_buffer) {
const auto& fifo_opts = mutable_cf_options.compaction_options_fifo;
assert(fifo_opts.use_kv_ratio_compaction);
assert(fifo_opts.max_data_files_size > 0);
// During migration from level/universal compaction to FIFO, non-L0 levels
// may still contain files. The ratio-based algorithm only operates on L0,
// so skip it until PickSizeCompaction has drained all non-L0 levels.
// Once levels collapse to L0-only, this algorithm will kick in.
for (int level = 1; level < vstorage->num_levels(); ++level) {
if (!vstorage->LevelFiles(level).empty()) {
ROCKS_LOG_BUFFER(log_buffer,
"[%s] FIFO kv-ratio compaction: skipping — non-L0 "
"level %d still has %" ROCKSDB_PRIszt
" files (migration in progress)",
cf_name.c_str(), level,
vstorage->LevelFiles(level).size());
return nullptr;
}
}
if (!level0_compactions_in_progress_.empty()) {
return nullptr;
}
const std::vector<FileMetaData*>& level0_files = vstorage->LevelFiles(0);
if (mutable_cf_options.level0_file_num_compaction_trigger <= 1) {
// trigger <= 0 is invalid; trigger == 1 means compact after every flush,
// which doesn't make sense for tiered merging (the tier boundary loop
// divides by trigger, so trigger == 1 would cause an infinite loop).
return nullptr;
}
const size_t trigger = static_cast<size_t>(
mutable_cf_options.level0_file_num_compaction_trigger);
if (level0_files.size() < trigger) {
return nullptr;
}
// Determine the target compacted file size.
//
// When max_compaction_bytes > 0 (explicitly set by user), use it directly
// as the target. This allows users to override the auto-calculated value.
//
// When max_compaction_bytes == 0 (default), auto-calculate from the data
// capacity and observed SST/blob ratio:
// target = max_data_files_size * sst_ratio / trigger
//
// This is recomputed on every PickCompaction call. The computation is
// trivial (sum file sizes + arithmetic) and PickCompaction is only called
// once per flush or compaction completion, so no caching is needed.
uint64_t target = 0;
if (mutable_cf_options.max_compaction_bytes > 0) {
// User explicitly set max_compaction_bytes — use it as target
target = mutable_cf_options.max_compaction_bytes;
} else {
// Auto-calculate from capacity and observed SST/blob ratio
uint64_t total_sst = GetTotalFilesSize(level0_files);
uint64_t total_blob = vstorage->GetBlobStats().total_file_size;
uint64_t total_data = total_sst + total_blob;
if (total_data == 0 || total_sst == 0) {
return nullptr;
}
// Compute sst_ratio (inverse of EstimateTotalDataForSST's proportion):
// when no blob files exist, sst_ratio is 1.0 and the target becomes
// max_data_files_size / trigger, which is large. The algorithm will
// naturally not find small enough files to compact.
double sst_ratio =
(total_blob > 0) ? static_cast<double>(total_sst) / total_data : 1.0;
uint64_t total_sst_at_cap =
static_cast<uint64_t>(fifo_opts.max_data_files_size * sst_ratio);
target = total_sst_at_cap / trigger;
ROCKS_LOG_BUFFER(log_buffer,
"[%s] FIFO ratio-based compaction: sst_ratio=%.4f, "
"target_file_size=%" PRIu64,
cf_name.c_str(), sst_ratio, target);
}
if (target == 0) {
return nullptr;
}
// Tiered size-based file selection.
//
// Tier boundaries form a geometric sequence descending from target:
// ..., target/trigger^2, target/trigger, target
// For each boundary (smallest first), find contiguous L0 files with
// size < boundary. If their accumulated bytes >= boundary, merge them.
// The output (~boundary bytes) advances to the next tier. Files that
// reach target are "graduated" and never compacted again.
//
// Trade-off: write amplification vs L0 file count.
//
// Write amp: O(log(target/flush) / log(trigger)) per byte, instead of
// O(target / (trigger * flush)) from flat merging. Each byte is
// rewritten once per tier crossing.
//
// L0 file count: trigger + k * (trigger - 1) at steady state, where
// k = ceil(log(target/flush) / log(trigger)). This is higher than
// the original trigger target because intermediate tier files
// accumulate while waiting for the next tier merge. The trade-off
// is explicit: more L0 files in exchange for logarithmic (instead
// of linear) write amplification.
// Build tier boundaries from smallest to largest.
// Stop at 10KB minimum — SST files of most workloads are larger than
// this, so lower boundaries would only waste CPU scanning L0 files.
// Files smaller than the lowest boundary simply merge at that boundary.
static constexpr uint64_t kMinTierBoundary = 10 * 1024; // 10KB
std::vector<uint64_t> boundaries;
for (uint64_t b = target; b >= kMinTierBoundary; b /= trigger) {
boundaries.push_back(b);
}
if (boundaries.empty()) {
// target itself is below kMinTierBoundary — use target as the
// sole boundary so we can still compact at the target size.
boundaries.push_back(target);
}
std::reverse(boundaries.begin(), boundaries.end());
// For each tier boundary (smallest first), scan L0 for mergeable batches.
// L0 files are stored newest-first; oldest is at the end.
for (const uint64_t boundary : boundaries) {
for (size_t scan = level0_files.size(); scan > 0;) {
// Skip files >= boundary (they belong to higher tiers) or in-progress
if (level0_files[scan - 1]->fd.file_size >= boundary ||
level0_files[scan - 1]->being_compacted) {
--scan;
continue;
}
// Found a file < boundary — collect contiguous batch
std::vector<FileMetaData*> batch;
uint64_t accumulated = 0;
size_t pos = scan;
while (pos > 0 && level0_files[pos - 1]->fd.file_size < boundary &&
!level0_files[pos - 1]->being_compacted) {
// Don't let output exceed 2x boundary (prevent tier-skipping)
if (accumulated >= boundary &&
accumulated + level0_files[pos - 1]->fd.file_size > boundary * 2) {
break;
}
batch.push_back(level0_files[pos - 1]);
accumulated += level0_files[pos - 1]->fd.file_size;
--pos;
}
// Viable: >= 2 files and accumulated >= boundary
if (batch.size() >= 2 && accumulated >= boundary) {
CompactionInputFiles comp_inputs;
comp_inputs.level = 0;
comp_inputs.files = std::move(batch);
ROCKS_LOG_BUFFER(
log_buffer,
"[%s] FIFO kv-ratio compaction: picking %" ROCKSDB_PRIszt
" files (%" PRIu64 " bytes) at tier boundary %" PRIu64
" for intra-L0 compaction, target=%" PRIu64,
cf_name.c_str(), comp_inputs.files.size(), accumulated, boundary,
target);
Compaction* c = new Compaction(
vstorage, ioptions_, mutable_cf_options, mutable_db_options,
{comp_inputs}, 0, boundary /* output file size limit */,
0 /* max compaction bytes, not applicable */,
0 /* output path ID */, mutable_cf_options.compression,
mutable_cf_options.compression_opts, Temperature::kUnknown,
0 /* max_subcompactions */, {},
/* earliest_snapshot */ std::nullopt,
/* snapshot_checker */ nullptr,
CompactionReason::kFIFOReduceNumFiles,
/* trim_ts */ "", vstorage->CompactionScore(0),
/* l0_files_might_overlap */ true);
return c;
}
// This batch wasn't enough — advance past it
scan = pos;
}
}
return nullptr;
}
// The full_history_ts_low parameter is used to control bottommost file marking
// for compaction when user-defined timestamps (UDT) are enabled.
@@ -441,10 +743,22 @@ Compaction* FIFOCompactionPicker::PickCompaction(
c = PickSizeCompaction(cf_name, mutable_cf_options, mutable_db_options,
vstorage, log_buffer);
}
// Intra-L0 compaction merges small files to reduce file count.
// It runs after size-based dropping: if PickSizeCompaction dropped files,
// it returned non-null and we skip this. Otherwise, we try to reduce
// L0 file count by merging small files together.
if (c == nullptr) {
c = PickIntraL0Compaction(cf_name, mutable_cf_options, mutable_db_options,
vstorage, log_buffer);
}
if (c == nullptr) {
c = PickTemperatureChangeCompaction(
cf_name, mutable_cf_options, mutable_db_options, vstorage, log_buffer);
}
if (c == nullptr) {
ROCKS_LOG_BUFFER(log_buffer, "[%s] FIFO compaction: no compaction picked",
cf_name.c_str());
}
RegisterCompaction(c);
return c;
}
+22
View File
@@ -55,6 +55,28 @@ class FIFOCompactionPicker : public CompactionPicker {
VersionStorageInfo* version,
LogBuffer* log_buffer);
// Intra-L0 compaction: merges small L0 files to reduce file count.
// Dispatches between two strategies based on configuration:
// - use_kv_ratio_compaction = true: PickRatioBasedIntraL0Compaction
// (BlobDB-optimized)
// - use_kv_ratio_compaction = false: PickCostBasedIntraL0Compaction
// (original)
// Only active when allow_compaction = true.
Compaction* PickIntraL0Compaction(const std::string& cf_name,
const MutableCFOptions& mutable_cf_options,
const MutableDBOptions& mutable_db_options,
VersionStorageInfo* vstorage,
LogBuffer* log_buffer);
// Capacity-derived intra-L0 compaction for BlobDB workloads.
// Uses the observed SST/blob ratio to compute a target file size,
// producing uniform files for predictable FIFO trimming.
// Called from PickIntraL0Compaction when use_kv_ratio_compaction = true.
Compaction* PickRatioBasedIntraL0Compaction(
const std::string& cf_name, const MutableCFOptions& mutable_cf_options,
const MutableDBOptions& mutable_db_options, VersionStorageInfo* vstorage,
LogBuffer* log_buffer);
// Will pick one file to compact at a time, starting from the oldest file.
Compaction* PickTemperatureChangeCompaction(
const std::string& cf_name, const MutableCFOptions& mutable_cf_options,
+24 -5
View File
@@ -36,6 +36,9 @@ bool LevelCompactionPicker::NeedsCompaction(
if (!vstorage->FilesMarkedForForcedBlobGC().empty()) {
return true;
}
if (!vstorage->ReadTriggeredCompactionFiles().empty()) {
return true;
}
for (int i = 0; i <= vstorage->MaxInputLevel(); i++) {
if (vstorage->CompactionScore(i) >= 1) {
return true;
@@ -325,6 +328,14 @@ void LevelCompactionBuilder::SetupInitialFiles() {
compaction_reason_ = CompactionReason::kForcedBlobGC;
return;
}
// Read-triggered compaction
PickFileToCompact(vstorage_->ReadTriggeredCompactionFiles(),
CompactToNextLevel::kYes);
if (!start_level_inputs_.empty()) {
compaction_reason_ = CompactionReason::kReadTriggered;
return;
}
}
bool LevelCompactionBuilder::SetupOtherL0FilesIfNeeded() {
@@ -404,7 +415,15 @@ void LevelCompactionBuilder::SetupOtherFilesWithRoundRobinExpansion() {
tmp_start_level_inputs = start_level_inputs_;
// TODO (zichen): Future parallel round-robin may also need to update this
// Constraint 1b (only expand till the end)
for (size_t i = start_index + 1; i < level_files.size(); i++) {
size_t next_index = start_index + 1;
const FileMetaData* last_file = start_level_inputs_.files.back();
for (size_t j = next_index; j < level_files.size(); j++) {
if (level_files[j] == last_file) {
next_index = j + 1;
break;
}
}
for (size_t i = next_index; i < level_files.size(); i++) {
auto* f = level_files[i];
if (f->being_compacted) {
// Constraint 1a
@@ -914,10 +933,10 @@ bool LevelCompactionBuilder::PickIntraL0Compaction() {
// resort to L0->L0 compaction yet.
return false;
}
return FindIntraL0Compaction(level_files, kMinFilesForIntraL0Compaction,
std::numeric_limits<uint64_t>::max(),
mutable_cf_options_.max_compaction_bytes,
&start_level_inputs_);
return PickCostBasedIntraL0Compaction(
level_files, kMinFilesForIntraL0Compaction,
std::numeric_limits<uint64_t>::max(),
mutable_cf_options_.max_compaction_bytes, &start_level_inputs_);
}
bool LevelCompactionBuilder::PickSizeBasedIntraL0Compaction() {
File diff suppressed because it is too large Load Diff
+178 -101
View File
@@ -275,6 +275,21 @@ class UniversalCompactionBuilder {
return c;
}
Compaction* MaybePickReadTriggeredCompaction(
Compaction* const prev_picked_c) {
if (prev_picked_c != nullptr ||
vstorage_->ReadTriggeredCompactionFiles().empty()) {
return prev_picked_c;
}
Compaction* c = PickReadTriggeredCompaction();
if (c != nullptr) {
ROCKS_LOG_BUFFER(log_buffer_,
"[%s] Universal: picked for read triggered compaction\n",
cf_name_.c_str());
}
return c;
}
// Pick Universal compaction to limit read amplification
Compaction* PickCompactionToReduceSortedRuns(
unsigned int ratio, unsigned int max_number_of_files_to_compact);
@@ -292,6 +307,14 @@ class UniversalCompactionBuilder {
Compaction* PickDeleteTriggeredCompaction();
Compaction* PickReadTriggeredCompaction();
// Given already-selected start_level_inputs, find the first non-empty output
// level, compute overlapping inputs, and create a Compaction. Can produce
// intra-L0 compaction if there is only level 0.
Compaction* BuildCompactionToNextLevel(
CompactionInputFiles& start_level_inputs, CompactionReason reason);
// Returns true if this given file (that is marked be compaction) should be
// skipped from being picked for now. We do this to best use standalone range
// tombstone files.
@@ -594,6 +617,9 @@ bool UniversalCompactionPicker::NeedsCompaction(
if (!vstorage->FilesMarkedForCompaction().empty()) {
return true;
}
if (!vstorage->ReadTriggeredCompactionFiles().empty()) {
return true;
}
return false;
}
@@ -759,6 +785,7 @@ Compaction* UniversalCompactionBuilder::PickCompaction() {
if (sorted_runs_.size() == 0 ||
(vstorage_->FilesMarkedForPeriodicCompaction().empty() &&
vstorage_->FilesMarkedForCompaction().empty() &&
vstorage_->ReadTriggeredCompactionFiles().empty() &&
sorted_runs_.size() < (unsigned int)file_num_compaction_trigger)) {
ROCKS_LOG_BUFFER(log_buffer_, "[%s] Universal: nothing to do\n",
cf_name_.c_str());
@@ -782,6 +809,7 @@ Compaction* UniversalCompactionBuilder::PickCompaction() {
c = MaybePickCompactionToReduceSortedRuns(c, file_num_compaction_trigger,
ratio);
c = MaybePickDeleteTriggeredCompaction(c);
c = MaybePickReadTriggeredCompaction(c);
if (c == nullptr) {
TEST_SYNC_POINT_CALLBACK(
@@ -1461,9 +1489,6 @@ Compaction* UniversalCompactionBuilder::PickIncrementalForReduceSizeAmp(
// CompactOnDeleteCollector due to the presence of tombstones.
Compaction* UniversalCompactionBuilder::PickDeleteTriggeredCompaction() {
CompactionInputFiles start_level_inputs;
int output_level;
std::vector<CompactionInputFiles> inputs;
std::vector<FileMetaData*> grandparents;
if (vstorage_->num_levels() == 1) {
// This is single level universal. Since we're basically trying to reclaim
@@ -1474,7 +1499,6 @@ Compaction* UniversalCompactionBuilder::PickDeleteTriggeredCompaction() {
start_level_inputs.level = 0;
start_level_inputs.files.clear();
output_level = 0;
// Find the first file marked for compaction. Ignore the last file
for (size_t loop = 0; loop + 1 < sorted_runs_.size(); loop++) {
SortedRun* sr = &sorted_runs_[loop];
@@ -1503,13 +1527,9 @@ Compaction* UniversalCompactionBuilder::PickDeleteTriggeredCompaction() {
FileMetaData* f = vstorage_->LevelFiles(0)[loop];
start_level_inputs.files.push_back(f);
}
if (start_level_inputs.size() <= 1) {
// If only the last file in L0 is marked for compaction, ignore it
return nullptr;
}
inputs.push_back(start_level_inputs);
} else {
int start_level;
int output_level;
// For multi-level universal, the strategy is to make this look more like
// leveled. We pick one of the files marked for compaction and compact with
@@ -1522,100 +1542,10 @@ Compaction* UniversalCompactionBuilder::PickDeleteTriggeredCompaction() {
if (start_level_inputs.empty()) {
return nullptr;
}
int max_output_level = vstorage_->MaxOutputLevel(allow_ingest_behind_);
// Pick the first non-empty level after the start_level
for (output_level = start_level + 1; output_level <= max_output_level;
output_level++) {
if (vstorage_->NumLevelFiles(output_level) != 0) {
break;
}
}
// If all higher levels are empty, pick the highest level as output level
if (output_level > max_output_level) {
if (start_level == 0) {
output_level = max_output_level;
} else {
// If start level is non-zero and all higher levels are empty, this
// compaction will translate into a trivial move. Since the idea is
// to reclaim space and trivial move doesn't help with that, we
// skip compaction in this case and return nullptr
return nullptr;
}
}
assert(output_level <= max_output_level);
if (!MeetsOutputLevelRequirements(output_level)) {
return nullptr;
}
if (output_level != 0) {
// For standalone range deletion, we don't want to compact it with newer
// L0 files that it doesn't cover.
const FileMetaData* starting_l0_file =
(start_level == 0 && start_level_inputs.size() == 1 &&
start_level_inputs.files[0]->FileIsStandAloneRangeTombstone())
? start_level_inputs.files[0]
: nullptr;
if (start_level == 0) {
if (!picker_->GetOverlappingL0Files(vstorage_, &start_level_inputs,
output_level, nullptr,
starting_l0_file)) {
return nullptr;
}
}
CompactionInputFiles output_level_inputs;
int parent_index = -1;
output_level_inputs.level = output_level;
if (!picker_->SetupOtherInputs(cf_name_, mutable_cf_options_, vstorage_,
&start_level_inputs, &output_level_inputs,
&parent_index, -1, false,
starting_l0_file)) {
return nullptr;
}
inputs.push_back(start_level_inputs);
if (!output_level_inputs.empty()) {
inputs.push_back(output_level_inputs);
}
if (picker_->FilesRangeOverlapWithCompaction(
inputs, output_level,
Compaction::EvaluateProximalLevel(vstorage_, mutable_cf_options_,
ioptions_, start_level,
output_level))) {
return nullptr;
}
picker_->GetGrandparents(vstorage_, start_level_inputs,
output_level_inputs, &grandparents);
} else {
inputs.push_back(start_level_inputs);
}
}
uint64_t estimated_total_size = 0;
// Use size of the output level as estimated file size
for (FileMetaData* f : vstorage_->LevelFiles(output_level)) {
estimated_total_size += f->fd.GetFileSize();
}
uint32_t path_id =
GetPathId(ioptions_, mutable_cf_options_, estimated_total_size);
return new Compaction(
vstorage_, ioptions_, mutable_cf_options_, mutable_db_options_,
std::move(inputs), output_level,
MaxFileSizeForLevel(mutable_cf_options_, output_level,
kCompactionStyleUniversal),
/* max_grandparent_overlap_bytes */ GetMaxOverlappingBytes(), path_id,
GetCompressionType(vstorage_, mutable_cf_options_, output_level, 1),
GetCompressionOptions(mutable_cf_options_, vstorage_, output_level),
Temperature::kUnknown,
/* max_subcompactions */ 0, grandparents, earliest_snapshot_,
snapshot_checker_, CompactionReason::kFilesMarkedForCompaction,
/* trim_ts */ "", score_,
/* l0_files_might_overlap */ true);
return BuildCompactionToNextLevel(
start_level_inputs, CompactionReason::kFilesMarkedForCompaction);
}
Compaction* UniversalCompactionBuilder::PickCompactionToOldest(
@@ -1782,6 +1712,153 @@ Compaction* UniversalCompactionBuilder::PickPeriodicCompaction() {
return c;
}
Compaction* UniversalCompactionBuilder::PickReadTriggeredCompaction() {
ROCKS_LOG_BUFFER(log_buffer_, "[%s] Universal: Read Triggered Compaction",
cf_name_.c_str());
if (vstorage_->ReadTriggeredCompactionFiles().empty()) {
return nullptr;
}
CompactionInputFiles start_level_inputs;
int start_level = -1;
for (const auto& level_file : vstorage_->ReadTriggeredCompactionFiles()) {
assert(!level_file.second->being_compacted);
start_level = level_file.first;
if (start_level == 0 &&
!picker_->level0_compactions_in_progress()->empty()) {
continue;
}
start_level_inputs.files = {level_file.second};
start_level_inputs.level = start_level;
if (picker_->ExpandInputsToCleanCut(cf_name_, vstorage_,
&start_level_inputs)) {
break;
}
start_level_inputs.files.clear();
}
if (start_level_inputs.empty()) {
return nullptr;
}
// For intra L0 compactions, pick up all overlapping files
if (vstorage_->num_levels() == 1 && start_level_inputs.level == 0) {
if (!picker_->GetOverlappingL0Files(vstorage_, &start_level_inputs,
/*output_level=*/0, nullptr, nullptr)) {
return nullptr;
}
}
return BuildCompactionToNextLevel(start_level_inputs,
CompactionReason::kReadTriggered);
}
Compaction* UniversalCompactionBuilder::BuildCompactionToNextLevel(
CompactionInputFiles& start_level_inputs, CompactionReason reason) {
int start_level = start_level_inputs.level;
int max_output_level = vstorage_->MaxOutputLevel(allow_ingest_behind_);
// Pick the first non-empty level after start_level as output.
int output_level;
for (output_level = start_level + 1; output_level <= max_output_level;
output_level++) {
if (vstorage_->NumLevelFiles(output_level) != 0) {
break;
}
}
// If all higher levels are empty, pick the highest level as output level
// for L0 start. For non-L0 start, a trivial move doesn't help reclaim
// space, so skip.
if (output_level > max_output_level) {
if (start_level == 0) {
output_level = max_output_level;
} else {
return nullptr;
}
}
if (start_level_inputs.size() <= 1 && output_level == 0) {
// If only the last file in L0 is marked for compaction, ignore it
return nullptr;
}
std::vector<CompactionInputFiles> inputs;
std::vector<FileMetaData*> grandparents;
if (output_level != 0) {
if (!MeetsOutputLevelRequirements(output_level)) {
return nullptr;
}
// For standalone range deletion, we don't want to compact it with newer
// L0 files that it doesn't cover.
const FileMetaData* starting_l0_file =
(start_level == 0 && start_level_inputs.size() == 1 &&
start_level_inputs.files[0]->FileIsStandAloneRangeTombstone())
? start_level_inputs.files[0]
: nullptr;
if (start_level == 0) {
if (!picker_->GetOverlappingL0Files(vstorage_, &start_level_inputs,
output_level, nullptr,
starting_l0_file)) {
return nullptr;
}
}
CompactionInputFiles output_level_inputs;
int parent_index = -1;
output_level_inputs.level = output_level;
if (!picker_->SetupOtherInputs(
cf_name_, mutable_cf_options_, vstorage_, &start_level_inputs,
&output_level_inputs, &parent_index, -1, false, starting_l0_file)) {
return nullptr;
}
inputs.push_back(start_level_inputs);
if (!output_level_inputs.empty()) {
inputs.push_back(output_level_inputs);
}
if (picker_->FilesRangeOverlapWithCompaction(
inputs, output_level,
Compaction::EvaluateProximalLevel(vstorage_, mutable_cf_options_,
ioptions_, start_level,
output_level))) {
return nullptr;
}
picker_->GetGrandparents(vstorage_, start_level_inputs, output_level_inputs,
&grandparents);
} else {
inputs.push_back(start_level_inputs);
}
uint64_t estimated_total_size = 0;
for (FileMetaData* f : vstorage_->LevelFiles(output_level)) {
estimated_total_size += f->fd.GetFileSize();
}
uint32_t path_id =
GetPathId(ioptions_, mutable_cf_options_, estimated_total_size);
return new Compaction(
vstorage_, ioptions_, mutable_cf_options_, mutable_db_options_,
std::move(inputs), output_level,
MaxFileSizeForLevel(mutable_cf_options_, output_level,
kCompactionStyleUniversal),
/* max_grandparent_overlap_bytes */ GetMaxOverlappingBytes(), path_id,
GetCompressionType(vstorage_, mutable_cf_options_, output_level, 1),
GetCompressionOptions(mutable_cf_options_, vstorage_, output_level),
Temperature::kUnknown,
/* max_subcompactions */ 0, grandparents, earliest_snapshot_,
snapshot_checker_, reason,
/* trim_ts */ "", score_,
/* l0_files_might_overlap */ true);
}
uint64_t UniversalCompactionBuilder::GetMaxOverlappingBytes() const {
if (!mutable_cf_options_.compaction_options_universal.incremental) {
return std::numeric_limits<uint64_t>::max();
+500 -40
View File
@@ -195,9 +195,11 @@ class MyTestCompactionService : public CompactionService {
std::vector<std::shared_ptr<EventListener>> listeners_;
std::vector<std::shared_ptr<TablePropertiesCollectorFactory>>
table_properties_collector_factories_;
std::atomic_bool canceled_{false};
std::atomic<CompactionServiceJobStatus> final_updated_status_{
CompactionServiceJobStatus::kUseLocal};
protected:
std::atomic_bool canceled_{false};
};
class CompactionServiceTest : public DBTestBase {
@@ -407,6 +409,34 @@ TEST_F(CompactionServiceTest, BasicCompactions) {
SyncPoint::GetInstance()->DisableProcessing();
}
TEST_F(CompactionServiceTest, SkipWALRecoveryInOpenAndCompact) {
// Verify that OpenAndCompact skips WAL recovery when opening the secondary
// instance. WAL replay is unnecessary for remote compaction since it only
// needs the LSM state from MANIFEST.
Options options = CurrentOptions();
ReopenWithCompactionService(&options);
// Track whether FindAndRecoverLogFiles is called during compaction
std::atomic_bool wal_recovery_called{false};
SyncPoint::GetInstance()->SetCallBack(
"DBImplSecondary::FindAndRecoverLogFiles:Begin",
[&](void* /* arg */) { wal_recovery_called.store(true); });
SyncPoint::GetInstance()->EnableProcessing();
// Generate data and trigger compaction (which uses OpenAndCompact)
GenerateTestData();
ASSERT_OK(dbfull()->TEST_WaitForCompact());
// WAL recovery should NOT have been called during OpenAndCompact
ASSERT_FALSE(wal_recovery_called.load());
// Data should still be correct (compaction worked without WAL recovery)
VerifyTestData();
SyncPoint::GetInstance()->DisableProcessing();
SyncPoint::GetInstance()->ClearAllCallBacks();
}
TEST_F(CompactionServiceTest, ManualCompaction) {
Options options = CurrentOptions();
options.disable_auto_compactions = true;
@@ -1111,6 +1141,8 @@ TEST_F(CompactionServiceTest, CorruptedOutputVerifyOutputFlags) {
VerifyOutputFlags::kVerifyBlockChecksum,
VerifyOutputFlags::kEnableForRemoteCompaction |
VerifyOutputFlags::kVerifyIteration,
VerifyOutputFlags::kEnableForRemoteCompaction |
VerifyOutputFlags::kVerifyFileChecksum,
VerifyOutputFlags::kVerifyAll}) {
SCOPED_TRACE(
"verify_output_flags=" +
@@ -1122,6 +1154,7 @@ TEST_F(CompactionServiceTest, CorruptedOutputVerifyOutputFlags) {
options.disable_auto_compactions = true;
options.paranoid_file_checks = false;
options.verify_output_flags = verify_output_flags;
options.file_checksum_gen_factory = GetFileChecksumGenCrc32cFactory();
ReopenWithCompactionService(&options);
GenerateTestData();
@@ -1157,10 +1190,13 @@ TEST_F(CompactionServiceTest, CorruptedOutputVerifyOutputFlags) {
!!(verify_output_flags & VerifyOutputFlags::kVerifyBlockChecksum);
const bool should_verify_iteration =
!!(verify_output_flags & VerifyOutputFlags::kVerifyIteration);
const bool should_verify_file_checksum =
!!(verify_output_flags & VerifyOutputFlags::kVerifyFileChecksum);
Status s = db_->CompactRange(CompactRangeOptions(), &start, &end);
if (is_enabled_for_remote_compaction &&
(should_verify_block_checksum || should_verify_iteration)) {
(should_verify_block_checksum || should_verify_iteration ||
should_verify_file_checksum)) {
ASSERT_NOK(s);
ASSERT_TRUE(s.IsCorruption());
} else {
@@ -1378,8 +1414,9 @@ TEST_F(CompactionServiceTest, CancelCompactionOnPrimarySide) {
// Primary DB calls CancelAllBackgroundWork() while the compaction is running
SyncPoint::GetInstance()->SetCallBack(
"CompactionJob::Run():Inprogress",
[&](void* /*arg*/) { CancelAllBackgroundWork(db_, false /*wait*/); });
"CompactionJob::Run():Inprogress", [&](void* /*arg*/) {
CancelAllBackgroundWork(db_.get(), false /*wait*/);
});
SyncPoint::GetInstance()->EnableProcessing();
@@ -2137,6 +2174,12 @@ class ResumableCompactionService : public MyTestCompactionService {
{} /* table_properties_collector_factories */),
scenario_(scenario) {}
// Set the user key where cancellation should happen.
void SetCancelAtKey(const std::string& key, SequenceNumber seqno) {
cancel_at_key_ = key;
cancel_at_seqno_ = seqno;
}
CompactionServiceJobStatus Wait(const std::string& scheduled_job_id,
std::string* result) override {
std::string compaction_input = ExtractCompactionInput(scheduled_job_id);
@@ -2149,25 +2192,50 @@ class ResumableCompactionService : public MyTestCompactionService {
// ASSUMPTION: This makes stats.count directly proportional to keys
// processed.
SyncPoint::GetInstance()->SetCallBack(
"CompactionOutputs::ShouldStopBefore::manual_decision", [](void* p) {
"CompactionOutputs::ShouldStopBefore::manual_decision",
[this](void* p) {
auto* pair = static_cast<std::pair<bool*, const Slice>*>(p);
*(pair->first) = true;
*(pair->first) = true; // Force file cut at every key
// If cancel_at_key_ is set, cancel when we encounter that key
if (!cancel_at_key_.empty() && !already_canceled_) {
ParsedInternalKey parsed_key;
if (ParseInternalKey(pair->second, &parsed_key, true).ok()) {
if (parsed_key.user_key.ToString() == cancel_at_key_) {
// Check sequence number if specified
if (cancel_at_seqno_ == kMaxSequenceNumber ||
parsed_key.sequence == cancel_at_seqno_) {
canceled_ = true;
already_canceled_ = true;
}
}
}
}
});
// If no cancel_at_key_ is set, use the original behavior:
// Simulate cancelled compaction by overriding status at completion. So
// compaction processes all keys before this point to make stats.count
// comparison straightforward.
SyncPoint::GetInstance()->SetCallBack(
"DBImplSecondary::CompactWithoutInstallation::End", [&](void* status) {
auto s = static_cast<Status*>(status);
*s = Status::Incomplete(Status::SubCode::kManualCompactionPaused);
});
if (cancel_at_key_.empty()) {
SyncPoint::GetInstance()->SetCallBack(
"DBImplSecondary::CompactWithoutInstallation::End",
[&](void* status) {
auto s = static_cast<Status*>(status);
*s = Status::Incomplete(Status::SubCode::kManualCompactionPaused);
});
}
SyncPoint::GetInstance()->EnableProcessing();
// Phase 1: Run compaction with resumption enabled and cancel it
// - Processes all input keys
// - Processes input keys until cancellation point
// - Creates output files and saves progress
// - Status overridden to "paused"
open_and_compaction_options.allow_resumption = true;
open_and_compaction_options.canceled = &canceled_;
already_canceled_ = false;
canceled_ = false;
auto phase1_stats =
RunCancelledCompaction(open_and_compaction_options, scheduled_job_id,
compaction_input, override_options);
@@ -2188,6 +2256,9 @@ class ResumableCompactionService : public MyTestCompactionService {
EXPECT_TRUE(cleanup_status.ok());
EXPECT_OK(override_options.env->CreateDir(output_dir));
already_canceled_ = false;
canceled_ = false;
phase2_stats =
RunCancelledCompaction(open_and_compaction_options, scheduled_job_id,
compaction_input, override_options);
@@ -2199,9 +2270,6 @@ class ResumableCompactionService : public MyTestCompactionService {
EXPECT_EQ(phase2_stats.count, phase1_stats.count);
}
SyncPoint::GetInstance()->ClearCallBack(
"DBImplSecondary::CompactWithoutInstallation::End");
// Final phase: Run compaction to completion (no cancellation)
if (scenario_ == TestScenario::kMultipleCancelToggleResumption) {
// Attempt to resume but it ends up starting fresh
@@ -2220,6 +2288,12 @@ class ResumableCompactionService : public MyTestCompactionService {
EXPECT_OK(override_options.env->CreateDir(output_dir));
}
// Prevent triggering of cancellation
SyncPoint::GetInstance()->ClearCallBack(
"DBImplSecondary::CompactWithoutInstallation::End");
already_canceled_ = true;
canceled_ = false;
auto final_phase_stats =
RunCompaction(open_and_compaction_options, scheduled_job_id,
compaction_input, override_options, result);
@@ -2227,36 +2301,38 @@ class ResumableCompactionService : public MyTestCompactionService {
SyncPoint::GetInstance()->DisableProcessing();
SyncPoint::GetInstance()->ClearAllCallBacks();
// Validate statistics based on scenario
if (scenario_ == TestScenario::kMultipleCancelToggleResumption) {
// ASSUMPTION: Phase 1 processes all keys before cancellation
EXPECT_GT(phase1_stats.count, 0);
// Validate statistics based on scenario (only when cancelling at end)
if (cancel_at_key_.empty()) {
if (scenario_ == TestScenario::kMultipleCancelToggleResumption) {
// ASSUMPTION: Phase 1 processes all keys before cancellation
EXPECT_GT(phase1_stats.count, 0);
// ASSUMPTION: Phase 2 runs with allow_resumption=false and an empty
// folder. Phase 2 then creates its own output files (but doesn't save
// progress). When Phase 3 starts with allow_resumption=true, it finds no
// progress file exists, so it cannot resume and must start from scratch,
// processing all input keys again.
// Result: Phase 3 does the same amount of work as Phase 1.
EXPECT_EQ(final_phase_stats.count, phase1_stats.count);
// ASSUMPTION: Phase 2 runs with allow_resumption=false and an empty
// folder. Phase 2 then creates its own output files (but doesn't save
// progress). When Phase 3 starts with allow_resumption=true, it finds
// no progress file exists, so it cannot resume and must start from
// scratch, processing all input keys again. Result: Phase 3 does the
// same amount of work as Phase 1.
EXPECT_EQ(final_phase_stats.count, phase1_stats.count);
} else if (scenario_ == TestScenario::kCancelThenResume) {
// ASSUMPTION: Phase 1 processes all keys before cancellation
EXPECT_GT(phase1_stats.count, 0);
} else if (scenario_ == TestScenario::kCancelThenResume) {
// ASSUMPTION: Phase 1 processes all keys before cancellation
EXPECT_GT(phase1_stats.count, 0);
// ASSUMPTION: Phase 1 processes all keys and saves progress before
// cancellation. Final phase resumes from Phase 1's saved progress.
// Since Phase 1 completed all processing before being cancelled, the
// final phase should do less work than Phase 1.
EXPECT_LT(final_phase_stats.count, phase1_stats.count);
// ASSUMPTION: Phase 1 processes all keys and saves progress before
// cancellation. Final phase resumes from Phase 1's saved progress.
// Since Phase 1 completed all processing before being cancelled, the
// final phase should do less work than Phase 1.
EXPECT_LT(final_phase_stats.count, phase1_stats.count);
} else { // kCancelThenFreshStart
// ASSUMPTION: Phase 1 processes all keys before cancellation
EXPECT_GT(phase1_stats.count, 0);
} else { // kCancelThenFreshStart
// ASSUMPTION: Phase 1 processes all keys before cancellation
EXPECT_GT(phase1_stats.count, 0);
// ASSUMPTION: Final phase starts fresh without resumption, so it
// processes all input keys again and creates the same number of files
EXPECT_EQ(final_phase_stats.count, phase1_stats.count);
// ASSUMPTION: Final phase starts fresh without resumption, so it
// processes all input keys again and creates the same number of files
EXPECT_EQ(final_phase_stats.count, phase1_stats.count);
}
}
StoreResult(*result);
@@ -2326,6 +2402,9 @@ class ResumableCompactionService : public MyTestCompactionService {
}
TestScenario scenario_;
std::string cancel_at_key_;
SequenceNumber cancel_at_seqno_ = kMaxSequenceNumber;
std::atomic<bool> already_canceled_{false};
};
class ResumableCompactionServiceTest : public CompactionServiceTest {
@@ -2432,6 +2511,387 @@ TEST_F(ResumableCompactionServiceTest,
RunCompactionCancelTest(ResumableCompactionService::TestScenario::
kMultipleCancelToggleResumption);
}
class ResumableCompactionKeyTypeTest : public CompactionServiceTest {
public:
explicit ResumableCompactionKeyTypeTest() : CompactionServiceTest() {}
protected:
void SetupResumableCompactionService(
Options& options, const std::string& cancel_at_key = "",
SequenceNumber cancel_at_seqno = kMaxSequenceNumber) {
options.disable_auto_compactions = true;
statistics_ = CreateDBStatistics();
resume_cs_ = std::make_shared<ResumableCompactionService>(
dbname_, options, statistics_,
ResumableCompactionService::TestScenario::kCancelThenResume);
if (!cancel_at_key.empty()) {
resume_cs_->SetCancelAtKey(cancel_at_key, cancel_at_seqno);
}
options.compaction_service = resume_cs_;
DestroyAndReopen(options);
}
void ResetStatistics() { ASSERT_OK(statistics_->Reset()); }
void VerifyResumeBytes() {
uint64_t resumed_bytes =
statistics_->getTickerCount(REMOTE_COMPACT_RESUMED_BYTES);
ASSERT_GT(resumed_bytes, 0);
}
private:
std::shared_ptr<ResumableCompactionService> resume_cs_;
std::shared_ptr<Statistics> statistics_;
};
// Cancel compaction right before processing key "c" to test resumption at a
// deletion at the non-bottom level. When resumed, compaction will continue
// from this deletion.
TEST_F(ResumableCompactionKeyTypeTest,
CancelAndResumeWithDeleteAtNonBottomLevel) {
Options options = CurrentOptions();
SetupResumableCompactionService(options, "c");
ASSERT_OK(Put("c", "old_value"));
ASSERT_OK(Put("c_placeholder", "placeholder"));
ASSERT_OK(Flush());
MoveFilesToLevel(options.num_levels - 1);
ASSERT_OK(Put("a", "val1"));
ASSERT_OK(Put("b", "val2"));
ASSERT_OK(Put("d", "val4"));
ASSERT_OK(Flush());
ASSERT_OK(Delete("c"));
ASSERT_OK(Flush());
std::vector<std::string> input_files;
ColumnFamilyMetaData cf_meta;
db_->GetColumnFamilyMetaData(&cf_meta);
for (const auto& file : cf_meta.levels[0].files) {
input_files.push_back(file.name);
}
ASSERT_EQ(input_files.size(), 2);
ResetStatistics();
CompactionOptions compact_options;
ASSERT_OK(
db_->CompactFiles(compact_options, input_files, 1 /* output_level*/));
ASSERT_EQ(Get("a"), "val1");
ASSERT_EQ(Get("b"), "val2");
ASSERT_EQ(Get("c"), "NOT_FOUND");
ASSERT_EQ(Get("d"), "val4");
VerifyResumeBytes();
}
// Cancel compaction right before processing key "c" to test resumption at a
// deletion at the ottom level. When resumed, compaction will continue from
// the last saved progress point before the delete.
TEST_F(ResumableCompactionKeyTypeTest, CancelAndResumeWithDeleteAtBottomLevel) {
Options options = CurrentOptions();
SetupResumableCompactionService(options, "c");
ASSERT_OK(Put("c", "old_value"));
const Snapshot* snapshot = db_->GetSnapshot();
ASSERT_OK(Delete("c"));
ASSERT_OK(Flush());
MoveFilesToLevel(options.num_levels - 1);
ASSERT_OK(Put("a", "val1"));
ASSERT_OK(Put("b", "val2"));
ASSERT_OK(Put("d", "val4"));
ASSERT_OK(Flush());
ResetStatistics();
ASSERT_OK(db_->CompactRange(CompactRangeOptions(), nullptr, nullptr));
ASSERT_EQ(Get("a"), "val1");
ASSERT_EQ(Get("b"), "val2");
ASSERT_EQ(Get("c"), "NOT_FOUND");
ASSERT_EQ(Get("c", snapshot), "old_value");
ASSERT_EQ(Get("d"), "val4");
db_->ReleaseSnapshot(snapshot);
VerifyResumeBytes();
}
// Cancel compaction right before processing key "c" to test resumption at a
// merge operand. When resumed, compaction will continue from the last saved
// progress point before the merge operand.
TEST_F(ResumableCompactionKeyTypeTest, CancelAndResumeWithMerge) {
Options options = CurrentOptions();
options.merge_operator = MergeOperators::CreateStringAppendOperator();
SetupResumableCompactionService(options, "c");
ASSERT_OK(Put("c", "old_value"));
ASSERT_OK(Put("c_placeholder", "placeholder"));
ASSERT_OK(Flush());
MoveFilesToLevel(options.num_levels - 1);
ASSERT_OK(Put("a", "val1"));
ASSERT_OK(Put("b", "val2"));
ASSERT_OK(Put("d", "val4"));
ASSERT_OK(Flush());
ASSERT_OK(Merge("c", "new_value"));
ASSERT_OK(Flush());
std::vector<std::string> input_files;
ColumnFamilyMetaData cf_meta;
db_->GetColumnFamilyMetaData(&cf_meta);
for (const auto& file : cf_meta.levels[0].files) {
input_files.push_back(file.name);
}
ASSERT_EQ(input_files.size(), 2);
ResetStatistics();
CompactionOptions compact_options;
ASSERT_OK(
db_->CompactFiles(compact_options, input_files, 1 /* output_level*/));
ASSERT_EQ(Get("a"), "val1");
ASSERT_EQ(Get("b"), "val2");
ASSERT_EQ(Get("c"), "old_value,new_value");
ASSERT_EQ(Get("d"), "val4");
VerifyResumeBytes();
}
// Cancel compaction right before processing key "c" to test resumption at a
// single delete. When resumed, compaction will continue from the last saved
// progress point before the single delete.
TEST_F(ResumableCompactionKeyTypeTest, CancelAndResumeWithSingleDelete) {
Options options = CurrentOptions();
SetupResumableCompactionService(options, "c");
ASSERT_OK(Put("c", "old_value"));
ASSERT_OK(Put("c_placeholder", "placeholder"));
ASSERT_OK(Flush());
MoveFilesToLevel(options.num_levels - 1);
ASSERT_OK(Put("a", "val1"));
ASSERT_OK(Put("b", "val2"));
ASSERT_OK(Put("d", "val4"));
ASSERT_OK(Flush());
ASSERT_OK(SingleDelete("c"));
ASSERT_OK(Flush());
std::vector<std::string> input_files;
ColumnFamilyMetaData cf_meta;
db_->GetColumnFamilyMetaData(&cf_meta);
for (const auto& file : cf_meta.levels[0].files) {
input_files.push_back(file.name);
}
ASSERT_EQ(input_files.size(), 2);
ResetStatistics();
CompactionOptions compact_options;
ASSERT_OK(
db_->CompactFiles(compact_options, input_files, 1 /* output_level*/));
ASSERT_EQ(Get("a"), "val1");
ASSERT_EQ(Get("b"), "val2");
ASSERT_EQ(Get("c"), "NOT_FOUND");
ASSERT_EQ(Get("d"), "val4");
VerifyResumeBytes();
}
// Cancel compaction right before processing key "c" to test resumption at a
// range delete. When resumed, compaction will continue from the last saved
// progress point before the range delete.
TEST_F(ResumableCompactionKeyTypeTest, CancelAndResumeWithRangeDelete) {
Options options = CurrentOptions();
SetupResumableCompactionService(options, "c");
ASSERT_OK(Put("c", "old_value"));
ASSERT_OK(Put("c_placeholder", "placeholder"));
ASSERT_OK(Flush());
MoveFilesToLevel(options.num_levels - 1);
ASSERT_OK(Put("a", "val1"));
ASSERT_OK(Put("b", "val2"));
ASSERT_OK(Put("d", "val4"));
ASSERT_OK(Flush());
ASSERT_OK(
db_->DeleteRange(WriteOptions(), db_->DefaultColumnFamily(), "c", "c_"));
ASSERT_OK(Flush());
std::vector<std::string> input_files;
ColumnFamilyMetaData cf_meta;
db_->GetColumnFamilyMetaData(&cf_meta);
for (const auto& file : cf_meta.levels[0].files) {
input_files.push_back(file.name);
}
ASSERT_EQ(input_files.size(), 2);
ResetStatistics();
CompactionOptions compact_options;
ASSERT_OK(
db_->CompactFiles(compact_options, input_files, 1 /* output_level*/));
ASSERT_EQ(Get("a"), "val1");
ASSERT_EQ(Get("b"), "val2");
ASSERT_EQ(Get("c"), "NOT_FOUND");
ASSERT_EQ(Get("d"), "val4");
VerifyResumeBytes();
}
// Test resumption when a key has multiple versions spanning across file
// boundaries (i.e., the same key exists in multiple SST files).
//
// Scenario:
// File 1 largest key: key "b"
// File 2 smallest key: key "c" with seqno=4 (older version)
// File 3 largest key: key "c" with seqno=5 (newer version)
//
// Cancel compaction right before processing the older version of key "c".
// Upon resumption, compaction continues from the saved progress point "b" and
// correctly processes both versions
TEST_F(ResumableCompactionKeyTypeTest,
CancelAndResumeWithKeySpanningFileBoundaries) {
Options options = CurrentOptions();
// Set up cancellation at the older version of the key which will have
// sequence number zero-ed out
SetupResumableCompactionService(options, "c" /*cancel_at_key*/, 0 /*seqno*/);
ASSERT_OK(Put("a", "val1"));
ASSERT_OK(Put("b", "val2"));
ASSERT_OK(Put("d", "val4"));
ASSERT_OK(Flush());
ASSERT_OK(Put("c", "old_value"));
const Snapshot* snapshot = db_->GetSnapshot();
ASSERT_OK(Put("c", "new_value"));
ASSERT_OK(Flush());
ResetStatistics();
ASSERT_OK(db_->CompactRange(CompactRangeOptions(), nullptr, nullptr));
ASSERT_EQ(Get("a"), "val1");
ASSERT_EQ(Get("b"), "val2");
ASSERT_EQ(Get("c"), "new_value");
ASSERT_EQ(Get("c", snapshot), "old_value");
ASSERT_EQ(Get("d"), "val4");
db_->ReleaseSnapshot(snapshot);
VerifyResumeBytes();
}
// Cancel compaction right before processing key "c" to test resumption at a
// wide column. When resumed, compaction will continue
// from the wide column.
TEST_F(ResumableCompactionKeyTypeTest, CancelAndResumeWithWideColumn) {
Options options = CurrentOptions();
SetupResumableCompactionService(options, "c" /*cancel_at_key*/);
ASSERT_OK(Put("a", "val1"));
ASSERT_OK(Put("b", "val2"));
ASSERT_OK(Put("d", "val4"));
ASSERT_OK(Flush());
WideColumns columns{{"col1", "value1"}, {"col2", "value2"}};
ASSERT_OK(
db_->PutEntity(WriteOptions(), db_->DefaultColumnFamily(), "c", columns));
ASSERT_OK(Flush());
ResetStatistics();
ASSERT_OK(db_->CompactRange(CompactRangeOptions(), nullptr, nullptr));
ASSERT_EQ(Get("a"), "val1");
ASSERT_EQ(Get("b"), "val2");
PinnableWideColumns result;
ASSERT_OK(
db_->GetEntity(ReadOptions(), db_->DefaultColumnFamily(), "c", &result));
WideColumns expected{{"col1", "value1"}, {"col2", "value2"}};
ASSERT_EQ(result.columns(), expected);
ASSERT_EQ(Get("d"), "val4");
VerifyResumeBytes();
}
// Cancel compaction right before processing key "c" to test resumption at a
// timed put. When resumed, compaction will continue
// from the timed put.
TEST_F(ResumableCompactionKeyTypeTest, CancelAndResumeWithTimedPut) {
Options options = CurrentOptions();
options.preclude_last_level_data_seconds = 86400; // Enable TimedPut feature
options.preserve_internal_time_seconds = 86400; // Preserve write time
SetupResumableCompactionService(options, "c" /*cancel_at_key*/);
ASSERT_OK(Put("c", "old_value"));
ASSERT_OK(Put("c_placeholder", "placeholder"));
ASSERT_OK(Flush());
MoveFilesToLevel(options.num_levels - 1);
ASSERT_OK(Put("a", "val1"));
ASSERT_OK(Put("b", "val2"));
ASSERT_OK(Put("d", "val4"));
ASSERT_OK(Flush());
// Use TimedPut for key "c" with current write time
uint64_t write_time = env_->NowMicros() / 1000000;
ASSERT_OK(TimedPut("c", "val3", write_time /*write_unix_time*/));
ASSERT_OK(Put("d", "val4"));
ASSERT_OK(Flush());
std::vector<std::string> input_files;
ColumnFamilyMetaData cf_meta;
db_->GetColumnFamilyMetaData(&cf_meta);
for (const auto& file : cf_meta.levels[0].files) {
input_files.push_back(file.name);
}
ASSERT_EQ(input_files.size(), 2);
ResetStatistics();
CompactionOptions compact_options;
ASSERT_OK(
db_->CompactFiles(compact_options, input_files, 1 /* output_level*/));
ASSERT_EQ(Get("a"), "val1");
ASSERT_EQ(Get("b"), "val2");
ASSERT_EQ(Get("c"), "val3");
ASSERT_EQ(Get("d"), "val4");
VerifyResumeBytes();
}
} // namespace ROCKSDB_NAMESPACE
int main(int argc, char** argv) {
+2 -2
View File
@@ -38,11 +38,11 @@ OutputIterator SubcompactionState::GetOutputs() const {
compaction_outputs_.outputs_);
}
void SubcompactionState::Cleanup(Cache* cache) {
void SubcompactionState::Cleanup(Cache* cache, const Status& overall_status) {
proximal_level_outputs_.Cleanup();
compaction_outputs_.Cleanup();
if (!status.ok()) {
if (!status.ok() || !overall_status.ok()) {
for (const auto& out : GetOutputs()) {
// If this file was inserted into the table cache then remove it here
// because this compaction was not committed. This is not strictly
+10 -1
View File
@@ -81,6 +81,15 @@ class SubcompactionState {
// it returns both the last level outputs and proximal level outputs.
OutputIterator GetOutputs() const;
// Get mutable access to outputs for post-processing (e.g., retrieving
// file open metadata).
std::vector<CompactionOutputs::Output>& GetMutableCompactionOutputs() {
return compaction_outputs_.GetMutableOutputs();
}
std::vector<CompactionOutputs::Output>& GetMutableProximalOutputs() {
return proximal_level_outputs_.GetMutableOutputs();
}
// Assign range dels aggregator. The various tombstones will potentially
// be filtered to different outputs.
void AssignRangeDelAggregator(
@@ -170,7 +179,7 @@ class SubcompactionState {
}
}
void Cleanup(Cache* cache);
void Cleanup(Cache* cache, const Status& overall_status);
void AggregateCompactionOutputStats(
InternalStats::CompactionStatsFull& internal_stats) const;
+4 -4
View File
@@ -1717,8 +1717,8 @@ TEST_P(PrecludeLastLevelTest, MigrationFromPreserveTimePartial) {
ASSERT_EQ("0,0,0,0,0,0,1", FilesPerLevel());
std::vector<KeyVersion> key_versions;
ASSERT_OK(GetAllKeyVersions(db_, {}, {}, std::numeric_limits<size_t>::max(),
&key_versions));
ASSERT_OK(GetAllKeyVersions(
db_.get(), {}, {}, std::numeric_limits<size_t>::max(), &key_versions));
// make sure there're more than 300 keys and first 100 keys are having seqno
// zeroed out, the last 100 key seqno not zeroed out
@@ -2319,8 +2319,8 @@ TEST_P(PrecludeLastLevelTest, LastLevelOnlyCompactionPartial) {
ASSERT_GT(GetSstSizeHelper(Temperature::kUnknown), 0);
std::vector<KeyVersion> key_versions;
ASSERT_OK(GetAllKeyVersions(db_, {}, {}, std::numeric_limits<size_t>::max(),
&key_versions));
ASSERT_OK(GetAllKeyVersions(
db_.get(), {}, {}, std::numeric_limits<size_t>::max(), &key_versions));
// make sure there're more than 300 keys and first 100 keys are having seqno
// zeroed out, the last 100 key seqno not zeroed out
+7 -9
View File
@@ -258,12 +258,12 @@ class ComparatorDBTest
private:
std::string dbname_;
Env* env_;
DB* db_;
std::unique_ptr<DB> db_;
Options last_options_;
std::unique_ptr<const Comparator> comparator_guard;
public:
ComparatorDBTest() : env_(Env::Default()), db_(nullptr) {
ComparatorDBTest() : env_(Env::Default()) {
kTestComparator = BytewiseComparator();
dbname_ = test::PerThreadDBPath("comparator_db_test");
BlockBasedTableOptions toptions;
@@ -274,12 +274,12 @@ class ComparatorDBTest
}
~ComparatorDBTest() override {
delete db_;
db_.reset();
EXPECT_OK(DestroyDB(dbname_, last_options_));
kTestComparator = BytewiseComparator();
}
DB* GetDB() { return db_; }
DB* GetDB() { return db_.get(); }
void SetOwnedComparator(const Comparator* cmp, bool owner = true) {
if (owner) {
@@ -301,14 +301,12 @@ class ComparatorDBTest
}
void Destroy() {
delete db_;
db_ = nullptr;
db_.reset();
ASSERT_OK(DestroyDB(dbname_, last_options_));
}
Status TryReopen() {
delete db_;
db_ = nullptr;
db_.reset();
last_options_.create_if_missing = true;
return DB::Open(last_options_, dbname_, &db_);
@@ -318,7 +316,7 @@ class ComparatorDBTest
INSTANTIATE_TEST_CASE_P(FormatDef, ComparatorDBTest,
testing::Values(test::kDefaultFormatVersion));
INSTANTIATE_TEST_CASE_P(FormatLatest, ComparatorDBTest,
testing::Values(kLatestFormatVersion));
testing::Values(kLatestBbtFormatVersion));
TEST_P(ComparatorDBTest, Bytewise) {
for (int rand_seed = 301; rand_seed < 306; rand_seed++) {
+4 -4
View File
@@ -65,7 +65,7 @@ Status VerifySstFileChecksum(const Options& options,
}
Status VerifySstFileChecksumInternal(const Options& options,
const EnvOptions& env_options,
const FileOptions& file_options,
const ReadOptions& read_options,
const std::string& file_path,
const SequenceNumber& largest_seqno) {
@@ -74,8 +74,8 @@ Status VerifySstFileChecksumInternal(const Options& options,
InternalKeyComparator internal_comparator(options.comparator);
ImmutableOptions ioptions(options);
Status s = ioptions.fs->NewRandomAccessFile(
file_path, FileOptions(env_options), &file, nullptr);
Status s =
ioptions.fs->NewRandomAccessFile(file_path, file_options, &file, nullptr);
if (s.ok()) {
s = ioptions.fs->GetFileSize(file_path, IOOptions(), &file_size, nullptr);
} else {
@@ -94,7 +94,7 @@ Status VerifySstFileChecksumInternal(const Options& options,
const bool kImmortal = true;
auto reader_options = TableReaderOptions(
ioptions, options.prefix_extractor, options.compression_manager.get(),
env_options, internal_comparator, options.block_protection_bytes_per_key,
file_options, internal_comparator, options.block_protection_bytes_per_key,
false /* skip_filters */, !kImmortal, false /* force_direct_prefetch */,
-1 /* level */);
reader_options.largest_seqno = largest_seqno;
+2 -1
View File
@@ -5,10 +5,11 @@
#pragma once
#include "rocksdb/db.h"
#include "rocksdb/file_system.h"
namespace ROCKSDB_NAMESPACE {
Status VerifySstFileChecksumInternal(const Options& options,
const EnvOptions& env_options,
const FileOptions& file_options,
const ReadOptions& read_options,
const std::string& file_path,
const SequenceNumber& largest_seqno = 0);

Some files were not shown because too many files have changed in this diff Show More