Verify CPU corruption after each op via --verify_cpu_corruption_dir (#14852)

Summary:

Detection layer of the CPU corruption injector (coming up). With `--verify_cpu_corruption_dir=<dir>`, db_stress reads back the full keyspace after every write/flush/compaction op and compares it to the expected-values model, classifying any mismatch by `kind`: `lost` / `resurrected` / `wrong-value` (silent data corruption) or `detected-corruption` (a status/checksum-caught error). Each finding is written to `<dir>/data_corruption.<tid>.json` ({kind, cf, key, value_from_db, value_from_expected, op_status}) and routed through db_stress's standard `VerificationAbort` for a clean exit-1. A startup guard requires `--threads=1` and all fault injection off so the read-back is single-writer and the only corruption present is the injected one

**Test plan:**
1.Startup guard rejects misconfiguration:
```
--threads=2           -> exit 1: "--verify_cpu_corruption_dir requires --threads=1"
--read_fault_one_in=5 -> exit 1: "requires all fault injection off"
```
2.No false positive (clean CORE preset run, no injection):
```
$ db_stress --verify_cpu_corruption_dir=<dir> --threads=1 (full protections, all *_fault_one_in=0) ...
exit 0; no data_corruption.<tid>.json produced; "Verification successful"
```
3.Write-path cpu corruption injection (coming up, e.g, gdb flips a register inside MemTable::Add), then the immediate post-op read-back catches it. Real `<dir>/data_corruption.<tid>.json`:

silent data corruption -- write returned OK but the key is gone on read-back:
```
{"kind":"lost","cf":0,"key":9814,"value_from_db":"","value_from_expected":"010000000504070609080B0A0D0C0F0E","op_status":"Get: NotFound"}
```
detected corruption -- read-back Get returns Corruption via the memtable per-key checksum:
```
{"kind":"detected-corruption","cf":0,"key":139,"value_from_db":"","value_from_expected":"","op_status":"Get: Corruption: Corrupted memtable entry, per key-value checksum verification failed."
```

Differential Revision: D107999834

.github/actions/windows-build-steps/action.yml

@@ -3,7 +3,7 @@ inputs:
   suite-run:
     description: Comma-separated list of test suites to run (empty to skip C++ tests)
     required: false
-    default: arena_test,db_basic_test,db_test,db_test2,db_merge_operand_test,bloom_test,c_test,coding_test,crc32c_test,dynamic_bloom_test,env_basic_test,env_test,hash_test,random_test
+    default: arena_test,db_basic_test,db_test,db_etc2_test,db_merge_operand_test,bloom_test,c_test,coding_test,crc32c_test,dynamic_bloom_test,env_basic_test,env_test,hash_test,random_test
   run-java:
     description: Whether to run Java tests
     required: false

.github/workflows/pr-jobs.yml

@@ -537,7 +537,7 @@ jobs:
           suite_run: db_test
           run_java: "false"
         - test_shard: other
-          suite_run: arena_test,db_basic_test,db_test2,db_merge_operand_test,bloom_test,c_test,coding_test,crc32c_test,dynamic_bloom_test,env_basic_test,env_test,hash_test,random_test
+          suite_run: arena_test,db_basic_test,db_etc2_test,db_merge_operand_test,bloom_test,c_test,coding_test,crc32c_test,dynamic_bloom_test,env_basic_test,env_test,hash_test,random_test
           run_java: "false"
         - test_shard: java
           suite_run: ""

.gitignore

@@ -52,7 +52,7 @@ GRTAGS
 GTAGS
 rocksdb_dump
 rocksdb_undump
-db_test2
+db_etc2_test
 trace_analyzer
 block_cache_trace_analyzer
 io_tracer_parser

BUCK

@@ -4868,6 +4868,12 @@ cpp_unittest_wrapper(name="db_encryption_test",
             extra_compiler_flags=[])
 
 
+cpp_unittest_wrapper(name="db_etc2_test",
+            srcs=["db/db_etc2_test.cc"],
+            deps=[":rocksdb_test_lib"],
+            extra_compiler_flags=[])
+
+
 cpp_unittest_wrapper(name="db_etc3_test",
             srcs=["db/db_etc3_test.cc"],
             deps=[":rocksdb_test_lib"],
@@ -5024,12 +5030,6 @@ cpp_unittest_wrapper(name="db_test",
             extra_compiler_flags=[])
 
 
-cpp_unittest_wrapper(name="db_test2",
-            srcs=["db/db_test2.cc"],
-            deps=[":rocksdb_test_lib"],
-            extra_compiler_flags=[])
-
-
 cpp_unittest_wrapper(name="db_universal_compaction_test",
             srcs=["db/db_universal_compaction_test.cc"],
             deps=[":rocksdb_test_lib"],

CLAUDE.md

@@ -149,6 +149,26 @@ Cache management is critical for RocksDB's performance.
 
 When reviewing RocksDB code (or preparing code for review), use this checklist:
 
+### Contract Boundaries
+- [ ] Is each behavior owned by the right layer? High-level policy (for example,
+  "compaction wants this I/O mode") should live at the caller/policy layer, while
+  lower layers should expose generic mechanisms (for example, "open a fresh
+  reader", "skip shared cache insertion", or "use these FileOptions").
+- [ ] Do comments and names describe local contracts rather than leaking a
+  specific caller's rationale into reusable APIs? Generic code should not need to
+  know about one current use case unless the API itself is intentionally
+  use-case-specific.
+- [ ] Does each flag or parameter control one coherent behavior? If one boolean
+  starts implying ownership, cache policy, I/O mode, prefetching, and caller
+  identity, split it into explicit flags or an options struct.
+- [ ] Could a future caller use this lower-level API without accidentally
+  inheriting assumptions from compaction, backup, user reads, or a particular
+  table format? If not, tighten the contract with assertions, clearer names, or
+  a narrower API.
+- [ ] Are implementation details not being used as policy signals? Prefer an
+  explicit contract over inferring behavior from incidental fields such as file
+  options, cache handles, or current table-reader state.
+
 ### Correctness
 - [ ] Does the change preserve database semantics (e.g., snapshot isolation, key ordering)?
 - [ ] Are all error cases handled appropriately?
@@ -204,17 +224,24 @@ The following patterns emerged as frequent sources of review feedback:
 
 10. **Platform Compatibility:** Ensure changes work correctly on all supported platforms (Linux, Windows, macOS) and with all supported compilers (GCC, Clang, MSVC).
 
+11. **Contract Boundary Leaks:** When a change plumbs a new option or use-case
+specific behavior through multiple subsystems, review the call chain for
+contract leaks. Caller-specific rationale belongs at the call site or public API
+documentation; reusable layers should expose precise, layer-local capabilities.
+Watch especially for comments mentioning one caller in generic code, booleans
+that silently bundle several behaviors, and downstream code inferring policy
+from an implementation detail instead of an explicit option.
+
 ---
 
 ## Important tips
 
 ### Build system
-* There are 3 build system. Make, CMake, BUCK(meta internal).
+* There are 3 build system. Make for git clones, BUCK (meta internal) for hg
+  clones, and CMake for some special cases.
 * When a new .cc file is added, update Makefile, CMakeLists.txt, src.mk, BUCK.
 * Don't manually edit BUCK file, after updating src.mk, run
     /usr/local/bin/python3 buckifier/buckify_rocksdb.py to update it
-* Use make to build and run the test. CMake and BUCK are not used locally.
-* Use `make dbg` command to build all of the unit test in debug mode.
 * For -j in make command, use the number of CPU cores to decide it.
 
 ### When to run `make clean` (avoid mixing build modes)
@@ -252,6 +279,29 @@ phantom bug.
     `dynamic_cast` in debug builds, plain `static_cast` in release).
 * Unit tests (`*_test.cc`) are built in debug mode with RTTI enabled.
 
+### Cross-platform / portability
+Local `make` only exercises Linux with GCC/Clang, but CI
+(`.github/workflows/pr-jobs.yml` and `nightly.yml`) gates on a much wider
+matrix, so portability breaks are invisible locally until CI fails. Code must
+build (and where noted, run tests) across:
+
+| Axis | Must support |
+|------|--------------|
+| OS | Linux (x86_64 + ARM), macOS, Windows |
+| Compiler | GCC, Clang (libstdc++ **and** libc++), AppleClang, **MSVC (VS2022)**, MinGW (Linux cross-compile, build-only, no gflags) |
+| Build system | Make, CMake, and BUCK (internal) -- keep all in sync (see "Build system" above) |
+| Config | release (`-fno-rtti`), `ASSERT_STATUS_CHECKED`, ASAN/UBSAN/TSAN, folly, unity build, JNI/Java |
+
+Treat these as constraints to satisfy and infer the specifics from them before
+adding any system header, libc call, or compiler-specific construct. The most
+common trap: anything that compiles under GCC/Clang on Linux but not under
+**MSVC/MinGW** -- e.g. unguarded POSIX-only headers/functions (`<unistd.h>`,
+`<sys/*.h>`, `getpid`, `_exit`, ...) or GCC/Clang extensions
+(`__attribute__`, `__builtin_*`, VLAs, `alloca`). Prefer the `port::`/`Env`
+abstractions; otherwise guard with `#ifdef OS_WIN` (POSIX `<unistd.h>` ->
+Windows `<process.h>`). Because libc++ is also tested, include what you use
+rather than relying on libstdc++ transitive includes.
+
 ### Unit Test
 * After all of the unit tests are added, review them and try to extract common
     reusable utility functions to reduce code duplication due to copy past between
@@ -267,6 +317,9 @@ phantom bug.
     COERCE_CONTEXT_SWITCH=1 make {test_binary}
     ./{test_binary} --gtest_filter="*YourTestName*" --gtest_repeat=5
     ```
+* For CI-style flaky tests that do not reproduce with `gtest_parallel.py`,
+    `--gtest_repeat`, or normal coerce-mode runs, inspect
+    `tools/gtest_parallel_repro.py --help`.
 
 ### Unit test dedup guidelines
 * Extract helper functions for repeated patterns such as object

CMakeLists.txt

@@ -1491,6 +1491,7 @@ if(WITH_TESTS)
         db/db_clip_test.cc
         db/db_dynamic_level_test.cc
         db/db_encryption_test.cc
+        db/db_etc2_test.cc
         db/db_etc3_test.cc
         db/db_flush_test.cc
         db/db_inplace_update_test.cc
@@ -1514,7 +1515,6 @@ if(WITH_TESTS)
         db/db_table_properties_test.cc
         db/db_tailing_iter_test.cc
         db/db_test.cc
-        db/db_test2.cc
         db/db_logical_block_size_cache_test.cc
         db/db_universal_compaction_test.cc
         db/db_wal_test.cc

HISTORY.md

@@ -1,6 +1,28 @@
 # Rocksdb Change Log
 > NOTE: Entries for next release do not go here. Follow instructions in `unreleased_history/README.txt`
 
+## 11.4.0 (06/02/2026)
+### Public API Changes
+* Added `rocksdb_options_set_memtable_batch_lookup_optimization()` and `rocksdb_options_get_memtable_batch_lookup_optimization()` to the C API, exposing the existing `AdvancedColumnFamilyOptions::memtable_batch_lookup_optimization` field. This allows C API users (and downstream language bindings) to enable the skip-list memtable's batch-lookup optimization for `MultiGet`, which caches the search path between consecutive keys and reduces per-key cost from O(log N) to O(log d) where d is the distance between consecutive keys.
+* Added `rocksdb_readoptions_set_optimize_multiget_for_io()` and `rocksdb_readoptions_get_optimize_multiget_for_io()` to the C API, exposing the existing `ReadOptions::optimize_multiget_for_io` field. This allows C API users (and downstream language bindings) to opt out of the multi-level parallel MultiGet path, which only takes effect when the library is built with `USE_COROUTINES`.
+* Added `rocksdb_block_based_table_index_block_search_type_auto` enum constant and the `rocksdb_block_based_options_set_uniform_cv_threshold()` setter to the C API. The constant exposes `BlockSearchType::kAuto`, and the setter exposes `BlockBasedTableOptions::uniform_cv_threshold`. Both are required for `kAuto` index-block search to take effect from the C API: without setting `uniform_cv_threshold >= 0`, the per-block `is_uniform` footer bit is never written, and `kAuto` falls back to binary search at read time.
+* Added `EventListener::OnDBShutdownBegin`, a callback that fires once when a DB begins shutdown, including when RocksDB cleans up a failed `DB::Open()` attempt.
+* Added a new PerfContext counter `blob_cache_read_byte` for blob cache bytes read.
+
+### Behavior Changes
+* Remote compaction now falls back to local compaction when the primary cannot deserialize a successful `CompactionServiceResult` before installing any remote output files.
+* StringToMap (rocksdb/convenience.h) now preserves the outer braces of nested values so each map entry is in self-contained form and can be embedded directly into another `key=value;` string (e.g. SetOptions) without losing inner ';' delimiters. A new symmetric MapToString utility is provided.
+
+### Bug Fixes
+* Fixed a bug where `AbortAllCompactions()` could leave automatic compaction work unscheduled when aborting before the background worker picked it, causing compaction and write stalls until DB restart.
+* Fix a bug where db had a false positive compaction corruption error due to remote compaction service result with `has_accurate_num_input_records=false` not being serialized.
+* Fixed WritePrepared TransactionDB cleanup after retryable commit or rollback write failures so prepared transactions remain rollbackable instead of leaving unresolved prepared state.
+* Fix a rare corruption bug for tiered compaction that incorrectly moved last level range tombstones into proximal level. This corruption error is surfaced when force_consistency_checks is enabled.
+
+### Performance Improvements
+* Reduce the likelihood of user-facing metadata operations blocking on MANIFEST rotation when file creation is slow, such as on remote storage. MANIFEST write batches containing foreground edits from `CreateColumnFamily()`, `DropColumnFamily()`, `CreateColumnFamilyWithImport()`, `IngestExternalFile()`, and `DeleteFilesInRanges()` now get a relaxed file size threshold for triggering MANIFEST rotation; background-only MANIFEST write batches, such as compaction and flush, continue to use the normal threshold. This change should make auto-tuning manifest file size more attractive (see `max_manifest_file_size` and `max_manifest_space_amp_pct` options).
+
+
 ## 11.3.0 (05/15/2026)
 ### New Features
 * Add experimental DB option `async_wal_precreate` to precreate the next WAL file in a background thread and reduce foreground WAL rotation latency. The option is sanitized to false when WAL recycling is enabled.

Makefile

@@ -967,8 +967,6 @@ gen_parallel_tests:
 #      total time across many shards (need early queueing for fan-out).
 # Each alternative is wrapped in `^.*NAME.*$$` so the *whole input line* is
 # captured into $$1; then `s,(...),100 $$1,` prepends "100 " to the line.
-# Use a trailing dash on prefix-of-other-name binaries (e.g. `db_test-` to
-# avoid matching `db_test2-...`).
 #
 # Tiers below are based on observed timings (see suggest-slow-tests).
 # Tier 1: max single-shard time >= 30s (critical-path bottlenecks).
@@ -976,7 +974,7 @@ gen_parallel_tests:
 # Tier 3: huge total time across many tiny shards; front-loading them keeps
 #         the tail of the run busy while big shards finish.
 slow_test_regexp = \
-	^.*point_lock_manager_stress_test.*$$|^.*db_test-.*$$|^.*external_sst_file_test.*$$|^.*compaction_service_test.*$$|^.*corruption_test.*$$|^.*comparator_db_test.*$$|^.*external_sst_file_basic_test.*$$|^.*rate_limiter_test.*$$|^.*db_compaction_test.*$$|^.*write_prepared_transaction_test.*$$|^.*db_merge_operator_test.*$$|\
+	^.*point_lock_manager_stress_test.*$$|^.*db_test.*$$|^.*external_sst_file_test.*$$|^.*compaction_service_test.*$$|^.*corruption_test.*$$|^.*comparator_db_test.*$$|^.*external_sst_file_basic_test.*$$|^.*rate_limiter_test.*$$|^.*db_compaction_test.*$$|^.*write_prepared_transaction_test.*$$|^.*db_merge_operator_test.*$$|\
 	^.*db_dynamic_level_test.*$$|^.*db_bloom_filter_test.*$$|^.*error_handler_fs_test.*$$|^.*merge_helper_test.*$$|^.*transaction_test.*$$|^.*db_kv_checksum_test.*$$|^.*inlineskiplist_test.*$$|\
 	^.*db_with_timestamp_basic_test.*$$|^.*table_test.*$$|^.*db_wal_test.*$$|^.*block_based_table_reader_test.*$$|^.*block_test.*$$
 prioritize_long_running_tests =						\
@@ -1586,7 +1584,7 @@ db_open_with_config_test: $(OBJ_DIR)/db/db_open_with_config_test.o $(TEST_LIBRAR
 db_test: $(OBJ_DIR)/db/db_test.o $(TEST_LIBRARY) $(LIBRARY)
 	$(AM_LINK)
 
-db_test2: $(OBJ_DIR)/db/db_test2.o $(TEST_LIBRARY) $(LIBRARY)
+db_etc2_test: $(OBJ_DIR)/db/db_etc2_test.o $(TEST_LIBRARY) $(LIBRARY)
 	$(AM_LINK)
 
 db_etc3_test: $(OBJ_DIR)/db/db_etc3_test.o $(TEST_LIBRARY) $(LIBRARY)

claude_md/add_option.md

@@ -46,7 +46,7 @@ This document provides guidance on how to add new options to RocksDB's public AP
 
 ## Pattern 1: Adding a Standard Column Family Option
 
-Example reference: commit `94e65a2e0b4f817aa4bfa4c96cdf867e7980d7bc` (memtable_veirfy_per_key_checksum_on_seek)
+Example reference: commit `94e65a2e0b4f817aa4bfa4c96cdf867e7980d7bc` (memtable_verify_per_key_checksum_on_seek)
 
 ### Step 1: Define the Option in Public Header
 
@@ -63,7 +63,7 @@ Add the option with documentation in `AdvancedColumnFamilyOptions` struct:
 // operation.
 // This option depends on memtable_protection_bytes_per_key to be non zero.
 // If memtable_protection_bytes_per_key is zero, no validation is performed.
-bool memtable_veirfy_per_key_checksum_on_seek = false;
+bool memtable_verify_per_key_checksum_on_seek = false;
 ```
 
 ### Step 2: Add to Internal Options Structs
@@ -74,14 +74,14 @@ Add to `MutableCFOptions` struct (or `ImmutableCFOptions` for immutable options)
 
 ```cpp
 // In MutableCFOptions constructor from Options:
-memtable_veirfy_per_key_checksum_on_seek(
-    options.memtable_veirfy_per_key_checksum_on_seek),
+memtable_verify_per_key_checksum_on_seek(
+    options.memtable_verify_per_key_checksum_on_seek),
 
 // In MutableCFOptions default constructor:
-memtable_veirfy_per_key_checksum_on_seek(false),
+memtable_verify_per_key_checksum_on_seek(false),
 
 // In MutableCFOptions struct member declarations:
-bool memtable_veirfy_per_key_checksum_on_seek;
+bool memtable_verify_per_key_checksum_on_seek;
 ```
 
 ### Step 3: Register for Serialization/Deserialization
@@ -91,9 +91,9 @@ bool memtable_veirfy_per_key_checksum_on_seek;
 Add to the options type info map for serialization:
 
 ```cpp
-{"memtable_veirfy_per_key_checksum_on_seek",
+{"memtable_verify_per_key_checksum_on_seek",
  {offsetof(struct MutableCFOptions,
-           memtable_veirfy_per_key_checksum_on_seek),
+           memtable_verify_per_key_checksum_on_seek),
   OptionType::kBoolean, OptionVerificationType::kNormal,
   OptionTypeFlags::kMutable}},
 ```
@@ -101,8 +101,8 @@ Add to the options type info map for serialization:
 Add logging in `MutableCFOptions::Dump()`:
 
 ```cpp
-ROCKS_LOG_INFO(log, "memtable_veirfy_per_key_checksum_on_seek: %d",
-               memtable_veirfy_per_key_checksum_on_seek);
+ROCKS_LOG_INFO(log, "memtable_verify_per_key_checksum_on_seek: %d",
+               memtable_verify_per_key_checksum_on_seek);
 ```
 
 ### Step 4: Update Options Helper
@@ -112,8 +112,8 @@ ROCKS_LOG_INFO(log, "memtable_veirfy_per_key_checksum_on_seek: %d",
 Add to `UpdateColumnFamilyOptions()`:
 
 ```cpp
-cf_opts->memtable_veirfy_per_key_checksum_on_seek =
-    moptions.memtable_veirfy_per_key_checksum_on_seek;
+cf_opts->memtable_verify_per_key_checksum_on_seek =
+    moptions.memtable_verify_per_key_checksum_on_seek;
 ```
 
 ### Step 5: Add to Options Settable Test
@@ -123,7 +123,7 @@ cf_opts->memtable_veirfy_per_key_checksum_on_seek =
 Add to the test string in `ColumnFamilyOptionsAllFieldsSettable`:
 
 ```cpp
-"memtable_veirfy_per_key_checksum_on_seek=1;"
+"memtable_verify_per_key_checksum_on_seek=1;"
 ```
 
 ### Step 6: Add db_stress Support
@@ -131,23 +131,23 @@ Add to the test string in `ColumnFamilyOptionsAllFieldsSettable`:
 **File: `db_stress_tool/db_stress_common.h`**
 
 ```cpp
-DECLARE_bool(memtable_veirfy_per_key_checksum_on_seek);
+DECLARE_bool(memtable_verify_per_key_checksum_on_seek);
 ```
 
 **File: `db_stress_tool/db_stress_gflags.cc`**
 
 ```cpp
 DEFINE_bool(
-    memtable_veirfy_per_key_checksum_on_seek,
-    ROCKSDB_NAMESPACE::Options().memtable_veirfy_per_key_checksum_on_seek,
-    "Sets CF option memtable_veirfy_per_key_checksum_on_seek.");
+    memtable_verify_per_key_checksum_on_seek,
+    ROCKSDB_NAMESPACE::Options().memtable_verify_per_key_checksum_on_seek,
+    "Sets CF option memtable_verify_per_key_checksum_on_seek.");
 ```
 
 **File: `db_stress_tool/db_stress_test_base.cc`**
 
 ```cpp
-options.memtable_veirfy_per_key_checksum_on_seek =
-    FLAGS_memtable_veirfy_per_key_checksum_on_seek;
+options.memtable_verify_per_key_checksum_on_seek =
+    FLAGS_memtable_verify_per_key_checksum_on_seek;
 ```
 
 ### Step 7: Add db_bench Support
@@ -156,12 +156,12 @@ options.memtable_veirfy_per_key_checksum_on_seek =
 
 ```cpp
 // Flag definition (near related flags):
-DEFINE_bool(memtable_veirfy_per_key_checksum_on_seek, false,
-            "Sets CF option memtable_veirfy_per_key_checksum_on_seek");
+DEFINE_bool(memtable_verify_per_key_checksum_on_seek, false,
+            "Sets CF option memtable_verify_per_key_checksum_on_seek");
 
 // Apply flag to options (in InitializeOptionsFromFlags or similar):
-options.memtable_veirfy_per_key_checksum_on_seek =
-    FLAGS_memtable_veirfy_per_key_checksum_on_seek;
+options.memtable_verify_per_key_checksum_on_seek =
+    FLAGS_memtable_verify_per_key_checksum_on_seek;
 ```
 
 ### Step 8: Add Crash Test Support
@@ -169,7 +169,7 @@ options.memtable_veirfy_per_key_checksum_on_seek =
 **File: `tools/db_crashtest.py`**
 
 ```python
-"memtable_veirfy_per_key_checksum_on_seek": lambda: random.choice([0] * 7 + [1]),
+"memtable_verify_per_key_checksum_on_seek": lambda: random.choice([0] * 7 + [1]),
 ```
 
 Also add constraint handling in `finalize_and_sanitize()` if needed:
@@ -177,7 +177,7 @@ Also add constraint handling in `finalize_and_sanitize()` if needed:
 ```python
 # only skip list memtable representation supports paranoid memory checks
 if dest_params.get("memtablerep") != "skip_list":
-    dest_params["memtable_veirfy_per_key_checksum_on_seek"] = 0
+    dest_params["memtable_verify_per_key_checksum_on_seek"] = 0
 ```
 
 ### Step 9: Add Release Note
@@ -185,7 +185,7 @@ if dest_params.get("memtablerep") != "skip_list":
 **File: `unreleased_history/new_features/<descriptive_name>.md`**
 
 ```markdown
-A new flag memtable_veirfy_per_key_checksum_on_seek is added to AdvancedColumnFamilyOptions. When it is enabled, it will validate key checksum along the binary search path on skiplist based memtable during seek operation.
+A new flag memtable_verify_per_key_checksum_on_seek is added to AdvancedColumnFamilyOptions. When it is enabled, it will validate key checksum along the binary search path on skiplist based memtable during seek operation.
 ```
 
 ---

db/arena_wrapped_db_iter.cc

@@ -30,13 +30,78 @@ inline static SequenceNumber GetSeqNum(const DBImpl* db, const Snapshot* s) {
 Status ArenaWrappedDBIter::GetProperty(std::string prop_name,
                                        std::string* prop) {
   if (prop_name == "rocksdb.iterator.super-version-number") {
+    if (!internal_iter_initialized_) {
+      *prop = std::to_string(sv_number_);
+      return Status::OK();
+    }
     // First try to pass the value returned from inner iterator.
     if (!db_iter_->GetProperty(prop_name, prop).ok()) {
       *prop = std::to_string(sv_number_);
     }
     return Status::OK();
   }
-  return db_iter_->GetProperty(prop_name, prop);
+  if (!EnsureInternalIteratorInitialized(nullptr).ok()) {
+    return db_iter_->status();
+  }
+  return db_iter_->GetProperty(std::move(prop_name), prop);
+}
+
+void ArenaWrappedDBIter::CleanupDeferredSuperVersion() {
+  if (deferred_sv_ != nullptr) {
+    assert(db_impl_ != nullptr);
+    db_impl_->CleanupIteratorSuperVersion(
+        deferred_sv_, read_options_.background_purge_on_iterator_cleanup);
+    deferred_sv_ = nullptr;
+  }
+}
+
+void ArenaWrappedDBIter::DestroyDBIter() {
+  db_iter_->~DBIter();
+  CleanupDeferredSuperVersion();
+}
+
+void ArenaWrappedDBIter::ColumnFamilyDataUnrefDeleter::operator()(
+    ColumnFamilyData* cfd) const {
+  if (cfd == nullptr) {
+    return;
+  }
+  assert(db_impl != nullptr);
+
+  InstrumentedMutexLock lock(db_impl->mutex());
+  cfd->UnrefAndTryDelete();
+}
+
+void ArenaWrappedDBIter::DestroyDBIterAndArena() {
+  DestroyDBIter();
+  arena_.~Arena();
+}
+
+Status ArenaWrappedDBIter::EnsureInternalIteratorInitialized(
+    const MultiScanArgs* scan_opts) {
+  if (internal_iter_initialized_) {
+    return Status::OK();
+  }
+  if (db_impl_ == nullptr || deferred_cfd_ == nullptr ||
+      deferred_sv_ == nullptr) {
+    Status s = Status::InvalidArgument(
+        "Internal iterator cannot be initialized without deferred DB state");
+    db_iter_->set_status(s);
+    db_iter_->set_valid(false);
+    return s;
+  }
+
+  const MultiScanArgs* pruning_scan_opts =
+      scan_opts != nullptr && scan_opts->HasBoundedScanRanges() ? scan_opts
+                                                                : nullptr;
+  child_read_options_ = read_options_;
+  child_read_options_.snapshot = nullptr;
+  InternalIterator* internal_iter = db_impl_->NewInternalIterator(
+      child_read_options_, deferred_cfd_, deferred_sv_, &arena_, sequence_,
+      /*allow_unprepared_value=*/true, this, pruning_scan_opts);
+  deferred_cfd_ = nullptr;
+  deferred_sv_ = nullptr;
+  SetIterUnderDBIterImpl(internal_iter);
+  return Status::OK();
 }
 
 void ArenaWrappedDBIter::Init(
@@ -44,18 +109,26 @@ void ArenaWrappedDBIter::Init(
     const MutableCFOptions& mutable_cf_options, const Version* version,
     const SequenceNumber& sequence, uint64_t version_number,
     ReadCallback* read_callback, ColumnFamilyHandleImpl* cfh,
-    bool expose_blob_index, bool allow_refresh, ReadOnlyMemTable* active_mem) {
+    bool expose_blob_index, bool allow_refresh, ReadOnlyMemTable* active_mem,
+    DBImpl* db_impl, ColumnFamilyData* cfd) {
   read_options_ = read_options;
+  child_read_options_ = read_options;
   if (!CheckFSFeatureSupport(env->GetFileSystem().get(),
                              FSSupportedOps::kAsyncIO)) {
     read_options_.async_io = false;
   }
   read_options_.total_order_seek |= ioptions.prefix_seek_opt_in_only;
 
-  db_iter_ = DBIter::NewIter(
-      env, read_options_, ioptions, mutable_cf_options,
-      ioptions.user_comparator, /*internal_iter=*/nullptr, version, sequence,
-      read_callback, active_mem, cfh, expose_blob_index, &arena_);
+  if (cfh != nullptr) {
+    db_impl = cfh->db();
+    cfd = cfh->cfd();
+  }
+
+  db_iter_ = DBIter::NewIter(env, read_options_, ioptions, mutable_cf_options,
+                             ioptions.user_comparator,
+                             /*internal_iter=*/nullptr, version, sequence,
+                             read_callback, active_mem, /*cfh=*/nullptr,
+                             expose_blob_index, &arena_, db_impl, cfd);
 
   sv_number_ = version_number;
   allow_refresh_ = allow_refresh;
@@ -65,13 +138,13 @@ void ArenaWrappedDBIter::Init(
 
 void ArenaWrappedDBIter::MaybeAutoRefresh(bool is_seek,
                                           DBIter::Direction direction) {
-  if (cfh_ != nullptr && read_options_.snapshot != nullptr && allow_refresh_ &&
-      read_options_.auto_refresh_iterator_with_snapshot) {
+  if (cfd_ref_ != nullptr && read_options_.snapshot != nullptr &&
+      allow_refresh_ && read_options_.auto_refresh_iterator_with_snapshot) {
     // The intent here is to capture the superversion number change
     // reasonably soon from the time it actually happened. As such,
     // we're fine with weaker synchronization / ordering guarantees
     // provided by relaxed atomic (in favor of less CPU / mem overhead).
-    uint64_t cur_sv_number = cfh_->cfd()->GetSuperVersionNumberRelaxed();
+    uint64_t cur_sv_number = cfd_ref_->GetSuperVersionNumberRelaxed();
     if ((sv_number_ != cur_sv_number) && status().ok()) {
       // Changing iterators' direction is pretty heavy-weight operation and
       // could have unintended consequences when it comes to prefix seek.
@@ -150,13 +223,13 @@ void ArenaWrappedDBIter::DoRefresh(const Snapshot* snapshot,
   // present in the error log, but won't be reflected in the iterator status.
   // This is by design as we expect compaction to clean up those obsolete files
   // eventually.
-  db_iter_->~DBIter();
-
-  arena_.~Arena();
+  DestroyDBIterAndArena();
   new (&arena_) Arena();
 
-  auto cfd = cfh_->cfd();
-  auto db_impl = cfh_->db();
+  auto cfd = cfd_ref_.get();
+  auto db_impl = db_impl_;
+  assert(cfd != nullptr);
+  assert(db_impl != nullptr);
 
   SuperVersion* sv = cfd->GetReferencedSuperVersion(db_impl);
   assert(sv->version_number >= sv_number);
@@ -164,23 +237,27 @@ void ArenaWrappedDBIter::DoRefresh(const Snapshot* snapshot,
   if (read_callback_) {
     read_callback_->Refresh(read_seq);
   }
+  // TODO: Preserve Prepare() scan options across Refresh() so a refreshed
+  // MultiScan iterator can rebuild the same pruned tree.
   Init(env, read_options_, cfd->ioptions(), sv->mutable_cf_options, sv->current,
-       read_seq, sv->version_number, read_callback_, cfh_, expose_blob_index_,
-       allow_refresh_, allow_mark_memtable_for_flush_ ? sv->mem : nullptr);
+       read_seq, sv->version_number, read_callback_, nullptr,
+       expose_blob_index_, allow_refresh_,
+       allow_mark_memtable_for_flush_ ? sv->mem : nullptr, db_impl, cfd);
 
   InternalIterator* internal_iter = db_impl->NewInternalIterator(
       read_options_, cfd, sv, &arena_, read_seq,
       /* allow_unprepared_value */ true, /* db_iter */ this);
   SetIterUnderDBIter(internal_iter);
+  internal_iter_initialized_ = true;
 }
 
 Status ArenaWrappedDBIter::Refresh(const Snapshot* snapshot) {
-  if (cfh_ == nullptr || !allow_refresh_) {
+  if (cfd_ref_ == nullptr || db_impl_ == nullptr || !allow_refresh_) {
     return Status::NotSupported("Creating renew iterator is not allowed.");
   }
   assert(db_iter_ != nullptr);
-  auto cfd = cfh_->cfd();
-  auto db_impl = cfh_->db();
+  auto cfd = cfd_ref_.get();
+  auto db_impl = db_impl_;
 
   // TODO(yiwu): For last_seq_same_as_publish_seq_==false, this is not the
   // correct behavior. Will be corrected automatically when we take a snapshot
@@ -193,6 +270,13 @@ Status ArenaWrappedDBIter::Refresh(const Snapshot* snapshot) {
   TEST_SYNC_POINT("ArenaWrappedDBIter::Refresh:1");
   TEST_SYNC_POINT("ArenaWrappedDBIter::Refresh:2");
 
+  if (!internal_iter_initialized_) {
+    Status s = EnsureInternalIteratorInitialized(nullptr);
+    if (!s.ok()) {
+      return s;
+    }
+  }
+
   while (true) {
     if (sv_number_ != cur_sv_number) {
       DoRefresh(snapshot, cur_sv_number);
@@ -250,25 +334,47 @@ Status ArenaWrappedDBIter::Refresh(const Snapshot* snapshot) {
   return Status::OK();
 }
 
+void ArenaWrappedDBIter::Prepare(const MultiScanArgs& scan_opts) {
+  if (prepare_called_) {
+    db_iter_->set_status(Status::InvalidArgument(
+        "Prepare called more than once on the same iterator"));
+    db_iter_->set_valid(false);
+    return;
+  }
+  prepare_called_ = true;
+
+  Status s = db_iter_->SetScanOptionsForPrepare(scan_opts);
+  if (!s.ok()) {
+    return;
+  }
+
+  const MultiScanArgs* pruning_scan_opts =
+      scan_opts.HasBoundedScanRanges() ? &scan_opts : nullptr;
+  s = EnsureInternalIteratorInitialized(pruning_scan_opts);
+  if (!s.ok()) {
+    return;
+  }
+
+  db_iter_->PrepareInternalChildren();
+}
+
 ArenaWrappedDBIter* NewArenaWrappedDbIterator(
     Env* env, const ReadOptions& read_options, ColumnFamilyHandleImpl* cfh,
     SuperVersion* sv, const SequenceNumber& sequence,
     ReadCallback* read_callback, DBImpl* db_impl, bool expose_blob_index,
     bool allow_refresh, bool allow_mark_memtable_for_flush) {
   ArenaWrappedDBIter* db_iter = new ArenaWrappedDBIter();
+  ColumnFamilyData* cfd = cfh->cfd();
   db_iter->Init(env, read_options, cfh->cfd()->ioptions(),
                 sv->mutable_cf_options, sv->current, sequence,
                 sv->version_number, read_callback, cfh, expose_blob_index,
                 allow_refresh,
                 allow_mark_memtable_for_flush ? sv->mem : nullptr);
-  if (cfh != nullptr && allow_refresh) {
-    db_iter->StoreRefreshInfo(cfh, read_callback, expose_blob_index);
+  if (allow_refresh && cfd != nullptr) {
+    db_iter->StoreRefreshInfo(read_callback, expose_blob_index);
   }
-
-  InternalIterator* internal_iter = db_impl->NewInternalIterator(
-      db_iter->GetReadOptions(), cfh->cfd(), sv, db_iter->GetArena(), sequence,
-      /*allow_unprepared_value=*/true, db_iter);
-  db_iter->SetIterUnderDBIter(internal_iter);
+  db_iter->StoreDeferredInitInfo(db_impl, cfd, sv, sequence,
+                                 allow_mark_memtable_for_flush);
 
   return db_iter;
 }

db/arena_wrapped_db_iter.h

@@ -10,6 +10,7 @@
 #pragma once
 #include <stdint.h>
 
+#include <memory>
 #include <string>
 
 #include "db/db_impl/db_impl.h"
@@ -33,10 +34,17 @@ class Version;
 // When using the class's Iterator interface, the behavior is exactly
 // the same as the inner DBIter.
 class ArenaWrappedDBIter : public Iterator {
+  struct ColumnFamilyDataUnrefDeleter {
+    DBImpl* db_impl = nullptr;
+    void operator()(ColumnFamilyData* cfd) const;
+  };
+  using ColumnFamilyDataRef =
+      std::unique_ptr<ColumnFamilyData, ColumnFamilyDataUnrefDeleter>;
+
  public:
   ~ArenaWrappedDBIter() override {
     if (db_iter_ != nullptr) {
-      db_iter_->~DBIter();
+      DestroyDBIter();
     } else {
       assert(false);
     }
@@ -51,7 +59,7 @@ class ArenaWrappedDBIter : public Iterator {
   // Set the internal iterator wrapped inside the DB Iterator. Usually it is
   // a merging iterator.
   virtual void SetIterUnderDBIter(InternalIterator* iter) {
-    db_iter_->SetIter(iter);
+    SetIterUnderDBIterImpl(iter);
   }
 
   void SetMemtableRangetombstoneIter(
@@ -60,26 +68,48 @@ class ArenaWrappedDBIter : public Iterator {
   }
 
   bool Valid() const override { return db_iter_->Valid(); }
-  void SeekToFirst() override { db_iter_->SeekToFirst(); }
-  void SeekToLast() override { db_iter_->SeekToLast(); }
+  void SeekToFirst() override {
+    if (!EnsureInternalIteratorInitialized(nullptr).ok()) {
+      return;
+    }
+    db_iter_->SeekToFirst();
+  }
+  void SeekToLast() override {
+    if (!EnsureInternalIteratorInitialized(nullptr).ok()) {
+      return;
+    }
+    db_iter_->SeekToLast();
+  }
   // 'target' does not contain timestamp, even if user timestamp feature is
   // enabled.
   void Seek(const Slice& target) override {
+    if (!EnsureInternalIteratorInitialized(nullptr).ok()) {
+      return;
+    }
     MaybeAutoRefresh(true /* is_seek */, DBIter::kForward);
     db_iter_->Seek(target);
   }
 
   void SeekForPrev(const Slice& target) override {
+    if (!EnsureInternalIteratorInitialized(nullptr).ok()) {
+      return;
+    }
     MaybeAutoRefresh(true /* is_seek */, DBIter::kReverse);
     db_iter_->SeekForPrev(target);
   }
 
   void Next() override {
+    if (!EnsureInternalIteratorInitialized(nullptr).ok()) {
+      return;
+    }
     db_iter_->Next();
     MaybeAutoRefresh(false /* is_seek */, DBIter::kForward);
   }
 
   void Prev() override {
+    if (!EnsureInternalIteratorInitialized(nullptr).ok()) {
+      return;
+    }
     db_iter_->Prev();
     MaybeAutoRefresh(false /* is_seek */, DBIter::kReverse);
   }
@@ -96,12 +126,13 @@ class ArenaWrappedDBIter : public Iterator {
   Status Refresh() override;
   Status Refresh(const Snapshot*) override;
 
-  bool PrepareValue() override { return db_iter_->PrepareValue(); }
-
-  void Prepare(const MultiScanArgs& scan_opts) override {
-    db_iter_->Prepare(scan_opts);
+  bool PrepareValue() override {
+    return EnsureInternalIteratorInitialized(nullptr).ok() &&
+           db_iter_->PrepareValue();
   }
 
+  void Prepare(const MultiScanArgs& scan_opts) override;
+
   // FIXME: we could just pass SV in for mutable cf option, version and version
   // number, but this is used by SstFileReader which does not have a SV.
   void Init(Env* env, const ReadOptions& read_options,
@@ -110,27 +141,53 @@ class ArenaWrappedDBIter : public Iterator {
             const SequenceNumber& sequence, uint64_t version_number,
             ReadCallback* read_callback, ColumnFamilyHandleImpl* cfh,
             bool expose_blob_index, bool allow_refresh,
-            ReadOnlyMemTable* active_mem);
+            ReadOnlyMemTable* active_mem, DBImpl* db_impl = nullptr,
+            ColumnFamilyData* cfd = nullptr);
 
-  // Store some parameters so we can refresh the iterator at a later point
-  // with these same params
-  void StoreRefreshInfo(ColumnFamilyHandleImpl* cfh,
-                        ReadCallback* read_callback, bool expose_blob_index) {
-    cfh_ = cfh;
+  // Store parameters used only by explicit/auto-refresh.
+  void StoreRefreshInfo(ReadCallback* read_callback, bool expose_blob_index) {
     read_callback_ = read_callback;
     expose_blob_index_ = expose_blob_index;
   }
 
+  void StoreDeferredInitInfo(DBImpl* db_impl, ColumnFamilyData* cfd,
+                             SuperVersion* sv, const SequenceNumber& sequence,
+                             bool allow_mark_memtable_for_flush) {
+    assert(cfd != nullptr);
+    db_impl_ = db_impl;
+    cfd->Ref();
+    cfd_ref_ = ColumnFamilyDataRef(cfd, ColumnFamilyDataUnrefDeleter{db_impl});
+    deferred_cfd_ = cfd;
+    deferred_sv_ = sv;
+    sequence_ = sequence;
+    allow_mark_memtable_for_flush_ = allow_mark_memtable_for_flush;
+  }
+
  private:
+  Status EnsureInternalIteratorInitialized(const MultiScanArgs* scan_opts);
+  void SetIterUnderDBIterImpl(InternalIterator* iter) {
+    db_iter_->SetIter(iter);
+    internal_iter_initialized_ = true;
+  }
+  void CleanupDeferredSuperVersion();
+  void DestroyDBIter();
+  void DestroyDBIterAndArena();
   void DoRefresh(const Snapshot* snapshot, uint64_t sv_number);
   void MaybeAutoRefresh(bool is_seek, DBIter::Direction direction);
 
   DBIter* db_iter_ = nullptr;
   Arena arena_;
-  uint64_t sv_number_;
-  ColumnFamilyHandleImpl* cfh_ = nullptr;
+  uint64_t sv_number_ = 0;
+  DBImpl* db_impl_ = nullptr;
+  ColumnFamilyDataRef cfd_ref_{nullptr, ColumnFamilyDataUnrefDeleter{}};
+  ColumnFamilyData* deferred_cfd_ = nullptr;
+  SuperVersion* deferred_sv_ = nullptr;
+  SequenceNumber sequence_ = kMaxSequenceNumber;
+  bool internal_iter_initialized_ = false;
+  bool prepare_called_ = false;
   ReadOptions read_options_;
-  ReadCallback* read_callback_;
+  ReadOptions child_read_options_;
+  ReadCallback* read_callback_ = nullptr;
   bool expose_blob_index_ = false;
   bool allow_refresh_ = true;
   bool allow_mark_memtable_for_flush_ = true;

db/blob/blob_file_builder.cc

@@ -22,6 +22,7 @@
 #include "logging/logging.h"
 #include "options/cf_options.h"
 #include "options/options_helper.h"
+#include "rocksdb/file_system.h"
 #include "rocksdb/slice.h"
 #include "rocksdb/status.h"
 #include "test_util/sync_point.h"
@@ -200,6 +201,8 @@ Status BlobFileBuilder::OpenBlobFileIfNeeded() {
   {
     assert(file_options_);
     fo_copy = *file_options_;
+    fo_copy.open_contract = FileOpenContract::kNoReopenForWrite |
+                            FileOpenContract::kNoReadersWhileOpenForWrite;
     fo_copy.write_hint = write_hint_;
     Status s = NewWritableFile(fs_, blob_file_path, &file, fo_copy);
 

db/blob/blob_file_cache.cc

@@ -123,6 +123,7 @@ Status BlobFileCache::OpenBlobFileReaderUncached(
   Statistics* const statistics = immutable_options_->stats;
   RecordTick(statistics, NO_FILE_OPENS);
 
+  assert(file_options_);
   Status s = BlobFileReader::Create(
       *immutable_options_, read_options, *file_options_, column_family_id_,
       blob_file_read_hist_, blob_file_number, io_tracer_,

db/blob/blob_file_partition_manager.cc

@@ -24,6 +24,7 @@
 #include "file/read_write_util.h"
 #include "file/writable_file_writer.h"
 #include "logging/logging.h"
+#include "rocksdb/file_system.h"
 #include "util/aligned_buffer.h"
 #include "util/compression.h"
 #include "util/mutexlock.h"
@@ -199,7 +200,9 @@ Status BlobFilePartitionManager::OpenNewBlobFile(Partition* partition,
   }
 
   std::unique_ptr<FSWritableFile> file;
-  Status s = NewWritableFile(fs_, blob_file_path, &file, file_options_);
+  FileOptions writer_file_options = file_options_;
+  writer_file_options.open_contract = FileOpenContract::kNoReopenForWrite;
+  Status s = NewWritableFile(fs_, blob_file_path, &file, writer_file_options);
   if (!s.ok()) {
     RemoveFilePartitionMapping(blob_file_number);
     return s;
@@ -208,7 +211,7 @@ Status BlobFilePartitionManager::OpenNewBlobFile(Partition* partition,
   const bool perform_data_verification =
       checksum_handoff_file_types_.Contains(FileType::kBlobFile);
   auto file_writer = std::make_unique<WritableFileWriter>(
-      std::move(file), blob_file_path, file_options_, clock_, io_tracer_,
+      std::move(file), blob_file_path, writer_file_options, clock_, io_tracer_,
       statistics_, Histograms::BLOB_DB_BLOB_FILE_WRITE_MICROS, listeners_,
       file_checksum_gen_factory_, perform_data_verification);
 

db/blob/blob_file_reader.cc

@@ -12,6 +12,7 @@
 #include "db/blob/blob_log_format.h"
 #include "file/file_prefetch_buffer.h"
 #include "file/filename.h"
+#include "file/read_write_util.h"
 #include "monitoring/statistics_impl.h"
 #include "options/cf_options.h"
 #include "rocksdb/file_system.h"
@@ -20,6 +21,7 @@
 #include "table/format.h"
 #include "table/multiget_context.h"
 #include "test_util/sync_point.h"
+#include "util/aligned_buffer.h"
 #include "util/compression.h"
 #include "util/stop_watch.h"
 
@@ -105,24 +107,6 @@ Status BlobFileReader::OpenFile(
 
   constexpr IODebugContext* dbg = nullptr;
 
-  {
-    TEST_SYNC_POINT("BlobFileReader::OpenFile:GetFileSize");
-
-    const Status s =
-        fs->GetFileSize(blob_file_path, IOOptions(), file_size, dbg);
-    if (!s.ok()) {
-      return s;
-    }
-  }
-
-  if (!skip_footer_size_check &&
-      *file_size < BlobLogHeader::kSize + BlobLogFooter::kSize) {
-    return Status::Corruption("Malformed blob file");
-  }
-  if (skip_footer_size_check && *file_size < BlobLogHeader::kSize) {
-    return Status::Corruption("Malformed blob file");
-  }
-
   std::unique_ptr<FSRandomAccessFile> file;
   FileOptions reader_file_opts = file_opts;
 
@@ -142,6 +126,24 @@ Status BlobFileReader::OpenFile(
 
   assert(file);
 
+  {
+    Status s = GetFileSizeFromOpenFileOrPath(
+        file.get(), fs, blob_file_path, file_size, dbg,
+        FileSizeFallback::kNotSupportedOnly,
+        []() { TEST_SYNC_POINT("BlobFileReader::OpenFile:GetFileSize"); });
+    if (!s.ok()) {
+      return s;
+    }
+  }
+
+  if (!skip_footer_size_check &&
+      *file_size < BlobLogHeader::kSize + BlobLogFooter::kSize) {
+    return Status::Corruption("Malformed blob file");
+  }
+  if (skip_footer_size_check && *file_size < BlobLogHeader::kSize) {
+    return Status::Corruption("Malformed blob file");
+  }
+
   if (immutable_options.advise_random_on_open) {
     file->Hint(FSRandomAccessFile::kRandom);
   }
@@ -165,7 +167,7 @@ Status BlobFileReader::ReadHeader(const RandomAccessFileReader* file_reader,
 
   Slice header_slice;
   Buffer buf;
-  AlignedBuf aligned_buf;
+  AlignedBuffer direct_io_buffer;
 
   {
     TEST_SYNC_POINT("BlobFileReader::ReadHeader:ReadFromFile");
@@ -175,7 +177,7 @@ Status BlobFileReader::ReadHeader(const RandomAccessFileReader* file_reader,
 
     const Status s =
         ReadFromFile(file_reader, read_options, read_offset, read_size,
-                     statistics, &header_slice, &buf, &aligned_buf);
+                     statistics, &header_slice, &buf, &direct_io_buffer);
     if (!s.ok()) {
       return s;
     }
@@ -216,7 +218,7 @@ Status BlobFileReader::ReadFooter(const RandomAccessFileReader* file_reader,
 
   Slice footer_slice;
   Buffer buf;
-  AlignedBuf aligned_buf;
+  AlignedBuffer direct_io_buffer;
 
   {
     TEST_SYNC_POINT("BlobFileReader::ReadFooter:ReadFromFile");
@@ -226,7 +228,7 @@ Status BlobFileReader::ReadFooter(const RandomAccessFileReader* file_reader,
 
     const Status s =
         ReadFromFile(file_reader, read_options, read_offset, read_size,
-                     statistics, &footer_slice, &buf, &aligned_buf);
+                     statistics, &footer_slice, &buf, &direct_io_buffer);
     if (!s.ok()) {
       return s;
     }
@@ -257,10 +259,11 @@ Status BlobFileReader::ReadFromFile(const RandomAccessFileReader* file_reader,
                                     const ReadOptions& read_options,
                                     uint64_t read_offset, size_t read_size,
                                     Statistics* statistics, Slice* slice,
-                                    Buffer* buf, AlignedBuf* aligned_buf) {
+                                    Buffer* buf,
+                                    AlignedBuffer* direct_io_buffer) {
   assert(slice);
   assert(buf);
-  assert(aligned_buf);
+  assert(direct_io_buffer);
 
   assert(file_reader);
 
@@ -277,15 +280,15 @@ Status BlobFileReader::ReadFromFile(const RandomAccessFileReader* file_reader,
 
   if (file_reader->use_direct_io()) {
     constexpr char* scratch = nullptr;
+    AlignedBufferAllocationContext direct_io_context{direct_io_buffer};
 
     s = file_reader->Read(io_options, read_offset, read_size, slice, scratch,
-                          aligned_buf, &dbg);
+                          &direct_io_context, &dbg);
   } else {
     buf->reset(new char[read_size]);
-    constexpr AlignedBuf* aligned_scratch = nullptr;
 
     s = file_reader->Read(io_options, read_offset, read_size, slice, buf->get(),
-                          aligned_scratch, &dbg);
+                          nullptr, &dbg);
   }
 
   if (!s.ok()) {
@@ -349,7 +352,7 @@ Status BlobFileReader::GetBlob(
 
   Slice record_slice;
   Buffer buf;
-  AlignedBuf aligned_buf;
+  AlignedBuffer direct_io_buffer;
 
   bool prefetched = false;
 
@@ -379,7 +382,7 @@ Status BlobFileReader::GetBlob(
     const Status s =
         ReadFromFile(file_reader_.get(), read_options, record_offset,
                      static_cast<size_t>(record_size), statistics_,
-                     &record_slice, &buf, &aligned_buf);
+                     &record_slice, &buf, &direct_io_buffer);
     if (!s.ok()) {
       return s;
     }
@@ -477,7 +480,7 @@ void BlobFileReader::MultiGetBlob(
   }
 
   Buffer buf;
-  AlignedBuf aligned_buf;
+  AlignedBuffer direct_io_buffer;
 
   Status s;
   bool direct_io = file_reader_->use_direct_io();
@@ -500,8 +503,9 @@ void BlobFileReader::MultiGetBlob(
   IODebugContext dbg;
   s = file_reader_->PrepareIOOptions(read_options, opts, &dbg);
   if (s.ok()) {
+    AlignedBufferAllocationContext direct_io_context{&direct_io_buffer};
     s = file_reader_->MultiRead(opts, read_reqs.data(), read_reqs.size(),
-                                direct_io ? &aligned_buf : nullptr, &dbg);
+                                &direct_io_context, &dbg);
   }
   if (!s.ok()) {
     for (auto& req : read_reqs) {

db/blob/blob_file_reader.h

@@ -108,7 +108,7 @@ class BlobFileReader {
                              const ReadOptions& read_options,
                              uint64_t read_offset, size_t read_size,
                              Statistics* statistics, Slice* slice, Buffer* buf,
-                             AlignedBuf* aligned_buf);
+                             AlignedBuffer* direct_io_buffer);
 
   static Status VerifyBlob(const Slice& record_slice, const Slice& user_key,
                            uint64_t value_size);

db/blob/blob_file_reader_test.cc

@@ -7,10 +7,12 @@
 
 #include <cassert>
 #include <string>
+#include <utility>
 
 #include "db/blob/blob_contents.h"
 #include "db/blob/blob_log_format.h"
 #include "db/blob/blob_log_writer.h"
+#include "env/composite_env_wrapper.h"
 #include "env/mock_env.h"
 #include "file/filename.h"
 #include "file/read_write_util.h"
@@ -136,6 +138,167 @@ class BlobFileReaderTest : public testing::Test {
   std::unique_ptr<Env> mock_env_;
 };
 
+namespace {
+
+// Returns a stale path-level size for one target blob file while leaving the
+// opened file handle untouched. This lets the test verify
+// BlobFileReader::Create prefers the opened handle's size when the path stat
+// lags behind.
+class StalePathSizeFileSystem : public FileSystemWrapper {
+ public:
+  static const char* kClassName() { return "StalePathSizeFileSystem"; }
+
+  StalePathSizeFileSystem(const std::shared_ptr<FileSystem>& target,
+                          std::string stale_path)
+      : FileSystemWrapper(target), stale_path_(std::move(stale_path)) {}
+
+  const char* Name() const override { return kClassName(); }
+
+  IOStatus GetFileSize(const std::string& fname, const IOOptions& options,
+                       uint64_t* file_size, IODebugContext* dbg) override {
+    if (fname == stale_path_) {
+      *file_size = 0;
+      return IOStatus::OK();
+    }
+    return FileSystemWrapper::GetFileSize(fname, options, file_size, dbg);
+  }
+
+ private:
+  const std::string stale_path_;
+};
+
+// Makes opened blob file handles fail GetFileSize() with a hard error so the
+// reader path can verify that such errors are propagated directly.
+class FailingOpenFileSizeRandomAccessFile
+    : public FSRandomAccessFileOwnerWrapper {
+ public:
+  explicit FailingOpenFileSizeRandomAccessFile(
+      std::unique_ptr<FSRandomAccessFile>&& target)
+      : FSRandomAccessFileOwnerWrapper(std::move(target)) {}
+
+  IOStatus GetFileSize(uint64_t* /*result*/) override {
+    return IOStatus::IOError("open file size failed");
+  }
+};
+
+// Makes opened blob file handles report GetFileSize() as unsupported so the
+// reader path exercises its path-level fallback logic.
+class NotSupportedOpenFileSizeRandomAccessFile
+    : public FSRandomAccessFileOwnerWrapper {
+ public:
+  explicit NotSupportedOpenFileSizeRandomAccessFile(
+      std::unique_ptr<FSRandomAccessFile>&& target)
+      : FSRandomAccessFileOwnerWrapper(std::move(target)) {}
+
+  IOStatus GetFileSize(uint64_t* /*result*/) override {
+    return IOStatus::NotSupported("open file size not supported");
+  }
+};
+
+class OpenFileSizeFileSystem : public FileSystemWrapper {
+ public:
+  OpenFileSizeFileSystem(const std::shared_ptr<FileSystem>& target,
+                         std::string target_path)
+      : FileSystemWrapper(target), target_path_(std::move(target_path)) {}
+
+  IOStatus NewRandomAccessFile(const std::string& fname,
+                               const FileOptions& options,
+                               std::unique_ptr<FSRandomAccessFile>* result,
+                               IODebugContext* dbg) override {
+    std::unique_ptr<FSRandomAccessFile> file;
+    IOStatus s =
+        FileSystemWrapper::NewRandomAccessFile(fname, options, &file, dbg);
+    if (!s.ok()) {
+      return IOStatus(std::move(s));
+    }
+    if (fname == target_path_) {
+      *result = WrapTargetFile(std::move(file));
+    } else {
+      *result = std::move(file);
+    }
+    return IOStatus::OK();
+  }
+
+ protected:
+  virtual std::unique_ptr<FSRandomAccessFile> WrapTargetFile(
+      std::unique_ptr<FSRandomAccessFile>&& file) = 0;
+
+ private:
+  const std::string target_path_;
+};
+
+// Wraps the target blob file in FailingOpenFileSizeRandomAccessFile while
+// leaving all other files alone.
+class FailingOpenFileSizeFileSystem : public OpenFileSizeFileSystem {
+ public:
+  static const char* kClassName() { return "FailingOpenFileSizeFileSystem"; }
+
+  FailingOpenFileSizeFileSystem(const std::shared_ptr<FileSystem>& target,
+                                std::string target_path)
+      : OpenFileSizeFileSystem(target, std::move(target_path)) {}
+
+  const char* Name() const override { return kClassName(); }
+
+ private:
+  std::unique_ptr<FSRandomAccessFile> WrapTargetFile(
+      std::unique_ptr<FSRandomAccessFile>&& file) override {
+    return std::unique_ptr<FSRandomAccessFile>(
+        new FailingOpenFileSizeRandomAccessFile(std::move(file)));
+  }
+};
+
+// Wraps the target blob file in NotSupportedOpenFileSizeRandomAccessFile while
+// leaving all other files alone.
+class NotSupportedOpenFileSizeFileSystem : public OpenFileSizeFileSystem {
+ public:
+  static const char* kClassName() {
+    return "NotSupportedOpenFileSizeFileSystem";
+  }
+
+  NotSupportedOpenFileSizeFileSystem(const std::shared_ptr<FileSystem>& target,
+                                     std::string target_path)
+      : OpenFileSizeFileSystem(target, std::move(target_path)) {}
+
+  const char* Name() const override { return kClassName(); }
+
+ private:
+  std::unique_ptr<FSRandomAccessFile> WrapTargetFile(
+      std::unique_ptr<FSRandomAccessFile>&& file) override {
+    return std::unique_ptr<FSRandomAccessFile>(
+        new NotSupportedOpenFileSizeRandomAccessFile(std::move(file)));
+  }
+};
+
+// Forces the target blob file onto the NotSupported open-handle-size branch and
+// then fails the fallback path-level GetFileSize() call.
+class FailingFallbackPathSizeFileSystem
+    : public NotSupportedOpenFileSizeFileSystem {
+ public:
+  static const char* kClassName() {
+    return "FailingFallbackPathSizeFileSystem";
+  }
+
+  FailingFallbackPathSizeFileSystem(const std::shared_ptr<FileSystem>& target,
+                                    std::string target_path)
+      : NotSupportedOpenFileSizeFileSystem(target, target_path),
+        target_path_(std::move(target_path)) {}
+
+  const char* Name() const override { return kClassName(); }
+
+  IOStatus GetFileSize(const std::string& fname, const IOOptions& options,
+                       uint64_t* file_size, IODebugContext* dbg) override {
+    if (fname == target_path_) {
+      return IOStatus::IOError("fallback path size failed");
+    }
+    return FileSystemWrapper::GetFileSize(fname, options, file_size, dbg);
+  }
+
+ private:
+  const std::string target_path_;
+};
+
+}  // namespace
+
 TEST_F(BlobFileReaderTest, CreateReaderAndGetBlob) {
   Options options;
   options.env = mock_env_.get();
@@ -428,6 +591,211 @@ TEST_F(BlobFileReaderTest, CreateReaderAndGetBlob) {
   }
 }
 
+// Verifies Create() prefers the file size reported by the opened handle over a
+// stale path-level size query by returning a readable reader even when the path
+// stat intentionally reports 0 for the blob file.
+TEST_F(BlobFileReaderTest, CreateReaderUsesOpenedFileSizeWhenPathSizeIsStale) {
+  const std::string db_path = test::PerThreadDBPath(
+      mock_env_.get(),
+      "BlobFileReaderTest_CreateReaderUsesOpenedFileSizeWhenPathSizeIsStale");
+
+  constexpr uint64_t blob_file_number = 1;
+  const std::string blob_file_path = BlobFileName(db_path, blob_file_number);
+
+  auto stale_size_fs = std::make_shared<StalePathSizeFileSystem>(
+      mock_env_->GetFileSystem(), blob_file_path);
+  CompositeEnvWrapper stale_size_env(mock_env_.get(), stale_size_fs);
+
+  Options options;
+  options.env = &stale_size_env;
+  options.cf_paths.emplace_back(db_path, 0);
+  options.enable_blob_files = true;
+
+  ImmutableOptions immutable_options(options);
+
+  constexpr uint32_t column_family_id = 1;
+  constexpr bool has_ttl = false;
+  constexpr ExpirationRange expiration_range;
+  constexpr char key[] = "key";
+  constexpr char blob[] = "blob";
+
+  uint64_t blob_offset = 0;
+  uint64_t blob_size = 0;
+  WriteBlobFile(immutable_options, column_family_id, has_ttl, expiration_range,
+                expiration_range, blob_file_number, key, blob, kNoCompression,
+                &blob_offset, &blob_size);
+
+  std::unique_ptr<BlobFileReader> reader;
+  ReadOptions read_options;
+  read_options.verify_checksums = false;
+  ASSERT_OK(BlobFileReader::Create(
+      immutable_options, read_options, FileOptions(), column_family_id,
+      /*blob_file_read_hist=*/nullptr, blob_file_number, nullptr /*IOTracer*/,
+      &reader));
+  ASSERT_NE(reader, nullptr);
+
+  std::unique_ptr<BlobContents> value;
+  uint64_t bytes_read = 0;
+  ASSERT_OK(reader->GetBlob(read_options, key, blob_offset, blob_size,
+                            kNoCompression,
+                            /*prefetch_buffer=*/nullptr,
+                            /*allocator=*/nullptr, &value, &bytes_read));
+  ASSERT_NE(value, nullptr);
+  ASSERT_EQ(value->data(), Slice(blob));
+  ASSERT_EQ(bytes_read, blob_size);
+}
+
+// Verifies Create() falls back to the path-level GetFileSize() call when the
+// opened file handle reports that size queries are unsupported.
+TEST_F(BlobFileReaderTest,
+       CreateReaderFallsBackToPathSizeWhenOpenedFileSizeNotSupported) {
+  const std::string db_path = test::PerThreadDBPath(
+      mock_env_.get(),
+      "BlobFileReaderTest_"
+      "CreateReaderFallsBackToPathSizeWhenOpenedFileSizeNotSupported");
+
+  constexpr uint64_t blob_file_number = 1;
+  const std::string blob_file_path = BlobFileName(db_path, blob_file_number);
+
+  auto not_supported_open_file_size_fs =
+      std::make_shared<NotSupportedOpenFileSizeFileSystem>(
+          mock_env_->GetFileSystem(), blob_file_path);
+  CompositeEnvWrapper not_supported_open_file_size_env(
+      mock_env_.get(), not_supported_open_file_size_fs);
+
+  Options options;
+  options.env = &not_supported_open_file_size_env;
+  options.cf_paths.emplace_back(db_path, 0);
+  options.enable_blob_files = true;
+
+  ImmutableOptions immutable_options(options);
+
+  constexpr uint32_t column_family_id = 1;
+  constexpr bool has_ttl = false;
+  constexpr ExpirationRange expiration_range;
+  constexpr char key[] = "key";
+  constexpr char blob[] = "blob";
+
+  uint64_t blob_offset = 0;
+  uint64_t blob_size = 0;
+  WriteBlobFile(immutable_options, column_family_id, has_ttl, expiration_range,
+                expiration_range, blob_file_number, key, blob, kNoCompression,
+                &blob_offset, &blob_size);
+
+  std::unique_ptr<BlobFileReader> reader;
+  ReadOptions read_options;
+  read_options.verify_checksums = false;
+  ASSERT_OK(BlobFileReader::Create(
+      immutable_options, read_options, FileOptions(), column_family_id,
+      /*blob_file_read_hist=*/nullptr, blob_file_number, nullptr /*IOTracer*/,
+      &reader));
+  ASSERT_NE(reader, nullptr);
+
+  std::unique_ptr<BlobContents> value;
+  uint64_t bytes_read = 0;
+  ASSERT_OK(reader->GetBlob(read_options, key, blob_offset, blob_size,
+                            kNoCompression,
+                            /*prefetch_buffer=*/nullptr,
+                            /*allocator=*/nullptr, &value, &bytes_read));
+  ASSERT_NE(value, nullptr);
+  ASSERT_EQ(value->data(), Slice(blob));
+  ASSERT_EQ(bytes_read, blob_size);
+}
+
+// Verifies Create() propagates a real error from the opened file handle's
+// GetFileSize() rather than masking it with any path-level fallback.
+TEST_F(BlobFileReaderTest, CreateReaderPropagatesOpenedFileSizeError) {
+  const std::string db_path = test::PerThreadDBPath(
+      mock_env_.get(),
+      "BlobFileReaderTest_CreateReaderPropagatesOpenedFileSizeError");
+
+  constexpr uint64_t blob_file_number = 1;
+  const std::string blob_file_path = BlobFileName(db_path, blob_file_number);
+
+  auto failing_open_file_size_fs =
+      std::make_shared<FailingOpenFileSizeFileSystem>(
+          mock_env_->GetFileSystem(), blob_file_path);
+  CompositeEnvWrapper failing_open_file_size_env(mock_env_.get(),
+                                                 failing_open_file_size_fs);
+
+  Options options;
+  options.env = &failing_open_file_size_env;
+  options.cf_paths.emplace_back(db_path, 0);
+  options.enable_blob_files = true;
+
+  ImmutableOptions immutable_options(options);
+
+  constexpr uint32_t column_family_id = 1;
+  constexpr bool has_ttl = false;
+  constexpr ExpirationRange expiration_range;
+  constexpr char key[] = "key";
+  constexpr char blob[] = "blob";
+
+  uint64_t blob_offset = 0;
+  uint64_t blob_size = 0;
+  WriteBlobFile(immutable_options, column_family_id, has_ttl, expiration_range,
+                expiration_range, blob_file_number, key, blob, kNoCompression,
+                &blob_offset, &blob_size);
+
+  std::unique_ptr<BlobFileReader> reader;
+  ReadOptions read_options;
+  Status s = BlobFileReader::Create(
+      immutable_options, read_options, FileOptions(), column_family_id,
+      /*blob_file_read_hist=*/nullptr, blob_file_number, nullptr /*IOTracer*/,
+      &reader);
+  ASSERT_TRUE(s.IsIOError());
+  ASSERT_EQ(reader, nullptr);
+}
+
+// Verifies Create() also propagates errors from the path-level fallback size
+// query when the opened file handle only reports NotSupported for GetFileSize.
+TEST_F(
+    BlobFileReaderTest,
+    CreateReaderPropagatesFallbackPathSizeErrorWhenOpenedFileSizeNotSupported) {
+  const std::string db_path =
+      test::PerThreadDBPath(mock_env_.get(),
+                            "BlobFileReaderTest_"
+                            "CreateReaderPropagatesFallbackPathSizeErrorWhenOpe"
+                            "nedFileSizeNotSupported");
+
+  constexpr uint64_t blob_file_number = 1;
+  const std::string blob_file_path = BlobFileName(db_path, blob_file_number);
+
+  auto failing_fallback_path_size_fs =
+      std::make_shared<FailingFallbackPathSizeFileSystem>(
+          mock_env_->GetFileSystem(), blob_file_path);
+  CompositeEnvWrapper failing_fallback_path_size_env(
+      mock_env_.get(), failing_fallback_path_size_fs);
+
+  Options options;
+  options.env = &failing_fallback_path_size_env;
+  options.cf_paths.emplace_back(db_path, 0);
+  options.enable_blob_files = true;
+
+  ImmutableOptions immutable_options(options);
+
+  constexpr uint32_t column_family_id = 1;
+  constexpr bool has_ttl = false;
+  constexpr ExpirationRange expiration_range;
+  constexpr char key[] = "key";
+  constexpr char blob[] = "blob";
+
+  uint64_t blob_offset = 0;
+  uint64_t blob_size = 0;
+  WriteBlobFile(immutable_options, column_family_id, has_ttl, expiration_range,
+                expiration_range, blob_file_number, key, blob, kNoCompression,
+                &blob_offset, &blob_size);
+
+  std::unique_ptr<BlobFileReader> reader;
+  ReadOptions read_options;
+  Status s = BlobFileReader::Create(
+      immutable_options, read_options, FileOptions(), column_family_id,
+      /*blob_file_read_hist=*/nullptr, blob_file_number, nullptr /*IOTracer*/,
+      &reader);
+  ASSERT_TRUE(s.IsIOError());
+  ASSERT_EQ(reader, nullptr);
+}
+
 TEST_F(BlobFileReaderTest, Malformed) {
   // Write a blob file consisting of nothing but a header, and make sure we
   // detect the error when we open it for reading
@@ -849,7 +1217,6 @@ class BlobFileReaderIOErrorTest
 
 INSTANTIATE_TEST_CASE_P(BlobFileReaderTest, BlobFileReaderIOErrorTest,
                         ::testing::ValuesIn(std::vector<std::string>{
-                            "BlobFileReader::OpenFile:GetFileSize",
                             "BlobFileReader::OpenFile:NewRandomAccessFile",
                             "BlobFileReader::ReadHeader:ReadFromFile",
                             "BlobFileReader::ReadFooter:ReadFromFile",

db/blob/db_blob_direct_write_test.cc

@@ -10,11 +10,14 @@
 
 #include <algorithm>
 #include <array>
+#include <functional>
 #include <limits>
 #include <mutex>
 #include <set>
 #include <sstream>
 #include <string>
+#include <unordered_set>
+#include <utility>
 #include <vector>
 
 #include "cache/compressed_secondary_cache.h"
@@ -26,10 +29,12 @@
 #include "db/db_test_util.h"
 #include "db/db_with_timestamp_test_util.h"
 #include "db/wide/wide_column_test_util.h"
+#include "env/composite_env_wrapper.h"
 #include "file/filename.h"
 #include "file/random_access_file_reader.h"
 #include "port/stack_trace.h"
 #include "rocksdb/convenience.h"
+#include "rocksdb/file_system.h"
 #include "rocksdb/trace_reader_writer.h"
 #include "rocksdb/trace_record.h"
 #include "rocksdb/utilities/replayer.h"
@@ -101,6 +106,206 @@ class RecordingBlobDirectWritePartitionStrategy
   std::vector<Call> calls_;
 };
 
+namespace {
+
+// Matches BlobDB file names so the test filesystem can special-case only active
+// blob files without changing unrelated file opens.
+bool IsBlobFilePath(const std::string& fname) {
+  const size_t basename_pos = fname.find_last_of("/\\");
+  const std::string basename = basename_pos == std::string::npos
+                                   ? fname
+                                   : fname.substr(basename_pos + 1);
+  uint64_t file_number = 0;
+  FileType file_type = kWalFile;
+  return ParseFileName(basename, &file_number, &file_type) &&
+         file_type == kBlobFile;
+}
+
+// Keeps track of when an active blob file stops being writer-visible so the
+// test filesystem can model current-size visibility only while the file remains
+// active.
+class ActiveBlobVisibilityWritableFile : public FSWritableFileOwnerWrapper {
+ public:
+  ActiveBlobVisibilityWritableFile(std::unique_ptr<FSWritableFile>&& target,
+                                   std::function<void()> on_close)
+      : FSWritableFileOwnerWrapper(std::move(target)),
+        on_close_(std::move(on_close)) {}
+
+  ~ActiveBlobVisibilityWritableFile() override { DeactivateOnce(); }
+
+  IOStatus Close(const IOOptions& options, IODebugContext* dbg) override {
+    IOStatus s = target()->Close(options, dbg);
+    DeactivateOnce();
+    return s;
+  }
+
+ private:
+  void DeactivateOnce() {
+    if (on_close_) {
+      on_close_();
+      on_close_ = nullptr;
+    }
+  }
+
+  std::function<void()> on_close_;
+};
+
+// Simulates a remote readable file whose GetFileSize() either exposes the live
+// on-disk size for default-contract active blobs or hides it for
+// append-only-no-readers files.
+class ActiveBlobVisibilityRandomAccessFile
+    : public FSRandomAccessFileOwnerWrapper {
+ public:
+  ActiveBlobVisibilityRandomAccessFile(
+      std::unique_ptr<FSRandomAccessFile>&& target, bool expose_current_size,
+      std::shared_ptr<FileSystem> underlying_fs, std::string fname)
+      : FSRandomAccessFileOwnerWrapper(std::move(target)),
+        expose_current_size_(expose_current_size),
+        underlying_fs_(std::move(underlying_fs)),
+        fname_(std::move(fname)) {}
+
+  IOStatus GetFileSize(uint64_t* result) override {
+    if (!expose_current_size_) {
+      *result = 0;
+      return IOStatus::OK();
+    }
+    // Delegate to the underlying FS path-level GetFileSize (bypassing the
+    // wrapper that reports 0 for active blobs) to simulate a remote FS that
+    // exposes the current file size through the open handle.
+    return underlying_fs_->GetFileSize(fname_, IOOptions(), result,
+                                       /*dbg=*/nullptr);
+  }
+
+ private:
+  const bool expose_current_size_;
+  std::shared_ptr<FileSystem> underlying_fs_;
+  std::string fname_;
+};
+
+// Models a remote filesystem where active direct-write blobs are only
+// reader-visible when the writer does not promise no concurrent readers.
+class RemoteBlobVisibilityFileSystem : public FileSystemWrapper {
+ public:
+  explicit RemoteBlobVisibilityFileSystem(const std::shared_ptr<FileSystem>& fs)
+      : FileSystemWrapper(fs) {}
+
+  static const char* kClassName() { return "RemoteBlobVisibilityFileSystem"; }
+  const char* Name() const override { return kClassName(); }
+
+  IOStatus NewWritableFile(const std::string& fname, const FileOptions& opts,
+                           std::unique_ptr<FSWritableFile>* result,
+                           IODebugContext* dbg) override {
+    std::unique_ptr<FSWritableFile> file;
+    IOStatus s = target()->NewWritableFile(fname, opts, &file, dbg);
+    if (!s.ok() || !IsBlobFilePath(fname)) {
+      *result = std::move(file);
+      return IOStatus(std::move(s));
+    }
+
+    const bool has_no_reopen_for_write_contract = HasFileOpenContract(
+        opts.open_contract, FileOpenContract::kNoReopenForWrite);
+    const bool has_no_readers_while_open_for_write_contract =
+        HasFileOpenContract(opts.open_contract,
+                            FileOpenContract::kNoReadersWhileOpenForWrite);
+    {
+      std::lock_guard<std::mutex> lock(mu_);
+      saw_no_reopen_for_write_writer_contract_ |=
+          has_no_reopen_for_write_contract;
+      saw_no_readers_while_open_for_write_writer_contract_ |=
+          has_no_readers_while_open_for_write_contract;
+      if (!has_no_readers_while_open_for_write_contract) {
+        active_blob_paths_.insert(fname);
+      }
+    }
+
+    if (has_no_readers_while_open_for_write_contract) {
+      *result = std::move(file);
+      return IOStatus::OK();
+    }
+
+    result->reset(new ActiveBlobVisibilityWritableFile(
+        std::move(file), [this, tracked_path = std::string(fname)]() {
+          DeactivateBlobPath(tracked_path);
+        }));
+    return IOStatus::OK();
+  }
+
+  IOStatus NewRandomAccessFile(const std::string& fname,
+                               const FileOptions& opts,
+                               std::unique_ptr<FSRandomAccessFile>* result,
+                               IODebugContext* dbg) override {
+    std::unique_ptr<FSRandomAccessFile> file;
+    IOStatus s = target()->NewRandomAccessFile(fname, opts, &file, dbg);
+    if (!s.ok() || !IsBlobFilePath(fname)) {
+      *result = std::move(file);
+      return IOStatus(std::move(s));
+    }
+
+    const bool has_no_readers_while_open_for_write_contract =
+        HasFileOpenContract(opts.open_contract,
+                            FileOpenContract::kNoReadersWhileOpenForWrite);
+    const bool active_blob = IsActiveBlobPath(fname);
+    {
+      std::lock_guard<std::mutex> lock(mu_);
+      saw_no_readers_while_open_for_write_reader_contract_ |=
+          has_no_readers_while_open_for_write_contract;
+    }
+
+    if (!active_blob) {
+      *result = std::move(file);
+      return IOStatus::OK();
+    }
+
+    result->reset(new ActiveBlobVisibilityRandomAccessFile(
+        std::move(file), !has_no_readers_while_open_for_write_contract, target_,
+        fname));
+    return IOStatus::OK();
+  }
+
+  IOStatus GetFileSize(const std::string& fname, const IOOptions& options,
+                       uint64_t* file_size, IODebugContext* dbg) override {
+    if (IsActiveBlobPath(fname)) {
+      *file_size = 0;
+      return IOStatus::OK();
+    }
+    return target()->GetFileSize(fname, options, file_size, dbg);
+  }
+
+  bool SawNoReopenForWriteWriterContract() const {
+    std::lock_guard<std::mutex> lock(mu_);
+    return saw_no_reopen_for_write_writer_contract_;
+  }
+
+  bool SawNoReadersWhileOpenForWriteWriterContract() const {
+    std::lock_guard<std::mutex> lock(mu_);
+    return saw_no_readers_while_open_for_write_writer_contract_;
+  }
+
+  bool SawNoReadersWhileOpenForWriteReaderContract() const {
+    std::lock_guard<std::mutex> lock(mu_);
+    return saw_no_readers_while_open_for_write_reader_contract_;
+  }
+
+ private:
+  void DeactivateBlobPath(const std::string& fname) {
+    std::lock_guard<std::mutex> lock(mu_);
+    active_blob_paths_.erase(fname);
+  }
+
+  bool IsActiveBlobPath(const std::string& fname) const {
+    std::lock_guard<std::mutex> lock(mu_);
+    return active_blob_paths_.find(fname) != active_blob_paths_.end();
+  }
+
+  mutable std::mutex mu_;
+  bool saw_no_reopen_for_write_writer_contract_ = false;
+  bool saw_no_readers_while_open_for_write_writer_contract_ = false;
+  bool saw_no_readers_while_open_for_write_reader_contract_ = false;
+  std::unordered_set<std::string> active_blob_paths_;
+};
+
+}  // namespace
+
 class DBBlobDirectWriteTest : public DBTestBase {
  protected:
   DBBlobDirectWriteTest()
@@ -816,6 +1021,30 @@ TEST_F(DBBlobDirectWriteTest, DirectWriteImmutableMemtableRead) {
   verify_reads();
 }
 
+// Verifies active blob direct-write files promise no write reopen without also
+// promising no concurrent readers.
+TEST_F(DBBlobDirectWriteTest, DirectWriteUsesReadableContractForRemoteFile) {
+  auto remote_fs =
+      std::make_shared<RemoteBlobVisibilityFileSystem>(env_->GetFileSystem());
+  std::unique_ptr<Env> remote_env(new CompositeEnvWrapper(env_, remote_fs));
+
+  Options options = GetDirectWriteOptions();
+  options.env = remote_env.get();
+  Reopen(options);
+
+  const std::string key = "remote_blob_key";
+  const std::string value(128, 'w');
+
+  ASSERT_OK(Put(key, value));
+  ASSERT_TRUE(remote_fs->SawNoReopenForWriteWriterContract());
+  ASSERT_FALSE(remote_fs->SawNoReadersWhileOpenForWriteWriterContract());
+
+  ASSERT_EQ(Get(key), value);
+  ASSERT_FALSE(remote_fs->SawNoReadersWhileOpenForWriteReaderContract());
+
+  Close();
+}
+
 TEST_F(DBBlobDirectWriteTest, DirectWriteRefreshesReaderAfterFlush) {
   Options options = GetDirectWriteOptions();
   options.blob_direct_write_partitions = 1;

db/builder.cc

@@ -179,6 +179,8 @@ Status BuildTable(
       TEST_SYNC_POINT_CALLBACK("BuildTable:create_file", &use_direct_writes);
 #endif  // !NDEBUG
       FileOptions fo_copy = file_options;
+      fo_copy.open_contract = FileOpenContract::kNoReopenForWrite |
+                              FileOpenContract::kNoReadersWhileOpenForWrite;
       fo_copy.write_hint = write_hint;
       IOStatus io_s = NewWritableFile(fs, fname, &file, fo_copy);
       assert(s.ok());

db/c.cc

@@ -148,6 +148,7 @@ using ROCKSDB_NAMESPACE::TransactionDB;
 using ROCKSDB_NAMESPACE::TransactionDBOptions;
 using ROCKSDB_NAMESPACE::TransactionLogIterator;
 using ROCKSDB_NAMESPACE::TransactionOptions;
+using ROCKSDB_NAMESPACE::TxnDBWritePolicy;
 using ROCKSDB_NAMESPACE::WaitForCompactOptions;
 using ROCKSDB_NAMESPACE::WALRecoveryMode;
 using ROCKSDB_NAMESPACE::WritableFile;
@@ -1384,6 +1385,10 @@ void rocksdb_backup_engine_info_destroy(
   delete info;
 }
 
+void rocksdb_backup_engine_stop_backup(rocksdb_backup_engine_t* be) {
+  be->rep->StopBackup();
+}
+
 void rocksdb_backup_engine_close(rocksdb_backup_engine_t* be) {
   delete be->rep;
   delete be;
@@ -1465,6 +1470,20 @@ uint64_t rocksdb_backup_engine_options_get_restore_rate_limit(
   return options->rep.restore_rate_limit;
 }
 
+void rocksdb_backup_engine_options_set_backup_rate_limiter(
+    rocksdb_backup_engine_options_t* options, rocksdb_ratelimiter_t* limiter) {
+  if (limiter) {
+    options->rep.backup_rate_limiter = limiter->rep;
+  }
+}
+
+void rocksdb_backup_engine_options_set_restore_rate_limiter(
+    rocksdb_backup_engine_options_t* options, rocksdb_ratelimiter_t* limiter) {
+  if (limiter) {
+    options->rep.restore_rate_limiter = limiter->rep;
+  }
+}
+
 void rocksdb_backup_engine_options_set_max_background_operations(
     rocksdb_backup_engine_options_t* options, int val) {
   options->rep.max_background_operations = val;
@@ -4247,6 +4266,15 @@ void rocksdb_set_options(rocksdb_t* db, int count, const char* const keys[],
   SaveError(errptr, db->rep->SetOptions(options_map));
 }
 
+void rocksdb_set_db_options(rocksdb_t* db, int count, const char* const keys[],
+                            const char* const values[], char** errptr) {
+  std::unordered_map<std::string, std::string> options_map;
+  for (int i = 0; i < count; i++) {
+    options_map[keys[i]] = values[i];
+  }
+  SaveError(errptr, db->rep->SetDBOptions(options_map));
+}
+
 void rocksdb_set_options_cf(rocksdb_t* db,
                             rocksdb_column_family_handle_t* handle, int count,
                             const char* const keys[],
@@ -5043,6 +5071,16 @@ unsigned char rocksdb_options_get_use_direct_io_for_flush_and_compaction(
   return opt->rep.use_direct_io_for_flush_and_compaction;
 }
 
+void rocksdb_options_set_use_direct_io_for_compaction_reads(
+    rocksdb_options_t* opt, unsigned char v) {
+  opt->rep.use_direct_io_for_compaction_reads = v;
+}
+
+unsigned char rocksdb_options_get_use_direct_io_for_compaction_reads(
+    rocksdb_options_t* opt) {
+  return opt->rep.use_direct_io_for_compaction_reads;
+}
+
 void rocksdb_options_set_allow_mmap_reads(rocksdb_options_t* opt,
                                           unsigned char v) {
   opt->rep.allow_mmap_reads = v;
@@ -7527,6 +7565,11 @@ void rocksdb_transactiondb_options_set_default_lock_timeout(
   opt->rep.default_lock_timeout = default_lock_timeout;
 }
 
+void rocksdb_transactiondb_options_set_write_policy(
+    rocksdb_transactiondb_options_t* opt, int write_policy) {
+  opt->rep.write_policy = static_cast<TxnDBWritePolicy>(write_policy);
+}
+
 void rocksdb_transactiondb_options_set_use_per_key_point_lock_mgr(
     rocksdb_transactiondb_options_t* opt, int use_per_key_point_lock_mgr) {
   opt->rep.use_per_key_point_lock_mgr = use_per_key_point_lock_mgr;

db/c_test.c

@@ -6,6 +6,7 @@
 #include "rocksdb/c.h"
 
 #include <assert.h>
+#include <inttypes.h>
 #include <stddef.h>
 #include <stdio.h>
 #include <stdlib.h>
@@ -14,7 +15,6 @@
 #ifndef OS_WIN
 #include <unistd.h>
 #endif
-#include <inttypes.h>
 
 // Can not use port/port.h macros as this is a c file
 #ifdef OS_WIN
@@ -32,6 +32,14 @@ int geteuid() {
 
 #endif
 
+static uint64_t GetTestId(void) {
+#ifdef OS_WIN
+  return (uint64_t)geteuid();
+#else
+  return ((uint64_t)geteuid() << 32) ^ (uint64_t)getpid();
+#endif
+}
+
 const char* phase = "";
 static char dbname[200];
 static char sstfilename[200];
@@ -848,20 +856,21 @@ int main(int argc, char** argv) {
   char* err = NULL;
   int run = -1;
 
-  snprintf(dbname, sizeof(dbname), "%s/rocksdb_c_test-%d", GetTempDir(),
-           ((int)geteuid()));
+  snprintf(dbname, sizeof(dbname), "%s/rocksdb_c_test-%" PRIu64, GetTempDir(),
+           GetTestId());
 
-  snprintf(dbbackupname, sizeof(dbbackupname), "%s/rocksdb_c_test-%d-backup",
-           GetTempDir(), ((int)geteuid()));
+  snprintf(dbbackupname, sizeof(dbbackupname),
+           "%s/rocksdb_c_test-%" PRIu64 "-backup", GetTempDir(), GetTestId());
 
   snprintf(dbcheckpointname, sizeof(dbcheckpointname),
-           "%s/rocksdb_c_test-%d-checkpoint", GetTempDir(), ((int)geteuid()));
+           "%s/rocksdb_c_test-%" PRIu64 "-checkpoint", GetTempDir(),
+           GetTestId());
 
-  snprintf(sstfilename, sizeof(sstfilename), "%s/rocksdb_c_test-%d-sst",
-           GetTempDir(), ((int)geteuid()));
+  snprintf(sstfilename, sizeof(sstfilename),
+           "%s/rocksdb_c_test-%" PRIu64 "-sst", GetTempDir(), GetTestId());
 
-  snprintf(dbpathname, sizeof(dbpathname), "%s/rocksdb_c_test-%d-dbpath",
-           GetTempDir(), ((int)geteuid()));
+  snprintf(dbpathname, sizeof(dbpathname),
+           "%s/rocksdb_c_test-%" PRIu64 "-dbpath", GetTempDir(), GetTestId());
 
   StartPhase("create_objects");
   cmp = rocksdb_comparator_create(NULL, CmpDestroy, CmpCompare, CmpName);
@@ -951,6 +960,19 @@ int main(int argc, char** argv) {
   CheckNoError(err);
   CheckGet(db, roptions, "foo", "hello");
 
+  StartPhase("set_db_options");
+  {
+    const char* keys[] = {"stats_dump_period_sec"};
+    const char* values[] = {"10"};
+    rocksdb_set_db_options(db, 1, keys, values, &err);
+    CheckNoError(err);
+
+    keys[0] = "not_a_db_option";
+    rocksdb_set_db_options(db, 1, keys, values, &err);
+    CheckCondition(err != NULL);
+    Free(&err);
+  }
+
   StartPhase("backup_and_restore");
   {
     rocksdb_destroy_db(options, dbbackupname, &err);
@@ -1018,6 +1040,12 @@ int main(int argc, char** argv) {
     Free(&before_db_id);
     Free(&after_db_id);
 
+    rocksdb_backup_engine_stop_backup(be);
+    rocksdb_backup_engine_create_new_backup(be, db, &err);
+    CheckCondition(err != NULL);
+    free(err);
+    err = NULL;
+
     rocksdb_backup_engine_close(be);
   }
 
@@ -1147,9 +1175,11 @@ int main(int argc, char** argv) {
     static char cf_export_path[200];
     static char db_import_path[200];
     snprintf(cf_export_path, sizeof(cf_export_path),
-             "%s/rocksdb_c_test-%d-cf_export", GetTempDir(), ((int)geteuid()));
+             "%s/rocksdb_c_test-%" PRIu64 "-cf_export", GetTempDir(),
+             GetTestId());
     snprintf(db_import_path, sizeof(db_import_path),
-             "%s/rocksdb_c_test-%d-db_import", GetTempDir(), ((int)geteuid()));
+             "%s/rocksdb_c_test-%" PRIu64 "-db_import", GetTempDir(),
+             GetTestId());
 
     rocksdb_options_t* db_options = rocksdb_options_create();
     rocksdb_column_family_handle_t* cf_export =
@@ -2058,8 +2088,9 @@ int main(int argc, char** argv) {
     // use dbpathname2 as the cf_path for "cf1"
     rocksdb_dbpath_t* dbpath2;
     char dbpathname2[200];
-    snprintf(dbpathname2, sizeof(dbpathname2), "%s/rocksdb_c_test-%d-dbpath2",
-             GetTempDir(), ((int)geteuid()));
+    snprintf(dbpathname2, sizeof(dbpathname2),
+             "%s/rocksdb_c_test-%" PRIu64 "-dbpath2", GetTempDir(),
+             GetTestId());
     dbpath2 = rocksdb_dbpath_create(dbpathname2, 1024 * 1024);
     const rocksdb_dbpath_t* cf_paths[1] = {dbpath2};
     rocksdb_options_set_cf_paths(cf_options_2, cf_paths, 1);
@@ -2760,6 +2791,10 @@ int main(int argc, char** argv) {
     CheckCondition(
         1 == rocksdb_options_get_use_direct_io_for_flush_and_compaction(o));
 
+    rocksdb_options_set_use_direct_io_for_compaction_reads(o, 1);
+    CheckCondition(1 ==
+                   rocksdb_options_get_use_direct_io_for_compaction_reads(o));
+
     rocksdb_options_set_is_fd_close_on_exec(o, 1);
     CheckCondition(1 == rocksdb_options_get_is_fd_close_on_exec(o));
 
@@ -2987,6 +3022,8 @@ int main(int argc, char** argv) {
     CheckCondition(1 == rocksdb_options_get_use_direct_reads(copy));
     CheckCondition(
         1 == rocksdb_options_get_use_direct_io_for_flush_and_compaction(copy));
+    CheckCondition(
+        1 == rocksdb_options_get_use_direct_io_for_compaction_reads(copy));
     CheckCondition(1 == rocksdb_options_get_is_fd_close_on_exec(copy));
     CheckCondition(18 == rocksdb_options_get_stats_dump_period_sec(copy));
     CheckCondition(5 == rocksdb_options_get_stats_persist_period_sec(copy));
@@ -3265,6 +3302,12 @@ int main(int argc, char** argv) {
     CheckCondition(
         1 == rocksdb_options_get_use_direct_io_for_flush_and_compaction(o));
 
+    rocksdb_options_set_use_direct_io_for_compaction_reads(copy, 0);
+    CheckCondition(
+        0 == rocksdb_options_get_use_direct_io_for_compaction_reads(copy));
+    CheckCondition(1 ==
+                   rocksdb_options_get_use_direct_io_for_compaction_reads(o));
+
     rocksdb_options_set_is_fd_close_on_exec(copy, 0);
     CheckCondition(0 == rocksdb_options_get_is_fd_close_on_exec(copy));
     CheckCondition(1 == rocksdb_options_get_is_fd_close_on_exec(o));
@@ -3697,6 +3740,52 @@ int main(int argc, char** argv) {
     CheckCondition(37 ==
                    rocksdb_backup_engine_options_get_restore_rate_limit(bdo));
 
+    {
+      rocksdb_ratelimiter_t* limiter = rocksdb_ratelimiter_create_with_mode(
+          1024 * 1024, 100 * 1000, 10, 2, 0);
+      rocksdb_backup_engine_options_set_backup_rate_limiter(bdo, limiter);
+      rocksdb_backup_engine_options_set_restore_rate_limiter(bdo, limiter);
+      rocksdb_ratelimiter_destroy(limiter);
+
+      rocksdb_backup_engine_options_t* rate_beo =
+          rocksdb_backup_engine_options_create(dbbackupname);
+      rocksdb_ratelimiter_t* backup_limiter =
+          rocksdb_ratelimiter_create_with_mode(1024 * 1024, 100 * 1000, 10, 2,
+                                               0);
+      rocksdb_ratelimiter_t* restore_limiter =
+          rocksdb_ratelimiter_create_with_mode(1024 * 1024, 100 * 1000, 10, 2,
+                                               0);
+      rocksdb_backup_engine_options_set_backup_rate_limiter(rate_beo,
+                                                            backup_limiter);
+      rocksdb_backup_engine_options_set_restore_rate_limiter(rate_beo,
+                                                             restore_limiter);
+      rocksdb_ratelimiter_destroy(backup_limiter);
+      rocksdb_ratelimiter_destroy(restore_limiter);
+      rocksdb_env_t* rate_benv = rocksdb_create_default_env();
+      rocksdb_backup_engine_t* rate_be =
+          rocksdb_backup_engine_open_opts(rate_beo, rate_benv, &err);
+      rocksdb_backup_engine_options_destroy(rate_beo);
+      rocksdb_env_destroy(rate_benv);
+      CheckNoError(err);
+      rocksdb_backup_engine_create_new_backup(rate_be, db, &err);
+      CheckNoError(err);
+      {
+        char rate_restore_path[220];
+        snprintf(rate_restore_path, sizeof(rate_restore_path),
+                 "%s.rate_restore", dbname);
+        rocksdb_restore_options_t* rate_restore_opts =
+            rocksdb_restore_options_create();
+        rocksdb_backup_engine_restore_db_from_latest_backup(
+            rate_be, rate_restore_path, rate_restore_path, rate_restore_opts,
+            &err);
+        CheckNoError(err);
+        rocksdb_restore_options_destroy(rate_restore_opts);
+        rocksdb_destroy_db(options, rate_restore_path, &err);
+        err = NULL;
+      }
+      rocksdb_backup_engine_close(rate_be);
+    }
+
     rocksdb_backup_engine_options_set_max_background_operations(bdo, 20);
     CheckCondition(
         20 == rocksdb_backup_engine_options_get_max_background_operations(bdo));
@@ -3808,6 +3897,43 @@ int main(int argc, char** argv) {
     }
   }
 
+  StartPhase("transactiondb_set_write_policy_prepared");
+  {
+    char wp_dbname[200];
+    rocksdb_transactiondb_t* wp_txn_db;
+    rocksdb_transactiondb_options_t* wp_txn_opts;
+    snprintf(wp_dbname, sizeof(wp_dbname), "%s/rocksdb_c_test-txnwp-%" PRIu64,
+             GetTempDir(), GetTestId());
+    rocksdb_destroy_db(options, wp_dbname, &err);
+    Free(&err);
+    wp_txn_opts = rocksdb_transactiondb_options_create();
+    rocksdb_transactiondb_options_set_write_policy(
+        wp_txn_opts, rocksdb_txndb_write_policy_write_prepared);
+    rocksdb_options_set_create_if_missing(options, 1);
+    // create new TransactionDB with write policy WRITE_PREPARED
+    wp_txn_db =
+        rocksdb_transactiondb_open(options, wp_txn_opts, wp_dbname, &err);
+    CheckNoError(err);
+    {
+      rocksdb_transaction_options_t* wp_txn_options =
+          rocksdb_transaction_options_create();
+      rocksdb_transaction_t* wp_txn =
+          rocksdb_transaction_begin(wp_txn_db, woptions, wp_txn_options, NULL);
+      // write one record within a transaction
+      rocksdb_transaction_put(wp_txn, "k", 1, "v", 1, &err);
+      CheckNoError(err);
+      rocksdb_transaction_commit(wp_txn, &err);
+      CheckNoError(err);
+      rocksdb_transaction_destroy(wp_txn);
+      rocksdb_transaction_options_destroy(wp_txn_options);
+    }
+    CheckTxnDBGet(wp_txn_db, roptions, "k", "v");
+    rocksdb_transactiondb_close(wp_txn_db);
+    rocksdb_transactiondb_options_destroy(wp_txn_opts);
+    rocksdb_destroy_db(options, wp_dbname, &err);
+    CheckNoError(err);
+  }
+
   StartPhase("transactions");
   {
     rocksdb_close(db);
@@ -4445,7 +4571,7 @@ int main(int argc, char** argv) {
     rocksdb_options_set_max_open_files(opts, -1);
     rocksdb_options_set_create_if_missing(opts, 1);
     snprintf(secondary_path, sizeof(secondary_path),
-             "%s/rocksdb_c_test_secondary-%d", GetTempDir(), ((int)geteuid()));
+             "%s/rocksdb_c_test_secondary-%" PRIu64, GetTempDir(), GetTestId());
     db1 = rocksdb_open_as_secondary(opts, dbname, secondary_path, &err);
     CheckNoError(err);
 
@@ -4904,7 +5030,13 @@ int main(int argc, char** argv) {
     rocksdb_options_set_create_if_missing(null_opts, 1);
     rocksdb_options_set_compaction_service(null_opts, null_service);
 
-    const char* null_db = "rocksdb_c_test_null_service";
+    char null_db[200];
+    int null_db_len = snprintf(null_db, sizeof(null_db),
+                               "%s/rocksdb_c_test_null_service-%" PRIu64,
+                               GetTempDir(), GetTestId());
+    if (null_db_len <= 0 || (size_t)null_db_len >= sizeof(null_db)) {
+      abort();
+    }
 
     rocksdb_t* null_db_handle = rocksdb_open(null_opts, null_db, &err);
     CheckNoError(err);

db/compaction/compaction_job.cc

@@ -173,6 +173,7 @@ CompactionJob::CompactionJob(
       fs_(db_options.fs, io_tracer),
       file_options_for_read_(
           fs_->OptimizeForCompactionTableRead(file_options, db_options_)),
+      file_options_for_compaction_input_read_(file_options_for_read_),
       versions_(versions),
       shutting_down_(shutting_down),
       manual_compaction_canceled_(manual_compaction_canceled),
@@ -203,6 +204,11 @@ CompactionJob::CompactionJob(
   assert(job_context);
   assert(job_context->snapshot_context_initialized);
 
+  if (db_options_.use_direct_io_for_compaction_reads &&
+      !db_options_.use_direct_reads) {
+    file_options_for_compaction_input_read_.use_direct_reads = true;
+  }
+
   const auto* cfd = compact_->compaction->column_family_data();
   ThreadStatusUtil::SetEnableTracking(db_options_.enable_thread_tracking);
   ThreadStatusUtil::SetColumnFamily(cfd);
@@ -1536,10 +1542,20 @@ InternalIterator* CompactionJob::CreateInputIterator(
 
   // Although the v2 aggregator is what the level iterator(s) know about,
   // the AddTombstones calls will be propagated down to the v1 aggregator.
+  const bool open_ephemeral_table_reader =
+      db_options_.use_direct_io_for_compaction_reads &&
+      !db_options_.use_direct_reads;
+  FileOptions& input_file_options =
+      open_ephemeral_table_reader ? file_options_for_compaction_input_read_
+                                  : file_options_for_read_;
+  TEST_SYNC_POINT_CALLBACK(
+      "CompactionJob::CreateInputIterator:InputFileOptions",
+      &input_file_options);
   iterators.raw_input =
       std::unique_ptr<InternalIterator>(versions_->MakeInputIterator(
           read_options, sub_compact->compaction, sub_compact->RangeDelAgg(),
-          file_options_for_read_, boundaries.start, boundaries.end));
+          input_file_options, boundaries.start, boundaries.end,
+          open_ephemeral_table_reader));
   InternalIterator* input = iterators.raw_input.get();
 
   if (boundaries.start.has_value() || boundaries.end.has_value()) {
@@ -2470,6 +2486,8 @@ Status CompactionJob::OpenCompactionOutputFile(SubcompactionState* sub_compact,
   auto temperature =
       sub_compact->compaction->GetOutputTemperature(outputs.IsProximalLevel());
   fo_copy.temperature = temperature;
+  fo_copy.open_contract = FileOpenContract::kNoReopenForWrite |
+                          FileOpenContract::kNoReadersWhileOpenForWrite;
   fo_copy.write_hint = write_hint_;
 
   Status s;

db/compaction/compaction_job.h

@@ -462,6 +462,8 @@ class CompactionJob {
   FileSystemPtr fs_;
   // env_option optimized for compaction table reads
   FileOptions file_options_for_read_;
+  // file_options_for_read_ plus compaction-input-only overrides.
+  FileOptions file_options_for_compaction_input_read_;
   VersionSet* versions_;
   const std::atomic<bool>* shutting_down_;
   const std::atomic<bool>& manual_compaction_canceled_;

db/compaction/compaction_service_job.cc

@@ -431,6 +431,14 @@ Status CompactionServiceCompactionJob::Run() {
   // Please note that input stats will be updated by primary host when all
   // subcompactions are finished
   UpdateCompactionJobOutputStatsFromInternalStats(status, internal_stats_);
+  // Whole-file filtering changes which keys are fed to the iterator, so the
+  // iterator-based input record count should not be treated as exact.
+  for (size_t level = 1; level < c->num_input_levels(); ++level) {
+    if (!c->filtered_input_levels(level).empty()) {
+      compaction_result_->stats.has_accurate_num_input_records = false;
+      break;
+    }
+  }
   // and set fields that are not propagated as part of the update
   compaction_result_->stats.is_manual_compaction = c->is_manual_compaction();
   compaction_result_->stats.is_full_compaction = c->is_full_compaction();
@@ -651,6 +659,10 @@ static std::unordered_map<std::string, OptionTypeInfo>
         {"cpu_micros",
          {offsetof(struct CompactionJobStats, cpu_micros), OptionType::kUInt64T,
           OptionVerificationType::kNormal, OptionTypeFlags::kNone}},
+        {"has_accurate_num_input_records",
+         {offsetof(struct CompactionJobStats, has_accurate_num_input_records),
+          OptionType::kBoolean, OptionVerificationType::kNormal,
+          OptionTypeFlags::kNone}},
         {"num_input_records",
          {offsetof(struct CompactionJobStats, num_input_records),
           OptionType::kUInt64T, OptionVerificationType::kNormal,

db/compaction/compaction_service_test.cc

@@ -611,6 +611,101 @@ TEST_F(CompactionServiceTest, StandaloneDeleteRangeTombstoneOptimization) {
   SyncPoint::GetInstance()->DisableProcessing();
 }
 
+TEST_F(CompactionServiceTest,
+       StandaloneDeleteRangeTombstoneMakesInputCountInaccurate) {
+  Options options = CurrentOptions();
+  options.compaction_style = CompactionStyle::kCompactionStyleUniversal;
+  options.compaction_verify_record_count = true;
+  ReopenWithCompactionService(&options);
+
+  std::vector<std::string> files;
+  {
+    SstFileWriter sst_file_writer(EnvOptions(), options);
+    std::string file1 = dbname_ + "file1.sst";
+    ASSERT_OK(sst_file_writer.Open(file1));
+    ASSERT_OK(sst_file_writer.Put("a", "a1"));
+    ASSERT_OK(sst_file_writer.Put("b", "b1"));
+    ExternalSstFileInfo file1_info;
+    ASSERT_OK(sst_file_writer.Finish(&file1_info));
+    files.push_back(std::move(file1));
+
+    std::string file2 = dbname_ + "file2.sst";
+    ASSERT_OK(sst_file_writer.Open(file2));
+    ASSERT_OK(sst_file_writer.Put("x", "x1"));
+    ASSERT_OK(sst_file_writer.Put("y", "y1"));
+    ExternalSstFileInfo file2_info;
+    ASSERT_OK(sst_file_writer.Finish(&file2_info));
+    files.push_back(std::move(file2));
+  }
+
+  IngestExternalFileOptions ifo;
+  ASSERT_OK(db_->IngestExternalFile(files, ifo));
+  ASSERT_EQ(Get("a"), "a1");
+  ASSERT_EQ(Get("b"), "b1");
+  ASSERT_EQ(Get("x"), "x1");
+  ASSERT_EQ(Get("y"), "y1");
+  ASSERT_EQ(2, NumTableFilesAtLevel(6));
+
+  auto my_cs = GetCompactionService();
+  uint64_t comp_num = my_cs->GetCompactionNum();
+
+  size_t num_files_after_filtered = 0;
+  ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->SetCallBack(
+      "VersionSet::MakeInputIterator:NewCompactionMergingIterator",
+      [&](void* arg) {
+        num_files_after_filtered = *static_cast<size_t*>(arg);
+      });
+  SyncPoint::GetInstance()->EnableProcessing();
+
+  {
+    // The standalone range tombstone can fully cover the old versioned files,
+    // so universal compaction may filter those files before iteration.
+    files.clear();
+    SstFileWriter sst_file_writer(EnvOptions(), options);
+    std::string file2 = dbname_ + "file2.sst";
+    ASSERT_OK(sst_file_writer.Open(file2));
+    ASSERT_OK(sst_file_writer.DeleteRange("a", "z"));
+    ExternalSstFileInfo file2_info;
+    ASSERT_OK(sst_file_writer.Finish(&file2_info));
+    files.push_back(std::move(file2));
+
+    std::string file3 = dbname_ + "file3.sst";
+    ASSERT_OK(sst_file_writer.Open(file3));
+    ASSERT_OK(sst_file_writer.Put("a", "a2"));
+    ASSERT_OK(sst_file_writer.Put("b", "b2"));
+    ExternalSstFileInfo file3_info;
+    ASSERT_OK(sst_file_writer.Finish(&file3_info));
+    files.push_back(std::move(file3));
+
+    std::string file4 = dbname_ + "file4.sst";
+    ASSERT_OK(sst_file_writer.Open(file4));
+    ASSERT_OK(sst_file_writer.Put("x", "x2"));
+    ASSERT_OK(sst_file_writer.Put("y", "y2"));
+    ExternalSstFileInfo file4_info;
+    ASSERT_OK(sst_file_writer.Finish(&file4_info));
+    files.push_back(std::move(file4));
+  }
+
+  ASSERT_OK(db_->IngestExternalFile(files, ifo));
+  ASSERT_OK(db_->WaitForCompact(WaitForCompactOptions()));
+  ASSERT_GE(my_cs->GetCompactionNum(), comp_num + 1);
+
+  CompactionServiceResult result;
+  my_cs->GetResult(&result);
+  ASSERT_OK(result.status);
+  ASSERT_TRUE(result.stats.is_remote_compaction);
+  ASSERT_FALSE(result.stats.has_accurate_num_input_records);
+  ASSERT_EQ(1, num_files_after_filtered);
+
+  ASSERT_EQ(Get("a"), "a2");
+  ASSERT_EQ(Get("b"), "b2");
+  ASSERT_EQ(Get("x"), "x2");
+  ASSERT_EQ(Get("y"), "y2");
+
+  SyncPoint::GetInstance()->DisableProcessing();
+  SyncPoint::GetInstance()->ClearAllCallBacks();
+}
+
 TEST_F(CompactionServiceTest, CompactionOutputFileIOError) {
   Options options = CurrentOptions();
   options.disable_auto_compactions = true;
@@ -943,6 +1038,37 @@ TEST_F(CompactionServiceTest, VerifyInputRecordCount) {
   SyncPoint::GetInstance()->ClearAllCallBacks();
 }
 
+TEST_F(CompactionServiceTest, InaccurateInputRecordCount) {
+  Options options = CurrentOptions();
+  options.disable_auto_compactions = true;
+  ReopenWithCompactionService(&options);
+  GenerateTestData();
+
+  auto my_cs = GetCompactionService();
+
+  std::string start_str = Key(15);
+  std::string end_str = Key(45);
+  Slice start(start_str);
+  Slice end(end_str);
+  uint64_t comp_num = my_cs->GetCompactionNum();
+
+  ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->SetCallBack(
+      "CompactionServiceCompactionJob::Run:0", [&](void* arg) {
+        CompactionServiceResult* compaction_result =
+            *(static_cast<CompactionServiceResult**>(arg));
+        ASSERT_TRUE(compaction_result != nullptr);
+        compaction_result->stats.has_accurate_num_input_records = false;
+        compaction_result->stats.num_input_records = 0;
+      });
+  SyncPoint::GetInstance()->EnableProcessing();
+
+  ASSERT_OK(db_->CompactRange(CompactRangeOptions(), &start, &end));
+  ASSERT_GE(my_cs->GetCompactionNum(), comp_num + 1);
+
+  SyncPoint::GetInstance()->DisableProcessing();
+  SyncPoint::GetInstance()->ClearAllCallBacks();
+}
+
 TEST_F(CompactionServiceTest, EmptyResult) {
   Options options = CurrentOptions();
   options.disable_auto_compactions = true;
@@ -1285,6 +1411,10 @@ TEST_F(CompactionServiceTest, TruncatedOutput) {
 
 TEST_F(CompactionServiceTest, CustomFileChecksum) {
   Options options = CurrentOptions();
+  // Pin compression so the auto-compacted LSM shape (and thus whether the
+  // manual CompactRange below schedules a remote compaction) doesn't depend on
+  // the default compression type. kNoCompression is always available.
+  options.compression = kNoCompression;
   options.file_checksum_gen_factory = GetFileChecksumGenCrc32cFactory();
   ReopenWithCompactionService(&options);
   GenerateTestData();

db/db_basic_test.cc

@@ -6331,7 +6331,7 @@ TEST_F(DBBasicTest, DisallowMemtableWrite) {
   Options options_disallow = options_allow;
   options_disallow.disallow_memtable_writes = true;
   options_disallow.paranoid_memory_checks = true;
-  options_disallow.memtable_veirfy_per_key_checksum_on_seek = true;
+  options_disallow.memtable_verify_per_key_checksum_on_seek = true;
 
   DestroyAndReopen(options_allow);
   // CFs allowing and disallowing memtable write

db/db_block_cache_test.cc

@@ -1482,10 +1482,13 @@ TEST_P(DBBlockCacheTypeTest, AddRedundantStats) {
   // Access just data, forcing redundant load+insert
   ReadOptions read_options;
   std::unique_ptr<Iterator> iter{db_->NewIterator(read_options)};
+  ASSERT_OK(iter->Refresh());
+
   cache->SetNthLookupNotFound(1);
   iter->SeekToFirst();
   ASSERT_TRUE(iter->Valid());
   ASSERT_EQ(iter->key(), "bar");
+  ASSERT_EQ(iter->value(), "value");
 
   EXPECT_EQ(2, TestGetTickerCount(options, BLOCK_CACHE_INDEX_ADD));
   EXPECT_EQ(2, TestGetTickerCount(options, BLOCK_CACHE_FILTER_ADD));

db/db_bloom_filter_test.cc

@@ -2022,7 +2022,11 @@ TEST_F(DBBloomFilterTest, MutableFilterPolicy) {
   double expected_bpk = 10.0;
   // Other configs to try
   std::vector<std::pair<std::string, double>> configs = {
-      {"ribbonfilter:10:-1", 7.0}, {"bloomfilter:5", 5.0}, {"nullptr", 0.0}};
+      {"ribbonfilter:10:-1", 7.0},
+      {"bloomfilter:5", 5.0},
+      {"nullptr", 0.0},
+      // As serialized in OPTIONS file
+      {"{id=ribbonfilter:12:-1;bloom_before_level=-1;}", 8.4}};
 
   table_options.cache_index_and_filter_blocks = true;
   options.table_factory.reset(NewBlockBasedTableFactory(table_options));

db/db_compaction_abort_test.cc

@@ -427,6 +427,38 @@ TEST_F(DBCompactionAbortTest, AbortAutomaticCompaction) {
   }
 }
 
+TEST_F(DBCompactionAbortTest, AbortScheduledAutomaticCompactionBeforePick) {
+  // The goal is to cover an automatic compaction that has been scheduled, but
+  // aborts before the worker picks and removes a column family from the
+  // compaction queue. This pins the scheduler bookkeeping invariant that
+  // ResumeAllCompactions() must be able to schedule that still-queued work.
+  constexpr int kCompactionTrigger = 4;
+  constexpr int kNumL0Files = kCompactionTrigger + 1;
+  Options options = GetOptionsWithStats();
+  options.level0_file_num_compaction_trigger = kCompactionTrigger;
+  options.max_background_compactions = 1;
+  options.max_subcompactions = 1;
+  options.disable_auto_compactions = false;
+  Reopen(options);
+
+  SyncPointAbortHelper helper("BackgroundCallCompaction:0");
+  helper.Setup(dbfull());
+
+  PopulateData(/*num_files=*/kNumL0Files, /*keys_per_file=*/100,
+               /*value_size=*/1000);
+  helper.CleanupAndWait();
+
+  const uint64_t compact_write_bytes_before_resume =
+      stats_->getTickerCount(COMPACT_WRITE_BYTES);
+  dbfull()->ResumeAllCompactions();
+  ASSERT_OK(dbfull()->TEST_WaitForCompact());
+
+  ASSERT_GT(stats_->getTickerCount(COMPACT_WRITE_BYTES),
+            compact_write_bytes_before_resume);
+  ASSERT_LT(NumTableFilesAtLevel(0), kNumL0Files);
+  VerifyDataIntegrity(/*num_keys=*/100);
+}
+
 TEST_F(DBCompactionAbortTest, AbortAndVerifyNoOutputFiles) {
   Options options = CurrentOptions();
   options.level0_file_num_compaction_trigger = 4;

db/db_compaction_test.cc

@@ -8,11 +8,13 @@
 // found in the LICENSE file. See the AUTHORS file for names of contributors.
 
 #include <tuple>
+#include <utility>
 
 #include "compaction/compaction_picker_universal.h"
 #include "db/blob/blob_index.h"
 #include "db/db_test_util.h"
 #include "db/dbformat.h"
+#include "db/table_cache.h"
 #include "env/mock_env.h"
 #include "file/filename.h"
 #include "port/port.h"
@@ -6651,6 +6653,572 @@ TEST_P(DBCompactionDirectIOTest, DirectIO) {
 INSTANTIATE_TEST_CASE_P(DBCompactionDirectIOTest, DBCompactionDirectIOTest,
                         testing::Bool());
 
+// With use_direct_io_for_compaction_reads OFF, compaction reads must stay
+// buffered: neither the compaction-input FileOptions nor a kernel O_DIRECT
+// open should fire. Runs on every platform (the sync points just don't fire
+// where O_DIRECT isn't reachable). Pairs with
+// UseDirectIoForCompactionReadsEndToEnd for the on case.
+TEST_F(DBCompactionTest, UseDirectIoForCompactionReadsOffStaysBuffered) {
+  Options options = CurrentOptions();
+  Destroy(options);
+  options.create_if_missing = true;
+  options.disable_auto_compactions = true;
+  options.use_direct_reads = false;
+  options.use_direct_io_for_compaction_reads = false;
+  options.use_direct_io_for_flush_and_compaction = false;
+
+  std::atomic<bool> observed_direct_compaction_read{false};
+  std::atomic<int> observed_callbacks{0};
+  std::atomic<int> observed_odirect_opens{0};
+  ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->SetCallBack(
+      "CompactionJob::CreateInputIterator:InputFileOptions", [&](void* arg) {
+        const auto* fo = static_cast<const FileOptions*>(arg);
+        observed_callbacks.fetch_add(1, std::memory_order_relaxed);
+        if (fo->use_direct_reads) {
+          observed_direct_compaction_read.store(true,
+                                                std::memory_order_relaxed);
+        }
+      });
+  ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->SetCallBack(
+      "NewRandomAccessFile:O_DIRECT", [&](void* /*arg*/) {
+        observed_odirect_opens.fetch_add(1, std::memory_order_relaxed);
+      });
+  ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->EnableProcessing();
+
+  ASSERT_OK(TryReopen(options));
+
+  const std::string value(4096, 'v');
+  for (int i = 0; i < 64; ++i) {
+    ASSERT_OK(Put(Key(i), value));
+  }
+  ASSERT_OK(Flush());
+  for (int i = 0; i < 64; ++i) {
+    ASSERT_OK(Put(Key(i), value));
+  }
+  ASSERT_OK(Flush());
+
+  ASSERT_OK(dbfull()->CompactRange(CompactRangeOptions(), nullptr, nullptr));
+  ASSERT_OK(dbfull()->TEST_WaitForCompact());
+
+  ASSERT_GT(observed_callbacks.load(), 0);
+  ASSERT_FALSE(observed_direct_compaction_read.load());
+  ASSERT_EQ(0, observed_odirect_opens.load());
+
+  ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->DisableProcessing();
+  ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->ClearAllCallBacks();
+  Destroy(options);
+}
+
+TEST_F(DBCompactionTest,
+       UseDirectIoForCompactionReadsUsesBoundedEphemeralReaders) {
+  auto fs = std::make_shared<MockFileSystem>(Env::Default()->GetSystemClock(),
+                                             /*supports_direct_io=*/true);
+  std::unique_ptr<Env> mock_env = NewCompositeEnv(fs);
+
+  Options options = CurrentOptions();
+  options.env = mock_env.get();
+  Destroy(options);
+  options.create_if_missing = true;
+  options.disable_auto_compactions = true;
+  options.use_direct_reads = false;
+  options.use_direct_io_for_compaction_reads = true;
+  options.use_direct_io_for_flush_and_compaction = false;
+  options.max_subcompactions = 3;
+  options.statistics = CreateDBStatistics();
+
+  BlockBasedTableOptions table_options;
+  table_options.block_cache = NewLRUCache(64 << 20);
+  table_options.cache_index_and_filter_blocks = true;
+  table_options.pin_l0_filter_and_index_blocks_in_cache = true;
+  table_options.filter_policy.reset(NewBloomFilterPolicy(10));
+  options.table_factory.reset(NewBlockBasedTableFactory(table_options));
+
+  std::atomic<int> fresh_reader_opens{0};
+  std::atomic<int> malformed_fresh_reader_options{0};
+  std::atomic<int> avoid_shared_metadata_cache_opens{0};
+  std::atomic<int> shared_metadata_cache_uses{0};
+  std::atomic<size_t> input_iterator_file_bound{0};
+
+  ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->SetCallBack(
+      "TableCache::FindTable:FreshTableReader", [&](void* arg) {
+        const auto* open_options = static_cast<TableCacheOpenOptions*>(arg);
+        fresh_reader_opens.fetch_add(1, std::memory_order_relaxed);
+        if (!open_options->open_ephemeral_table_reader ||
+            !open_options->avoid_shared_metadata_cache ||
+            !open_options->skip_filters) {
+          malformed_fresh_reader_options.fetch_add(1,
+                                                   std::memory_order_relaxed);
+        }
+      });
+  ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->SetCallBack(
+      "BlockBasedTable::PrefetchIndexAndFilterBlocks:SharedMetadataCache",
+      [&](void* arg) {
+        const auto* context = static_cast<std::pair<bool, bool>*>(arg);
+        const bool avoid_shared_metadata_cache = context->first;
+        const bool use_cache = context->second;
+        if (avoid_shared_metadata_cache) {
+          avoid_shared_metadata_cache_opens.fetch_add(
+              1, std::memory_order_relaxed);
+          if (use_cache) {
+            shared_metadata_cache_uses.fetch_add(1, std::memory_order_relaxed);
+          }
+        }
+      });
+  ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->SetCallBack(
+      "VersionSet::MakeInputIterator:NewCompactionMergingIterator",
+      [&](void* arg) {
+        input_iterator_file_bound.fetch_add(*static_cast<size_t*>(arg),
+                                            std::memory_order_relaxed);
+      });
+  ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->EnableProcessing();
+
+  ASSERT_OK(TryReopen(options));
+
+  const std::string value(4096, 'v');
+  for (int i = 0; i < 64; ++i) {
+    ASSERT_OK(Put(Key(i), value));
+  }
+  ASSERT_OK(Flush());
+  for (int i = 0; i < 64; ++i) {
+    ASSERT_OK(Put(Key(i), value));
+  }
+  ASSERT_OK(Flush());
+
+  ASSERT_OK(dbfull()->CompactRange(CompactRangeOptions(), nullptr, nullptr));
+  ASSERT_OK(dbfull()->TEST_WaitForCompact());
+
+  ASSERT_GT(fresh_reader_opens.load(), 0);
+  EXPECT_EQ(0, malformed_fresh_reader_options.load());
+  ASSERT_GT(avoid_shared_metadata_cache_opens.load(), 0);
+  EXPECT_EQ(0, shared_metadata_cache_uses.load());
+  ASSERT_GT(input_iterator_file_bound.load(), 0);
+  EXPECT_LE(static_cast<size_t>(fresh_reader_opens.load()),
+            input_iterator_file_bound.load());
+
+  ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->DisableProcessing();
+  ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->ClearAllCallBacks();
+  Destroy(options);
+}
+
+// End-to-end check that use_direct_io_for_compaction_reads opens compaction
+// inputs with O_DIRECT while use_direct_reads stays off (user reads buffered).
+// The NewRandomAccessFile:O_DIRECT sync point in env/fs_posix.cc fires once
+// per fresh open with the O_DIRECT flag, so this proves the kernel path, not
+// just the FileOptions. Only runs on platforms that take the O_DIRECT path.
+#if !defined(OS_MACOSX) && !defined(OS_OPENBSD) && !defined(OS_SOLARIS) && \
+    !defined(OS_WIN)
+TEST_F(DBCompactionTest, UseDirectIoForCompactionReadsEndToEnd) {
+  if (!IsDirectIOSupported()) {
+    ROCKSDB_GTEST_BYPASS("Direct IO not supported");
+    return;
+  }
+
+  Options options = CurrentOptions();
+  Destroy(options);
+  options.create_if_missing = true;
+  options.disable_auto_compactions = true;
+  // User reads stay buffered, compaction reads should switch to O_DIRECT.
+  options.use_direct_reads = false;
+  options.use_direct_io_for_compaction_reads = true;
+  // Isolate the read-side change; leave the compaction write path buffered.
+  options.use_direct_io_for_flush_and_compaction = false;
+
+  // Sync-point callbacks fire on compaction threads while the test thread
+  // reads these counters, so use atomics to avoid a data race.
+  std::atomic<int> observed_run_starts{0};
+  std::atomic<int> observed_odirect_opens{0};
+  std::atomic<bool> observed_direct_compaction_read{false};
+  std::atomic<int> observed_callbacks{0};
+  ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->LoadDependency({});
+  // Plumbing-level probe: the compaction-input FileOptions should carry
+  // use_direct_reads = true when the new flag is enabled.
+  ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->SetCallBack(
+      "CompactionJob::CreateInputIterator:InputFileOptions", [&](void* arg) {
+        const auto* fo = static_cast<const FileOptions*>(arg);
+        observed_callbacks.fetch_add(1, std::memory_order_relaxed);
+        if (fo != nullptr && fo->use_direct_reads) {
+          observed_direct_compaction_read.store(true,
+                                                std::memory_order_relaxed);
+        }
+      });
+  ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->SetCallBack(
+      "CompactionJob::Run():Start", [&](void* /*arg*/) {
+        observed_run_starts.fetch_add(1, std::memory_order_relaxed);
+      });
+  // Kernel-level probe: this fires only when open() is issued with O_DIRECT,
+  // proving we change the actual cache mode, not just the FileOptions struct.
+  ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->SetCallBack(
+      "NewRandomAccessFile:O_DIRECT", [&](void* /*arg*/) {
+        observed_odirect_opens.fetch_add(1, std::memory_order_relaxed);
+      });
+  ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->EnableProcessing();
+
+  Status s = TryReopen(options);
+  if (s.IsNotSupported() || s.IsInvalidArgument()) {
+    ROCKSDB_GTEST_BYPASS(
+        "Direct IO reads not supported in this test environment");
+    ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->DisableProcessing();
+    ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->ClearAllCallBacks();
+    return;
+  }
+  ASSERT_OK(s);
+
+  // Produce two L0 files with OVERLAPPING key ranges so that CompactRange has
+  // actual merge work to do (otherwise RocksDB performs a trivial file move
+  // and never constructs a CompactionJob).
+  const std::string value(4096, 'v');
+  for (int i = 0; i < 64; ++i) {
+    ASSERT_OK(Put(Key(i), value));
+  }
+  ASSERT_OK(Flush());
+  for (int i = 0; i < 64; ++i) {
+    ASSERT_OK(Put(Key(i), value));
+  }
+  ASSERT_OK(Flush());
+
+  // User reads should still go through the buffered path. Confirm that the
+  // option does not silently flip use_direct_reads for user reads.
+  for (int i = 0; i < 8; ++i) {
+    std::string actual;
+    ASSERT_OK(db_->Get(ReadOptions(), Key(i), &actual));
+    ASSERT_EQ(value, actual);
+  }
+
+  ASSERT_OK(dbfull()->CompactRange(CompactRangeOptions(), nullptr, nullptr));
+  // Wait for compaction to complete and CompactionJob to be constructed.
+  ASSERT_OK(dbfull()->TEST_WaitForCompact());
+
+  // Confirm the compaction actually ran; otherwise the missing sync-point hits
+  // would be a test-setup problem, not an option regression.
+  ASSERT_GT(observed_run_starts.load(), 0)
+      << "CompactionJob::Run():Start never fired; CompactRange did not "
+         "schedule a compaction.";
+  ASSERT_GT(observed_callbacks.load(), 0);
+  ASSERT_TRUE(observed_direct_compaction_read.load());
+  // At least one compaction-input open went through O_DIRECT. Without the
+  // TableCache bypass this would be zero, since compaction would reuse the
+  // buffered handles cached for user reads.
+  EXPECT_GT(observed_odirect_opens.load(), 0)
+      << "no compaction-input opens went through O_DIRECT; "
+         "observed_odirect_opens="
+      << observed_odirect_opens.load();
+
+  // Quick sanity sweep after compaction to confirm data is intact.
+  for (int i = 0; i < 64; ++i) {
+    std::string actual;
+    ASSERT_OK(db_->Get(ReadOptions(), Key(i), &actual));
+    ASSERT_EQ(value, actual);
+  }
+
+  ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->DisableProcessing();
+  ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->ClearAllCallBacks();
+  Destroy(options);
+}
+
+// Exercise the LevelIterator bypass path (L1+ compactions) with range
+// tombstones, where the ephemeral TableReader's lifetime is coupled to the
+// range_tombstone_iter the file iterator returns. The end-to-end test above
+// only makes two L0 files, which take the direct NewIterator path and never
+// hit LevelIterator. Here we build L1/L2 with tombstones and compact L1->L2 so
+// LevelIterator::NewFileIterator drives the bypass; a wrong reader/iterator
+// lifetime would crash or trip sanitizers as LevelIterator switches files.
+// Correctness is checked by computing the expected state of every key and
+// asserting Get() matches: wave-2 puts beat wave-1, and each key's most-recent
+// covering tombstone shadows older puts.
+TEST_F(DBCompactionTest,
+       UseDirectIoForCompactionReadsLevelIteratorWithTombstones) {
+  if (!IsDirectIOSupported()) {
+    ROCKSDB_GTEST_BYPASS("Direct IO not supported");
+    return;
+  }
+
+  Options options = CurrentOptions();
+  Destroy(options);
+  options.create_if_missing = true;
+  options.disable_auto_compactions = true;
+  options.use_direct_reads = false;
+  options.use_direct_io_for_compaction_reads = true;
+  options.use_direct_io_for_flush_and_compaction = false;
+  // Small files / small level base so we can pack data into L1 and L2 with
+  // a few flushes and CompactRange calls instead of needing millions of keys.
+  options.write_buffer_size = 64 * 1024;
+  options.target_file_size_base = 64 * 1024;
+  options.max_bytes_for_level_base = 256 * 1024;
+  options.level0_file_num_compaction_trigger = 100;  // never auto-trigger
+
+  std::atomic<int> observed_odirect_opens{0};
+  std::atomic<int> observed_run_starts{0};
+  ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->SetCallBack(
+      "NewRandomAccessFile:O_DIRECT", [&](void* /*arg*/) {
+        observed_odirect_opens.fetch_add(1, std::memory_order_relaxed);
+      });
+  ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->SetCallBack(
+      "CompactionJob::Run():Start", [&](void* /*arg*/) {
+        observed_run_starts.fetch_add(1, std::memory_order_relaxed);
+      });
+  ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->EnableProcessing();
+
+  Status s = TryReopen(options);
+  if (s.IsNotSupported() || s.IsInvalidArgument()) {
+    ROCKSDB_GTEST_BYPASS(
+        "Direct IO reads not supported in this test environment");
+    ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->DisableProcessing();
+    ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->ClearAllCallBacks();
+    return;
+  }
+  ASSERT_OK(s);
+
+  // Use distinguishable values per wave so we can verify which wave's put
+  // won the merge for each key, not just "some put won".
+  const std::string wave1_value(1024, '1');
+  const std::string wave2_value(1024, '2');
+
+  auto write_batch = [&](int begin, int end, const std::string& value,
+                         bool with_range_tombstone) {
+    for (int i = begin; i < end; ++i) {
+      ASSERT_OK(Put(Key(i), value));
+    }
+    if (with_range_tombstone) {
+      // Drop a slice in the middle of the just-written range. The
+      // DeleteRange follows the puts in this batch, so its sequence number
+      // is higher and it shadows the puts on [del_lo, del_hi) within this
+      // SST.
+      ASSERT_OK(db_->DeleteRange(WriteOptions(), db_->DefaultColumnFamily(),
+                                 Key(begin + (end - begin) / 4),
+                                 Key(begin + 3 * (end - begin) / 4)));
+    }
+    ASSERT_OK(Flush());
+  };
+
+  // Wave 1: four flushed SSTs covering [0, 800), each with a tombstone
+  // covering the middle half of its own range. Then compact down so the
+  // next phase exercises LevelIterator over L1+ files.
+  constexpr int kWave1Begin = 0;
+  constexpr int kWave1End = 800;
+  constexpr int kWave1BatchSize = 200;
+  for (int batch_begin = kWave1Begin; batch_begin < kWave1End;
+       batch_begin += kWave1BatchSize) {
+    write_batch(batch_begin, batch_begin + kWave1BatchSize, wave1_value,
+                /*with_range_tombstone=*/true);
+  }
+  ASSERT_OK(dbfull()->CompactRange(CompactRangeOptions(), nullptr, nullptr));
+  ASSERT_OK(dbfull()->TEST_WaitForCompact());
+
+  // Wave 2: two more flushed SSTs at L0 that overlap wave 1, each with their
+  // own range tombstone. The puts in wave 2 have higher seqno than wave 1,
+  // so they win when not tombstoned.
+  struct Wave2Range {
+    int begin;
+    int end;
+  };
+  const Wave2Range wave2_ranges[] = {{50, 250}, {350, 550}};
+  for (const auto& r : wave2_ranges) {
+    write_batch(r.begin, r.end, wave2_value, /*with_range_tombstone=*/true);
+  }
+
+  const int run_starts_before = observed_run_starts.load();
+  const int odirect_before = observed_odirect_opens.load();
+
+  // Compact everything together, forcing a LevelIterator over the lower-level
+  // files on the bypass path. Wrong ephemeral reader / tombstone-iter
+  // lifetimes should trip sanitizers here.
+  ASSERT_OK(dbfull()->CompactRange(CompactRangeOptions(), nullptr, nullptr));
+  ASSERT_OK(dbfull()->TEST_WaitForCompact());
+
+  ASSERT_GT(observed_run_starts.load(), run_starts_before)
+      << "expected at least one compaction to run during the L1+ phase";
+  EXPECT_GT(observed_odirect_opens.load(), odirect_before)
+      << "no compaction-input opens went through O_DIRECT during L1+ "
+         "compaction; LevelIterator bypass path may be broken";
+
+  // Compute the precise expected state per key from the test parameters.
+  // For each k in the wave-1 range:
+  //   - If k is inside a wave-2 batch: wave-2 wins. NotFound iff k is in
+  //     that batch's tombstone range; otherwise present with wave2_value.
+  //   - Else: wave 1 wins. NotFound iff k is in its batch's tombstone
+  //     range; otherwise present with wave1_value.
+  enum class Expectation { kAbsent, kWave1Value, kWave2Value };
+  auto classify = [&](int k) -> Expectation {
+    // Wave 2 first (it shadows wave 1 wherever it covers).
+    for (const auto& r : wave2_ranges) {
+      if (k >= r.begin && k < r.end) {
+        const int width = r.end - r.begin;
+        const int del_lo = r.begin + width / 4;
+        const int del_hi = r.begin + 3 * width / 4;
+        if (k >= del_lo && k < del_hi) {
+          return Expectation::kAbsent;
+        }
+        return Expectation::kWave2Value;
+      }
+    }
+    // Fall back to wave 1.
+    const int batch = (k - kWave1Begin) / kWave1BatchSize;
+    const int batch_begin = kWave1Begin + batch * kWave1BatchSize;
+    const int del_lo = batch_begin + kWave1BatchSize / 4;
+    const int del_hi = batch_begin + 3 * kWave1BatchSize / 4;
+    if (k >= del_lo && k < del_hi) {
+      return Expectation::kAbsent;
+    }
+    return Expectation::kWave1Value;
+  };
+
+  int present_w1 = 0;
+  int present_w2 = 0;
+  int absent = 0;
+  std::string actual;
+  for (int k = kWave1Begin; k < kWave1End; ++k) {
+    const Status get_s = db_->Get(ReadOptions(), Key(k), &actual);
+    const Expectation exp = classify(k);
+    switch (exp) {
+      case Expectation::kAbsent:
+        EXPECT_TRUE(get_s.IsNotFound())
+            << "key " << k << " expected NotFound (covered by tombstone); "
+            << "got status=" << get_s.ToString()
+            << " value_len=" << (get_s.ok() ? actual.size() : 0);
+        ++absent;
+        break;
+      case Expectation::kWave1Value:
+        ASSERT_OK(get_s) << "key " << k << " expected wave-1 value";
+        EXPECT_EQ(wave1_value, actual)
+            << "key " << k << " expected wave-1 value but got a different one";
+        ++present_w1;
+        break;
+      case Expectation::kWave2Value:
+        ASSERT_OK(get_s) << "key " << k << " expected wave-2 value";
+        EXPECT_EQ(wave2_value, actual)
+            << "key " << k << " expected wave-2 value but got a different one";
+        ++present_w2;
+        break;
+    }
+  }
+  // All three buckets should be non-empty, so the test really exercises both
+  // tombstone paths and the wave-2-wins path. A zero here means the setup
+  // drifted and the setup (not these checks) should be fixed.
+  EXPECT_GT(present_w1, 0);
+  EXPECT_GT(present_w2, 0);
+  EXPECT_GT(absent, 0);
+
+  ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->DisableProcessing();
+  ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->ClearAllCallBacks();
+  Destroy(options);
+}
+
+// With the option enabled, run reader threads against the live DB while manual
+// compactions run in the background. Each compaction-input file is open through
+// an ephemeral O_DIRECT handle while the shared TableCache still serves user
+// reads through a buffered handle, so the same SST is open in two cache modes
+// at once. This stresses that coexistence under TSAN/ASAN/UBSAN; it asserts
+// reads stay consistent and final values match the last writes, not anything
+// about timing.
+TEST_F(DBCompactionTest, UseDirectIoForCompactionReadsConcurrentReadStress) {
+  if (!IsDirectIOSupported()) {
+    ROCKSDB_GTEST_BYPASS("Direct IO not supported");
+    return;
+  }
+
+  Options options = CurrentOptions();
+  Destroy(options);
+  options.create_if_missing = true;
+  options.disable_auto_compactions = true;
+  options.use_direct_reads = false;
+  options.use_direct_io_for_compaction_reads = true;
+  options.use_direct_io_for_flush_and_compaction = false;
+  options.write_buffer_size = 64 * 1024;
+  options.target_file_size_base = 64 * 1024;
+  options.max_bytes_for_level_base = 256 * 1024;
+  options.level0_file_num_compaction_trigger = 100;  // never auto-trigger
+
+  Status s = TryReopen(options);
+  if (s.IsNotSupported() || s.IsInvalidArgument()) {
+    ROCKSDB_GTEST_BYPASS(
+        "Direct IO reads not supported in this test environment");
+    return;
+  }
+  ASSERT_OK(s);
+
+  constexpr int kNumKeys = 512;
+  constexpr int kValueSize = 256;
+  const std::string base_value(kValueSize, 'b');
+
+  // Initial population, flushed into a few L0 SSTs.
+  for (int i = 0; i < kNumKeys; ++i) {
+    ASSERT_OK(Put(Key(i), base_value));
+    if (i > 0 && i % 128 == 0) {
+      ASSERT_OK(Flush());
+    }
+  }
+  ASSERT_OK(Flush());
+
+  std::atomic<bool> stop{false};
+  std::atomic<int64_t> reads_observed{0};
+  std::atomic<int64_t> read_errors{0};
+
+  // Reader threads loop until stop is set, recording any status that isn't OK
+  // or NotFound. The point is to surface Corruption/IOError/use-after-free, so
+  // the main thread can fail the test.
+  constexpr int kNumReaders = 4;
+  std::vector<port::Thread> readers;
+  readers.reserve(kNumReaders);
+  for (int t = 0; t < kNumReaders; ++t) {
+    readers.emplace_back([&, t]() {
+      std::string value;
+      while (!stop.load(std::memory_order_acquire)) {
+        const int k =
+            (t * 7919 +
+             static_cast<int>(reads_observed.load(std::memory_order_relaxed))) %
+            kNumKeys;
+        Status read_s = db_->Get(ReadOptions(), Key(k), &value);
+        if (!read_s.ok() && !read_s.IsNotFound()) {
+          read_errors.fetch_add(1, std::memory_order_relaxed);
+        }
+        reads_observed.fetch_add(1, std::memory_order_relaxed);
+      }
+    });
+  }
+
+  // Drive a few rounds of writes and compactions on the main thread while
+  // the readers hammer Get(). Each round overwrites every key with a
+  // round-tagged value and then compacts.
+  constexpr int kRounds = 4;
+  std::string last_value = base_value;
+  for (int round = 0; round < kRounds; ++round) {
+    const std::string round_value(kValueSize,
+                                  static_cast<char>('A' + (round % 26)));
+    for (int i = 0; i < kNumKeys; ++i) {
+      ASSERT_OK(Put(Key(i), round_value));
+      if (i > 0 && i % 64 == 0) {
+        ASSERT_OK(Flush());
+      }
+    }
+    ASSERT_OK(Flush());
+    ASSERT_OK(dbfull()->CompactRange(CompactRangeOptions(), nullptr, nullptr));
+    ASSERT_OK(dbfull()->TEST_WaitForCompact());
+    last_value = round_value;
+  }
+
+  stop.store(true, std::memory_order_release);
+  for (auto& reader : readers) {
+    reader.join();
+  }
+
+  // Every key must now hold the final round's value. Catches wholesale
+  // corruption the sampling readers might miss, and confirms compaction
+  // finished under the bypass path.
+  std::string value;
+  for (int i = 0; i < kNumKeys; ++i) {
+    ASSERT_OK(db_->Get(ReadOptions(), Key(i), &value));
+    EXPECT_EQ(last_value, value);
+  }
+  EXPECT_GT(reads_observed.load(), 0)
+      << "reader threads never observed a single read; test was a no-op";
+  EXPECT_EQ(0, read_errors.load())
+      << "concurrent reads against compaction with bypass-path saw "
+      << read_errors.load() << " non-OK/non-NotFound status returns";
+
+  Destroy(options);
+}
+#endif  // !defined(OS_MACOSX) && !defined(OS_OPENBSD) && ...
+
 class CompactionPriTest : public DBTestBase,
                           public testing::WithParamInterface<uint32_t> {
  public:

db/db_etc2_test.cc

@@ -35,7 +35,7 @@ namespace ROCKSDB_NAMESPACE {
 
 class DBTest2 : public DBTestBase {
  public:
-  DBTest2() : DBTestBase("db_test2", /*env_do_fsync=*/true) {}
+  DBTest2() : DBTestBase("db_etc2_test", /*env_do_fsync=*/true) {}
 };
 
 TEST_F(DBTest2, OpenForReadOnly) {

db/db_etc3_test.cc

@@ -4,6 +4,9 @@
 //  (found in the LICENSE.Apache file in the root directory).
 
 #include "db/db_test_util.h"
+#include "rocksdb/convenience.h"
+#include "rocksdb/metadata.h"
+#include "rocksdb/sst_file_writer.h"
 
 namespace ROCKSDB_NAMESPACE {
 
@@ -43,50 +46,98 @@ TEST_F(DBEtc3Test, AutoTuneManifestSize) {
   ASSERT_EQ(DBOptions{}.max_manifest_space_amp_pct, 500);
 
   Options options = CurrentOptions();
-  ASSERT_OK(db_->SetOptions({{"level0_file_num_compaction_trigger", "20"}}));
+  // Test setup: create many flushed files. Keep level compaction semantics so
+  // DeleteFilesInRanges can remove non-L0 files, but prevent automatic
+  // compactions and write stalls from adding unrelated behavior.
+  options.disable_auto_compactions = true;
+  options.level0_slowdown_writes_trigger = 100;
+  options.level0_stop_writes_trigger = 200;
 
-  // Use large column family names to essentially control the amount of payload
-  // data needed for the manifest file. Drop manifest entries don't include the
-  // CF name so are small.
+  // Test strategy: use large column family names to control the rough amount
+  // of payload added to the MANIFEST. Drop manifest entries do not include the
+  // CF name, so they are small.
+  //
+  // Most CF helper calls piggy-back a background manifest write so the main
+  // auto-tuning phases continue to test the unrelaxed background threshold
+  // even though CF manipulation itself is foreground. Phase-specific foreground
+  // checks disable that piggy-backed background write.
   uint64_t prev_manifest_num = 0, cur_manifest_num = 0;
   std::deque<ColumnFamilyHandle*> handles;
   int counter = 5;
-  auto AddCfFn = [&]() {
+  auto UpdateManifestNumsFrom = [&](uint64_t before_manifest_num) {
+    prev_manifest_num = before_manifest_num;
+    cur_manifest_num = dbfull()->TEST_Current_Manifest_FileNo();
+  };
+  auto BackgroundManifestWriteFn = [&]() {
+    uint64_t before_manifest_num = cur_manifest_num;
+    ASSERT_OK(Put("x", std::to_string(counter++)));
+    ASSERT_OK(Flush());
+    UpdateManifestNumsFrom(before_manifest_num);
+  };
+  auto AddCfFn = [&](bool include_background_manifest_write = true) {
+    uint64_t before_manifest_num = cur_manifest_num;
     std::string name = "cf" + std::to_string(counter++);
     name.resize(1000, 'a');
     ASSERT_OK(db_->CreateColumnFamily(options, name, &handles.emplace_back()));
-    prev_manifest_num = cur_manifest_num;
     cur_manifest_num = dbfull()->TEST_Current_Manifest_FileNo();
+    if (include_background_manifest_write) {
+      BackgroundManifestWriteFn();
+    }
+    UpdateManifestNumsFrom(before_manifest_num);
   };
-  auto DropCfFn = [&]() {
+  auto DropCfFn = [&](bool include_background_manifest_write = true) {
+    uint64_t before_manifest_num = cur_manifest_num;
     ASSERT_OK(db_->DropColumnFamily(handles.front()));
     ASSERT_OK(db_->DestroyColumnFamilyHandle(handles.front()));
     handles.pop_front();
-    prev_manifest_num = cur_manifest_num;
-    cur_manifest_num = dbfull()->TEST_Current_Manifest_FileNo();
-  };
-  auto TrivialManifestWriteFn = [&]() {
-    ASSERT_OK(Put("x", std::to_string(counter++)));
-    ASSERT_OK(Flush());
-    prev_manifest_num = cur_manifest_num;
     cur_manifest_num = dbfull()->TEST_Current_Manifest_FileNo();
+    if (include_background_manifest_write) {
+      BackgroundManifestWriteFn();
+    }
+    UpdateManifestNumsFrom(before_manifest_num);
   };
 
+  // ---- Phase 1: foreground threshold relaxation is bounded ----
+  //
+  // Foreground operations should only get about 25% extra headroom, not an
+  // unbounded threshold. With 1000-byte CF names and a 3000-byte normal limit,
+  // the relaxed limit should allow the first four foreground-only CF additions
+  // but require rotation on the fifth.
+  options.max_manifest_file_size = 3000;
+  options.max_manifest_space_amp_pct = 0;
+  DestroyAndReopen(options);
+  cur_manifest_num = dbfull()->TEST_Current_Manifest_FileNo();
+  prev_manifest_num = cur_manifest_num;
+  for (int i = 1; i <= 4; ++i) {
+    AddCfFn(/*include_background_manifest_write=*/false);
+    ASSERT_EQ(prev_manifest_num, cur_manifest_num);
+  }
+  AddCfFn(/*include_background_manifest_write=*/false);
+  ASSERT_LT(prev_manifest_num, cur_manifest_num);
+  while (!handles.empty()) {
+    ASSERT_OK(db_->DestroyColumnFamilyHandle(handles.front()));
+    handles.pop_front();
+  }
+
+  // ---- Phase 2: no auto-tuning means frequent rotation ----
+  //
   options.max_manifest_file_size = 1000000;
   options.max_manifest_space_amp_pct = 0;  // no auto-tuning yet
   DestroyAndReopen(options);
+  cur_manifest_num = dbfull()->TEST_Current_Manifest_FileNo();
+  prev_manifest_num = cur_manifest_num;
 
-  // With the generous (minimum) maximum manifest size, should not be rotated
+  // With the generous minimum manifest size, should not be rotated.
   AddCfFn();
   AddCfFn();
   AddCfFn();
   ASSERT_EQ(prev_manifest_num, cur_manifest_num);
 
-  // Change options for small max and (still) no auto-tuning
+  // Lower the minimum while still disabling auto-tuning.
   ASSERT_OK(db_->SetDBOptions({{"max_manifest_file_size", "3000"}}));
 
   // Takes effect on the next manifest write
-  TrivialManifestWriteFn();
+  BackgroundManifestWriteFn();
   ASSERT_LT(prev_manifest_num, cur_manifest_num);
 
   // Now we have to rewrite the whole manifest on each write because the
@@ -97,28 +148,31 @@ TEST_F(DBEtc3Test, AutoTuneManifestSize) {
   ASSERT_LT(prev_manifest_num, cur_manifest_num);
   AddCfFn();
   ASSERT_LT(prev_manifest_num, cur_manifest_num);
-  TrivialManifestWriteFn();
+  BackgroundManifestWriteFn();
   ASSERT_LT(prev_manifest_num, cur_manifest_num);
 
+  // ---- Phase 3: auto-tuning raises the background threshold ----
+  //
   // Enabling auto-tuning should fix this, immediately for next manifest writes.
-  // This will allow up to double-ish the size of the compacted manifest,
-  // which last should have been 4000 + some bytes.
+  // This will allow up to roughly double the size of the compacted manifest,
+  // which now includes CF entries plus the piggy-backed background writes.
   ASSERT_EQ(handles.size(), 4U);
-  ASSERT_OK(db_->SetDBOptions({{"max_manifest_space_amp_pct", "105"}}));
+  ASSERT_OK(db_->SetDBOptions({{"max_manifest_space_amp_pct", "95"}}));
 
-  // After 9 CF names should be enough to rotate the manifest
-  for (int i = 1; i <= 5; ++i) {
+  // Auto-tuning lets several more CF names accumulate in the MANIFEST before
+  // the piggy-backed background write crosses the threshold.
+  for (int i = 1; i <= 3; ++i) {
     if ((i % 2) == 1) {
       DropCfFn();
     }
     AddCfFn();
     ASSERT_EQ(prev_manifest_num, cur_manifest_num);
   }
-  TrivialManifestWriteFn();
+  AddCfFn();
   ASSERT_LT(prev_manifest_num, cur_manifest_num);
 
-  // We now have a different last compacted manifest size, should be
-  // able to go beyond 9 CFs named in manifest this time.
+  // We now have a different last compacted manifest size, so the next
+  // threshold should be based on the newly compacted MANIFEST.
   ASSERT_EQ(handles.size(), 6U);
 
   DropCfFn();
@@ -128,11 +182,10 @@ TEST_F(DBEtc3Test, AutoTuneManifestSize) {
     AddCfFn();
     ASSERT_EQ(prev_manifest_num, cur_manifest_num);
   }
-  // We've written 10 named CFs to the manifest. We should be able to
-  // dynamically change the auto-tuning still based on the last "compacted"
-  // manifest size of 7000 + some bytes.
+  // We should be able to dynamically change the auto-tuning still based on
+  // the last "compacted" manifest size.
   ASSERT_OK(db_->SetDBOptions({{"max_manifest_space_amp_pct", "51"}}));
-  TrivialManifestWriteFn();
+  BackgroundManifestWriteFn();
   ASSERT_LT(prev_manifest_num, cur_manifest_num);
   // And the "compacted" manifest size has reset again, so should be changed
   // again sooner.
@@ -141,16 +194,129 @@ TEST_F(DBEtc3Test, AutoTuneManifestSize) {
     AddCfFn();
     ASSERT_EQ(prev_manifest_num, cur_manifest_num);
   }
-  // Enough for manifest change
-  AddCfFn();
+
+  // ---- Phase 4: foreground operations use relaxed threshold ----
+  //
+  // The current MANIFEST is now large enough for the next background manifest
+  // write to rotate it, but still small enough for foreground operations to use
+  // their 25% extra headroom. Assert that each foreground operation stays on
+  // the same MANIFEST, then verify the next background write rotates.
+  const std::string external_files_dir = dbname_ + "/external_files";
+  ASSERT_OK(env_->CreateDirIfMissing(external_files_dir));
+  auto WriteExternalSstFile = [&](const std::string& file_name,
+                                  const std::string& key,
+                                  const std::string& value) {
+    const std::string file_path = external_files_dir + "/" + file_name;
+    Status ignored = env_->DeleteFile(file_path);
+    ignored.PermitUncheckedError();
+
+    SstFileWriter sst_file_writer(EnvOptions(), options);
+    ASSERT_OK(sst_file_writer.Open(file_path));
+    ASSERT_OK(sst_file_writer.Put(key, value));
+    ASSERT_OK(sst_file_writer.Finish());
+  };
+  auto IngestExternalFileForegroundManifestWriteFn =
+      [&](std::string* ingested_key) {
+        uint64_t before_manifest_num = cur_manifest_num;
+        const std::string key = "z" + std::to_string(counter++);
+        const std::string value = "v" + std::to_string(counter++);
+        const std::string file_name =
+            "ingest_" + std::to_string(counter++) + ".sst";
+        const std::string file_path = external_files_dir + "/" + file_name;
+        WriteExternalSstFile(file_name, key, value);
+
+        ASSERT_OK(
+            db_->IngestExternalFile({file_path}, IngestExternalFileOptions()));
+        ASSERT_EQ(value, Get(key));
+        *ingested_key = key;
+        Status ignored = env_->DeleteFile(file_path);
+        ignored.PermitUncheckedError();
+        UpdateManifestNumsFrom(before_manifest_num);
+      };
+  auto CreateColumnFamilyWithImportForegroundManifestWriteFn = [&]() {
+    uint64_t before_manifest_num = cur_manifest_num;
+    const std::string cf_name = "import_cf" + std::to_string(counter++);
+    const std::string key = "import" + std::to_string(counter++);
+    const std::string value = "v" + std::to_string(counter++);
+    const std::string file_name =
+        "import_" + std::to_string(counter++) + ".sst";
+    const std::string file_path = external_files_dir + "/" + file_name;
+    WriteExternalSstFile(file_name, key, value);
+
+    LiveFileMetaData file_metadata;
+    file_metadata.name = file_name;
+    file_metadata.db_path = external_files_dir;
+    file_metadata.smallest_seqno = 0;
+    file_metadata.largest_seqno = 0;
+    file_metadata.level = 0;
+    ExportImportFilesMetaData metadata;
+    metadata.files.push_back(file_metadata);
+    metadata.db_comparator_name = options.comparator->Name();
+
+    ColumnFamilyHandle* import_handle = nullptr;
+    ASSERT_OK(db_->CreateColumnFamilyWithImport(options, cf_name,
+                                                ImportColumnFamilyOptions(),
+                                                metadata, &import_handle));
+    ASSERT_NE(import_handle, nullptr);
+    handles.push_back(import_handle);
+
+    std::string result;
+    ASSERT_OK(db_->Get(ReadOptions(), import_handle, key, &result));
+    ASSERT_EQ(value, result);
+    Status ignored = env_->DeleteFile(file_path);
+    ignored.PermitUncheckedError();
+    UpdateManifestNumsFrom(before_manifest_num);
+  };
+  auto DeleteFilesInRangesForegroundManifestWriteFn =
+      [&](const std::string& key) {
+        uint64_t before_manifest_num = cur_manifest_num;
+        const std::string limit = key + "\xff";
+        std::vector<RangeOpt> ranges;
+        ranges.emplace_back(key, limit);
+        ASSERT_OK(DeleteFilesInRanges(db_.get(), db_->DefaultColumnFamily(),
+                                      ranges.data(), ranges.size(),
+                                      /*include_end=*/false));
+
+        std::string result;
+        ASSERT_TRUE(db_->Get(ReadOptions(), key, &result).IsNotFound());
+        UpdateManifestNumsFrom(before_manifest_num);
+      };
+
+  // Column family manipulation.
+  AddCfFn(/*include_background_manifest_write=*/false);
+  ASSERT_EQ(prev_manifest_num, cur_manifest_num);
+
+  // SetOptions should not write to the MANIFEST. If that regresses, the write
+  // should be treated as a background write and rotate here.
+  {
+    ASSERT_OK(db_->SetOptions({{"level0_slowdown_writes_trigger", "101"}}));
+    UpdateManifestNumsFrom(cur_manifest_num);
+  }
+  ASSERT_EQ(prev_manifest_num, cur_manifest_num);
+
+  // External file ingestion.
+  std::string ingested_key;
+  IngestExternalFileForegroundManifestWriteFn(&ingested_key);
+  ASSERT_EQ(prev_manifest_num, cur_manifest_num);
+
+  // Imported column family creation.
+  CreateColumnFamilyWithImportForegroundManifestWriteFn();
+  ASSERT_EQ(prev_manifest_num, cur_manifest_num);
+
+  // Physical file deletion by range.
+  DeleteFilesInRangesForegroundManifestWriteFn(ingested_key);
+  ASSERT_EQ(prev_manifest_num, cur_manifest_num);
+
+  // Background flush.
+  BackgroundManifestWriteFn();
   ASSERT_LT(prev_manifest_num, cur_manifest_num);
 
-  // ---- Verify persisted compacted manifest size survives close/reopen ----
+  // ---- Phase 5: persisted compacted manifest size survives close/reopen ----
   // Close with CFs still live. Reopen with reuse_manifest_on_open so the
   // manifest is NOT rewritten from scratch. The persisted compacted size
   // should be loaded and used for auto-tuning.
-  // At this point we have 7 CF handles plus default.
-  ASSERT_EQ(handles.size(), 7U);
+  // At this point we have 8 CF handles plus default.
+  ASSERT_EQ(handles.size(), 8U);
 
   // Collect CF names for reopen, then release handles (Close needs this)
   std::vector<std::string> cf_names = {"default"};
@@ -163,18 +329,16 @@ TEST_F(DBEtc3Test, AutoTuneManifestSize) {
   handles.clear();
 
   Close();
-  // Use a large max_manifest_file_size so the reused manifest (which is
-  // already ~10KB) does NOT trigger rotation on the first few writes.
-  // Auto-tuning with the persisted compacted size (~5KB) at 200% amp
-  // gives a tuned threshold of ~15KB. Without persistence, the threshold
-  // would be max(max_manifest_file_size, 0 * anything) =
-  // max_manifest_file_size.
+  // Use a max_manifest_file_size below the reused manifest size. Auto-tuning
+  // with the persisted compacted size at 200% amp keeps the tuned threshold
+  // high enough. Without persistence, the threshold would be
+  // max(max_manifest_file_size, 0 * anything) = max_manifest_file_size.
   //
-  // We set max_manifest_file_size to two values to distinguish:
-  // - 3000: if persisted compacted size is NOT loaded, tuned = 3000,
-  //   and the first AddCf will rotate (manifest is already ~10KB > 3000)
-  // - With persisted compacted size loaded, tuned = max(3000, 5000*3) = 15000,
-  //   so no rotation until we exceed 15KB
+  // With max_manifest_file_size set to 3000, missing persisted compacted size
+  // would keep tuned = 3000 and the first AddCf would rotate because the
+  // reused manifest is already larger. With persisted compacted size loaded,
+  // the tuned threshold is based on the compacted size, so no rotation until
+  // the manifest grows well beyond the minimum.
   options.max_manifest_file_size = 3000;
   options.max_manifest_space_amp_pct = 200;
   options.reuse_manifest_on_open = true;
@@ -186,11 +350,11 @@ TEST_F(DBEtc3Test, AutoTuneManifestSize) {
   // during reopen, because the persisted compacted size keeps the tuned
   // threshold high enough. Without persistence, last_compacted = 0, so
   // tuned = max_manifest_file_size = 3000, and the first LogAndApply
-  // during Open rotates the manifest because it's already ~10KB > 3000.
+  // during Open rotates the manifest because it is already larger than 3000.
   ASSERT_EQ(manifest_num_before_reopen, cur_manifest_num);
 
-  // Adding CFs should still not trigger rotation because the tuned
-  // threshold (~15KB) exceeds the current manifest size (~10KB + adds).
+  // Adding CFs should still not trigger rotation because the tuned threshold
+  // from the persisted compacted size exceeds the current manifest size.
   for (int i = 1; i <= 4; ++i) {
     AddCfFn();
     ASSERT_EQ(prev_manifest_num, cur_manifest_num);
@@ -202,7 +366,7 @@ TEST_F(DBEtc3Test, AutoTuneManifestSize) {
 
   // Wrap up
   while (!handles.empty()) {
-    DropCfFn();
+    DropCfFn(/*include_background_manifest_write=*/false);
   }
 }
 

db/db_impl/db_impl.cc

@@ -102,6 +102,7 @@
 #include "table/get_context.h"
 #include "table/merging_iterator.h"
 #include "table/multiget_context.h"
+#include "table/prepared_file_info.h"
 #include "table/sst_file_dumper.h"
 #include "table/table_builder.h"
 #include "table/two_level_iterator.h"
@@ -832,6 +833,10 @@ Status DBImpl::CloseHelper() {
   assert(!immutable_db_options_.open_files_async || !opened_successfully_ ||
          bg_async_file_open_state_ != AsyncFileOpenState::kNotScheduled);
 
+  // No FileIngestionHandle from PrepareFileIngestion() may still be
+  // outstanding
+  assert(num_outstanding_prepared_ingestions_.load() == 0);
+
   TEST_SYNC_POINT_CALLBACK("DBImpl::CloseHelper:PendingPurgeFinished",
                            &files_grabbed_for_purge_);
   EraseThreadStatusDbInfo();
@@ -853,6 +858,8 @@ Status DBImpl::CloseHelper() {
   if (default_cf_handle_ != nullptr || persist_stats_cf_handle_ != nullptr) {
     // we need to delete handle outside of lock because it does its own locking
     mutex_.Unlock();
+    TEST_SYNC_POINT("DBImpl::CloseHelper:CFHandleCleanupUnlocked");
+    TEST_SYNC_POINT("DBImpl::CloseHelper:CFHandleCleanupAllowed");
     if (default_cf_handle_) {
       delete default_cf_handle_;
       default_cf_handle_ = nullptr;
@@ -920,25 +927,6 @@ Status DBImpl::CloseHelper() {
     logs_.clear();
   }
 
-  // Table cache may have table handles holding blocks from the block cache.
-  // We need to release them before the block cache is destroyed. The block
-  // cache may be destroyed inside versions_.reset(), when column family data
-  // list is destroyed, so leaving handles in table cache after
-  // versions_.reset() may cause issues. Here we clean all unreferenced handles
-  // in table cache, and (for certain builds/conditions) assert that no obsolete
-  // files are hanging around unreferenced (leak) in the table/blob file cache.
-  // Now we assume all user queries have finished, so only version set itself
-  // can possibly hold the blocks from block cache. After releasing unreferenced
-  // handles here, only handles held by version set left and inside
-  // versions_.reset(), we will release them. There, we need to make sure every
-  // time a handle is released, we erase it from the cache too. By doing that,
-  // we can guarantee that after versions_.reset(), table cache is empty
-  // so the cache can be safely destroyed.
-#ifndef NDEBUG
-  TEST_VerifyNoObsoleteFilesCached(/*db_mutex_already_held=*/true);
-#endif  // !NDEBUG
-  table_cache_->EraseUnRefEntries();
-
   for (auto& txn_entry : recovered_transactions_) {
     delete txn_entry.second;
   }
@@ -964,6 +952,42 @@ Status DBImpl::CloseHelper() {
     }
   }
 
+  // Drain obsolete-file purge work started AFTER the early CloseHelper wait
+  // near the top of this method. A late SuperVersion cleanup -- a dropped
+  // column family handle, or an in-flight iterator/Get whose ReadOptions or
+  // immutable_db_options_.avoid_unnecessary_blocking_io selected background
+  // purge -- can run in one of the mutex-unlocked windows above and enter the
+  // FindObsoleteFiles() -> PurgeObsoleteFiles(..., true) handoff. During that
+  // handoff pending_purge_obsolete_files_ is already nonzero, but
+  // bg_purge_scheduled_ may still be zero until SchedulePurge() runs at the end
+  // of PurgeObsoleteFiles(). Wait for both states while mutex_/this are still
+  // alive; bg_cv_.Wait() releases mutex_ so pending and scheduled purges can
+  // finish and signal.
+  TEST_SYNC_POINT("DBImpl::CloseHelper:BeforeFinalPurgeDrain");
+  while (pending_purge_obsolete_files_ || bg_purge_scheduled_) {
+    TEST_SYNC_POINT("DBImpl::CloseHelper:FinalPurgeDrainWait");
+    bg_cv_.Wait();
+  }
+
+  // Table cache may have table handles holding blocks from the block cache.
+  // We need to release them before the block cache is destroyed. The block
+  // cache may be destroyed inside versions_.reset(), when column family data
+  // list is destroyed, so leaving handles in table cache after
+  // versions_.reset() may cause issues. Here we clean all unreferenced handles
+  // in table cache, and (for certain builds/conditions) assert that no obsolete
+  // files are hanging around unreferenced (leak) in the table/blob file cache.
+  // Now we assume all user queries have finished, and close-time purge work has
+  // settled, so only version set itself can possibly hold the blocks from block
+  // cache. After releasing unreferenced handles here, only handles held by
+  // version set left and inside versions_.reset(), we will release them. There,
+  // we need to make sure every time a handle is released, we erase it from the
+  // cache too. By doing that, we can guarantee that after versions_.reset(),
+  // table cache is empty so the cache can be safely destroyed.
+#ifndef NDEBUG
+  TEST_VerifyNoObsoleteFilesCached(/*db_mutex_already_held=*/true);
+#endif  // !NDEBUG
+  table_cache_->EraseUnRefEntries();
+
   versions_.reset();
   mutex_.Unlock();
   if (db_lock_ != nullptr) {
@@ -1842,6 +1866,8 @@ Status DBImpl::SetDBOptions(
       // TODO(xiez): clarify why apply optimize for read to write options
       file_options_for_compaction_ = fs_->OptimizeForCompactionTableRead(
           file_options_for_compaction_, immutable_db_options_);
+      TEST_SYNC_POINT_CALLBACK("DBImpl::SetDBOptions:FileOptionsForCompaction",
+                               &file_options_for_compaction_);
       if (wal_other_option_changed || wal_size_option_changed) {
         WriteThread::Writer w;
         write_thread_.EnterUnbatched(&w, &mutex_);
@@ -2473,30 +2499,8 @@ struct SuperVersionHandle {
 static void CleanupSuperVersionHandle(void* arg1, void* /*arg2*/) {
   SuperVersionHandle* sv_handle = static_cast<SuperVersionHandle*>(arg1);
 
-  if (sv_handle->super_version->Unref()) {
-    // Job id == 0 means that this is not our background process, but rather
-    // user thread
-    JobContext job_context(0);
-
-    sv_handle->mu->Lock();
-    sv_handle->super_version->Cleanup();
-    sv_handle->db->FindObsoleteFiles(&job_context, false, true);
-    if (sv_handle->background_purge) {
-      sv_handle->db->ScheduleBgLogWriterClose(&job_context);
-      sv_handle->db->AddSuperVersionsToFreeQueue(sv_handle->super_version);
-      sv_handle->db->SchedulePurge();
-    }
-    sv_handle->mu->Unlock();
-
-    if (!sv_handle->background_purge) {
-      delete sv_handle->super_version;
-    }
-    if (job_context.HaveSomethingToDelete()) {
-      sv_handle->db->PurgeObsoleteFiles(job_context,
-                                        sv_handle->background_purge);
-    }
-    job_context.Clean();
-  }
+  sv_handle->db->CleanupIteratorSuperVersion(sv_handle->super_version,
+                                             sv_handle->background_purge);
 
   delete sv_handle;
 }
@@ -2518,7 +2522,8 @@ static void CleanupGetMergeOperandsState(void* arg1, void* /*arg2*/) {
 InternalIterator* DBImpl::NewInternalIterator(
     const ReadOptions& read_options, ColumnFamilyData* cfd,
     SuperVersion* super_version, Arena* arena, SequenceNumber sequence,
-    bool allow_unprepared_value, ArenaWrappedDBIter* db_iter) {
+    bool allow_unprepared_value, ArenaWrappedDBIter* db_iter,
+    const MultiScanArgs* scan_opts) {
   InternalIterator* internal_iter;
   assert(arena != nullptr);
   auto prefix_extractor =
@@ -2530,47 +2535,62 @@ InternalIterator* DBImpl::NewInternalIterator(
       // here, and no unit test cares about the value provided here.
       !read_options.total_order_seek && prefix_extractor != nullptr,
       read_options.iterate_upper_bound);
-  // Collect iterator for mutable memtable
-  auto mem_iter = super_version->mem->NewIterator(
-      read_options, super_version->GetSeqnoToTimeMapping(), arena,
-      super_version->mutable_cf_options.prefix_extractor.get(),
-      /*for_flush=*/false);
   Status s;
-  if (!read_options.ignore_range_deletions) {
-    std::unique_ptr<TruncatedRangeDelIterator> mem_tombstone_iter;
-    auto range_del_iter = super_version->mem->NewRangeTombstoneIterator(
-        read_options, sequence, false /* immutable_memtable */);
-    if (range_del_iter == nullptr || range_del_iter->empty()) {
-      delete range_del_iter;
+  const Comparator* user_comparator = cfd->user_comparator();
+  const bool mem_intersects =
+      !super_version->mem->IsEmpty() &&
+      MultiScanIntersectsMemTable(super_version->mem, read_options,
+                                  super_version->GetSeqnoToTimeMapping(),
+                                  prefix_extractor, scan_opts, user_comparator);
+  if (scan_opts == nullptr || mem_intersects) {
+    // Collect iterator for mutable memtable
+    auto mem_iter = super_version->mem->NewIterator(
+        read_options, super_version->GetSeqnoToTimeMapping(), arena,
+        super_version->mutable_cf_options.prefix_extractor.get(),
+        /*for_flush=*/false);
+    if (!read_options.ignore_range_deletions) {
+      std::unique_ptr<TruncatedRangeDelIterator> mem_tombstone_iter;
+      std::unique_ptr<FragmentedRangeTombstoneIterator> range_del_iter(
+          super_version->mem->NewRangeTombstoneIterator(
+              read_options, sequence, false /* immutable_memtable */));
+      if (range_del_iter != nullptr && !range_del_iter->empty()) {
+        mem_tombstone_iter = std::make_unique<TruncatedRangeDelIterator>(
+            std::move(range_del_iter), &cfd->ioptions().internal_comparator,
+            nullptr /* smallest */, nullptr /* largest */);
+      }
+      merge_iter_builder.AddPointAndTombstoneIterator(
+          mem_iter, std::move(mem_tombstone_iter));
     } else {
-      mem_tombstone_iter = std::make_unique<TruncatedRangeDelIterator>(
-          std::unique_ptr<FragmentedRangeTombstoneIterator>(range_del_iter),
-          &cfd->ioptions().internal_comparator, nullptr /* smallest */,
-          nullptr /* largest */);
+      merge_iter_builder.AddIterator(mem_iter);
     }
-    merge_iter_builder.AddPointAndTombstoneIterator(
-        mem_iter, std::move(mem_tombstone_iter));
-  } else {
-    merge_iter_builder.AddIterator(mem_iter);
+  } else if (scan_opts != nullptr) {
+    merge_iter_builder.SetMemtablePruned(true);
   }
 
-  // Collect all needed child iterators for immutable memtables
-  if (s.ok()) {
+  if (s.ok() &&
+      (scan_opts == nullptr || super_version->imm->GetTotalNumEntries() > 0)) {
+    // Collect all needed child iterators for immutable memtables. When scan
+    // ranges are provided, AddIterators prunes each immutable memtable
+    // individually.
     super_version->imm->AddIterators(
         read_options, super_version->GetSeqnoToTimeMapping(),
         super_version->mutable_cf_options.prefix_extractor.get(),
-        &merge_iter_builder, !read_options.ignore_range_deletions);
+        &merge_iter_builder, !read_options.ignore_range_deletions, sequence,
+        scan_opts, user_comparator);
   }
   TEST_SYNC_POINT_CALLBACK("DBImpl::NewInternalIterator:StatusCallback", &s);
   if (s.ok()) {
     // Collect iterators for files in L0 - Ln
     if (read_options.read_tier != kMemtableTier) {
-      super_version->current->AddIterators(read_options, file_options_,
-                                           &merge_iter_builder,
-                                           allow_unprepared_value);
+      super_version->current->AddIterators(
+          read_options, file_options_, &merge_iter_builder,
+          allow_unprepared_value, sequence, scan_opts);
     }
     internal_iter = merge_iter_builder.Finish(
         read_options.ignore_range_deletions ? nullptr : db_iter);
+    if (internal_iter == nullptr) {
+      internal_iter = NewEmptyInternalIterator<Slice>(arena);
+    }
     SuperVersionHandle* cleanup = new SuperVersionHandle(
         this, &mutex_, super_version,
         read_options.background_purge_on_iterator_cleanup ||
@@ -2584,6 +2604,35 @@ InternalIterator* DBImpl::NewInternalIterator(
   return NewErrorInternalIterator<Slice>(s, arena);
 }
 
+void DBImpl::CleanupIteratorSuperVersion(SuperVersion* super_version,
+                                         bool background_purge) {
+  background_purge =
+      background_purge || immutable_db_options_.avoid_unnecessary_blocking_io;
+  if (super_version->Unref()) {
+    // Job id == 0 means that this is not our background process, but rather
+    // user thread
+    JobContext job_context(0);
+
+    mutex_.Lock();
+    super_version->Cleanup();
+    FindObsoleteFiles(&job_context, false, true);
+    if (background_purge) {
+      ScheduleBgLogWriterClose(&job_context);
+      AddSuperVersionsToFreeQueue(super_version);
+      SchedulePurge();
+    }
+    mutex_.Unlock();
+
+    if (!background_purge) {
+      delete super_version;
+    }
+    if (job_context.HaveSomethingToDelete()) {
+      PurgeObsoleteFiles(job_context, background_purge);
+    }
+    job_context.Clean();
+  }
+}
+
 ColumnFamilyHandle* DBImpl::DefaultColumnFamily() const {
   return default_cf_handle_;
 }
@@ -5853,6 +5902,7 @@ Status DBImpl::DeleteFilesInRanges(ColumnFamilyHandle* column_family,
   }
 
   VersionEdit edit;
+  edit.MarkForegroundOperation();
   std::set<FileMetaData*> deleted_files;
   JobContext job_context(next_job_id_.fetch_add(1), true);
   {
@@ -5950,6 +6000,79 @@ Status DBImpl::GetLiveFilesChecksumInfo(FileChecksumList* checksum_list) {
   return versions_->GetLiveFilesChecksumInfo(checksum_list);
 }
 
+Status DBImpl::GetPreparedFileInfoForExternalSstIngestion(
+    const std::string& file_path,
+    std::shared_ptr<const PreparedFileInfo>* file_info) {
+  if (file_info == nullptr) {
+    return Status::InvalidArgument("file_info must not be null");
+  }
+  file_info->reset();
+
+  const size_t file_name_pos = file_path.find_last_of("/\\");
+  const std::string file_name = file_name_pos == std::string::npos
+                                    ? file_path
+                                    : file_path.substr(file_name_pos + 1);
+  uint64_t file_number = 0;
+  FileType file_type;
+  if (!ParseFileName(file_name, &file_number, &file_type) ||
+      file_type != kTableFile || file_number == 0) {
+    return Status::InvalidArgument("Invalid table file name: " + file_path);
+  }
+
+  int file_level = -1;
+  FileMetaData* file_meta = nullptr;
+  ColumnFamilyData* file_cfd = nullptr;
+  Version* file_version = nullptr;
+  std::shared_ptr<const TableProperties> table_properties;
+  ReadOptions read_options;
+
+  {
+    InstrumentedMutexLock l(&mutex_);
+    Status s = versions_->GetMetadataForFile(file_number, &file_level,
+                                             &file_meta, &file_cfd);
+    if (!s.ok()) {
+      return s;
+    }
+
+    const std::string expected_file_path = TableFileName(
+        file_cfd->ioptions().cf_paths, file_number, file_meta->fd.GetPathId());
+    if (file_path != expected_file_path) {
+      return Status::InvalidArgument("Path does not match live table file: " +
+                                     file_path);
+    }
+
+    file_cfd->Ref();
+    file_version = file_cfd->current();
+    file_version->Ref();
+  }
+  const Defer cleanup_refs([&]() {
+    InstrumentedMutexLock l(&mutex_);
+    file_version->Unref();
+    file_cfd->UnrefAndTryDelete();
+  });
+
+  Status s = file_cfd->table_cache()->GetTableProperties(
+      file_options_, read_options, file_cfd->internal_comparator(), *file_meta,
+      &table_properties, file_version->GetMutableCFOptions(),
+      false /* no_io */);
+  if (!s.ok()) {
+    return s;
+  }
+  assert(table_properties != nullptr);
+
+  auto prepared_file_info = std::make_shared<PreparedFileInfo>();
+  prepared_file_info->file_size = file_meta->fd.GetFileSize();
+  prepared_file_info->smallest = file_meta->smallest;
+  prepared_file_info->largest = file_meta->largest;
+  prepared_file_info->table_properties = *table_properties;
+  prepared_file_info->table_properties.key_largest_seqno =
+      file_meta->fd.largest_seqno;
+  prepared_file_info->table_properties.key_smallest_seqno =
+      file_meta->fd.smallest_seqno;
+  *file_info = std::move(prepared_file_info);
+  return s;
+}
+
 void DBImpl::GetColumnFamilyMetaData(ColumnFamilyHandle* column_family,
                                      ColumnFamilyMetaData* cf_meta) {
   assert(column_family);
@@ -6672,11 +6795,76 @@ Status DBImpl::IngestExternalFile(
   return IngestExternalFiles({arg});
 }
 
-Status DBImpl::IngestExternalFiles(
-    const std::vector<IngestExternalFileArg>& args) {
-  PERF_TIMER_GUARD(file_ingestion_nanos);
-  // TODO: plumb Env::IOActivity, Env::IOPriority
-  const WriteOptions write_options;
+class FileIngestionHandleImpl : public FileIngestionHandle {
+ public:
+  explicit FileIngestionHandleImpl(DBImpl* db) : db_(db) {}
+  ~FileIngestionHandleImpl() override;
+
+  FileIngestionHandleImpl(const FileIngestionHandleImpl&) = delete;
+  FileIngestionHandleImpl& operator=(const FileIngestionHandleImpl&) = delete;
+  FileIngestionHandleImpl(FileIngestionHandleImpl&&) = delete;
+  FileIngestionHandleImpl& operator=(FileIngestionHandleImpl&&) = delete;
+
+  Status Abort() override;
+
+  DBImpl* const db_;
+  std::vector<ExternalSstFileIngestionJob> jobs_;
+  std::unique_ptr<std::list<uint64_t>::iterator> pending_output_elem_;
+  bool fill_cache_ = true;
+  // Set true once committed or aborted, so the destructor does not roll back.
+  bool consumed_ = false;
+};
+
+FileIngestionHandleImpl::~FileIngestionHandleImpl() {
+  if (consumed_ || db_ == nullptr) {
+    return;
+  }
+  // Dropped without commit or abort: roll back as a safety net.
+  db_->RollbackPreparedFileIngestion(this);
+  ROCKS_LOG_WARN(
+      db_->immutable_db_options_.info_log,
+      "[%zu CF(s)] File ingestion handle destroyed without commit or "
+      "abort; prepared files were rolled back.",
+      jobs_.size());
+}
+
+void DBImpl::RollbackPreparedFileIngestion(FileIngestionHandleImpl* const h) {
+  // Delete the staged internal files and release the reserved file numbers /
+  // pending-output protection, leaving the DB unchanged.
+  const Status rollback_status = Status::Incomplete("file ingestion aborted");
+  for (auto& job : h->jobs_) {
+    job.Cleanup(rollback_status);
+  }
+  {
+    InstrumentedMutexLock l(&mutex_);
+    ReleaseFileNumberFromPendingOutputs(h->pending_output_elem_);
+  }
+  h->consumed_ = true;
+  num_outstanding_prepared_ingestions_.fetch_sub(1);
+}
+
+Status FileIngestionHandleImpl::Abort() {
+  if (consumed_) {
+    return Status::InvalidArgument(
+        "file ingestion handle has already been committed or aborted");
+  }
+  db_->RollbackPreparedFileIngestion(this);
+  return Status::OK();
+}
+
+Status DBImpl::PrepareFileIngestion(
+    const std::vector<IngestExternalFileArg>& args,
+    std::unique_ptr<FileIngestionHandle>* handle) {
+  if (handle == nullptr) {
+    return Status::InvalidArgument("file ingestion handle output is null");
+  }
+  handle->reset();
+  // Recorded as INGEST_EXTERNAL_FILE_PREPARE_TIME on success below.
+  const bool record_ingest_micros =
+      stats_ != nullptr &&
+      stats_->get_stats_level() > StatsLevel::kExceptTimers;
+  const uint64_t prepare_start_micros =
+      record_ingest_micros ? immutable_db_options_.clock->NowMicros() : 0;
 
   if (args.empty()) {
     return Status::InvalidArgument("ingestion arg list is empty");
@@ -6701,6 +6889,17 @@ Status DBImpl::IngestExternalFiles(
           "external_files[" + std::to_string(i) + "] is empty";
       return Status::InvalidArgument(err_msg);
     }
+    if (!args[i].file_infos.empty()) {
+      if (args[i].file_infos.size() != args[i].external_files.size()) {
+        return Status::InvalidArgument("file_infos[" + std::to_string(i) +
+                                       "] size must match external_files[" +
+                                       std::to_string(i) + "] size");
+      }
+      if (args[i].options.write_global_seqno) {
+        return Status::InvalidArgument(
+            "write_global_seqno is not supported when file_infos is set");
+      }
+    }
     if (i && args[i].options.fill_cache != args[i - 1].options.fill_cache) {
       return Status::InvalidArgument(
           "fill_cache should be the same across ingestion options.");
@@ -6777,6 +6976,7 @@ Status DBImpl::IngestExternalFiles(
   }
 
   std::vector<ExternalSstFileIngestionJob> ingestion_jobs;
+  ingestion_jobs.reserve(num_cfs);
   for (const auto& arg : args) {
     auto* cfd = static_cast<ColumnFamilyHandleImpl*>(arg.column_family)->cfd();
     ingestion_jobs.emplace_back(versions_.get(), cfd, immutable_db_options_,
@@ -6794,8 +6994,9 @@ Status DBImpl::IngestExternalFiles(
             this);
     Status es = ingestion_jobs[i].Prepare(
         args[i].external_files, args[i].files_checksums,
-        args[i].files_checksum_func_names, args[i].atomic_replace_range,
-        args[i].file_temperature, start_file_number, super_version);
+        args[i].files_checksum_func_names, args[i].file_infos,
+        args[i].atomic_replace_range, args[i].file_temperature,
+        start_file_number, super_version);
     // capture first error only
     if (!es.ok() && status.ok()) {
       status = es;
@@ -6810,8 +7011,9 @@ Status DBImpl::IngestExternalFiles(
             this);
     Status es = ingestion_jobs[0].Prepare(
         args[0].external_files, args[0].files_checksums,
-        args[0].files_checksum_func_names, args[0].atomic_replace_range,
-        args[0].file_temperature, next_file_number, super_version);
+        args[0].files_checksum_func_names, args[0].file_infos,
+        args[0].atomic_replace_range, args[0].file_temperature,
+        next_file_number, super_version);
     if (!es.ok()) {
       status = es;
     }
@@ -6826,8 +7028,86 @@ Status DBImpl::IngestExternalFiles(
     return status;
   }
 
+  auto handle_impl = std::make_unique<FileIngestionHandleImpl>(this);
+  handle_impl->jobs_ = std::move(ingestion_jobs);
+  handle_impl->pending_output_elem_ = std::move(pending_output_elem);
+  handle_impl->fill_cache_ = args[0].options.fill_cache;
+  if (record_ingest_micros) {
+    RecordTimeToHistogram(
+        stats_, INGEST_EXTERNAL_FILE_PREPARE_TIME,
+        immutable_db_options_.clock->NowMicros() - prepare_start_micros);
+  }
+  num_outstanding_prepared_ingestions_.fetch_add(1);
+  *handle = std::move(handle_impl);
+  return Status::OK();
+}
+
+Status DBImpl::CommitFileIngestionHandles(
+    std::vector<std::unique_ptr<FileIngestionHandle>> handles) {
+  if (handles.empty()) {
+    return Status::InvalidArgument("no file ingestion handles to commit");
+  }
+  // Validate the handles and group their jobs by column family, merging same-CF
+  // jobs into one so each column family commits via a single atomic version
+  // edit and one SuperVersion install.
+  Status status;
+  std::vector<FileIngestionHandleImpl*> hs;
+  hs.reserve(handles.size());
+  std::vector<ExternalSstFileIngestionJob*> ingestion_jobs;
+  const bool fill_cache =
+      static_cast<FileIngestionHandleImpl*>(handles[0].get())->fill_cache_;
+  int max_file_opening_threads = 1;
+
+  {
+    UnorderedMap<ColumnFamilyData*, ExternalSstFileIngestionJob*>
+        primary_job_for_cfd;
+    for (const auto& handle : handles) {
+      assert(handle);
+      auto* h = static_cast<FileIngestionHandleImpl*>(handle.get());
+      assert(h->db_ == this);
+      if (h->consumed_) {
+        status = Status::InvalidArgument(
+            "file ingestion handle has already been committed or aborted");
+      }
+      if (h->fill_cache_ != fill_cache) {
+        status = Status::InvalidArgument(
+            "fill cache arg must be consistent across all handles");
+      }
+      if (!status.ok()) {
+        return status;
+      }
+
+      hs.push_back(h);
+      for (auto& job : h->jobs_) {
+        max_file_opening_threads =
+            std::max(max_file_opening_threads, job.file_opening_threads());
+        auto [it, inserted] =
+            primary_job_for_cfd.try_emplace(job.GetColumnFamilyData(), &job);
+        if (inserted) {
+          ingestion_jobs.push_back(&job);
+        } else {
+          status = it->second->MergeForSameColumnFamily(&job);
+          if (!status.ok()) {
+            return status;
+          }
+        }
+      }
+    }
+  }
+  const size_t num_jobs = ingestion_jobs.size();
+
+  // TODO: plumb Env::IOActivity, Env::IOPriority
+  const WriteOptions write_options;
+  // Decided here (not in Prepare) so the histograms reflect the stats level at
+  // commit time; recorded only on success below.
+  const bool record_ingest_micros =
+      stats_ != nullptr &&
+      stats_->get_stats_level() > StatsLevel::kExceptTimers;
+  const uint64_t run_start_micros =
+      record_ingest_micros ? immutable_db_options_.clock->NowMicros() : 0;
+
   std::vector<SuperVersionContext> sv_ctxs;
-  for (size_t i = 0; i != num_cfs; ++i) {
+  for (size_t i = 0; i != num_jobs; ++i) {
     sv_ctxs.emplace_back(true /* create_superversion */);
   }
   TEST_SYNC_POINT("DBImpl::IngestExternalFiles:BeforeJobsRun:0");
@@ -6838,9 +7118,9 @@ Status DBImpl::IngestExternalFiles(
   // mutex so the lock-acquisition order is ingest_sst_lock -> DB mutex
   // throughout. Use readlock so we still allow concurrent ingestions.
   std::vector<std::unique_ptr<ReadLock>> ingest_read_locks;
-  ingest_read_locks.reserve(num_cfs);
-  for (size_t i = 0; i != num_cfs; ++i) {
-    auto* cfd = ingestion_jobs[i].GetColumnFamilyData();
+  ingest_read_locks.reserve(num_jobs);
+  for (auto* job : ingestion_jobs) {
+    auto* cfd = job->GetColumnFamilyData();
     if (!cfd->IsDropped()) {
       ingest_read_locks.emplace_back(
           std::make_unique<ReadLock>(&cfd->GetIngestSstLock()));
@@ -6866,14 +7146,14 @@ Status DBImpl::IngestExternalFiles(
     // So wait here to ensure there is no pending write to memtable.
     WaitForPendingWrites();
 
-    num_running_ingest_file_ += static_cast<int>(num_cfs);
+    num_running_ingest_file_ += static_cast<int>(num_jobs);
     TEST_SYNC_POINT("DBImpl::IngestExternalFile:AfterIncIngestFileCounter");
     TEST_SYNC_POINT("DBImpl::IngestExternalFile:AfterIncIngestFileCounter:2");
 
     bool at_least_one_cf_need_flush = false;
-    std::vector<bool> need_flush(num_cfs, false);
-    for (size_t i = 0; i != num_cfs; ++i) {
-      auto* cfd = ingestion_jobs[i].GetColumnFamilyData();
+    std::vector<bool> need_flush(num_jobs, false);
+    for (size_t i = 0; i != num_jobs; ++i) {
+      auto* cfd = ingestion_jobs[i]->GetColumnFamilyData();
       if (cfd->IsDropped()) {
         // TODO (yanqin) investigate whether we should abort ingestion or
         // proceed with other non-dropped column families.
@@ -6882,7 +7162,7 @@ Status DBImpl::IngestExternalFiles(
         break;
       }
       bool tmp = false;
-      status = ingestion_jobs[i].NeedsFlush(&tmp, cfd->GetSuperVersion());
+      status = ingestion_jobs[i]->NeedsFlush(&tmp, cfd->GetSuperVersion());
       need_flush[i] = tmp;
       at_least_one_cf_need_flush = (at_least_one_cf_need_flush || tmp);
       if (!status.ok()) {
@@ -6902,11 +7182,11 @@ Status DBImpl::IngestExternalFiles(
             {} /* provided_candidate_cfds */, true /* entered_write_thread */);
         mutex_.Lock();
       } else {
-        for (size_t i = 0; i != num_cfs; ++i) {
+        for (size_t i = 0; i != num_jobs; ++i) {
           if (need_flush[i]) {
             mutex_.Unlock();
             status =
-                FlushMemTable(ingestion_jobs[i].GetColumnFamilyData(),
+                FlushMemTable(ingestion_jobs[i]->GetColumnFamilyData(),
                               flush_opts, FlushReason::kExternalFileIngestion,
                               true /* entered_write_thread */);
             mutex_.Lock();
@@ -6917,22 +7197,22 @@ Status DBImpl::IngestExternalFiles(
         }
       }
       if (status.ok()) {
-        for (size_t i = 0; i != num_cfs; ++i) {
+        for (size_t i = 0; i != num_jobs; ++i) {
           if (immutable_db_options_.atomic_flush || need_flush[i]) {
-            ingestion_jobs[i].SetFlushedBeforeRun();
+            ingestion_jobs[i]->SetFlushedBeforeRun();
           }
         }
       }
     }
     // Run ingestion jobs.
     if (status.ok()) {
-      for (size_t i = 0; i != num_cfs; ++i) {
+      for (size_t i = 0; i != num_jobs; ++i) {
         mutex_.AssertHeld();
-        status = ingestion_jobs[i].Run();
+        status = ingestion_jobs[i]->Run();
         if (!status.ok()) {
           break;
         }
-        ingestion_jobs[i].RegisterRange();
+        ingestion_jobs[i]->RegisterRange();
       }
     }
     // Now that Run() has assigned the actual seqno for each ingested file,
@@ -6944,9 +7224,9 @@ Status DBImpl::IngestExternalFiles(
     // next conversion observes the new barrier and refuses any insert
     // with insert_seq < assigned.
     if (status.ok()) {
-      for (size_t i = 0; i != num_cfs; ++i) {
-        auto* cfd = ingestion_jobs[i].GetColumnFamilyData();
-        SequenceNumber assigned = ingestion_jobs[i].MaxAssignedSequenceNumber();
+      for (auto* job : ingestion_jobs) {
+        auto* cfd = job->GetColumnFamilyData();
+        SequenceNumber assigned = job->MaxAssignedSequenceNumber();
         if (assigned > 0) {
           cfd->mem()->BumpIngestSeqnoBarrier(assigned);
         }
@@ -6954,16 +7234,18 @@ Status DBImpl::IngestExternalFiles(
     }
     if (status.ok()) {
       ReadOptions read_options;
-      read_options.fill_cache = args[0].options.fill_cache;
+      read_options.fill_cache = fill_cache;
       autovector<ColumnFamilyData*> cfds_to_commit;
       autovector<autovector<VersionEdit*>> edit_lists;
       uint32_t num_entries = 0;
-      for (size_t i = 0; i != num_cfs; ++i) {
-        auto* cfd = ingestion_jobs[i].GetColumnFamilyData();
+      for (auto* job : ingestion_jobs) {
+        auto* cfd = job->GetColumnFamilyData();
         assert(!cfd->IsDropped());
         cfds_to_commit.push_back(cfd);
         autovector<VersionEdit*> edit_list;
-        edit_list.push_back(ingestion_jobs[i].edit());
+        auto* edit = job->edit();
+        edit->MarkForegroundOperation();
+        edit_list.push_back(edit);
         edit_lists.push_back(edit_list);
         ++num_entries;
       }
@@ -6976,10 +7258,11 @@ Status DBImpl::IngestExternalFiles(
         }
         assert(0 == num_entries);
       }
-      status =
-          versions_->LogAndApply(cfds_to_commit, read_options, write_options,
-
-                                 edit_lists, &mutex_, directories_.GetDbDir());
+      status = versions_->LogAndApply(
+          cfds_to_commit, read_options, write_options, edit_lists, &mutex_,
+          directories_.GetDbDir(), false /* new_descriptor_log */,
+          nullptr /* new_cf_options */, {} /* manifest_wcbs */, {} /* pre_cb */,
+          max_file_opening_threads);
       // It is safe to update VersionSet last seqno here after LogAndApply since
       // LogAndApply persists last sequence number from VersionEdits,
       // which are from file's largest seqno and not from VersionSet.
@@ -6988,11 +7271,10 @@ Status DBImpl::IngestExternalFiles(
       // mutex when persisting MANIFEST file, and the snapshots taken during
       // that period will not be stable if VersionSet last seqno is updated
       // before LogAndApply.
-      SequenceNumber max_assigned_seqno =
-          ingestion_jobs[0].MaxAssignedSequenceNumber();
-      for (size_t i = 1; i != num_cfs; ++i) {
-        max_assigned_seqno = std::max(
-            max_assigned_seqno, ingestion_jobs[i].MaxAssignedSequenceNumber());
+      SequenceNumber max_assigned_seqno = 0;
+      for (auto* job : ingestion_jobs) {
+        max_assigned_seqno =
+            std::max(max_assigned_seqno, job->MaxAssignedSequenceNumber());
       }
       if (max_assigned_seqno > 0) {
         const SequenceNumber last_seqno = versions_->LastSequence();
@@ -7004,17 +7286,17 @@ Status DBImpl::IngestExternalFiles(
       }
     }
 
-    for (auto& job : ingestion_jobs) {
-      job.UnregisterRange();
+    for (auto* job : ingestion_jobs) {
+      job->UnregisterRange();
     }
 
     if (status.ok()) {
-      for (size_t i = 0; i != num_cfs; ++i) {
-        auto* cfd = ingestion_jobs[i].GetColumnFamilyData();
+      for (size_t i = 0; i != num_jobs; ++i) {
+        auto* cfd = ingestion_jobs[i]->GetColumnFamilyData();
         assert(!cfd->IsDropped());
         InstallSuperVersionAndScheduleWork(cfd, &sv_ctxs[i]);
 #ifndef NDEBUG
-        if (0 == i && num_cfs > 1) {
+        if (0 == i && num_jobs > 1) {
           TEST_SYNC_POINT("DBImpl::IngestExternalFiles:InstallSVForFirstCF:0");
           TEST_SYNC_POINT("DBImpl::IngestExternalFiles:InstallSVForFirstCF:1");
         }
@@ -7039,12 +7321,14 @@ Status DBImpl::IngestExternalFiles(
     PERF_TIMER_STOP(file_ingestion_blocking_live_writes_nanos);
 
     if (status.ok()) {
-      for (auto& job : ingestion_jobs) {
-        job.UpdateStats();
+      for (auto* job : ingestion_jobs) {
+        job->UpdateStats();
+      }
+      for (auto* h : hs) {
+        ReleaseFileNumberFromPendingOutputs(h->pending_output_elem_);
       }
     }
-    ReleaseFileNumberFromPendingOutputs(pending_output_elem);
-    num_running_ingest_file_ -= static_cast<int>(num_cfs);
+    num_running_ingest_file_ -= static_cast<int>(num_jobs);
     if (0 == num_running_ingest_file_) {
       bg_cv_.SignalAll();
     }
@@ -7052,24 +7336,46 @@ Status DBImpl::IngestExternalFiles(
   }
   // mutex_ is unlocked here
 
-  // Cleanup
-  for (size_t i = 0; i != num_cfs; ++i) {
-    sv_ctxs[i].Clean();
-    // This may rollback jobs that have completed successfully. This is
-    // intended for atomicity.
-    ingestion_jobs[i].Cleanup(status);
+  for (auto& sv_ctx : sv_ctxs) {
+    sv_ctx.Clean();
   }
-  if (status.ok()) {
-    for (size_t i = 0; i != num_cfs; ++i) {
-      auto* cfd = ingestion_jobs[i].GetColumnFamilyData();
-      if (!cfd->IsDropped()) {
-        NotifyOnExternalFileIngested(cfd, ingestion_jobs[i]);
-      }
+  if (!status.ok()) {
+    // The atomic commit failed; nothing was made visible. The handles are NOT
+    // consumed, so each one's destructor rolls it back
+    return status;
+  }
+
+  for (auto* job : ingestion_jobs) {
+    job->Cleanup(status);
+    auto* cfd = job->GetColumnFamilyData();
+    if (!cfd->IsDropped()) {
+      NotifyOnExternalFileIngested(cfd, *job);
     }
   }
+  for (auto* h : hs) {
+    h->consumed_ = true;
+    num_outstanding_prepared_ingestions_.fetch_sub(1);
+  }
+  // Record commit latency.
+  if (record_ingest_micros) {
+    RecordTimeToHistogram(
+        stats_, INGEST_EXTERNAL_FILE_RUN_TIME,
+        immutable_db_options_.clock->NowMicros() - run_start_micros);
+  }
   return status;
 }
 
+Status DBImpl::IngestExternalFiles(
+    const std::vector<IngestExternalFileArg>& args) {
+  PERF_TIMER_GUARD(file_ingestion_nanos);
+  std::unique_ptr<FileIngestionHandle> handle;
+  Status status = PrepareFileIngestion(args, &handle);
+  if (!status.ok()) {
+    return status;
+  }
+  return CommitFileIngestionHandle(std::move(handle));
+}
+
 Status DBImpl::CreateColumnFamilyWithImport(
     const ColumnFamilyOptions& options, const std::string& column_family_name,
     const ImportColumnFamilyOptions& import_options,
@@ -7111,6 +7417,7 @@ Status DBImpl::CreateColumnFamilyWithImport(
 
   SuperVersionContext dummy_sv_ctx(/* create_superversion */ true);
   VersionEdit dummy_edit;
+  dummy_edit.MarkForegroundOperation();
   uint64_t next_file_number = 0;
   std::unique_ptr<std::list<uint64_t>::iterator> pending_output_elem;
   {
@@ -7168,9 +7475,10 @@ Status DBImpl::CreateColumnFamilyWithImport(
 
       // Install job edit [Mutex will be unlocked here]
       if (status.ok()) {
-        status = versions_->LogAndApply(cfd, read_options, write_options,
-                                        import_job.edit(), &mutex_,
-                                        directories_.GetDbDir());
+        auto* edit = import_job.edit();
+        edit->MarkForegroundOperation();
+        status = versions_->LogAndApply(cfd, read_options, write_options, edit,
+                                        &mutex_, directories_.GetDbDir());
         if (status.ok()) {
           InstallSuperVersionForConfigChange(cfd, &sv_context);
         }
@@ -7601,6 +7909,7 @@ Status DBImpl::ReserveFileNumbersBeforeIngestion(
   // reuse the file number that has already assigned to the internal file,
   // and this will overwrite the external file. To protect the external
   // file, we have to make sure the file number will never being reused.
+  dummy_edit.MarkForegroundOperation();
   s = versions_->LogAndApply(cfd, read_options, write_options, &dummy_edit,
                              &mutex_, directories_.GetDbDir());
   if (s.ok()) {

db/db_impl/db_impl.h

@@ -76,6 +76,7 @@ namespace ROCKSDB_NAMESPACE {
 
 class Arena;
 class ArenaWrappedDBIter;
+class FileIngestionHandleImpl;
 class InMemoryStatsHistoryIterator;
 class MemTable;
 class PersistentStatsHistoryIterator;
@@ -575,6 +576,10 @@ class DBImpl : public DB {
       const LiveFilesStorageInfoOptions& opts,
       std::vector<LiveFileStorageInfo>* files) override;
 
+  Status GetPreparedFileInfoForExternalSstIngestion(
+      const std::string& file_path,
+      std::shared_ptr<const PreparedFileInfo>* file_info) override;
+
   // Obtains the meta data of the specified column family of the DB.
   // TODO(yhchiang): output parameter is placed in the end in this codebase.
   void GetColumnFamilyMetaData(ColumnFamilyHandle* column_family,
@@ -604,6 +609,14 @@ class DBImpl : public DB {
   Status IngestExternalFiles(
       const std::vector<IngestExternalFileArg>& args) override;
 
+  using DB::PrepareFileIngestion;
+  Status PrepareFileIngestion(
+      const std::vector<IngestExternalFileArg>& args,
+      std::unique_ptr<FileIngestionHandle>* handle) override;
+
+  Status CommitFileIngestionHandles(
+      std::vector<std::unique_ptr<FileIngestionHandle>> handles) override;
+
   using DB::CreateColumnFamilyWithImport;
   Status CreateColumnFamilyWithImport(
       const ColumnFamilyOptions& options, const std::string& column_family_name,
@@ -857,12 +870,22 @@ class DBImpl : public DB {
   // memtable range tombstone iterator used by the underlying merging iterator.
   // This range tombstone iterator can be refreshed later by db_iter.
   // @param read_options Must outlive the returned iterator.
-  InternalIterator* NewInternalIterator(const ReadOptions& read_options,
-                                        ColumnFamilyData* cfd,
-                                        SuperVersion* super_version,
-                                        Arena* arena, SequenceNumber sequence,
-                                        bool allow_unprepared_value,
-                                        ArenaWrappedDBIter* db_iter = nullptr);
+  // @param sequence The snapshot sequence captured when the DB iterator was
+  // created. Child iterators must use this instead of dereferencing
+  // read_options.snapshot, which may be released before lazy initialization.
+  // @param scan_opts Optional bounded scan ranges used only to prune the
+  // iterator tree during lazy Prepare() initialization.
+  InternalIterator* NewInternalIterator(
+      const ReadOptions& read_options, ColumnFamilyData* cfd,
+      SuperVersion* super_version, Arena* arena, SequenceNumber sequence,
+      bool allow_unprepared_value, ArenaWrappedDBIter* db_iter = nullptr,
+      const MultiScanArgs* scan_opts = nullptr);
+
+  // Release a SuperVersion held by an iterator. This preserves the cleanup
+  // behavior used by materialized internal iterators even when the DB iterator
+  // never needed to lazily build its child iterator tree.
+  void CleanupIteratorSuperVersion(SuperVersion* super_version,
+                                   bool background_purge);
 
   LogsWithPrepTracker* logs_with_prep_tracker() {
     return &logs_with_prep_tracker_;
@@ -1906,6 +1929,7 @@ class DBImpl : public DB {
   friend class WriteBatchWithIndex;
   friend class WriteUnpreparedTxnDB;
   friend class WriteUnpreparedTxn;
+  friend class FileIngestionHandleImpl;
 
   friend class ForwardIterator;
   friend struct SuperVersion;
@@ -2243,6 +2267,10 @@ class DBImpl : public DB {
   void ReleaseFileNumberFromPendingOutputs(
       std::unique_ptr<std::list<uint64_t>::iterator>& v);
 
+  // Rolls back one prepared file ingestion (delete its staged files, release
+  // the reserved file numbers)
+  void RollbackPreparedFileIngestion(FileIngestionHandleImpl* const h);
+
   // Similar to pending_outputs, preserve OPTIONS file. Used for remote
   // compaction.
   std::list<uint64_t>::iterator CaptureOptionsFileNumber();
@@ -3462,6 +3490,10 @@ class DBImpl : public DB {
   // REQUIRES: mutex held
   int num_running_ingest_file_ = 0;
 
+  // Number of FileIngestionHandle objects produced by PrepareFileIngestion()
+  // that have not been committed or destroyed yet.
+  std::atomic<uint32_t> num_outstanding_prepared_ingestions_{0};
+
   WalManager wal_manager_;
 
   // A value of > 0 temporarily disables scheduling of background work

db/db_impl/db_impl_compaction_flush.cc

@@ -4067,6 +4067,13 @@ Status DBImpl::BackgroundCompaction(bool* made_progress,
       status = Status::ShutdownInProgress();
     } else if (compaction_aborted_.load(std::memory_order_acquire) > 0) {
       status = Status::Incomplete(Status::SubCode::kCompactionAborted);
+      if (!is_prepicked) {
+        // This automatic compaction was scheduled from compaction_queue_, but
+        // abort happened before PickCompactionFromQueue() could remove the
+        // queued CF. Restore the unscheduled count so ResumeAllCompactions()
+        // can schedule the still-queued work.
+        unscheduled_compactions_++;
+      }
     } else if (is_manual &&
                manual_compaction->canceled.load(std::memory_order_acquire)) {
       status = Status::Incomplete(Status::SubCode::kManualCompactionPaused);
@@ -4074,10 +4081,14 @@ Status DBImpl::BackgroundCompaction(bool* made_progress,
   } else {
     status = error_handler_.GetBGError();
     // If we get here, it means a hard error happened after this compaction
-    // was scheduled by MaybeScheduleFlushOrCompaction(), but before it got
-    // a chance to execute. Since we didn't pop a cfd from the compaction
-    // queue, increment unscheduled_compactions_
-    unscheduled_compactions_++;
+    // was scheduled by MaybeScheduleFlushOrCompaction(), but before it got a
+    // chance to execute. Since non-prepicked work consumed an unscheduled
+    // compaction credit without popping a cfd from the compaction queue,
+    // restore the credit here. Prepicked work was scheduled directly and did
+    // not consume unscheduled_compactions_.
+    if (!is_prepicked) {
+      unscheduled_compactions_++;
+    }
   }
 
   if (!status.ok()) {

db/db_impl/db_impl_files.cc

@@ -833,6 +833,7 @@ void DBImpl::PurgeObsoleteFiles(JobContext& state, bool schedule_only) {
   }
   wal_manager_.PurgeObsoleteWALFiles();
   LogFlush(immutable_db_options_.info_log);
+  TEST_SYNC_POINT("DBImpl::PurgeObsoleteFiles:BeforePendingPurgeFinished");
   InstrumentedMutexLock l(&mutex_);
   --pending_purge_obsolete_files_;
   assert(pending_purge_obsolete_files_ >= 0);

db/db_impl/db_impl_open.cc

@@ -255,6 +255,17 @@ Status DBImpl::ValidateOptions(const DBOptions& db_options) {
         "then direct I/O reads (use_direct_reads) must be disabled. ");
   }
 
+  if (db_options.allow_mmap_reads &&
+      db_options.use_direct_io_for_compaction_reads) {
+    // mmap reads and direct I/O share the same EnvOptions field, so enabling
+    // both would try to mmap and O_DIRECT the same reads. Reject it here rather
+    // than tripping a lower-level assert.
+    return Status::NotSupported(
+        "If memory mapped reads (allow_mmap_reads) are enabled "
+        "then compaction-only direct I/O reads "
+        "(use_direct_io_for_compaction_reads) must be disabled. ");
+  }
+
   if (db_options.allow_mmap_writes &&
       db_options.use_direct_io_for_flush_and_compaction) {
     return Status::NotSupported(
@@ -506,7 +517,8 @@ Status DBImpl::Recover(
       std::unique_ptr<FSRandomAccessFile> idfile;
       FileOptions customized_fs(file_options_);
       customized_fs.use_direct_reads |=
-          immutable_db_options_.use_direct_io_for_flush_and_compaction;
+          immutable_db_options_.use_direct_io_for_flush_and_compaction ||
+          immutable_db_options_.use_direct_io_for_compaction_reads;
       const std::string& fname =
           manifest_path.empty() ? current_fname : manifest_path;
       s = fs_->NewRandomAccessFile(fname, customized_fs, &idfile, nullptr);

db/db_impl/db_impl_readonly.h

@@ -134,6 +134,19 @@ class DBImplReadOnly : public DBImpl {
     return Status::NotSupported("Not supported operation in read only mode.");
   }
 
+  using DB::PrepareFileIngestion;
+  Status PrepareFileIngestion(
+      const std::vector<IngestExternalFileArg>& /*args*/,
+      std::unique_ptr<FileIngestionHandle>* /*handle*/) override {
+    return Status::NotSupported("Not supported operation in read only mode.");
+  }
+
+  using DB::CommitFileIngestionHandles;
+  Status CommitFileIngestionHandles(
+      std::vector<std::unique_ptr<FileIngestionHandle>> /*handles*/) override {
+    return Status::NotSupported("Not supported operation in read only mode.");
+  }
+
   using DB::CreateColumnFamilyWithImport;
   Status CreateColumnFamilyWithImport(
       const ColumnFamilyOptions& /*options*/,

db/db_impl/db_impl_secondary.h

@@ -239,6 +239,19 @@ class DBImplSecondary : public DBImpl {
     return Status::NotSupported("Not supported operation in secondary mode.");
   }
 
+  using DB::PrepareFileIngestion;
+  Status PrepareFileIngestion(
+      const std::vector<IngestExternalFileArg>& /*args*/,
+      std::unique_ptr<FileIngestionHandle>* /*handle*/) override {
+    return Status::NotSupported("Not supported operation in secondary mode.");
+  }
+
+  using DB::CommitFileIngestionHandles;
+  Status CommitFileIngestionHandles(
+      std::vector<std::unique_ptr<FileIngestionHandle>> /*handles*/) override {
+    return Status::NotSupported("Not supported operation in secondary mode.");
+  }
+
   // Try to catch up with the primary by reading as much as possible from the
   // log files until there is nothing more to read or encounters an error. If
   // the amount of information in the log files to process is huge, this

db/db_iter.cc

@@ -67,8 +67,9 @@ DBIter::DBIter(Env* _env, const ReadOptions& read_options,
                const MutableCFOptions& mutable_cf_options,
                const Comparator* cmp, InternalIterator* iter,
                const Version* version, SequenceNumber s, bool arena_mode,
-               ReadCallback* read_callback, ColumnFamilyHandleImpl* cfh,
-               bool expose_blob_index, ReadOnlyMemTable* active_mem)
+               ReadCallback* read_callback, DBImpl* db_impl,
+               ColumnFamilyData* cfd, bool expose_blob_index,
+               ReadOnlyMemTable* active_mem)
     : prefix_extractor_(mutable_cf_options.prefix_extractor.get()),
       env_(_env),
       clock_(ioptions.clock),
@@ -76,21 +77,26 @@ DBIter::DBIter(Env* _env, const ReadOptions& read_options,
       user_comparator_(cmp),
       merge_operator_(ioptions.merge_operator.get()),
       iter_(iter),
-      blob_state_(
-          version, read_options.read_tier, read_options.verify_checksums,
-          read_options.fill_cache, read_options.io_activity,
-          cfh ? cfh->cfd()->blob_file_cache() : nullptr,
-          cfh != nullptr && cfh->cfd()->blob_partition_manager() != nullptr),
+      blob_state_(version, read_options.read_tier,
+                  read_options.verify_checksums, read_options.fill_cache,
+                  read_options.io_activity,
+                  cfd ? cfd->blob_file_cache() : nullptr,
+                  cfd != nullptr && cfd->blob_partition_manager() != nullptr),
       read_callback_(read_callback),
       sequence_(s),
-      value_columns_state_(version, read_options, cfh),
+      value_columns_state_(version, read_options, cfd),
       statistics_(ioptions.stats),
       max_skip_(mutable_cf_options.max_sequential_skip_in_iterations),
       max_skippable_internal_keys_(read_options.max_skippable_internal_keys),
       num_internal_keys_skipped_(0),
       iterate_lower_bound_(read_options.iterate_lower_bound),
       iterate_upper_bound_(read_options.iterate_upper_bound),
-      cfh_(cfh),
+      trace_db_(db_impl),
+      trace_cf_id_(cfd != nullptr ? cfd->GetID() : 0),
+      has_trace_state_(db_impl != nullptr && cfd != nullptr),
+      allow_blob_write_path_fallback_(cfd != nullptr &&
+                                      cfd->blob_partition_manager() != nullptr),
+      ingest_sst_lock_(cfd != nullptr ? &cfd->GetIngestSstLock() : nullptr),
       timestamp_ub_(read_options.timestamp),
       timestamp_lb_(read_options.iter_start_ts),
       timestamp_size_(timestamp_ub_ ? timestamp_ub_->size() : 0),
@@ -305,10 +311,8 @@ bool DBIter::SetValueAndColumnsFromBlobImpl(const Slice& user_key,
   // Keep the non-BDW iterator path on the pre-existing Version::GetBlob()
   // fast path. Only enable the direct-write fallback when this CF actually
   // has a write-path partition manager.
-  const bool allow_write_path_fallback =
-      cfh_ != nullptr && cfh_->cfd()->blob_partition_manager() != nullptr;
   const Status s = blob_state_.mut()->reader.RetrieveAndSetBlobValue(
-      user_key, blob_index, allow_write_path_fallback);
+      user_key, blob_index, allow_blob_write_path_fallback_);
   if (!s.ok()) {
     status_ = s;
     valid_ = false;
@@ -1617,10 +1621,8 @@ bool DBIter::MergeWithBlobBaseValue(const Slice& blob_index,
     return false;
   }
 
-  const bool allow_write_path_fallback =
-      cfh_ != nullptr && cfh_->cfd()->blob_partition_manager() != nullptr;
   const Status s = blob_state_.mut()->reader.RetrieveAndSetBlobValue(
-      user_key, blob_index, allow_write_path_fallback);
+      user_key, blob_index, allow_blob_write_path_fallback_);
   if (!s.ok()) {
     status_ = s;
     valid_ = false;
@@ -1816,10 +1818,10 @@ void DBIter::MaybeInsertRangeTombstone(const Slice& end_key) {
     }
   }
 
-  assert(cfh_ != nullptr);
+  assert(ingest_sst_lock_ != nullptr);
   if (active_mem_->AddLogicallyRedundantRangeTombstone(
           insert_seq, range_tomb_first_key_.GetUserKey(), end_key,
-          cfh_->cfd()->GetIngestSstLock())) {
+          *ingest_sst_lock_)) {
     RecordTick(statistics_, READ_PATH_RANGE_TOMBSTONES_INSERTED);
     ROCKS_LOG_DEBUG(logger_,
                     "Inserted range tombstone [%s, %s) @ seq %" PRIu64
@@ -1959,10 +1961,10 @@ Status DBIter::ValidateScanOptions(const MultiScanArgs& multiscan_opts) const {
   return Status::OK();
 }
 
-void DBIter::Prepare(const MultiScanArgs& scan_opts) {
+Status DBIter::SetScanOptionsForPrepare(const MultiScanArgs& scan_opts) {
   status_ = ValidateScanOptions(scan_opts);
   if (!status_.ok()) {
-    return;
+    return status_;
   }
   std::optional<MultiScanArgs> new_scan_opts;
   new_scan_opts.emplace(scan_opts);
@@ -1975,14 +1977,27 @@ void DBIter::Prepare(const MultiScanArgs& scan_opts) {
   if (!scan_opts_.value().io_dispatcher) {
     scan_opts_->io_dispatcher.reset(NewIODispatcher());
   }
+  return Status::OK();
+}
 
-  if (!scan_opts.empty()) {
+void DBIter::PrepareInternalChildren() {
+  if (!scan_opts_.has_value() || !status_.ok()) {
+    return;
+  }
+
+  if (scan_opts_.value().HasBoundedScanRanges()) {
     iter_.Prepare(&scan_opts_.value());
   } else {
     iter_.Prepare(nullptr);
   }
 }
 
+void DBIter::Prepare(const MultiScanArgs& scan_opts) {
+  if (SetScanOptionsForPrepare(scan_opts).ok()) {
+    PrepareInternalChildren();
+  }
+}
+
 void DBIter::Seek(const Slice& target) {
   PERF_COUNTER_ADD(iter_seek_count, 1);
   PERF_CPU_TIMER_GUARD(iter_seek_cpu_nanos, clock_);
@@ -2029,7 +2044,7 @@ void DBIter::Seek(const Slice& target) {
     scan_index_++;
   }
 
-  if (cfh_ != nullptr) {
+  if (has_trace_state_) {
     // TODO: What do we do if this returns an error?
     Slice lower_bound, upper_bound;
     if (iterate_lower_bound_ != nullptr) {
@@ -2042,9 +2057,7 @@ void DBIter::Seek(const Slice& target) {
     } else {
       upper_bound = Slice("");
     }
-    cfh_->db()
-        ->TraceIteratorSeek(cfh_->cfd()->GetID(), target, lower_bound,
-                            upper_bound)
+    trace_db_->TraceIteratorSeek(trace_cf_id_, target, lower_bound, upper_bound)
         .PermitUncheckedError();
   }
 
@@ -2097,7 +2110,7 @@ void DBIter::SeekForPrev(const Slice& target) {
   PERF_CPU_TIMER_GUARD(iter_seek_cpu_nanos, clock_);
   StopWatch sw(clock_, statistics_, DB_SEEK);
 
-  if (cfh_ != nullptr) {
+  if (has_trace_state_) {
     // TODO: What do we do if this returns an error?
     Slice lower_bound, upper_bound;
     if (iterate_lower_bound_ != nullptr) {
@@ -2110,8 +2123,8 @@ void DBIter::SeekForPrev(const Slice& target) {
     } else {
       upper_bound = Slice("");
     }
-    cfh_->db()
-        ->TraceIteratorSeekForPrev(cfh_->cfd()->GetID(), target, lower_bound,
+    trace_db_
+        ->TraceIteratorSeekForPrev(trace_cf_id_, target, lower_bound,
                                    upper_bound)
         .PermitUncheckedError();
   }

db/db_iter.h

@@ -32,6 +32,9 @@
 namespace ROCKSDB_NAMESPACE {
 class BlobFileCache;
 class Version;
+namespace port {
+class RWMutex;
+}
 
 // This file declares the factory functions of DBIter, in its original form
 // or a wrapped form with class ArenaWrappedDBIter, which is defined here.
@@ -83,14 +86,19 @@ class DBIter final : public Iterator {
                          ReadCallback* read_callback,
                          ReadOnlyMemTable* active_mem,
                          ColumnFamilyHandleImpl* cfh = nullptr,
-                         bool expose_blob_index = false,
-                         Arena* arena = nullptr) {
+                         bool expose_blob_index = false, Arena* arena = nullptr,
+                         DBImpl* db_impl = nullptr,
+                         ColumnFamilyData* cfd = nullptr) {
+    if (cfh != nullptr) {
+      db_impl = cfh->db();
+      cfd = cfh->cfd();
+    }
     void* mem = arena ? arena->AllocateAligned(sizeof(DBIter))
                       : operator new(sizeof(DBIter));
     DBIter* db_iter = new (mem)
         DBIter(env, read_options, ioptions, mutable_cf_options,
                user_key_comparator, internal_iter, version, sequence, arena,
-               read_callback, cfh, expose_blob_index, active_mem);
+               read_callback, db_impl, cfd, expose_blob_index, active_mem);
     return db_iter;
   }
 
@@ -207,7 +215,7 @@ class DBIter final : public Iterator {
   }
 
   Status status() const override {
-    if (status_.ok()) {
+    if (status_.ok() && iter_.iter() != nullptr) {
       return iter_.status();
     } else {
       assert(!valid_);
@@ -248,19 +256,22 @@ class DBIter final : public Iterator {
     iter_.SetRangeDelReadSeqno(s);
   }
   void set_valid(bool v) { valid_ = v; }
+  void set_status(Status s) { status_ = std::move(s); }
 
   bool PrepareValue() override;
 
   void Prepare(const MultiScanArgs& scan_opts) override;
   Status ValidateScanOptions(const MultiScanArgs& multiscan_opts) const;
+  Status SetScanOptionsForPrepare(const MultiScanArgs& scan_opts);
+  void PrepareInternalChildren();
 
  private:
   DBIter(Env* _env, const ReadOptions& read_options,
          const ImmutableOptions& ioptions,
          const MutableCFOptions& mutable_cf_options, const Comparator* cmp,
          InternalIterator* iter, const Version* version, SequenceNumber s,
-         bool arena_mode, ReadCallback* read_callback,
-         ColumnFamilyHandleImpl* cfh, bool expose_blob_index,
+         bool arena_mode, ReadCallback* read_callback, DBImpl* db_impl,
+         ColumnFamilyData* cfd, bool expose_blob_index,
          ReadOnlyMemTable* active_mem);
 
   class BlobReader {
@@ -318,13 +329,12 @@ class DBIter final : public Iterator {
   class ValueColumnsState {
    public:
     ValueColumnsState(const Version* version, const ReadOptions& read_options,
-                      ColumnFamilyHandleImpl* cfh)
+                      ColumnFamilyData* cfd)
         : entity_blob_resolver_(
               version, read_options.read_tier, read_options.verify_checksums,
               read_options.fill_cache, read_options.io_activity,
-              cfh ? cfh->cfd()->blob_file_cache() : nullptr,
-              cfh != nullptr &&
-                  cfh->cfd()->blob_partition_manager() != nullptr) {}
+              cfd ? cfd->blob_file_cache() : nullptr,
+              cfd != nullptr && cfd->blob_partition_manager() != nullptr) {}
 
     Slice& value() { return value_; }
     const Slice& value() const { return value_; }
@@ -706,7 +716,11 @@ class DBIter final : public Iterator {
   MergeContext merge_context_;
   LocalStatistics local_stats_;
   PinnedIteratorsManager pinned_iters_mgr_;
-  ColumnFamilyHandleImpl* cfh_;
+  DBImpl* trace_db_;
+  uint32_t trace_cf_id_;
+  bool has_trace_state_;
+  bool allow_blob_write_path_fallback_;
+  port::RWMutex* ingest_sst_lock_;
   const Slice* const timestamp_ub_;
   const Slice* const timestamp_lb_;
   const size_t timestamp_size_;

db/db_iter_test.cc

@@ -216,12 +216,19 @@ class TestIterator : public InternalIterator {
   bool IsKeyPinned() const override { return true; }
   bool IsValuePinned() const override { return true; }
 
+  void Prepare(const MultiScanArgs* /*scan_opts*/) override {
+    ++prepare_call_count_;
+  }
+
+  size_t prepare_call_count() const { return prepare_call_count_; }
+
  private:
   bool initialized_;
   bool valid_;
   size_t sequence_number_;
   size_t iter_;
   size_t steps_ = 0;
+  size_t prepare_call_count_ = 0;
 
   InternalKeyComparator cmp;
   std::vector<std::pair<std::string, std::string>> data_;
@@ -243,6 +250,53 @@ class DBIteratorTest : public testing::Test {
   DBIteratorTest() : env_(Env::Default()) {}
 };
 
+TEST_F(DBIteratorTest, PrepareForwardsValidatedScanRanges) {
+  Options options;
+  ImmutableOptions ioptions = ImmutableOptions(options);
+  MutableCFOptions mutable_cf_options = MutableCFOptions(options);
+  TestIterator* internal_iter = new TestIterator(BytewiseComparator());
+  internal_iter->AddPut("a", "val_a");
+  internal_iter->AddPut("b", "val_b");
+  internal_iter->Finish();
+
+  ReadOptions ro;
+  std::unique_ptr<Iterator> db_iter(DBIter::NewIter(
+      env_, ro, ioptions, mutable_cf_options, BytewiseComparator(),
+      internal_iter, nullptr /* version */, 10 /* sequence */,
+      nullptr /* read_callback */, /*active_mem=*/nullptr));
+
+  MultiScanArgs scan_opts(BytewiseComparator());
+  scan_opts.insert("a", "c");
+
+  db_iter->Prepare(scan_opts);
+  ASSERT_OK(db_iter->status());
+  EXPECT_EQ(internal_iter->prepare_call_count(), 1);
+}
+
+TEST_F(DBIteratorTest, PrepareDoesNotForwardInvalidScanRanges) {
+  Options options;
+  ImmutableOptions ioptions = ImmutableOptions(options);
+  MutableCFOptions mutable_cf_options = MutableCFOptions(options);
+  TestIterator* internal_iter = new TestIterator(BytewiseComparator());
+  internal_iter->AddPut("a", "val_a");
+  internal_iter->AddPut("b", "val_b");
+  internal_iter->Finish();
+
+  ReadOptions ro;
+  std::unique_ptr<Iterator> db_iter(DBIter::NewIter(
+      env_, ro, ioptions, mutable_cf_options, BytewiseComparator(),
+      internal_iter, nullptr /* version */, 10 /* sequence */,
+      nullptr /* read_callback */, /*active_mem=*/nullptr));
+
+  MultiScanArgs scan_opts(BytewiseComparator());
+  scan_opts.insert("b", "d");
+  scan_opts.insert("a", "c");
+
+  db_iter->Prepare(scan_opts);
+  EXPECT_TRUE(db_iter->status().IsInvalidArgument());
+  EXPECT_EQ(internal_iter->prepare_call_count(), 0);
+}
+
 TEST_F(DBIteratorTest, DBIteratorPrevNext) {
   Options options;
   ImmutableOptions ioptions = ImmutableOptions(options);

db/db_iterator_test.cc

@@ -10,6 +10,9 @@
 #include <functional>
 #include <iomanip>
 #include <iostream>
+#include <memory>
+#include <utility>
+#include <vector>
 
 #include "db/arena_wrapped_db_iter.h"
 #include "db/db_iter.h"
@@ -87,6 +90,392 @@ TEST_F(DBIteratorBaseTest, APICallsWithPerfContext) {
   delete iter;
 }
 
+TEST_F(DBIteratorBaseTest, PrepareWithMultiScanPrunesNonIntersectingFiles) {
+  Options options = CurrentOptions();
+  options.disable_auto_compactions = true;
+  DestroyAndReopen(options);
+
+  ASSERT_OK(Put("a", "va"));
+  ASSERT_OK(Flush());
+  ASSERT_OK(Put("z", "vz"));
+  ASSERT_OK(Flush());
+  ASSERT_EQ(2, NumTableFilesAtLevel(0));
+
+  int table_iterators_created = 0;
+  int files_added = 0;
+  int block_based_iterators = 0;
+  int level_iterators = 0;
+  SyncPoint::GetInstance()->SetCallBack(
+      "TableCache::NewIterator::BeforeFindTable",
+      [&](void* /*arg*/) { ++table_iterators_created; });
+  SyncPoint::GetInstance()->SetCallBack(
+      "Version::AddIteratorsForLevel:AddedFile",
+      [&](void* /*arg*/) { ++files_added; });
+  SyncPoint::GetInstance()->SetCallBack(
+      "Version::AddIteratorsForLevel:IteratorType", [&](void* arg) {
+        auto* iterator_type = static_cast<std::pair<bool, bool>*>(arg);
+        if (iterator_type->first) {
+          ++block_based_iterators;
+        }
+        if (iterator_type->second) {
+          ++level_iterators;
+        }
+      });
+  SyncPoint::GetInstance()->EnableProcessing();
+
+  MultiScanArgs scan_opts(BytewiseComparator());
+  scan_opts.insert(Slice("a"), Slice("b"));
+  Slice upper_bound("b");
+  ReadOptions read_options;
+  read_options.iterate_upper_bound = &upper_bound;
+  std::unique_ptr<Iterator> iter(
+      db_->NewIterator(read_options, db_->DefaultColumnFamily()));
+
+  ASSERT_EQ(0, table_iterators_created);
+  ASSERT_EQ(0, files_added);
+  ASSERT_EQ(0, block_based_iterators);
+  ASSERT_EQ(0, level_iterators);
+  iter->Prepare(scan_opts);
+  ASSERT_EQ(1, table_iterators_created);
+  ASSERT_EQ(1, files_added);
+  ASSERT_EQ(1, block_based_iterators);
+  ASSERT_EQ(0, level_iterators);
+  std::vector<std::string> keys;
+  for (iter->Seek("a"); iter->Valid(); iter->Next()) {
+    keys.push_back(iter->key().ToString());
+  }
+  ASSERT_EQ(keys, std::vector<std::string>({"a"}));
+  ASSERT_FALSE(iter->Valid());
+  ASSERT_OK(iter->status());
+
+  SyncPoint::GetInstance()->DisableProcessing();
+  SyncPoint::GetInstance()->ClearAllCallBacks();
+}
+
+TEST_F(DBIteratorBaseTest, PrepareWithMultiScanPrunesNonIntersectingLevels) {
+  Options options = CurrentOptions();
+  options.disable_auto_compactions = true;
+  DestroyAndReopen(options);
+
+  ASSERT_OK(Put("z", "vz"));
+  ASSERT_OK(Flush());
+  MoveFilesToLevel(2);
+  ASSERT_OK(Put("a", "va"));
+  ASSERT_OK(Flush());
+  MoveFilesToLevel(1);
+  ASSERT_OK(Put("m", "vm"));
+  ASSERT_OK(Flush());
+  MoveFilesToLevel(1);
+
+  int table_iterators_created = 0;
+  int files_added = 0;
+  int block_based_iterators = 0;
+  int level_iterators = 0;
+  SyncPoint::GetInstance()->SetCallBack(
+      "TableCache::NewIterator::BeforeFindTable",
+      [&](void* /*arg*/) { ++table_iterators_created; });
+  SyncPoint::GetInstance()->SetCallBack(
+      "Version::AddIteratorsForLevel:AddedFile",
+      [&](void* /*arg*/) { ++files_added; });
+  SyncPoint::GetInstance()->SetCallBack(
+      "Version::AddIteratorsForLevel:IteratorType", [&](void* arg) {
+        auto* iterator_type = static_cast<std::pair<bool, bool>*>(arg);
+        if (iterator_type->first) {
+          ++block_based_iterators;
+        }
+        if (iterator_type->second) {
+          ++level_iterators;
+        }
+      });
+  SyncPoint::GetInstance()->EnableProcessing();
+
+  MultiScanArgs scan_opts(BytewiseComparator());
+  scan_opts.insert(Slice("a"), Slice("n"));
+  Slice upper_bound("n");
+  ReadOptions read_options;
+  read_options.iterate_upper_bound = &upper_bound;
+  std::unique_ptr<Iterator> iter(
+      db_->NewIterator(read_options, db_->DefaultColumnFamily()));
+
+  ASSERT_EQ(0, table_iterators_created);
+  ASSERT_EQ(0, files_added);
+  ASSERT_EQ(0, block_based_iterators);
+  ASSERT_EQ(0, level_iterators);
+  iter->Prepare(scan_opts);
+  ASSERT_EQ(0, table_iterators_created);
+  ASSERT_EQ(2, files_added);
+  ASSERT_EQ(0, block_based_iterators);
+  ASSERT_EQ(1, level_iterators);
+  std::vector<std::string> keys;
+  for (iter->Seek("a"); iter->Valid(); iter->Next()) {
+    keys.push_back(iter->key().ToString());
+  }
+  ASSERT_EQ(keys, std::vector<std::string>({"a", "m"}));
+  ASSERT_FALSE(iter->Valid());
+  ASSERT_OK(iter->status());
+
+  SyncPoint::GetInstance()->DisableProcessing();
+  SyncPoint::GetInstance()->ClearAllCallBacks();
+}
+
+TEST_F(DBIteratorBaseTest, PrepareWithMultiScanAllowsSingleUnboundedRange) {
+  Options options = CurrentOptions();
+  options.disable_auto_compactions = true;
+  DestroyAndReopen(options);
+
+  ASSERT_OK(Put("a", "va"));
+  ASSERT_OK(Put("b", "vb"));
+  ASSERT_OK(Put("c", "vc"));
+  ASSERT_OK(Flush());
+  MoveFilesToLevel(1);
+
+  MultiScanArgs scan_opts(BytewiseComparator());
+  scan_opts.insert(Slice("b"));
+  ReadOptions read_options;
+  std::unique_ptr<Iterator> iter(
+      db_->NewIterator(read_options, db_->DefaultColumnFamily()));
+
+  iter->Prepare(scan_opts);
+  std::vector<std::string> keys;
+  for (iter->Seek("b"); iter->Valid(); iter->Next()) {
+    keys.push_back(iter->key().ToString());
+  }
+  ASSERT_EQ(keys, std::vector<std::string>({"b", "c"}));
+  ASSERT_FALSE(iter->Valid());
+  ASSERT_OK(iter->status());
+}
+
+TEST_F(DBIteratorBaseTest, PrepareWithMultiScanRejectsRepeatedPrepare) {
+  Options options = CurrentOptions();
+  options.disable_auto_compactions = true;
+  DestroyAndReopen(options);
+
+  ASSERT_OK(Put("a", "va"));
+  ASSERT_OK(Flush());
+  MoveFilesToLevel(1);
+
+  MultiScanArgs scan_opts(BytewiseComparator());
+  scan_opts.insert(Slice("a"), Slice("b"));
+  Slice upper_bound("b");
+  ReadOptions read_options;
+  read_options.iterate_upper_bound = &upper_bound;
+  std::unique_ptr<Iterator> iter(
+      db_->NewIterator(read_options, db_->DefaultColumnFamily()));
+
+  iter->Prepare(scan_opts);
+  ASSERT_OK(iter->status());
+  iter->Prepare(scan_opts);
+  ASSERT_NOK(iter->status());
+}
+
+TEST_F(DBIteratorBaseTest, PrepareWithMultiScanDedupsMultipleRangesInSameFile) {
+  Options options = CurrentOptions();
+  options.disable_auto_compactions = true;
+  DestroyAndReopen(options);
+
+  ASSERT_OK(Put("a", "va"));
+  ASSERT_OK(Put("m", "vm"));
+  ASSERT_OK(Put("z", "vz"));
+  ASSERT_OK(Flush());
+  MoveFilesToLevel(1);
+
+  int table_iterators_created = 0;
+  int files_added = 0;
+  int block_based_iterators = 0;
+  int level_iterators = 0;
+  SyncPoint::GetInstance()->SetCallBack(
+      "TableCache::NewIterator::BeforeFindTable",
+      [&](void* /*arg*/) { ++table_iterators_created; });
+  SyncPoint::GetInstance()->SetCallBack(
+      "Version::AddIteratorsForLevel:AddedFile",
+      [&](void* /*arg*/) { ++files_added; });
+  SyncPoint::GetInstance()->SetCallBack(
+      "Version::AddIteratorsForLevel:IteratorType", [&](void* arg) {
+        auto* iterator_type = static_cast<std::pair<bool, bool>*>(arg);
+        if (iterator_type->first) {
+          ++block_based_iterators;
+        }
+        if (iterator_type->second) {
+          ++level_iterators;
+        }
+      });
+  SyncPoint::GetInstance()->EnableProcessing();
+
+  MultiScanArgs scan_opts(BytewiseComparator());
+  scan_opts.insert(Slice("a"), Slice("b"));
+  scan_opts.insert(Slice("m"), Slice("n"));
+  Slice upper_bound("b");
+  ReadOptions read_options;
+  read_options.iterate_upper_bound = &upper_bound;
+  std::unique_ptr<Iterator> iter(
+      db_->NewIterator(read_options, db_->DefaultColumnFamily()));
+
+  ASSERT_EQ(0, table_iterators_created);
+  ASSERT_EQ(0, files_added);
+  ASSERT_EQ(0, block_based_iterators);
+  ASSERT_EQ(0, level_iterators);
+  iter->Prepare(scan_opts);
+  ASSERT_EQ(1, table_iterators_created);
+  ASSERT_EQ(1, files_added);
+  ASSERT_EQ(1, block_based_iterators);
+  ASSERT_EQ(0, level_iterators);
+
+  std::vector<std::string> keys;
+  for (iter->Seek("a"); iter->Valid(); iter->Next()) {
+    keys.push_back(iter->key().ToString());
+  }
+  ASSERT_OK(iter->status());
+  ASSERT_EQ(keys, std::vector<std::string>({"a"}));
+
+  upper_bound = "n";
+  for (iter->Seek("m"); iter->Valid(); iter->Next()) {
+    keys.push_back(iter->key().ToString());
+  }
+  ASSERT_EQ(keys, std::vector<std::string>({"a", "m"}));
+  ASSERT_FALSE(iter->Valid());
+  ASSERT_OK(iter->status());
+
+  SyncPoint::GetInstance()->DisableProcessing();
+  SyncPoint::GetInstance()->ClearAllCallBacks();
+}
+
+TEST_F(DBIteratorBaseTest, PrepareWithMultiScanPrunesOverlappingL0Files) {
+  Options options = CurrentOptions();
+  options.disable_auto_compactions = true;
+  DestroyAndReopen(options);
+
+  ASSERT_OK(Put("a", "va"));
+  ASSERT_OK(Flush());
+  ASSERT_OK(Put("m", "vm"));
+  ASSERT_OK(Flush());
+  ASSERT_EQ(2, NumTableFilesAtLevel(0));
+
+  int table_iterators_created = 0;
+  int files_added = 0;
+  int block_based_iterators = 0;
+  int level_iterators = 0;
+  SyncPoint::GetInstance()->SetCallBack(
+      "TableCache::NewIterator::BeforeFindTable",
+      [&](void* /*arg*/) { ++table_iterators_created; });
+  SyncPoint::GetInstance()->SetCallBack(
+      "Version::AddIteratorsForLevel:AddedFile",
+      [&](void* /*arg*/) { ++files_added; });
+  SyncPoint::GetInstance()->SetCallBack(
+      "Version::AddIteratorsForLevel:IteratorType", [&](void* arg) {
+        auto* iterator_type = static_cast<std::pair<bool, bool>*>(arg);
+        if (iterator_type->first) {
+          ++block_based_iterators;
+        }
+        if (iterator_type->second) {
+          ++level_iterators;
+        }
+      });
+  SyncPoint::GetInstance()->EnableProcessing();
+
+  MultiScanArgs scan_opts(BytewiseComparator());
+  scan_opts.insert(Slice("a"), Slice("n"));
+  Slice upper_bound("n");
+  ReadOptions read_options;
+  read_options.iterate_upper_bound = &upper_bound;
+  std::unique_ptr<Iterator> iter(
+      db_->NewIterator(read_options, db_->DefaultColumnFamily()));
+
+  ASSERT_EQ(0, table_iterators_created);
+  ASSERT_EQ(0, files_added);
+  ASSERT_EQ(0, block_based_iterators);
+  ASSERT_EQ(0, level_iterators);
+  iter->Prepare(scan_opts);
+  ASSERT_EQ(2, table_iterators_created);
+  ASSERT_EQ(2, files_added);
+  ASSERT_EQ(2, block_based_iterators);
+  ASSERT_EQ(0, level_iterators);
+  std::vector<std::string> keys;
+  for (iter->Seek("a"); iter->Valid(); iter->Next()) {
+    keys.push_back(iter->key().ToString());
+  }
+  ASSERT_EQ(keys, std::vector<std::string>({"a", "m"}));
+  ASSERT_FALSE(iter->Valid());
+  ASSERT_OK(iter->status());
+
+  SyncPoint::GetInstance()->DisableProcessing();
+  SyncPoint::GetInstance()->ClearAllCallBacks();
+}
+
+TEST_F(DBIteratorBaseTest, PrepareWithMultiScanPrunesNonIntersectingMemTables) {
+  Options options = CurrentOptions();
+  options.disable_auto_compactions = true;
+  DestroyAndReopen(options);
+
+  ASSERT_OK(Put("a", "va"));
+  ASSERT_OK(Flush());
+  MoveFilesToLevel(1);
+
+  ASSERT_OK(db_->PauseBackgroundWork());
+  ASSERT_OK(Put("z_imm", "vz"));
+  ASSERT_OK(dbfull()->TEST_SwitchMemtable());
+  ASSERT_OK(Put("z_mem", "vz"));
+
+  int table_iterators_created = 0;
+  int files_added = 0;
+  int block_based_iterators = 0;
+  int level_iterators = 0;
+  int merging_iterators = 0;
+  SyncPoint::GetInstance()->SetCallBack(
+      "TableCache::NewIterator::BeforeFindTable",
+      [&](void* /*arg*/) { ++table_iterators_created; });
+  SyncPoint::GetInstance()->SetCallBack(
+      "Version::AddIteratorsForLevel:AddedFile",
+      [&](void* /*arg*/) { ++files_added; });
+  SyncPoint::GetInstance()->SetCallBack(
+      "Version::AddIteratorsForLevel:IteratorType", [&](void* arg) {
+        auto* iterator_type = static_cast<std::pair<bool, bool>*>(arg);
+        if (iterator_type->first) {
+          ++block_based_iterators;
+        }
+        if (iterator_type->second) {
+          ++level_iterators;
+        }
+      });
+  SyncPoint::GetInstance()->SetCallBack(
+      "MergeIteratorBuilder::Finish:UseMergingIterator", [&](void* arg) {
+        if (*static_cast<bool*>(arg)) {
+          ++merging_iterators;
+        }
+      });
+  SyncPoint::GetInstance()->EnableProcessing();
+
+  MultiScanArgs scan_opts(BytewiseComparator());
+  scan_opts.insert(Slice("a"), Slice("b"));
+  Slice upper_bound("b");
+  ReadOptions read_options;
+  read_options.iterate_upper_bound = &upper_bound;
+  std::unique_ptr<Iterator> iter(
+      db_->NewIterator(read_options, db_->DefaultColumnFamily()));
+
+  ASSERT_EQ(0, table_iterators_created);
+  ASSERT_EQ(0, files_added);
+  ASSERT_EQ(0, block_based_iterators);
+  ASSERT_EQ(0, level_iterators);
+  ASSERT_EQ(0, merging_iterators);
+  iter->Prepare(scan_opts);
+  ASSERT_EQ(1, table_iterators_created);
+  ASSERT_EQ(1, files_added);
+  ASSERT_EQ(1, block_based_iterators);
+  ASSERT_EQ(0, level_iterators);
+  ASSERT_EQ(0, merging_iterators);
+  std::vector<std::string> keys;
+  for (iter->Seek("a"); iter->Valid(); iter->Next()) {
+    keys.push_back(iter->key().ToString());
+  }
+  ASSERT_EQ(keys, std::vector<std::string>({"a"}));
+  ASSERT_FALSE(iter->Valid());
+  ASSERT_OK(iter->status());
+
+  SyncPoint::GetInstance()->DisableProcessing();
+  SyncPoint::GetInstance()->ClearAllCallBacks();
+  ASSERT_OK(db_->ContinueBackgroundWork());
+}
+
 // Test param:
 //   bool: whether to pass read_callback to NewIterator().
 class DBIteratorTest : public DBIteratorBaseTest,
@@ -945,6 +1334,68 @@ TEST_P(DBIteratorTest, IteratorDeleteAfterCfDelete) {
   delete iter;
 }
 
+TEST_P(DBIteratorTest, IteratorSeekAfterCfDelete) {
+  CreateAndReopenWithCF({"pikachu"}, CurrentOptions());
+
+  ASSERT_OK(Put(1, "foo", "delete-cf-then-seek-iter"));
+  ASSERT_OK(Put(1, "hello", "value2"));
+
+  ColumnFamilyHandle* cf = handles_[1];
+  ReadOptions ro;
+
+  auto* iter = db_->NewIterator(ro, cf);
+
+  // Delete the CF handle before the lazy iterator tree is materialized.
+  EXPECT_OK(db_->DestroyColumnFamilyHandle(cf));
+  handles_.erase(std::begin(handles_) + 1);
+
+  iter->Seek("foo");
+  ASSERT_EQ(IterStatus(iter), "foo->delete-cf-then-seek-iter");
+  iter->SeekForPrev("hello");
+  ASSERT_EQ(IterStatus(iter), "hello->value2");
+  iter->SeekToFirst();
+  ASSERT_EQ(IterStatus(iter), "foo->delete-cf-then-seek-iter");
+  iter->Next();
+  ASSERT_EQ(IterStatus(iter), "hello->value2");
+  delete iter;
+}
+
+TEST_P(DBIteratorTest, IteratorAutoRefreshAfterCfDeleteBeforeLazyInit) {
+  CreateAndReopenWithCF({"pikachu"}, CurrentOptions());
+
+  ASSERT_OK(Put(1, "foo", "delete-cf-then-auto-refresh"));
+  ASSERT_OK(Flush(1));
+
+  ColumnFamilyHandle* cf = handles_[1];
+  const Snapshot* snapshot = db_->GetSnapshot();
+
+  ReadOptions ro;
+  ro.snapshot = snapshot;
+  ro.auto_refresh_iterator_with_snapshot = true;
+
+  auto* iter = db_->NewIterator(ro, cf);
+
+  ASSERT_OK(Put(1, "zzz", "after-snapshot"));
+  ASSERT_OK(Flush(1));
+
+  // Delete the CF handle before the lazy iterator tree is materialized. The
+  // following operations force auto-refresh to acquire a newer SuperVersion.
+  EXPECT_OK(db_->DestroyColumnFamilyHandle(cf));
+  handles_.erase(std::begin(handles_) + 1);
+
+  iter->Seek("foo");
+  ASSERT_EQ(IterStatus(iter), "foo->delete-cf-then-auto-refresh");
+  ASSERT_OK(iter->status());
+  iter->SeekForPrev("foo");
+  ASSERT_EQ(IterStatus(iter), "foo->delete-cf-then-auto-refresh");
+  ASSERT_OK(iter->status());
+  iter->Next();
+  ASSERT_OK(iter->status());
+
+  delete iter;
+  db_->ReleaseSnapshot(snapshot);
+}
+
 TEST_P(DBIteratorTest, IteratorDeleteAfterCfDrop) {
   CreateAndReopenWithCF({"pikachu"}, CurrentOptions());
 
@@ -966,6 +1417,30 @@ TEST_P(DBIteratorTest, IteratorDeleteAfterCfDrop) {
   delete iter;
 }
 
+TEST_P(DBIteratorTest, IteratorSeekAfterCfDrop) {
+  CreateAndReopenWithCF({"pikachu"}, CurrentOptions());
+
+  ASSERT_OK(Put(1, "foo", "drop-cf-then-seek-iter"));
+
+  ReadOptions ro;
+  ColumnFamilyHandle* cf = handles_[1];
+
+  auto* iter = db_->NewIterator(ro, cf);
+
+  // Drop and delete the CF before the lazy iterator tree is materialized.
+  EXPECT_OK(db_->DropColumnFamily(cf));
+  EXPECT_OK(db_->DestroyColumnFamilyHandle(cf));
+  handles_.erase(std::begin(handles_) + 1);
+
+  iter->Seek("foo");
+  ASSERT_EQ(IterStatus(iter), "foo->drop-cf-then-seek-iter");
+  iter->SeekForPrev("foo");
+  ASSERT_EQ(IterStatus(iter), "foo->drop-cf-then-seek-iter");
+  iter->SeekToFirst();
+  ASSERT_EQ(IterStatus(iter), "foo->drop-cf-then-seek-iter");
+  delete iter;
+}
+
 // SetOptions not defined in ROCKSDB LITE
 TEST_P(DBIteratorTest, DBIteratorBoundTest) {
   Options options = CurrentOptions();
@@ -2277,6 +2752,10 @@ TEST_P(DBIteratorTest, ReadAhead) {
   options.env = env_;
   options.disable_auto_compactions = true;
   options.write_buffer_size = 4 << 20;
+  // Pin compression so the readahead byte thresholds below don't depend on the
+  // default compression type. kNoCompression keeps the SST files large enough
+  // to exercise readahead and is always available.
+  options.compression = kNoCompression;
   options.statistics = ROCKSDB_NAMESPACE::CreateDBStatistics();
   BlockBasedTableOptions table_options;
   table_options.block_size = 1024;
@@ -2693,6 +3172,8 @@ TEST_P(DBIteratorTest, CreationFailure) {
 
   Iterator* iter = NewIterator(ReadOptions());
   ASSERT_FALSE(iter->Valid());
+  iter->SeekToFirst();
+  ASSERT_FALSE(iter->Valid());
   ASSERT_TRUE(iter->status().IsCorruption());
   delete iter;
 }
@@ -4395,6 +4876,71 @@ TEST_P(DBMultiScanIteratorTest, RangeAcrossFiles) {
   iter.reset();
 }
 
+TEST_P(DBMultiScanIteratorTest, SortedRangesSkipIODispatcherSort) {
+  auto options = CurrentOptions();
+  options.target_file_size_base = 100 << 10;
+  options.compaction_style = kCompactionStyleUniversal;
+  options.num_levels = 50;
+  options.compression = kNoCompression;
+  DestroyAndReopen(options);
+
+  auto key = [](int i) {
+    std::stringstream ss;
+    ss << "k" << std::setw(5) << std::setfill('0') << i;
+    return ss.str();
+  };
+
+  auto rnd = Random::GetTLSInstance();
+  for (int i = 0; i < 100; ++i) {
+    ASSERT_OK(Put(key(i), rnd->RandomString(2 << 10)));
+  }
+  ASSERT_OK(Flush());
+
+  ASSERT_OK(db_->CompactRange({}, nullptr, nullptr));
+  ASSERT_EQ(2, NumTableFilesAtLevel(49));
+
+  int sort_count = 0;
+  SyncPoint::GetInstance()->SetCallBack(
+      "IODispatcherImpl::SubmitJob:SortBlockHandles",
+      [&](void* /*arg*/) { ++sort_count; });
+  SyncPoint::GetInstance()->EnableProcessing();
+
+  auto tracking_dispatcher = std::make_shared<TrackingIODispatcher>();
+  std::vector<std::string> key_ranges({key(10), key(90)});
+  MultiScanArgs scan_options(BytewiseComparator());
+  scan_options.io_dispatcher = tracking_dispatcher;
+  scan_options.use_async_io = false;
+  scan_options.insert(key_ranges[0], key_ranges[1]);
+
+  ReadOptions ro;
+  ro.fill_cache = GetParam();
+  ColumnFamilyHandle* cfh = dbfull()->DefaultColumnFamily();
+  std::unique_ptr<MultiScan> iter =
+      dbfull()->NewMultiScan(ro, cfh, scan_options);
+
+  try {
+    int i = 10;
+    for (auto range : *iter) {
+      for (auto it : range) {
+        ASSERT_EQ(it.first.ToString(), key(i));
+        ++i;
+      }
+    }
+    ASSERT_EQ(i, 90);
+  } catch (MultiScanException& ex) {
+    ASSERT_NOK(ex.status());
+    std::cerr << "Iterator returned status " << ex.what();
+    abort();
+  } catch (std::logic_error& ex) {
+    std::cerr << "Iterator returned logic error " << ex.what();
+    abort();
+  }
+
+  ASSERT_GT(tracking_dispatcher->GetReadSets().size(), 0);
+  EXPECT_EQ(sort_count, 0);
+  iter.reset();
+}
+
 TEST_P(DBMultiScanIteratorTest, FailureTest) {
   auto options = CurrentOptions();
   options.compression = kNoCompression;

db/db_memtable_test.cc

@@ -359,7 +359,7 @@ class DBMemTableTestForSeek : public DBMemTableTest,
 TEST_P(DBMemTableTestForSeek, IntegrityChecks) {
   // Validate key corruption could be detected during seek.
   // We insert many keys into skiplist. Then we corrupt the each key one at a
-  // time. With memtable_veirfy_per_key_checksum_on_seek enabled, when the
+  // time. With memtable_verify_per_key_checksum_on_seek enabled, when the
   // corrupted key is searched, the checksum of every key visited during the
   // seek is validated. It will report data corruption. Otherwise seek returns
   // not found.
@@ -367,7 +367,7 @@ TEST_P(DBMemTableTestForSeek, IntegrityChecks) {
   Options options = CurrentOptions();
   options.allow_data_in_errors = allow_data_in_error;
   options.paranoid_memory_checks = std::get<1>(GetParam());
-  options.memtable_veirfy_per_key_checksum_on_seek = std::get<2>(GetParam());
+  options.memtable_verify_per_key_checksum_on_seek = std::get<2>(GetParam());
   options.memtable_protection_bytes_per_key = 8;
   DestroyAndReopen(options);
 
@@ -410,7 +410,7 @@ TEST_P(DBMemTableTestForSeek, IntegrityChecks) {
   ASSERT_EQ(raw_data_pointer.size(), key_count);
 
   bool enable_key_validation_on_seek =
-      options.memtable_veirfy_per_key_checksum_on_seek;
+      options.memtable_verify_per_key_checksum_on_seek;
 
   // For each key, corrupt it, validate corruption is detected correctly, then
   // revert it.

db/db_open_with_config_test.cc

@@ -151,8 +151,8 @@ TEST_F(DBOpenWithConfigTest, OpenWithComprehensiveConfig) {
   // memtable_op_scan_flush_trigger=0
   options.memtable_op_scan_flush_trigger = 0;
 
-  // memtable_veirfy_per_key_checksum_on_seek=false
-  options.memtable_veirfy_per_key_checksum_on_seek = false;
+  // memtable_verify_per_key_checksum_on_seek=false
+  options.memtable_verify_per_key_checksum_on_seek = false;
 
   // Level and compaction settings
   // num_levels=50

db/db_options_test.cc

@@ -13,6 +13,7 @@
 #include "db/column_family.h"
 #include "db/db_impl/db_impl.h"
 #include "db/db_test_util.h"
+#include "env/mock_env.h"
 #include "options/options_helper.h"
 #include "port/stack_trace.h"
 #include "rocksdb/cache.h"
@@ -1763,6 +1764,171 @@ TEST_F(DBOptionsTest, SetOptionsMultipleColumnFamilies) {
   ASSERT_TRUE(dbfull()->GetOptions(handles_[2]).disable_auto_compactions);
 }
 
+// Confirms the default value and serialization/parse round-trip of the new
+// option. No DB open required; exercises only the options-metadata layer.
+TEST_F(DBOptionsTest, UseDirectIoForCompactionReadsRoundTrip) {
+  // Default value must remain false to preserve existing semantics.
+  ASSERT_FALSE(DBOptions().use_direct_io_for_compaction_reads);
+
+  DBOptions parsed;
+  ConfigOptions config_options;
+  ASSERT_OK(GetDBOptionsFromString(config_options, DBOptions(),
+                                   "use_direct_io_for_compaction_reads=true",
+                                   &parsed));
+  ASSERT_TRUE(parsed.use_direct_io_for_compaction_reads);
+  ASSERT_OK(GetDBOptionsFromString(config_options, DBOptions(),
+                                   "use_direct_io_for_compaction_reads=false",
+                                   &parsed));
+  ASSERT_FALSE(parsed.use_direct_io_for_compaction_reads);
+}
+
+// Validates that Open rejects the documented incompatible combination.
+TEST_F(DBOptionsTest, UseDirectIoForCompactionReadsValidation) {
+  // mmap_reads + use_direct_io_for_compaction_reads is rejected at Open
+  // time, the same way mmap_reads + use_direct_reads has always been
+  // rejected.
+  Options bad_options = CurrentOptions();
+  bad_options.create_if_missing = true;
+  bad_options.allow_mmap_reads = true;
+  bad_options.use_direct_io_for_compaction_reads = true;
+  Status bad_status = TryReopen(bad_options);
+  ASSERT_TRUE(bad_status.IsNotSupported()) << bad_status.ToString();
+}
+
+// Confirms the option is plumbed all the way to the live DB's options API:
+// after opening with the flag set, GetDBOptions() reports it back. Only runs
+// when the environment supports direct I/O.
+TEST_F(DBOptionsTest, UseDirectIoForCompactionReadsLiveReopen) {
+  Options options = CurrentOptions();
+  options.create_if_missing = true;
+  options.use_direct_io_for_compaction_reads = true;
+  // Use a buffered user-read setup so the new flag is the one doing the work.
+  options.use_direct_reads = false;
+  options.use_direct_io_for_flush_and_compaction = false;
+  Status s = TryReopen(options);
+  if (s.IsNotSupported() || s.IsInvalidArgument()) {
+    ROCKSDB_GTEST_BYPASS(
+        "Direct I/O not supported in this test environment; live reopen "
+        "cannot be exercised.");
+    return;
+  }
+  ASSERT_OK(s);
+  ASSERT_TRUE(dbfull()->GetDBOptions().use_direct_io_for_compaction_reads);
+  Close();
+}
+
+TEST_F(DBOptionsTest, UseDirectIoForCompactionReadsUnsupportedFileSystem) {
+  auto fs = std::make_shared<MockFileSystem>(Env::Default()->GetSystemClock(),
+                                             /*supports_direct_io=*/false);
+  std::unique_ptr<Env> mock_env = NewCompositeEnv(fs);
+
+  Options options = CurrentOptions();
+  options.env = mock_env.get();
+  options.create_if_missing = true;
+  options.use_direct_reads = false;
+  options.use_direct_io_for_flush_and_compaction = false;
+  options.use_direct_io_for_compaction_reads = true;
+
+  Status s = TryReopen(options);
+  ASSERT_TRUE(s.IsInvalidArgument()) << s.ToString();
+  ASSERT_NE(s.ToString().find("Direct I/O is not supported"), std::string::npos)
+      << s.ToString();
+}
+
+TEST_F(
+    DBOptionsTest,
+    UseDirectIoForCompactionReadsSetDBOptionsKeepsVerificationReadsBuffered) {
+  auto fs = std::make_shared<MockFileSystem>(Env::Default()->GetSystemClock(),
+                                             /*supports_direct_io=*/true);
+  std::unique_ptr<Env> mock_env = NewCompositeEnv(fs);
+
+  Options options = CurrentOptions();
+  options.env = mock_env.get();
+  options.create_if_missing = true;
+  options.use_direct_reads = false;
+  options.use_direct_io_for_flush_and_compaction = false;
+  options.use_direct_io_for_compaction_reads = true;
+
+  ASSERT_OK(TryReopen(options));
+
+  bool observed = false;
+  FileOptions observed_file_options;
+  ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->SetCallBack(
+      "DBImpl::SetDBOptions:FileOptionsForCompaction", [&](void* arg) {
+        observed = true;
+        observed_file_options = *static_cast<FileOptions*>(arg);
+      });
+  ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->EnableProcessing();
+
+  ASSERT_OK(dbfull()->SetDBOptions({{"bytes_per_sync", "1024"}}));
+  ASSERT_TRUE(observed);
+  ASSERT_FALSE(observed_file_options.use_direct_reads);
+
+  ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->DisableProcessing();
+  ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->ClearAllCallBacks();
+  Close();
+}
+
+// Exercises the FileSystem::OptimizeForCompactionTableRead and
+// OptimizeForBlobFileRead helpers directly, asserting that the compaction-only
+// flag does not affect shared FileSystem hooks.
+TEST_F(DBOptionsTest, OptimizeForCompactionTableReadUsesGlobalDirectReadsOnly) {
+  FileOptions in_opts;
+  in_opts.use_direct_reads = false;
+
+  {
+    Options check_options;
+    check_options.use_direct_reads = false;
+    check_options.use_direct_io_for_compaction_reads = true;
+    check_options.use_direct_io_for_flush_and_compaction = false;
+    ImmutableDBOptions immutable(check_options);
+    FileOptions sst_read =
+        env_->GetFileSystem()->OptimizeForCompactionTableRead(in_opts,
+                                                              immutable);
+    FileOptions blob_read =
+        env_->GetFileSystem()->OptimizeForBlobFileRead(in_opts, immutable);
+    EXPECT_FALSE(sst_read.use_direct_reads);
+    EXPECT_FALSE(blob_read.use_direct_reads);
+    EXPECT_FALSE(sst_read.use_direct_writes);
+  }
+
+  {
+    Options off_options;
+    off_options.use_direct_reads = false;
+    off_options.use_direct_io_for_compaction_reads = false;
+    off_options.use_direct_io_for_flush_and_compaction = false;
+    ImmutableDBOptions immutable_off(off_options);
+    FileOptions sst_read_off =
+        env_->GetFileSystem()->OptimizeForCompactionTableRead(in_opts,
+                                                              immutable_off);
+    EXPECT_FALSE(sst_read_off.use_direct_reads);
+  }
+
+  {
+    Options global_on_options;
+    global_on_options.use_direct_reads = true;
+    global_on_options.use_direct_io_for_compaction_reads = false;
+    global_on_options.use_direct_io_for_flush_and_compaction = false;
+    ImmutableDBOptions immutable_global(global_on_options);
+    FileOptions sst_read_global =
+        env_->GetFileSystem()->OptimizeForCompactionTableRead(in_opts,
+                                                              immutable_global);
+    EXPECT_TRUE(sst_read_global.use_direct_reads);
+  }
+
+  {
+    Options both_on;
+    both_on.use_direct_reads = true;
+    both_on.use_direct_io_for_compaction_reads = true;
+    both_on.use_direct_io_for_flush_and_compaction = false;
+    ImmutableDBOptions immutable_both(both_on);
+    FileOptions sst_read_both =
+        env_->GetFileSystem()->OptimizeForCompactionTableRead(in_opts,
+                                                              immutable_both);
+    EXPECT_TRUE(sst_read_both.use_direct_reads);
+  }
+}
+
 }  // namespace ROCKSDB_NAMESPACE
 
 int main(int argc, char** argv) {

db/db_test.cc

@@ -3516,6 +3516,18 @@ class ModelDB : public DB {
     return Status::NotSupported("Not implemented");
   }
 
+  using DB::PrepareFileIngestion;
+  Status PrepareFileIngestion(
+      const std::vector<IngestExternalFileArg>& /*args*/,
+      std::unique_ptr<FileIngestionHandle>* /*handle*/) override {
+    return Status::NotSupported("Not implemented.");
+  }
+
+  Status CommitFileIngestionHandles(
+      std::vector<std::unique_ptr<FileIngestionHandle>> /*handles*/) override {
+    return Status::NotSupported("Not implemented.");
+  }
+
   using DB::CreateColumnFamilyWithImport;
   Status CreateColumnFamilyWithImport(
       const ColumnFamilyOptions& /*options*/,
@@ -3773,6 +3785,12 @@ class ModelDB : public DB {
     return Status::OK();
   }
 
+  Status GetPreparedFileInfoForExternalSstIngestion(
+      const std::string& /*file_path*/,
+      std::shared_ptr<const PreparedFileInfo>* /*file_info*/) override {
+    return Status::NotSupported();
+  }
+
   Status GetSortedWalFiles(VectorLogPtr& /*files*/) override {
     return Status::OK();
   }

db/db_wal_test.cc

@@ -860,6 +860,8 @@ TEST_F(DBWALTest, WALWithChecksumHandoff) {
   std::shared_ptr<FaultInjectionTestFS> fault_fs(
       new FaultInjectionTestFS(FileSystem::Default()));
   std::unique_ptr<Env> fault_fs_env(NewCompositeEnv(fault_fs));
+  // Close the DB before the local Env is destroyed if an ASSERT exits early.
+  Defer close_db_on_exit([this]() { Close(); });
   do {
     Options options = CurrentOptions();
 

db/deletefile_test.cc

@@ -7,6 +7,7 @@
 // Use of this source code is governed by a BSD-style license that can be
 // found in the LICENSE file. See the AUTHORS file for names of contributors.
 
+#include <atomic>
 #include <cstdlib>
 #include <map>
 #include <string>
@@ -258,6 +259,108 @@ TEST_F(DeleteFileTest, BackgroundPurgeIteratorTest) {
   CheckFileTypeCounts(dbname_, 0, 1, 1);
 }
 
+// Regression test for a use-after-free where an obsolete-file purge started
+// DURING DB close -- after CloseHelper's early purge drain -- continued using
+// DBImpl after ~DBImpl destroyed mutex_. Seen as a vhost-user-blk SIGABRT when
+// many RocksDB instances closed concurrently under a Warm Storage fault, with
+// avoid_unnecessary_blocking_io=true on a shared, process-wide Env.
+//
+// Here we make the straggler deterministic: keep a dropped column family handle
+// alive, delete it only after CloseHelper has passed its early purge drain,
+// then park that cleanup after FindObsoleteFiles() has incremented
+// pending_purge_obsolete_files_ but before PurgeObsoleteFiles(..., true)
+// schedules BGWorkPurge. Without the fix, the final drain sees
+// bg_purge_scheduled_ == 0 and can destroy DBImpl while the pending cleanup is
+// still about to lock/use it. With the fix, CloseHelper waits through the
+// pending handoff and then through the scheduled purge.
+TEST_F(DeleteFileTest, CloseWaitsForPurgeScheduledDuringClose) {
+  Options options = CurrentOptions();
+  SetOptions(&options);
+  Destroy(options);
+  options.create_if_missing = true;
+  // Route obsolete-file deletion through the background purge queue, matching
+  // the production config that hit the bug.
+  options.avoid_unnecessary_blocking_io = true;
+  Reopen(options);
+
+  // A dropped CF whose handle we keep alive: deleting the handle schedules a
+  // *background* purge (the dropped-CF teardown path).
+  ColumnFamilyHandle* cfh = nullptr;
+  ASSERT_OK(db_->CreateColumnFamily(ColumnFamilyOptions(), "dropme", &cfh));
+  ASSERT_OK(db_->Put(WriteOptions(), cfh, "key", "value"));
+  ASSERT_OK(db_->Flush(FlushOptions(), cfh));
+  ASSERT_OK(db_->DropColumnFamily(cfh));
+  ASSERT_OK(dbfull()->TEST_WaitForPurge());  // settle setup purges
+
+  std::atomic<bool> block_pending_handoff{false};
+  SyncPoint::GetInstance()->DisableProcessing();
+  SyncPoint::GetInstance()->ClearAllCallBacks();
+  SyncPoint::GetInstance()->SetCallBack(
+      "DBImpl::PurgeObsoleteFiles:BeforePendingPurgeFinished", [&](void*) {
+        if (!block_pending_handoff.exchange(false)) {
+          return;
+        }
+        TEST_SYNC_POINT(
+            "DeleteFileTest::CloseWaitsForPurge:PendingHandoffReady");
+        TEST_SYNC_POINT(
+            "DeleteFileTest::CloseWaitsForPurge:PendingHandoffBlocked");
+      });
+  SyncPoint::GetInstance()->LoadDependency({
+      // Delete the dropped CF handle only after CloseHelper has passed its
+      // early purge drain and entered a mutex-unlocked close window.
+      {"DBImpl::CloseHelper:CFHandleCleanupUnlocked",
+       "DeleteFileTest::CloseWaitsForPurge:DropDuringCloseUnlock"},
+      // Let the test thread observe that the deleter is parked in the pending
+      // handoff window.
+      {"DeleteFileTest::CloseWaitsForPurge:PendingHandoffReady",
+       "DeleteFileTest::CloseWaitsForPurge:WaitForPendingHandoff"},
+      // Keep CloseHelper in the mutex-unlocked close window until the
+      // dropped-CF cleanup reaches the pending-to-scheduled handoff.
+      {"DeleteFileTest::CloseWaitsForPurge:PendingHandoffReady",
+       "DBImpl::CloseHelper:CFHandleCleanupAllowed"},
+      // Observe that CloseHelper is actually waiting in the final purge drain
+      // while the dropped-CF cleanup is still parked in the pending handoff.
+      {"DBImpl::CloseHelper:FinalPurgeDrainWait",
+       "DeleteFileTest::CloseWaitsForPurge:WaitForFinalPurgeDrain"},
+      // Keep the dropped-CF cleanup parked in the pending handoff window until
+      // the test releases it.
+      {"DeleteFileTest::CloseWaitsForPurge:ReleasePendingHandoff",
+       "DeleteFileTest::CloseWaitsForPurge:PendingHandoffBlocked"},
+      // Park the scheduled BGWorkPurge before it locks mutex_ until we release
+      // it, so CloseHelper also has to wait for bg_purge_scheduled_.
+      {"DeleteFileTest::CloseWaitsForPurge:ReleaseBackgroundPurge",
+       "DBImpl::BackgroundCallPurge:beforeMutexLock"},
+  });
+  SyncPoint::GetInstance()->EnableProcessing();
+
+  port::Thread closer([&]() { db_.reset(); });
+
+  // Unblocks once CloseHelper is in a mutex-unlocked close window; then
+  // deleting the dropped CF handle starts purge work the early drain missed.
+  TEST_SYNC_POINT("DeleteFileTest::CloseWaitsForPurge:DropDuringCloseUnlock");
+  block_pending_handoff.store(true);
+  ColumnFamilyHandle* dropped_cfh = cfh;
+  cfh = nullptr;
+  port::Thread dropped_cf_deleter([dropped_cfh]() { delete dropped_cfh; });
+
+  TEST_SYNC_POINT("DeleteFileTest::CloseWaitsForPurge:WaitForPendingHandoff");
+
+  // Wait until CloseHelper is blocked in the final drain with no scheduled
+  // purge yet. Without the fix, it would proceed while the dropped-CF cleanup
+  // is still pending. With the fix, it waits for pending_purge_obsolete_files_.
+  TEST_SYNC_POINT("DeleteFileTest::CloseWaitsForPurge:WaitForFinalPurgeDrain");
+
+  // Release the pending cleanup, then the BGWorkPurge it schedules. With the
+  // fix, the closer waits for both transitions and completes cleanly.
+  TEST_SYNC_POINT("DeleteFileTest::CloseWaitsForPurge:ReleasePendingHandoff");
+  dropped_cf_deleter.join();
+  TEST_SYNC_POINT("DeleteFileTest::CloseWaitsForPurge:ReleaseBackgroundPurge");
+  closer.join();
+
+  SyncPoint::GetInstance()->DisableProcessing();
+  SyncPoint::GetInstance()->ClearAllCallBacks();
+}
+
 TEST_F(DeleteFileTest, PurgeDuringOpen) {
   Options options = CurrentOptions();
   CheckFileTypeCounts(dbname_, -1, 0, -1);

db/external_sst_file_basic_test.cc

@@ -1311,7 +1311,7 @@ TEST_F(ExternalSSTFileBasicTest, SyncFailure) {
     });
     if (i == 0) {
       SyncPoint::GetInstance()->SetCallBack(
-          "ExternalSstFileIngestionJob::Prepare:Reopen", [&](void* s) {
+          "ExternalSstFileIngestionJob::CheckSyncReturnCode", [&](void* s) {
             Status* status = static_cast<Status*>(s);
             if (status->IsNotSupported()) {
               no_sync = true;
@@ -1367,13 +1367,13 @@ TEST_F(ExternalSSTFileBasicTest, SyncFailure) {
   }
 }
 
-TEST_F(ExternalSSTFileBasicTest, ReopenNotSupported) {
+TEST_F(ExternalSSTFileBasicTest, SyncFileNotSupported) {
   Options options;
   options.create_if_missing = true;
   options.env = env_;
 
   SyncPoint::GetInstance()->SetCallBack(
-      "ExternalSstFileIngestionJob::Prepare:Reopen", [&](void* arg) {
+      "ExternalSstFileIngestionJob::CheckSyncReturnCode", [&](void* arg) {
         Status* s = static_cast<Status*>(arg);
         *s = Status::NotSupported();
       });
@@ -1386,7 +1386,7 @@ TEST_F(ExternalSSTFileBasicTest, ReopenNotSupported) {
   std::unique_ptr<SstFileWriter> sst_file_writer(
       new SstFileWriter(EnvOptions(), sst_file_writer_options));
   std::string file_name =
-      sst_files_dir_ + "reopen_not_supported_test_" + ".sst";
+      sst_files_dir_ + "sync_file_not_supported_test_" + ".sst";
   ASSERT_OK(sst_file_writer->Open(file_name));
   ASSERT_OK(sst_file_writer->Put("bar", "v2"));
   ASSERT_OK(sst_file_writer->Finish());

db/external_sst_file_ingestion_job.cc

@@ -19,6 +19,7 @@
 #include "logging/logging.h"
 #include "monitoring/statistics_impl.h"
 #include "table/merging_iterator.h"
+#include "table/prepared_file_info.h"
 #include "table/sst_file_writer_collectors.h"
 #include "table/table_builder.h"
 #include "table/unique_id_impl.h"
@@ -31,18 +32,22 @@ Status ExternalSstFileIngestionJob::Prepare(
     const std::vector<std::string>& external_files_paths,
     const std::vector<std::string>& files_checksums,
     const std::vector<std::string>& files_checksum_func_names,
+    const std::vector<const PreparedFileInfo*>& file_infos,
     const std::optional<RangeOpt>& atomic_replace_range,
     const Temperature& file_temperature, uint64_t next_file_number,
     SuperVersion* sv) {
   Status status;
 
-  // Read the information of files we are ingesting
-  for (const std::string& file_path : external_files_paths) {
+  // Read the information of files we are ingesting.
+  for (size_t i = 0; i < external_files_paths.size(); i++) {
+    const std::string& file_path = external_files_paths[i];
     IngestedFileInfo file_to_ingest;
     // For temperature, first assume it matches provided hint
     file_to_ingest.file_temperature = file_temperature;
-    status =
-        GetIngestedFileInfo(file_path, next_file_number++, &file_to_ingest, sv);
+    const PreparedFileInfo* prepared_file_info =
+        file_infos.empty() ? nullptr : file_infos[i];
+    status = GetIngestedFileInfo(file_path, next_file_number++,
+                                 prepared_file_info, &file_to_ingest, sv);
     if (!status.ok()) {
       ROCKS_LOG_WARN(db_options_.info_log,
                      "Failed to get ingested file info: %s: %s",
@@ -76,23 +81,10 @@ Status ExternalSstFileIngestionJob::Prepare(
   auto num_files = files_to_ingest_.size();
   if (num_files == 0) {
     return Status::InvalidArgument("The list of files is empty");
-  } else if (num_files > 1) {
-    // Verify that passed files don't have overlapping ranges
-    autovector<const IngestedFileInfo*> sorted_files;
-    for (size_t i = 0; i < num_files; i++) {
-      sorted_files.push_back(&files_to_ingest_[i]);
-    }
-
-    std::sort(sorted_files.begin(), sorted_files.end(), file_range_checker_);
-
-    for (size_t i = 0; i + 1 < num_files; i++) {
-      if (file_range_checker_.Overlaps(*sorted_files[i], *sorted_files[i + 1],
-                                       /* known_sorted= */ true)) {
-        files_overlap_ = true;
-        break;
-      }
-    }
   }
+  // Detect whether the input files overlap one another; this drives how they
+  // are divided into batches below.
+  files_overlap_ = ComputeFilesOverlap(files_to_ingest_);
 
   if (atomic_replace_range.has_value()) {
     atomic_replace_range_.emplace();
@@ -165,35 +157,21 @@ Status ExternalSstFileIngestionJob::Prepare(
         // It is unsafe to assume application had sync the file and file
         // directory before ingest the file. For integrity of RocksDB we need
         // to sync the file.
-
-        // TODO(xingbo), We should in general be moving away from production
-        // uses of ReuseWritableFile (except explicitly for WAL recycling),
-        // ReopenWritableFile, and NewRandomRWFile. We should create a
-        // FileSystem::SyncFile/FsyncFile API that by default does the
-        // re-open+sync+close combo but can (a) be reused easily, and (b) be
-        // overridden to do that more cleanly, e.g. in EncryptedEnv.
-        // https://github.com/facebook/rocksdb/issues/13741
-        std::unique_ptr<FSWritableFile> file_to_sync;
-        Status s = fs_->ReopenWritableFile(path_inside_db, env_options_,
-                                           &file_to_sync, nullptr);
-        TEST_SYNC_POINT_CALLBACK("ExternalSstFileIngestionJob::Prepare:Reopen",
-                                 &s);
+        TEST_SYNC_POINT("ExternalSstFileIngestionJob::BeforeSyncIngestedFile");
+        Status s = fs_->SyncFile(path_inside_db, env_options_, IOOptions(),
+                                 db_options_.use_fsync, nullptr);
+        TEST_SYNC_POINT("ExternalSstFileIngestionJob::AfterSyncIngestedFile");
+        TEST_SYNC_POINT_CALLBACK(
+            "ExternalSstFileIngestionJob::CheckSyncReturnCode", &s);
         // Some file systems (especially remote/distributed) don't support
-        // reopening a file for writing and don't require reopening and
-        // syncing the file. Ignore the NotSupported error in that case.
+        // explicitly syncing the file and don't require it. Ignore the
+        // NotSupported error in that case.
         if (!s.IsNotSupported()) {
           status = s;
-          if (status.ok()) {
-            TEST_SYNC_POINT(
-                "ExternalSstFileIngestionJob::BeforeSyncIngestedFile");
-            status = SyncIngestedFile(file_to_sync.get());
-            TEST_SYNC_POINT(
-                "ExternalSstFileIngestionJob::AfterSyncIngestedFile");
-            if (!status.ok()) {
-              ROCKS_LOG_WARN(db_options_.info_log,
-                             "Failed to sync ingested file %s: %s",
-                             path_inside_db.c_str(), status.ToString().c_str());
-            }
+          if (!status.ok()) {
+            ROCKS_LOG_WARN(db_options_.info_log,
+                           "Failed to sync ingested file %s: %s",
+                           path_inside_db.c_str(), status.ToString().c_str());
           }
         }
       } else if (status.IsNotSupported() &&
@@ -447,6 +425,60 @@ void ExternalSstFileIngestionJob::DivideInputFilesIntoBatches() {
   }
 }
 
+bool ExternalSstFileIngestionJob::ComputeFilesOverlap(
+    const autovector<IngestedFileInfo>& files) const {
+  const size_t num_files = files.size();
+  if (num_files <= 1) {
+    return false;
+  }
+  // Verify whether the files have overlapping ranges by sorting copies of the
+  // file ranges and checking adjacent pairs.
+  autovector<const IngestedFileInfo*> sorted_files;
+  for (size_t i = 0; i < num_files; i++) {
+    sorted_files.push_back(&files[i]);
+  }
+  std::sort(sorted_files.begin(), sorted_files.end(), file_range_checker_);
+  for (size_t i = 0; i + 1 < num_files; i++) {
+    if (file_range_checker_.Overlaps(*sorted_files[i], *sorted_files[i + 1],
+                                     /* known_sorted= */ true)) {
+      return true;
+    }
+  }
+  return false;
+}
+
+Status ExternalSstFileIngestionJob::MergeForSameColumnFamily(
+    ExternalSstFileIngestionJob* other) {
+  assert(other != nullptr);
+  assert(other != this);
+  assert(cfd_ == other->cfd_);
+  if (atomic_replace_range_.has_value() ||
+      other->atomic_replace_range_.has_value()) {
+    return Status::NotSupported(
+        "cannot merge file ingestion handles for the same column family when "
+        "atomic_replace_range is used");
+  }
+  if (!(ingestion_options_ == other->ingestion_options_)) {
+    return Status::InvalidArgument(
+        "file ingestion handles for the same column family must be prepared "
+        "with the same IngestExternalFileOptions");
+  }
+  // Append the other job's prepared files after this job's so that, for any
+  // overlapping keys, the other job's data wins via a higher assigned sequence
+  // number -- the same semantics as passing all the files to a single ingestion
+  // call in this order. Recompute overlap and rebuild the batches over the
+  // union.
+  for (IngestedFileInfo& file : other->files_to_ingest_) {
+    files_to_ingest_.push_back(std::move(file));
+  }
+  other->files_to_ingest_.clear();
+  other->file_batches_to_ingest_.clear();
+  files_overlap_ = ComputeFilesOverlap(files_to_ingest_);
+  file_batches_to_ingest_.clear();
+  DivideInputFilesIntoBatches();
+  return Status::OK();
+}
+
 Status ExternalSstFileIngestionJob::NeedsFlush(bool* flush_needed,
                                                SuperVersion* super_version) {
   Status status;
@@ -970,18 +1002,14 @@ Status ExternalSstFileIngestionJob::ResetTableReader(
 }
 
 Status ExternalSstFileIngestionJob::SanityCheckTableProperties(
-    const std::string& external_file, uint64_t new_file_number,
-    SuperVersion* sv, IngestedFileInfo* file_to_ingest,
-    std::unique_ptr<TableReader>* table_reader) {
-  // Get the external file properties
-  auto props = table_reader->get()->GetTableProperties();
-  assert(props.get());
-  const auto& uprops = props->user_collected_properties;
+    const std::string& external_file, const TableProperties& props,
+    IngestedFileInfo* file_to_ingest) {
+  const auto& uprops = props.user_collected_properties;
 
   // Get table version
   auto version_iter = uprops.find(ExternalSstFilePropertyNames::kVersion);
   if (version_iter == uprops.end()) {
-    assert(!SstFileWriter::CreatedBySstFileWriter(*props));
+    assert(!SstFileWriter::CreatedBySstFileWriter(props));
     if (!ingestion_options_.allow_db_generated_files) {
       return Status::Corruption("External file version not found");
     } else {
@@ -990,7 +1018,7 @@ Status ExternalSstFileIngestionJob::SanityCheckTableProperties(
       file_to_ingest->version = 0;
     }
   } else {
-    assert(SstFileWriter::CreatedBySstFileWriter(*props));
+    assert(SstFileWriter::CreatedBySstFileWriter(props));
     file_to_ingest->version = DecodeFixed32(version_iter->second.c_str());
   }
 
@@ -1004,12 +1032,17 @@ Status ExternalSstFileIngestionJob::SanityCheckTableProperties(
 
     // Set the global sequence number
     file_to_ingest->original_seqno = DecodeFixed64(seqno_iter->second.c_str());
-    if (props->external_sst_file_global_seqno_offset == 0) {
-      file_to_ingest->global_seqno_offset = 0;
+    file_to_ingest->global_seqno_offset =
+        static_cast<size_t>(props.external_sst_file_global_seqno_offset);
+    // The on-disk offset is only needed if we will write the global seqno back
+    // into the file (write_global_seqno). The metadata fast-path does not open
+    // the file, and its in-memory table properties do not carry the offset
+    // (it is only computed while reading the file back); that is fine as long
+    // as write_global_seqno is not requested.
+    if (ingestion_options_.write_global_seqno &&
+        file_to_ingest->global_seqno_offset == 0) {
       return Status::Corruption("Was not able to find file global seqno field");
     }
-    file_to_ingest->global_seqno_offset =
-        static_cast<size_t>(props->external_sst_file_global_seqno_offset);
   } else if (file_to_ingest->version == 1) {
     // SST file V1 should not have global seqno field
     assert(seqno_iter == uprops.end());
@@ -1030,23 +1063,20 @@ Status ExternalSstFileIngestionJob::SanityCheckTableProperties(
                                    " is not supported");
   }
 
-  file_to_ingest->cf_id = static_cast<uint32_t>(props->column_family_id);
-  // This assignment works fine even though `table_reader` may later be reset,
-  // since that will not affect how table properties are parsed, and this
-  // assignment is making a copy.
-  file_to_ingest->table_properties = *props;
+  file_to_ingest->cf_id = static_cast<uint32_t>(props.column_family_id);
+  file_to_ingest->table_properties = props;
 
   // Get number of entries in table
-  file_to_ingest->num_entries = props->num_entries;
-  file_to_ingest->num_range_deletions = props->num_range_deletions;
+  file_to_ingest->num_entries = props.num_entries;
+  file_to_ingest->num_range_deletions = props.num_range_deletions;
 
   // Validate table properties related to comparator name and user defined
   // timestamps persisted flag.
   file_to_ingest->user_defined_timestamps_persisted =
-      static_cast<bool>(props->user_defined_timestamps_persisted);
+      static_cast<bool>(props.user_defined_timestamps_persisted);
   bool mark_sst_file_has_no_udt = false;
   Status s = ValidateUserDefinedTimestampsOptions(
-      cfd_->user_comparator(), props->comparator_name,
+      cfd_->user_comparator(), props.comparator_name,
       cfd_->ioptions().persist_user_defined_timestamps,
       file_to_ingest->user_defined_timestamps_persisted,
       &mark_sst_file_has_no_udt);
@@ -1054,7 +1084,9 @@ Status ExternalSstFileIngestionJob::SanityCheckTableProperties(
     // A column family that enables user-defined timestamps in Memtable only
     // feature can also ingest external files created by a setting that disables
     // user-defined timestamps. In that case, we need to re-mark the
-    // user_defined_timestamps_persisted flag for the file.
+    // user_defined_timestamps_persisted flag for the file. The open-and-scan
+    // caller is then responsible for reopening its `TableReader` with the
+    // updated flag.
     file_to_ingest->user_defined_timestamps_persisted = false;
   } else if (!s.ok()) {
     ROCKS_LOG_WARN(
@@ -1064,22 +1096,84 @@ Status ExternalSstFileIngestionJob::SanityCheckTableProperties(
     return s;
   }
 
-  // `TableReader` is initialized with `user_defined_timestamps_persisted` flag
-  // to be true. If its value changed to false after this sanity check, we
-  // need to reset the `TableReader`.
-  if (ucmp_->timestamp_size() > 0 &&
-      !file_to_ingest->user_defined_timestamps_persisted) {
-    s = ResetTableReader(external_file, new_file_number,
-                         file_to_ingest->user_defined_timestamps_persisted, sv,
-                         file_to_ingest, table_reader);
-  }
   return s;
 }
 
-Status ExternalSstFileIngestionJob::GetIngestedFileInfo(
+Status ExternalSstFileIngestionJob::GetIngestedFileInfoFromFileInfo(
+    const std::string& external_file,
+    const PreparedFileInfo& prepared_file_info,
+    IngestedFileInfo* file_to_ingest) {
+  // Boundaries, size, and table properties were obtained without opening the
+  // file (e.g. produced by SstFileWriter::Finish). We reuse them directly.
+  file_to_ingest->file_size = prepared_file_info.file_size;
+  Status status = SanityCheckTableProperties(
+      external_file, prepared_file_info.table_properties, file_to_ingest);
+  if (!status.ok()) {
+    ROCKS_LOG_WARN(
+        db_options_.info_log,
+        "Failed to sanity check table properties for external file %s: %s",
+        external_file.c_str(), status.ToString().c_str());
+    return status;
+  }
+
+  const size_t ts_sz = ucmp_->timestamp_size();
+  if (ts_sz > 0 && !file_to_ingest->user_defined_timestamps_persisted) {
+    auto pad_timestamp = [ts_sz](std::string* result, const Slice& key) {
+      assert(result->empty());
+      if (ExtractValueType(key) == kTypeRangeDeletion) {
+        PadInternalKeyWithMaxTimestamp(result, key, ts_sz);
+      } else {
+        PadInternalKeyWithMinTimestamp(result, key, ts_sz);
+      }
+    };
+    pad_timestamp(file_to_ingest->smallest_internal_key.rep(),
+                  prepared_file_info.smallest.Encode());
+    pad_timestamp(file_to_ingest->largest_internal_key.rep(),
+                  prepared_file_info.largest.Encode());
+  } else {
+    file_to_ingest->smallest_internal_key = prepared_file_info.smallest;
+    file_to_ingest->largest_internal_key = prepared_file_info.largest;
+  }
+
+  if (ingestion_options_.allow_db_generated_files) {
+    // Sequence numbers are preserved (not reassigned), so the bounds must be
+    // known before we skip the GetSeqnoBoundaryForFile scan.
+    if (!file_to_ingest->table_properties.HasKeyLargestSeqno()) {
+      return Status::Corruption("Unknown largest seqno for db generated file.");
+    }
+    file_to_ingest->largest_seqno =
+        file_to_ingest->table_properties.key_largest_seqno;
+    if (file_to_ingest->largest_seqno == 0) {
+      file_to_ingest->smallest_seqno = 0;
+    } else {
+      if (!file_to_ingest->table_properties.HasKeySmallestSeqno()) {
+        return Status::Corruption(
+            "Unknown smallest seqno for db generated file.");
+      }
+      file_to_ingest->smallest_seqno =
+          file_to_ingest->table_properties.key_smallest_seqno;
+    }
+  } else {
+    // Normal ingestion reassigns a global sequence number later, so the file's
+    // keys must currently be at seqno 0 (mirror of the open-and-scan check).
+    SequenceNumber largest_seqno =
+        file_to_ingest->table_properties.key_largest_seqno;
+    // UINT64_MAX means unknown and the file is generated before table property
+    // `key_largest_seqno` is introduced.
+    if (largest_seqno != UINT64_MAX && largest_seqno > 0) {
+      return Status::Corruption(
+          "External file has non zero largest sequence number " +
+          std::to_string(largest_seqno));
+    }
+  }
+  return Status::OK();
+}
+
+Status ExternalSstFileIngestionJob::GetIngestedFileInfoFromFile(
     const std::string& external_file, uint64_t new_file_number,
-    IngestedFileInfo* file_to_ingest, SuperVersion* sv) {
-  file_to_ingest->external_file_path = external_file;
+    IngestedFileInfo* file_to_ingest, SuperVersion* sv,
+    std::unique_ptr<TableReader>* out_table_reader) {
+  TEST_SYNC_POINT("ExternalSstFileIngestionJob::GetIngestedFileInfo:ReadPath");
 
   // Get external file size
   Status status = fs_->GetFileSize(external_file, IOOptions(),
@@ -1091,15 +1185,11 @@ Status ExternalSstFileIngestionJob::GetIngestedFileInfo(
     return status;
   }
 
-  // Assign FD with number
-  file_to_ingest->fd =
-      FileDescriptor(new_file_number, 0, file_to_ingest->file_size);
-
-  // Create TableReader for external file
+  // Create TableReader for external file.
   std::unique_ptr<TableReader> table_reader;
   // Initially create the `TableReader` with flag
-  // `user_defined_timestamps_persisted` to be true since that's the most common
-  // case
+  // `user_defined_timestamps_persisted` to be true since that's the most
+  // common case
   status = ResetTableReader(external_file, new_file_number,
                             /*user_defined_timestamps_persisted=*/true, sv,
                             file_to_ingest, &table_reader);
@@ -1110,8 +1200,8 @@ Status ExternalSstFileIngestionJob::GetIngestedFileInfo(
     return status;
   }
 
-  status = SanityCheckTableProperties(external_file, new_file_number, sv,
-                                      file_to_ingest, &table_reader);
+  status = SanityCheckTableProperties(
+      external_file, *table_reader->GetTableProperties(), file_to_ingest);
   if (!status.ok()) {
     ROCKS_LOG_WARN(
         db_options_.info_log,
@@ -1120,6 +1210,23 @@ Status ExternalSstFileIngestionJob::GetIngestedFileInfo(
     return status;
   }
 
+  // The `TableReader` above was opened with `user_defined_timestamps_persisted`
+  // assumed true. If the sanity check determined the file has no persisted
+  // timestamps (UDT-in-Memtable-only feature), reopen it with the corrected
+  // flag so keys are parsed properly by the scan below.
+  if (ucmp_->timestamp_size() > 0 &&
+      !file_to_ingest->user_defined_timestamps_persisted) {
+    status = ResetTableReader(external_file, new_file_number,
+                              file_to_ingest->user_defined_timestamps_persisted,
+                              sv, file_to_ingest, &table_reader);
+    if (!status.ok()) {
+      ROCKS_LOG_WARN(db_options_.info_log,
+                     "Failed to reset table reader for external file %s: %s",
+                     external_file.c_str(), status.ToString().c_str());
+      return status;
+    }
+  }
+
   const bool allow_data_in_errors = db_options_.allow_data_in_errors;
   ParsedInternalKey key;
   if (ingestion_options_.allow_db_generated_files) {
@@ -1141,8 +1248,8 @@ Status ExternalSstFileIngestionJob::GetIngestedFileInfo(
   } else {
     SequenceNumber largest_seqno =
         table_reader.get()->GetTableProperties()->key_largest_seqno;
-    // UINT64_MAX means unknown and the file is generated before table property
-    // `key_largest_seqno` is introduced.
+    // UINT64_MAX means unknown and the file is generated before table
+    // property `key_largest_seqno` is introduced.
     if (largest_seqno != UINT64_MAX && largest_seqno > 0) {
       return Status::Corruption(
           "External file has non zero largest sequence number " +
@@ -1150,24 +1257,6 @@ Status ExternalSstFileIngestionJob::GetIngestedFileInfo(
     }
   }
 
-  if (ingestion_options_.verify_checksums_before_ingest) {
-    // If customized readahead size is needed, we can pass a user option
-    // all the way to here. Right now we just rely on the default readahead
-    // to keep things simple.
-    // TODO: plumb Env::IOActivity, Env::IOPriority
-    ReadOptions ro;
-    ro.readahead_size = ingestion_options_.verify_checksums_readahead_size;
-    ro.fill_cache = ingestion_options_.fill_cache;
-    status = table_reader->VerifyChecksum(
-        ro, TableReaderCaller::kExternalSSTIngestion);
-    if (!status.ok()) {
-      ROCKS_LOG_WARN(db_options_.info_log,
-                     "Failed to verify checksum for table reader: %s",
-                     status.ToString().c_str());
-      return status;
-    }
-  }
-
   // TODO: plumb Env::IOActivity, Env::IOPriority
   ReadOptions ro;
   ro.fill_cache = ingestion_options_.fill_cache;
@@ -1247,6 +1336,22 @@ Status ExternalSstFileIngestionJob::GetIngestedFileInfo(
             "External file has a range deletion with non zero sequence "
             "number.");
       }
+#ifndef NDEBUG
+      // To keep aligned with the fast path that expects range deletion keys to
+      // always use max ts.
+      const size_t ts_sz = ucmp_->timestamp_size();
+      if (ts_sz > 0) {
+        const std::string max_ts(ts_sz, '\xff');
+        assert(key.user_key.size() >= ts_sz);
+        assert(ucmp_->CompareTimestamp(
+                   ExtractTimestampFromUserKey(key.user_key, ts_sz), max_ts) ==
+               0);
+        assert(range_del_iter->value().size() >= ts_sz);
+        assert(ucmp_->CompareTimestamp(
+                   ExtractTimestampFromUserKey(range_del_iter->value(), ts_sz),
+                   max_ts) == 0);
+      }
+#endif
       RangeTombstone tombstone(key, range_del_iter->value());
       file_range_checker_.MaybeUpdateRange(tombstone.SerializeKey(),
                                            tombstone.SerializeEndKey(),
@@ -1254,6 +1359,75 @@ Status ExternalSstFileIngestionJob::GetIngestedFileInfo(
     }
   }
 
+  *out_table_reader = std::move(table_reader);
+  return Status::OK();
+}
+
+Status ExternalSstFileIngestionJob::GetIngestedFileInfo(
+    const std::string& external_file, uint64_t new_file_number,
+    const PreparedFileInfo* prepared_file_info,
+    IngestedFileInfo* file_to_ingest, SuperVersion* sv) {
+  file_to_ingest->external_file_path = external_file;
+
+  std::unique_ptr<TableReader> table_reader;
+  Status status =
+      prepared_file_info != nullptr
+          ? GetIngestedFileInfoFromFileInfo(external_file, *prepared_file_info,
+                                            file_to_ingest)
+          : GetIngestedFileInfoFromFile(external_file, new_file_number,
+                                        file_to_ingest, sv, &table_reader);
+  if (!status.ok()) {
+    return status;
+  }
+
+  assert(file_to_ingest->file_size > 0);
+  assert(!file_to_ingest->unset());
+  assert(file_to_ingest->table_properties.num_entries > 0 ||
+         file_to_ingest->table_properties.num_range_deletions > 0);
+  if (ingestion_options_.allow_db_generated_files) {
+    // These files keep their original sequence numbers (derived, not
+    // reassigned), so the bounds must be valid here.
+    assert(file_to_ingest->smallest_seqno <= file_to_ingest->largest_seqno);
+    assert(file_to_ingest->largest_seqno < kMaxSequenceNumber);
+  }
+
+  // Verify the file checksum if requested. The open-and-scan path already has a
+  // `TableReader` open; the fast-path opens one here only when verification is
+  // requested -- otherwise it performs no file I/O.
+  if (ingestion_options_.verify_checksums_before_ingest) {
+    if (table_reader == nullptr) {
+      status =
+          ResetTableReader(external_file, new_file_number,
+                           file_to_ingest->user_defined_timestamps_persisted,
+                           sv, file_to_ingest, &table_reader);
+      if (!status.ok()) {
+        ROCKS_LOG_WARN(db_options_.info_log,
+                       "Failed to reset table reader for external file %s: %s",
+                       external_file.c_str(), status.ToString().c_str());
+        return status;
+      }
+    }
+    // If customized readahead size is needed, we can pass a user option all the
+    // way to here. Right now we just rely on the default readahead to keep
+    // things simple.
+    // TODO: plumb Env::IOActivity, Env::IOPriority
+    ReadOptions ro;
+    ro.readahead_size = ingestion_options_.verify_checksums_readahead_size;
+    ro.fill_cache = ingestion_options_.fill_cache;
+    status = table_reader->VerifyChecksum(
+        ro, TableReaderCaller::kExternalSSTIngestion);
+    if (!status.ok()) {
+      ROCKS_LOG_WARN(db_options_.info_log,
+                     "Failed to verify checksum for external file %s: %s",
+                     external_file.c_str(), status.ToString().c_str());
+      return status;
+    }
+  }
+
+  // Assign FD with number.
+  file_to_ingest->fd =
+      FileDescriptor(new_file_number, 0, file_to_ingest->file_size);
+
   const size_t ts_sz = ucmp_->timestamp_size();
   Slice smallest = file_to_ingest->smallest_internal_key.user_key();
   Slice largest = file_to_ingest->largest_internal_key.user_key();

db/external_sst_file_ingestion_job.h

@@ -248,6 +248,7 @@ class ExternalSstFileIngestionJob {
   Status Prepare(const std::vector<std::string>& external_files_paths,
                  const std::vector<std::string>& files_checksums,
                  const std::vector<std::string>& files_checksum_func_names,
+                 const std::vector<const PreparedFileInfo*>& file_infos,
                  const std::optional<RangeOpt>& atomic_replace_range,
                  const Temperature& file_temperature, uint64_t next_file_number,
                  SuperVersion* sv);
@@ -301,6 +302,20 @@ class ExternalSstFileIngestionJob {
     return max_assigned_seqno_;
   }
 
+  // Max threads requested for opening this job's files during commit, from the
+  // per-CF IngestExternalFileOptions.
+  int file_opening_threads() const {
+    return ingestion_options_.file_opening_threads;
+  }
+
+  // Merge another already-Prepare()d job for the SAME column family into this
+  // one so both sets of files are committed by a single Run(). The other job's
+  // files are appended after this job's, so for any overlapping keys the other
+  // job's data wins via a higher assigned sequence number -- the same semantics
+  // as passing all the files to a single ingestion call in this order. The
+  // other job is left empty.
+  Status MergeForSameColumnFamily(ExternalSstFileIngestionJob* other);
+
  private:
   Status ResetTableReader(const std::string& external_file,
                           uint64_t new_file_number,
@@ -315,17 +330,37 @@ class ExternalSstFileIngestionJob {
   // different options. For example: when external file does not contain
   // timestamps while column family enables UDT in Memtables only feature.
   Status SanityCheckTableProperties(const std::string& external_file,
-                                    uint64_t new_file_number, SuperVersion* sv,
-                                    IngestedFileInfo* file_to_ingest,
-                                    std::unique_ptr<TableReader>* table_reader);
+                                    const TableProperties& props,
+                                    IngestedFileInfo* file_to_ingest);
 
   // Open the external file and populate `file_to_ingest` with all the
-  // external information we need to ingest this file.
+  // external information we need to ingest this file. When
+  // `prepared_file_info` is non-null, its caller-supplied metadata is reused
+  // instead of opening and scanning the file.
   Status GetIngestedFileInfo(const std::string& external_file,
                              uint64_t new_file_number,
+                             const PreparedFileInfo* prepared_file_info,
                              IngestedFileInfo* file_to_ingest,
                              SuperVersion* sv);
 
+  // Acquire the per-file metadata from the caller-supplied opaque
+  // `PreparedFileInfo` (produced by SstFileWriter::Finish) instead of opening
+  // the file.
+  Status GetIngestedFileInfoFromFileInfo(
+      const std::string& external_file,
+      const PreparedFileInfo& prepared_file_info,
+      IngestedFileInfo* file_to_ingest);
+
+  // Acquire the per-file metadata by opening the external file and scanning it
+  // (table properties, sequence number bounds, and boundary keys including any
+  // range-tombstone extensions). Used when no file_info is available. The
+  // opened `TableReader` is returned via `*table_reader` so the caller can
+  // reuse it (e.g. to verify the file checksum) without re-opening the file.
+  Status GetIngestedFileInfoFromFile(
+      const std::string& external_file, uint64_t new_file_number,
+      IngestedFileInfo* file_to_ingest, SuperVersion* sv,
+      std::unique_ptr<TableReader>* out_table_reader);
+
   // If the input files' key range overlaps themselves, this function divides
   // them in the user specified order into multiple batches. Where the files
   // within a batch do not overlap with each other, but key range could overlap
@@ -334,6 +369,10 @@ class ExternalSstFileIngestionJob {
   // make one batch.
   void DivideInputFilesIntoBatches();
 
+  // Returns whether any two files in `files` have overlapping key ranges, by
+  // sorting the file ranges and checking adjacent pairs.
+  bool ComputeFilesOverlap(const autovector<IngestedFileInfo>& files) const;
+
   // Assign level for the files in one batch. The files within one batch are not
   // overlapping, and we assign level to each file one after another.
   // If `prev_batch_uppermost_level` is specified, all files in this batch will
@@ -407,7 +446,7 @@ class ExternalSstFileIngestionJob {
   SnapshotList* db_snapshots_;
   autovector<IngestedFileInfo> files_to_ingest_;
   std::vector<FileBatchInfo> file_batches_to_ingest_;
-  const IngestExternalFileOptions& ingestion_options_;
+  const IngestExternalFileOptions ingestion_options_;
   std::optional<KeyRangeInfo> atomic_replace_range_;
   Directories* directories_;
   EventLogger* event_logger_;

db/external_sst_file_test.cc

@@ -5,6 +5,7 @@
 
 #include <table/block_based/block_based_table_factory.h>
 
+#include <atomic>
 #include <functional>
 #include <memory>
 #include <sstream>
@@ -17,6 +18,7 @@
 #include "port/stack_trace.h"
 #include "rocksdb/sst_file_reader.h"
 #include "rocksdb/sst_file_writer.h"
+#include "table/prepared_file_info.h"
 #include "test_util/testutil.h"
 #include "util/random.h"
 #include "util/thread_guard.h"
@@ -92,10 +94,17 @@ class ExternSSTFileLinkFailFallbackTest
 
 class ExternalSSTFileTest
     : public ExternalSSTFileTestBase,
-      public ::testing::WithParamInterface<std::tuple<bool, bool>> {
+      public ::testing::WithParamInterface<std::tuple<bool, bool, bool>> {
  public:
   ExternalSSTFileTest() = default;
 
+  void SetUp() override {
+    const auto* info = ::testing::UnitTest::GetInstance()->current_test_info();
+    if (info != nullptr && info->value_param() != nullptr) {
+      two_phase_ingest_ = std::get<2>(GetParam());
+    }
+  }
+
   Status GenerateOneExternalFile(
       const Options& options, ColumnFamilyHandle* cfh,
       std::vector<std::pair<std::string, std::string>>& data, int file_id,
@@ -152,7 +161,8 @@ class ExternalSSTFileTest
       bool verify_checksums_before_ingest = true, bool ingest_behind = false,
       bool sort_data = false,
       std::map<std::string, std::string>* true_data = nullptr,
-      ColumnFamilyHandle* cfh = nullptr, bool fill_cache = false) {
+      ColumnFamilyHandle* cfh = nullptr, bool fill_cache = false,
+      bool ingest_with_file_info = false) {
     // Generate a file id if not provided
     if (file_id == -1) {
       file_id = last_file_id_ + 1;
@@ -188,7 +198,12 @@ class ExternalSSTFileTest
         return s;
       }
     }
-    s = sst_file_writer.Finish();
+    ExternalSstFileInfo file_info;
+    if (ingest_with_file_info) {
+      s = sst_file_writer.Finish(&file_info);
+    } else {
+      s = sst_file_writer.Finish();
+    }
 
     if (s.ok()) {
       IngestExternalFileOptions ifo;
@@ -197,7 +212,21 @@ class ExternalSSTFileTest
       ifo.verify_checksums_before_ingest = verify_checksums_before_ingest;
       ifo.ingest_behind = ingest_behind;
       ifo.fill_cache = fill_cache;
-      if (cfh) {
+      if (two_phase_ingest_) {
+        std::unique_ptr<FileIngestionHandle> handle;
+        s = db_->PrepareFileIngestion(cfh ? cfh : db_->DefaultColumnFamily(),
+                                      {file_path}, ifo, &handle);
+        if (s.ok()) {
+          s = db_->CommitFileIngestionHandle(std::move(handle));
+        }
+      } else if (ingest_with_file_info) {
+        IngestExternalFileArg arg;
+        arg.column_family = cfh ? cfh : db_->DefaultColumnFamily();
+        arg.external_files = {file_path};
+        arg.file_infos = {file_info.prepared_file_info.get()};
+        arg.options = ifo;
+        s = db_->IngestExternalFiles({arg});
+      } else if (cfh) {
         s = db_->IngestExternalFile(cfh, {file_path}, ifo);
       } else {
         s = db_->IngestExternalFile({file_path}, ifo);
@@ -243,6 +272,14 @@ class ExternalSSTFileTest
       args[i].external_files.push_back(external_file_path);
       args[i].options = ifos[i];
     }
+    if (two_phase_ingest_) {
+      std::unique_ptr<FileIngestionHandle> handle;
+      Status s = db_->PrepareFileIngestion(args, &handle);
+      if (!s.ok()) {
+        return s;
+      }
+      return db_->CommitFileIngestionHandle(std::move(handle));
+    }
     return db_->IngestExternalFiles(args);
   }
 
@@ -294,10 +331,540 @@ class ExternalSSTFileTest
     return db_->IngestExternalFile(files, opts);
   }
 
+  // Writes `data` (sorted by user key) into a new external SST and returns its
+  // path in *file_path WITHOUT ingesting it, so a test can drive the two-phase
+  // PrepareFileIngestion()/CommitFileIngestion() API directly.
+  Status GenerateExternalFileOnly(
+      const Options& options,
+      std::vector<std::pair<std::string, std::string>> data,
+      std::string* file_path) {
+    std::sort(data.begin(), data.end(),
+              [&](const std::pair<std::string, std::string>& e1,
+                  const std::pair<std::string, std::string>& e2) {
+                return options.comparator->Compare(e1.first, e2.first) < 0;
+              });
+    *file_path = sst_files_dir_ + "only_" + std::to_string(++last_file_id_);
+    SstFileWriter sst_file_writer(EnvOptions(), options);
+    Status s = sst_file_writer.Open(*file_path);
+    if (!s.ok()) {
+      return s;
+    }
+    for (const auto& entry : data) {
+      s = sst_file_writer.Put(entry.first, entry.second);
+      if (!s.ok()) {
+        sst_file_writer.Finish().PermitUncheckedError();
+        return s;
+      }
+    }
+    return sst_file_writer.Finish();
+  }
+
  protected:
   int last_file_id_ = 0;
+  bool two_phase_ingest_ = false;
 };
 
+TEST_F(ExternalSSTFileTest, IngestionTimingHistogram) {
+  Options options = CurrentOptions();
+  options.statistics = CreateDBStatistics();
+  options.statistics->set_stats_level(StatsLevel::kAll);  // enable timers
+  DestroyAndReopen(options);
+
+  ASSERT_OK(GenerateAndAddExternalFile(
+      options, {{"k1", "v1"}, {"k2", "v2"}, {"k3", "v3"}}, /*file_id=*/-1,
+      /*allow_global_seqno=*/true, /*write_global_seqno=*/false,
+      /*verify_checksums_before_ingest=*/true, /*ingest_behind=*/false,
+      /*sort_data=*/true));
+
+  auto hist = [&](Histograms h) {
+    HistogramData hd;
+    options.statistics->histogramData(h, &hd);
+    return hd;
+  };
+
+  // One sample per call (not per CF) in each phase histogram.
+  HistogramData prepare_hd = hist(INGEST_EXTERNAL_FILE_PREPARE_TIME);
+  HistogramData run_hd = hist(INGEST_EXTERNAL_FILE_RUN_TIME);
+  ASSERT_EQ(1, prepare_hd.count);
+  ASSERT_EQ(1, run_hd.count);
+  ASSERT_GT(prepare_hd.max, 0.0);
+  ASSERT_GT(run_hd.max, 0.0);
+
+  // A second call adds exactly one more sample to each histogram.
+  ASSERT_OK(GenerateAndAddExternalFile(
+      options, {{"k4", "v4"}, {"k5", "v5"}}, /*file_id=*/-1,
+      /*allow_global_seqno=*/true, /*write_global_seqno=*/false,
+      /*verify_checksums_before_ingest=*/true, /*ingest_behind=*/false,
+      /*sort_data=*/true));
+  ASSERT_EQ(2, hist(INGEST_EXTERNAL_FILE_PREPARE_TIME).count);
+  ASSERT_EQ(2, hist(INGEST_EXTERNAL_FILE_RUN_TIME).count);
+
+  // A failed ingestion records nothing (latency is logged only on success).
+  ASSERT_NOK(db_->IngestExternalFile({"/path/does/not/exist.sst"},
+                                     IngestExternalFileOptions()));
+  ASSERT_EQ(2, hist(INGEST_EXTERNAL_FILE_PREPARE_TIME).count);
+  ASSERT_EQ(2, hist(INGEST_EXTERNAL_FILE_RUN_TIME).count);
+}
+
+TEST_F(ExternalSSTFileTest, PrepareThenCommit) {
+  Options options = CurrentOptions();
+  DestroyAndReopen(options);
+
+  std::string file_path;
+  ASSERT_OK(GenerateExternalFileOnly(
+      options, {{"k1", "v1"}, {"k2", "v2"}, {"k3", "v3"}}, &file_path));
+
+  IngestExternalFileArg arg;
+  arg.column_family = db_->DefaultColumnFamily();
+  arg.external_files = {file_path};
+
+  std::unique_ptr<FileIngestionHandle> handle;
+  ASSERT_OK(db_->PrepareFileIngestion({arg}, &handle));
+  ASSERT_TRUE(handle != nullptr);
+
+  // The prepared file is staged in the DB but not yet visible.
+  ASSERT_EQ("NOT_FOUND", Get("k1"));
+
+  ASSERT_OK(db_->CommitFileIngestionHandle(std::move(handle)));
+  ASSERT_EQ("v1", Get("k1"));
+  ASSERT_EQ("v2", Get("k2"));
+  ASSERT_EQ("v3", Get("k3"));
+}
+
+TEST_F(ExternalSSTFileTest, ParallelFileOpenWithFileOpeningThreads) {
+  // Ingest many non-overlapping files with file_opening_threads > 1 so commit
+  // opens table readers with multiple threads. max_open_files == -1 makes
+  // commit open every new file.
+  Options options = CurrentOptions();
+  options.max_open_files = -1;
+  DestroyAndReopen(options);
+
+  constexpr int kNumFiles = 8;
+  constexpr int kKeysPerFile = 50;
+  std::vector<std::string> files;
+  std::map<std::string, std::string> true_data;
+  for (int f = 0; f < kNumFiles; f++) {
+    std::vector<std::pair<std::string, std::string>> data;
+    for (int k = 0; k < kKeysPerFile; k++) {
+      const int key = f * kKeysPerFile + k;
+      const std::string value = "val" + std::to_string(key);
+      data.emplace_back(Key(key), value);
+      true_data[Key(key)] = value;
+    }
+    std::string file_path;
+    ASSERT_OK(GenerateExternalFileOnly(options, data, &file_path));
+    files.push_back(file_path);
+  }
+
+  IngestExternalFileOptions ifo;
+  ifo.file_opening_threads = 4;
+  ASSERT_OK(db_->IngestExternalFile(files, ifo));
+
+  for (const auto& [key, value] : true_data) {
+    ASSERT_EQ(value, Get(key));
+  }
+}
+
+TEST_F(ExternalSSTFileTest, AbortPreparedIngestion) {
+  Options options = CurrentOptions();
+  DestroyAndReopen(options);
+
+  std::string file_path;
+  ASSERT_OK(GenerateExternalFileOnly(options, {{"x", "1"}}, &file_path));
+  IngestExternalFileArg arg;
+  arg.column_family = db_->DefaultColumnFamily();
+  arg.external_files = {file_path};
+
+  std::unique_ptr<FileIngestionHandle> handle;
+  ASSERT_OK(db_->PrepareFileIngestion({arg}, &handle));
+  ASSERT_OK(handle->Abort());
+  ASSERT_EQ("NOT_FOUND", Get("x"));
+
+  // Committing an already-aborted handle returns an error.
+  ASSERT_TRUE(
+      db_->CommitFileIngestionHandle(std::move(handle)).IsInvalidArgument());
+
+  // The DB remains healthy after an abort: a fresh prepare + commit works.
+  std::unique_ptr<FileIngestionHandle> handle2;
+  ASSERT_OK(db_->PrepareFileIngestion({arg}, &handle2));
+  ASSERT_OK(db_->CommitFileIngestionHandle(std::move(handle2)));
+  ASSERT_EQ("1", Get("x"));
+}
+
+TEST_F(ExternalSSTFileTest, MultiHandleAtomicCommit) {
+  Options options = CurrentOptions();
+  CreateAndReopenWithCF({"cf1", "cf2", "cf3"}, options);
+
+  // Two handles whose column families overlap on cf2:
+  //   handle 1 -> cf1, cf2     handle 2 -> cf2, cf3
+  // Committing them together installs cf1 and cf3 each from one handle, while
+  // cf2 receives files from BOTH handles, merged into a single per-CF commit.
+  std::string f_cf1;
+  std::string f_cf2_h1;
+  std::string f_cf2_h2;
+  std::string f_cf3;
+  ASSERT_OK(GenerateExternalFileOnly(options, {{"a", "1"}}, &f_cf1));
+  // The cf2 files share key "b": handle 2 is committed later, so its value must
+  // win via a higher assigned sequence number.
+  ASSERT_OK(GenerateExternalFileOnly(options, {{"b", "h1"}, {"b1", "10"}},
+                                     &f_cf2_h1));
+  ASSERT_OK(GenerateExternalFileOnly(options, {{"b", "h2"}, {"b2", "20"}},
+                                     &f_cf2_h2));
+  ASSERT_OK(GenerateExternalFileOnly(options, {{"c", "3"}}, &f_cf3));
+
+  std::vector<IngestExternalFileArg> args1(2);
+  args1[0].column_family = handles_[1];  // cf1
+  args1[0].external_files = {f_cf1};
+  args1[1].column_family = handles_[2];  // cf2
+  args1[1].external_files = {f_cf2_h1};
+
+  std::vector<IngestExternalFileArg> args2(2);
+  args2[0].column_family =
+      handles_[2];  // cf2 -> merges with handle 1's cf2 job
+  args2[0].external_files = {f_cf2_h2};
+  args2[1].column_family = handles_[3];  // cf3
+  args2[1].external_files = {f_cf3};
+
+  std::unique_ptr<FileIngestionHandle> h1;
+  std::unique_ptr<FileIngestionHandle> h2;
+  ASSERT_OK(db_->PrepareFileIngestion(args1, &h1));
+  ASSERT_OK(db_->PrepareFileIngestion(args2, &h2));
+  ASSERT_EQ("NOT_FOUND", Get(1, "a"));
+  ASSERT_EQ("NOT_FOUND", Get(2, "b1"));
+  ASSERT_EQ("NOT_FOUND", Get(3, "c"));
+
+  // Commit both handles atomically.
+  std::vector<std::unique_ptr<FileIngestionHandle>> batch;
+  batch.push_back(std::move(h1));
+  batch.push_back(std::move(h2));
+  ASSERT_OK(db_->CommitFileIngestionHandles(std::move(batch)));
+
+  ASSERT_EQ("1", Get(1, "a"));
+  ASSERT_EQ("h2", Get(2, "b"));   // overlapping key: later handle (h2) wins
+  ASSERT_EQ("10", Get(2, "b1"));  // cf2 from handle 1
+  ASSERT_EQ("20", Get(2, "b2"));  // cf2 from handle 2 (merged)
+  ASSERT_EQ("3", Get(3, "c"));
+}
+
+TEST_F(ExternalSSTFileTest, MultiHandleSameColumnFamilyIncompatibleOptions) {
+  Options options = CurrentOptions();
+  DestroyAndReopen(options);
+
+  std::string f1;
+  std::string f2;
+  ASSERT_OK(GenerateExternalFileOnly(options, {{"a", "1"}}, &f1));
+  ASSERT_OK(GenerateExternalFileOnly(options, {{"b", "2"}}, &f2));
+
+  // Same column family but with options the merged Run() would consult set
+  // differently across the two handles.
+  IngestExternalFileOptions opts_fill;
+  IngestExternalFileOptions opts_no_fill;
+  opts_no_fill.fill_cache = false;
+  std::unique_ptr<FileIngestionHandle> h1;
+  std::unique_ptr<FileIngestionHandle> h2;
+  ASSERT_OK(db_->PrepareFileIngestion(db_->DefaultColumnFamily(), {f1},
+                                      opts_fill, &h1));
+  ASSERT_OK(db_->PrepareFileIngestion(db_->DefaultColumnFamily(), {f2},
+                                      opts_no_fill, &h2));
+
+  std::vector<std::unique_ptr<FileIngestionHandle>> batch;
+  batch.push_back(std::move(h1));
+  batch.push_back(std::move(h2));
+  ASSERT_TRUE(
+      db_->CommitFileIngestionHandles(std::move(batch)).IsInvalidArgument());
+
+  // The rejected commit rolled both handles back; the CF is still usable.
+  ASSERT_EQ("NOT_FOUND", Get("a"));
+  ASSERT_EQ("NOT_FOUND", Get("b"));
+  std::unique_ptr<FileIngestionHandle> h3;
+  ASSERT_OK(db_->PrepareFileIngestion(db_->DefaultColumnFamily(), {f1},
+                                      IngestExternalFileOptions(), &h3));
+  ASSERT_OK(db_->CommitFileIngestionHandle(std::move(h3)));
+  ASSERT_EQ("1", Get("a"));
+}
+
+TEST_F(ExternalSSTFileTest, PrepareFileIngestionValidationErrors) {
+  Options options = CurrentOptions();
+  DestroyAndReopen(options);
+
+  std::unique_ptr<FileIngestionHandle> handle;
+
+  ASSERT_TRUE(
+      db_->PrepareFileIngestion(std::vector<IngestExternalFileArg>{}, nullptr)
+          .IsInvalidArgument());
+
+  // Empty arg list.
+  ASSERT_TRUE(
+      db_->PrepareFileIngestion(std::vector<IngestExternalFileArg>{}, &handle)
+          .IsInvalidArgument());
+  ASSERT_TRUE(handle == nullptr);
+
+  // move_files and link_files are mutually exclusive.
+  std::string file_path;
+  ASSERT_OK(GenerateExternalFileOnly(options, {{"k", "v"}}, &file_path));
+  IngestExternalFileArg arg;
+  arg.column_family = db_->DefaultColumnFamily();
+  arg.external_files = {file_path};
+  arg.options.move_files = true;
+  arg.options.link_files = true;
+  ASSERT_TRUE(db_->PrepareFileIngestion({arg}, &handle).IsInvalidArgument());
+  ASSERT_TRUE(handle == nullptr);
+}
+
+// Ingestion that reuses the SstFileWriter's metadata via
+// IngestExternalFileArg::file_infos skips the open-and-scan path and still
+// produces correct data, including overlapping data that is reassigned a new
+// global sequence number.
+TEST_F(ExternalSSTFileTest, IngestWithFileInfo) {
+  Options options = CurrentOptions();
+  DestroyAndReopen(options);
+  std::map<std::string, std::string> true_data;
+
+  // Count entries into the open-and-scan path so we can assert it is skipped.
+  std::atomic<int> read_path_count{0};
+  SyncPoint::GetInstance()->SetCallBack(
+      "ExternalSstFileIngestionJob::GetIngestedFileInfo:ReadPath",
+      [&](void*) { read_path_count.fetch_add(1); });
+  SyncPoint::GetInstance()->EnableProcessing();
+
+  // Reuse the writer's metadata, with the normal ingestion options: a global
+  // sequence number is assigned (allow_db_generated_files is NOT set) and the
+  // checksum is verified (the fast-path opens the file only to verify it, which
+  // is not a metadata scan).
+  auto ingest_fi = [&](std::vector<std::pair<std::string, std::string>> data) {
+    return GenerateAndAddExternalFile(
+        options, std::move(data), /*file_id=*/-1, /*allow_global_seqno=*/true,
+        /*write_global_seqno=*/false, /*verify_checksums_before_ingest=*/true,
+        /*ingest_behind=*/false, /*sort_data=*/true, &true_data,
+        /*cfh=*/nullptr, /*fill_cache=*/false, /*ingest_with_file_info=*/true);
+  };
+
+  // Non-overlapping ingest: data reads back and the file is not scanned.
+  ASSERT_OK(ingest_fi({{Key(1), "a1"}, {Key(2), "a2"}, {Key(3), "a3"}}));
+  ASSERT_EQ(read_path_count.load(), 0);
+
+  // Overlap existing (flushed) data so a non-zero global seqno is assigned and
+  // the newer values win -- the normal seqno-reassignment path through
+  // file_infos.
+  ASSERT_OK(db_->Put(WriteOptions(), Key(2), "old2"));
+  true_data[Key(2)] = "old2";
+  ASSERT_OK(Flush());
+  ASSERT_OK(ingest_fi({{Key(2), "new2"}, {Key(4), "new4"}}));
+  ASSERT_EQ(read_path_count.load(), 0);
+
+  // Control: ingesting the same way but WITHOUT file_infos does enter the scan.
+  ASSERT_OK(GenerateAndAddExternalFile(
+      options, {{Key(8), "c8"}}, /*file_id=*/-1, /*allow_global_seqno=*/true,
+      /*write_global_seqno=*/false, /*verify_checksums_before_ingest=*/true,
+      /*ingest_behind=*/false, /*sort_data=*/true, &true_data));
+  ASSERT_GT(read_path_count.load(), 0);
+
+  SyncPoint::GetInstance()->DisableProcessing();
+  SyncPoint::GetInstance()->ClearAllCallBacks();
+  size_t kcnt = 0;
+  VerifyDBFromMap(true_data, &kcnt, false);
+}
+
+// Range deletions are honored on the file_infos fast-path: the writer's
+// range-del bounds are carried in the metadata, so a DeleteRange covering
+// existing keys still shadows them.
+TEST_F(ExternalSSTFileTest, IngestWithFileInfoRangeDeletion) {
+  Options options = CurrentOptions();
+  DestroyAndReopen(options);
+  ASSERT_OK(db_->Put(WriteOptions(), Key(20), "live20"));
+  ASSERT_OK(db_->Put(WriteOptions(), Key(25), "live25"));
+  ASSERT_OK(Flush());
+
+  const std::string f = sst_files_dir_ + "rangedel.sst";
+  SstFileWriter w(EnvOptions(), options);
+  ASSERT_OK(w.Open(f));
+  ASSERT_OK(w.Put(Key(30), "v30"));
+  ASSERT_OK(w.DeleteRange(Key(20), Key(26)));  // covers keys 20..25
+  ExternalSstFileInfo file_info;
+  ASSERT_OK(w.Finish(&file_info));
+
+  IngestExternalFileArg arg;
+  arg.column_family = db_->DefaultColumnFamily();
+  arg.external_files = {f};
+  arg.file_infos = {file_info.prepared_file_info.get()};
+  arg.options.allow_global_seqno = true;
+  ASSERT_OK(db_->IngestExternalFiles({arg}));
+
+  ReadOptions ro;
+  std::string val;
+  ASSERT_TRUE(db_->Get(ro, Key(20), &val).IsNotFound());
+  ASSERT_TRUE(db_->Get(ro, Key(25), &val).IsNotFound());
+  ASSERT_OK(db_->Get(ro, Key(30), &val));
+  ASSERT_EQ(val, "v30");
+}
+
+// verify_checksums_before_ingest is honored on the file_infos fast-path: it
+// opens the file just to verify, so corruption is still detected.
+TEST_F(ExternalSSTFileTest, IngestWithFileInfoVerifiesChecksum) {
+  Options options = CurrentOptions();
+  DestroyAndReopen(options);
+
+  const std::string f = sst_files_dir_ + "corrupt.sst";
+  SstFileWriter w(EnvOptions(), options);
+  ASSERT_OK(w.Open(f));
+  for (int k = 0; k < 100; k++) {
+    ASSERT_OK(w.Put(Key(k), "v" + Key(k)));
+  }
+  ExternalSstFileInfo file_info;
+  ASSERT_OK(w.Finish(&file_info));
+  ASSERT_OK(test::CorruptFile(options.env, f, /*offset=*/64,
+                              /*bytes_to_corrupt=*/8));
+
+  IngestExternalFileArg arg;
+  arg.column_family = db_->DefaultColumnFamily();
+  arg.external_files = {f};
+  arg.file_infos = {file_info.prepared_file_info.get()};
+  arg.options.verify_checksums_before_ingest = true;
+  ASSERT_TRUE(db_->IngestExternalFiles({arg}).IsCorruption());
+}
+
+// write_global_seqno is incompatible with file_infos: the file is not opened,
+// so the seqno cannot be written back into it.
+TEST_F(ExternalSSTFileTest, IngestWithFileInfoRejectsWriteGlobalSeqno) {
+  Options options = CurrentOptions();
+  DestroyAndReopen(options);
+
+  const std::string f = sst_files_dir_ + "wgs.sst";
+  SstFileWriter w(EnvOptions(), options);
+  ASSERT_OK(w.Open(f));
+  ASSERT_OK(w.Put(Key(1), "v1"));
+  ExternalSstFileInfo file_info;
+  ASSERT_OK(w.Finish(&file_info));
+
+  IngestExternalFileArg arg;
+  arg.column_family = db_->DefaultColumnFamily();
+  arg.external_files = {f};
+  arg.file_infos = {file_info.prepared_file_info.get()};
+  arg.options.write_global_seqno = true;
+  ASSERT_TRUE(db_->IngestExternalFiles({arg}).IsInvalidArgument());
+}
+
+TEST_F(ExternalSSTFileTest, GetPreparedFileInfoForExternalSstIngestion) {
+  Options options = CurrentOptions();
+  DestroyAndReopen(options);
+
+  const std::string source_dbname = dbname_ + "_source";
+  ASSERT_OK(DestroyDB(source_dbname, options));
+  std::unique_ptr<DB> source_db;
+  ASSERT_OK(DB::Open(options, source_dbname, &source_db));
+
+  ASSERT_OK(source_db->Put(WriteOptions(), Key(10), "v10"));
+  ASSERT_OK(source_db->Flush(FlushOptions()));
+
+  std::vector<LiveFileMetaData> live_meta;
+  source_db->GetLiveFilesMetaData(&live_meta);
+  ASSERT_EQ(live_meta.size(), 1);
+  ASSERT_GT(live_meta[0].largest_seqno, 0);
+  const std::string file_path =
+      live_meta[0].directory + "/" + live_meta[0].relative_filename;
+
+  std::shared_ptr<const PreparedFileInfo> prepared_file_info;
+  ASSERT_TRUE(source_db
+                  ->GetPreparedFileInfoForExternalSstIngestion(
+                      live_meta[0].relative_filename, &prepared_file_info)
+                  .IsInvalidArgument());
+  ASSERT_EQ(prepared_file_info, nullptr);
+  prepared_file_info = std::make_shared<PreparedFileInfo>();
+  ASSERT_TRUE(source_db
+                  ->GetPreparedFileInfoForExternalSstIngestion(
+                      dbname_ + "_wrong/" + live_meta[0].relative_filename,
+                      &prepared_file_info)
+                  .IsInvalidArgument());
+  ASSERT_EQ(prepared_file_info, nullptr);
+
+  ASSERT_OK(source_db->GetPreparedFileInfoForExternalSstIngestion(
+      file_path, &prepared_file_info));
+  ASSERT_TRUE(prepared_file_info != nullptr);
+  ASSERT_EQ(prepared_file_info->file_size, live_meta[0].size);
+  ASSERT_EQ(prepared_file_info->table_properties.key_largest_seqno,
+            live_meta[0].largest_seqno);
+
+  IngestExternalFileArg arg;
+  arg.column_family = db_->DefaultColumnFamily();
+  arg.external_files = {file_path};
+  arg.file_infos = {prepared_file_info.get()};
+  arg.options.allow_db_generated_files = true;
+  arg.options.snapshot_consistency = false;
+
+  SyncPoint::GetInstance()->SetCallBack(
+      "ExternalSstFileIngestionJob::GetIngestedFileInfo:ReadPath",
+      [](void*) { FAIL() << "Prepared file info should skip the read path"; });
+  SyncPoint::GetInstance()->EnableProcessing();
+
+  ASSERT_OK(db_->IngestExternalFiles({arg}));
+
+  ASSERT_EQ(Get(Key(10)), "v10");
+
+  ASSERT_OK(source_db->Close());
+  source_db.reset();
+  ASSERT_OK(DestroyDB(source_dbname, options));
+
+  SyncPoint::GetInstance()->DisableProcessing();
+  SyncPoint::GetInstance()->ClearAllCallBacks();
+}
+
+TEST_F(ExternalSSTFileTest,
+       IngestWithFileInfoRejectsMissingDbGeneratedSeqnoBounds) {
+  Options options = CurrentOptions();
+  DestroyAndReopen(options);
+
+  const std::string f = sst_files_dir_ + "missing_seqno_bounds.sst";
+  SstFileWriter w(EnvOptions(), options);
+  ASSERT_OK(w.Open(f));
+  ASSERT_OK(w.Put(Key(1), "v1"));
+  ExternalSstFileInfo file_info;
+  ASSERT_OK(w.Finish(&file_info));
+  auto prepared_file_info = file_info.prepared_file_info;
+
+  auto missing_seqno_bounds =
+      std::make_shared<PreparedFileInfo>(*prepared_file_info);
+  missing_seqno_bounds->table_properties.key_smallest_seqno = UINT64_MAX;
+  missing_seqno_bounds->table_properties.key_largest_seqno = UINT64_MAX;
+
+  IngestExternalFileArg arg;
+  arg.column_family = db_->DefaultColumnFamily();
+  arg.external_files = {f};
+  arg.file_infos = {missing_seqno_bounds.get()};
+  arg.options.allow_db_generated_files = true;
+  arg.options.snapshot_consistency = false;
+  Status s = db_->IngestExternalFiles({arg});
+  ASSERT_TRUE(s.IsCorruption());
+  ASSERT_NE(s.ToString().find("Unknown largest seqno"), std::string::npos);
+}
+
+TEST_F(ExternalSSTFileTest, IngestWithFileInfoPreparedKeyBoundTypes) {
+  Options options = CurrentOptions();
+  DestroyAndReopen(options);
+
+  const std::string f = sst_files_dir_ + "prepared_key_bound_types.sst";
+  SstFileWriter w(EnvOptions(), options);
+  ASSERT_OK(w.Open(f));
+  ASSERT_OK(w.Put(Key(10), "v10"));
+  ASSERT_OK(w.DeleteRange(Key(20), Key(30)));
+  ExternalSstFileInfo file_info;
+  ASSERT_OK(w.Finish(&file_info));
+  auto prepared_file_info = file_info.prepared_file_info;
+
+  ParsedInternalKey smallest;
+  ASSERT_OK(ParseInternalKey(prepared_file_info->smallest.Encode(), &smallest,
+                             false /* log_err_key */));
+  ASSERT_EQ(0, smallest.sequence);
+  ASSERT_EQ(kTypeValue, smallest.type);
+  ASSERT_EQ(Key(10), smallest.user_key.ToString());
+
+  ParsedInternalKey largest;
+  ASSERT_OK(ParseInternalKey(prepared_file_info->largest.Encode(), &largest,
+                             false /* log_err_key */));
+  ASSERT_EQ(kMaxSequenceNumber, largest.sequence);
+  ASSERT_EQ(kTypeRangeDeletion, largest.type);
+  ASSERT_EQ(Key(30), largest.user_key.ToString());
+}
+
 TEST_F(ExternalSSTFileTest, ComparatorMismatch) {
   Options options = CurrentOptions();
   Options options_diff_ucmp = options;
@@ -1816,7 +2383,8 @@ TEST_P(ExternalSSTFileTest, IngestFileWithGlobalSeqnoRandomized) {
     for (int i = 0; i < 500; i++) {
       std::vector<std::pair<std::string, std::string>> random_data;
       for (int j = 0; j < 100; j++) {
-        std::string k = rnd.RandomString(rnd.Next() % 20);
+        std::string k =
+            rnd.RandomString(rnd.Next() % 20 + 1);  // requires non-empty keys
         std::string v = rnd.RandomString(rnd.Next() % 50);
         random_data.emplace_back(k, v);
       }
@@ -3247,6 +3815,20 @@ class ExternalSSTFileWithTimestampTest : public ExternalSSTFileTest {
     return dst;
   }
 
+  std::string MaxTimestamp() {
+    Slice ts = MaxU64Ts();
+    return ts.ToString();
+  }
+
+  Options UDTMemtableOnlyOptions() {
+    Options options = CurrentOptions();
+    options.comparator = test::BytewiseComparatorWithU64TsWrapper();
+    options.persist_user_defined_timestamps = false;
+    options.allow_concurrent_memtable_write = false;
+    options.disable_auto_compactions = true;
+    return options;
+  }
+
   Status IngestExternalUDTFile(const std::vector<std::string>& files,
                                bool allow_global_seqno = true) {
     IngestExternalFileOptions opts;
@@ -3815,8 +4397,74 @@ TEST_F(ExternalSSTFileWithTimestampTest, TimestampsNotPersistedBasic) {
                          kRangeDelSkipConfigs));
 }
 
+TEST_F(ExternalSSTFileWithTimestampTest,
+       TimestampsPersistedIngestWithFileInfoRangeDeletionBounds) {
+  Options options = CurrentOptions();
+  options.comparator = test::BytewiseComparatorWithU64TsWrapper();
+  options.persist_user_defined_timestamps = true;
+  options.disable_auto_compactions = true;
+
+  DestroyAndReopen(options);
+
+  const std::string ts = EncodeAsUint64(10);
+  const std::string f = sst_files_dir_ + "udt_persisted_rangedel_bounds.sst";
+  SstFileWriter w(EnvOptions(), options);
+  ASSERT_OK(w.Open(f));
+  ASSERT_OK(w.Put(Key(30), ts, "v30"));
+  ASSERT_OK(w.DeleteRange(Key(20), Key(26), ts));
+  ExternalSstFileInfo file_info;
+  ASSERT_OK(w.Finish(&file_info));
+  auto prepared_file_info = file_info.prepared_file_info;
+  ASSERT_EQ(Key(20) + MaxTimestamp(),
+            prepared_file_info->smallest.user_key().ToString());
+  ASSERT_EQ(Key(30) + ts, prepared_file_info->largest.user_key().ToString());
+
+  IngestExternalFileArg arg;
+  arg.column_family = db_->DefaultColumnFamily();
+  arg.external_files = {f};
+  arg.file_infos = {file_info.prepared_file_info.get()};
+  ASSERT_OK(db_->IngestExternalFiles({arg}));
+
+  std::vector<LiveFileMetaData> metadata;
+  db_->GetLiveFilesMetaData(&metadata);
+  ASSERT_EQ(1, metadata.size());
+  ASSERT_EQ(Key(20) + MaxTimestamp(), metadata[0].smallestkey);
+  ASSERT_EQ(Key(30) + ts, metadata[0].largestkey);
+}
+
+TEST_F(ExternalSSTFileWithTimestampTest,
+       TimestampsNotPersistedIngestWithFileInfoRangeDeletionBounds) {
+  Options options = UDTMemtableOnlyOptions();
+
+  DestroyAndReopen(options);
+
+  const std::string f = sst_files_dir_ + "udt_rangedel_bounds.sst";
+  SstFileWriter w(EnvOptions(), options);
+  ASSERT_OK(w.Open(f));
+  ASSERT_OK(w.Put(Key(30), EncodeAsUint64(0), "v30"));
+  ASSERT_OK(w.DeleteRange(Key(20), Key(26), EncodeAsUint64(0)));
+  ExternalSstFileInfo file_info;
+  ASSERT_OK(w.Finish(&file_info));
+  auto prepared_file_info = file_info.prepared_file_info;
+  ASSERT_EQ(Key(20), prepared_file_info->smallest.user_key().ToString());
+  ASSERT_EQ(Key(30), prepared_file_info->largest.user_key().ToString());
+
+  IngestExternalFileArg arg;
+  arg.column_family = db_->DefaultColumnFamily();
+  arg.external_files = {f};
+  arg.file_infos = {file_info.prepared_file_info.get()};
+  ASSERT_OK(db_->IngestExternalFiles({arg}));
+
+  std::vector<LiveFileMetaData> metadata;
+  db_->GetLiveFilesMetaData(&metadata);
+  ASSERT_EQ(1, metadata.size());
+  ASSERT_EQ(Key(20) + MaxTimestamp(), metadata[0].smallestkey);
+  ASSERT_EQ(Key(30) + EncodeAsUint64(0), metadata[0].largestkey);
+}
+
 INSTANTIATE_TEST_CASE_P(ExternalSSTFileTest, ExternalSSTFileTest,
-                        testing::Combine(testing::Bool(), testing::Bool()));
+                        testing::Combine(testing::Bool(), testing::Bool(),
+                                         testing::Bool()));
 
 class IngestDBGeneratedFileTest
     : public ExternalSSTFileTestBase,

db/fault_injection_test.cc

@@ -646,6 +646,78 @@ TEST(FaultInjectionFSTest, ReadUnsyncedData) {
     ASSERT_EQ(0, sl.compare(Slice(data.data() + first_read_len, sl.size())));
   }
 }
+
+TEST(FaultInjectionFSTest, FileOpenContractRejectsReadersWhileOpenForWrite) {
+  std::shared_ptr<FaultInjectionTestFS> fault_fs =
+      std::make_shared<FaultInjectionTestFS>(FileSystem::Default());
+  const std::string fname = test::PerThreadDBPath(
+      "file_open_contract_no_readers_while_open_for_write");
+
+  FileOptions writer_options;
+  writer_options.open_contract = FileOpenContract::kNoReadersWhileOpenForWrite;
+
+  std::unique_ptr<FSWritableFile> writer;
+  ASSERT_OK(fault_fs->NewWritableFile(fname, writer_options, &writer, nullptr));
+
+  std::unique_ptr<FSRandomAccessFile> reader;
+  IOStatus s =
+      fault_fs->NewRandomAccessFile(fname, FileOptions(), &reader, nullptr);
+  ASSERT_TRUE(s.IsNotSupported());
+
+  ASSERT_OK(writer->Close(IOOptions(), nullptr));
+  ASSERT_OK(
+      fault_fs->NewRandomAccessFile(fname, FileOptions(), &reader, nullptr));
+}
+
+TEST(FaultInjectionFSTest,
+     FileOpenContractAllowsSyncFileButRejectsReopenedWrite) {
+  std::shared_ptr<FaultInjectionTestFS> fault_fs =
+      std::make_shared<FaultInjectionTestFS>(FileSystem::Default());
+  const std::string fname =
+      test::PerThreadDBPath("file_open_contract_no_reopen_for_write");
+
+  FileOptions writer_options;
+  writer_options.open_contract = FileOpenContract::kNoReopenForWrite;
+
+  std::unique_ptr<FSWritableFile> writer;
+  ASSERT_OK(fault_fs->NewWritableFile(fname, writer_options, &writer, nullptr));
+  ASSERT_OK(writer->Append("abc", IOOptions(), nullptr));
+  ASSERT_OK(writer->Close(IOOptions(), nullptr));
+
+  ASSERT_OK(fault_fs->SyncFile(fname, FileOptions(), IOOptions(),
+                               /*use_fsync=*/false, nullptr));
+
+  std::unique_ptr<FSWritableFile> reopened_writer;
+  ASSERT_OK(fault_fs->ReopenWritableFile(fname, FileOptions(), &reopened_writer,
+                                         nullptr));
+  IOStatus s = reopened_writer->Append("d", IOOptions(), nullptr);
+  ASSERT_TRUE(s.IsNotSupported());
+  ASSERT_OK(reopened_writer->Close(IOOptions(), nullptr));
+}
+
+TEST(FaultInjectionFSTest, FileOpenContractAllowsCreateAfterDelete) {
+  std::shared_ptr<FaultInjectionTestFS> fault_fs =
+      std::make_shared<FaultInjectionTestFS>(FileSystem::Default());
+  const std::string fname =
+      test::PerThreadDBPath("file_open_contract_recreate_after_delete");
+
+  FileOptions writer_options;
+  writer_options.open_contract = FileOpenContract::kNoReopenForWrite;
+
+  std::unique_ptr<FSWritableFile> writer;
+  ASSERT_OK(fault_fs->NewWritableFile(fname, writer_options, &writer, nullptr));
+  ASSERT_OK(writer->Append("abc", IOOptions(), nullptr));
+  ASSERT_OK(writer->Close(IOOptions(), nullptr));
+
+  ASSERT_OK(FileSystem::Default()->DeleteFile(fname, IOOptions(), nullptr));
+
+  std::unique_ptr<FSWritableFile> recreated_writer;
+  ASSERT_OK(fault_fs->NewWritableFile(fname, writer_options, &recreated_writer,
+                                      nullptr));
+  ASSERT_OK(recreated_writer->Append("def", IOOptions(), nullptr));
+  ASSERT_OK(recreated_writer->Close(IOOptions(), nullptr));
+  ASSERT_OK(fault_fs->DeleteFile(fname, IOOptions(), nullptr));
+}
 }  // namespace ROCKSDB_NAMESPACE
 
 int main(int argc, char** argv) {

db/memtable.cc

@@ -159,8 +159,8 @@ ImmutableMemTableOptions::ImmutableMemTableOptions(
           mutable_cf_options.memtable_protection_bytes_per_key),
       allow_data_in_errors(ioptions.allow_data_in_errors),
       paranoid_memory_checks(mutable_cf_options.paranoid_memory_checks),
-      memtable_veirfy_per_key_checksum_on_seek(
-          mutable_cf_options.memtable_veirfy_per_key_checksum_on_seek),
+      memtable_verify_per_key_checksum_on_seek(
+          mutable_cf_options.memtable_verify_per_key_checksum_on_seek),
       memtable_batch_lookup_optimization(
           ioptions.memtable_batch_lookup_optimization) {}
 
@@ -256,7 +256,7 @@ MemTable::MemTable(const InternalKeyComparator& cmp,
           mutable_cf_options.memtable_max_range_deletions),
       key_validation_callback_(
           (moptions_.protection_bytes_per_key != 0 &&
-           moptions_.memtable_veirfy_per_key_checksum_on_seek)
+           moptions_.memtable_verify_per_key_checksum_on_seek)
               ? std::bind(&MemTable::ValidateKey, this, std::placeholders::_1,
                           std::placeholders::_2)
               : std::function<Status(const char*, bool)>(nullptr)) {
@@ -528,7 +528,7 @@ class MemTableIterator : public InternalIterator {
         paranoid_memory_checks_(mem.moptions_.paranoid_memory_checks),
         validate_on_seek_(
             mem.moptions_.paranoid_memory_checks ||
-            mem.moptions_.memtable_veirfy_per_key_checksum_on_seek),
+            mem.moptions_.memtable_verify_per_key_checksum_on_seek),
         allow_data_in_error_(mem.moptions_.allow_data_in_errors),
         key_validation_callback_(mem.key_validation_callback_) {
     if (kind == kRangeDelEntries) {
@@ -1682,7 +1682,7 @@ void MemTable::GetFromTable(const LookupKey& key,
   saver.protection_bytes_per_key = moptions_.protection_bytes_per_key;
 
   if (!moptions_.paranoid_memory_checks &&
-      !moptions_.memtable_veirfy_per_key_checksum_on_seek) {
+      !moptions_.memtable_verify_per_key_checksum_on_seek) {
     table_->Get(key, &saver, SaveValue);
   } else {
     Status check_s = table_->GetAndValidate(
@@ -1750,7 +1750,7 @@ void MemTable::MultiGet(const ReadOptions& read_options, MultiGetRange* range,
   // Use batch lookup optimization when enabled
   bool use_batch_optimization = moptions_.memtable_batch_lookup_optimization;
   bool validate = moptions_.paranoid_memory_checks ||
-                  moptions_.memtable_veirfy_per_key_checksum_on_seek;
+                  moptions_.memtable_verify_per_key_checksum_on_seek;
 
   if (use_batch_optimization) {
     // Phase 1: Handle range tombstones and set up Savers for batched lookup

db/memtable.h

@@ -68,7 +68,7 @@ struct ImmutableMemTableOptions {
   uint32_t protection_bytes_per_key;
   bool allow_data_in_errors;
   bool paranoid_memory_checks;
-  bool memtable_veirfy_per_key_checksum_on_seek;
+  bool memtable_verify_per_key_checksum_on_seek;
   bool memtable_batch_lookup_optimization;
 };
 

db/memtable_list.cc

@@ -17,6 +17,7 @@
 #include "db/version_set.h"
 #include "logging/log_buffer.h"
 #include "logging/logging.h"
+#include "memory/arena.h"
 #include "monitoring/thread_status_util.h"
 #include "rocksdb/db.h"
 #include "rocksdb/env.h"
@@ -31,6 +32,78 @@ class InternalKeyComparator;
 class Mutex;
 class VersionSet;
 
+bool MultiScanOverlapsUserKeyRange(const MultiScanArgs* scan_opts,
+                                   const Comparator* user_comparator,
+                                   const Slice& smallest_user_key,
+                                   const Slice& largest_user_key) {
+  if (scan_opts == nullptr || !scan_opts->HasBoundedScanRanges()) {
+    return true;
+  }
+
+  for (const ScanOptions& scan_range : scan_opts->GetScanRanges()) {
+    if (user_comparator->CompareWithoutTimestamp(
+            scan_range.range.limit.value(), /*a_has_ts=*/false,
+            smallest_user_key, /*b_has_ts=*/true) <= 0) {
+      continue;
+    }
+    if (user_comparator->CompareWithoutTimestamp(
+            scan_range.range.start.value(), /*a_has_ts=*/false,
+            largest_user_key, /*b_has_ts=*/true) > 0) {
+      continue;
+    }
+    return true;
+  }
+  return false;
+}
+
+bool MultiScanIteratorOverlapsUserKeyRange(InternalIterator* iter,
+                                           const MultiScanArgs* scan_opts,
+                                           const Comparator* user_comparator) {
+  iter->SeekToFirst();
+  if (!iter->status().ok()) {
+    return true;
+  }
+  if (!iter->Valid()) {
+    return true;
+  }
+  const std::string smallest_user_key = ExtractUserKey(iter->key()).ToString();
+  iter->SeekToLast();
+  if (!iter->status().ok()) {
+    return true;
+  }
+  if (!iter->Valid()) {
+    return true;
+  }
+  const std::string largest_user_key = ExtractUserKey(iter->key()).ToString();
+  return MultiScanOverlapsUserKeyRange(scan_opts, user_comparator,
+                                       smallest_user_key, largest_user_key);
+}
+
+bool MultiScanIntersectsMemTable(
+    ReadOnlyMemTable* memtable, const ReadOptions& read_options,
+    UnownedPtr<const SeqnoToTimeMapping> seqno_to_time_mapping,
+    const SliceTransform* prefix_extractor, const MultiScanArgs* scan_opts,
+    const Comparator* user_comparator) {
+  if (scan_opts == nullptr || !scan_opts->HasBoundedScanRanges()) {
+    return true;
+  }
+  if (memtable->NumRangeDeletion() > 0) {
+    return true;
+  }
+
+  Arena arena;
+  ReadOptions intersect_read_options(read_options);
+  intersect_read_options.total_order_seek = true;
+  intersect_read_options.iterate_lower_bound = nullptr;
+  intersect_read_options.iterate_upper_bound = nullptr;
+  InternalIterator* iter =
+      memtable->NewIterator(intersect_read_options, seqno_to_time_mapping,
+                            &arena, prefix_extractor, /*for_flush=*/false);
+  ScopedArenaPtr<InternalIterator> iter_guard(iter);
+  return MultiScanIteratorOverlapsUserKeyRange(iter, scan_opts,
+                                               user_comparator);
+}
+
 void MemTableListVersion::AddMemTable(ReadOnlyMemTable* m) {
   if (!memlist_.empty()) {
     // ID can be equal for MemPurge
@@ -229,33 +302,42 @@ void MemTableListVersion::AddIterators(
     const ReadOptions& options,
     UnownedPtr<const SeqnoToTimeMapping> seqno_to_time_mapping,
     const SliceTransform* prefix_extractor,
-    MergeIteratorBuilder* merge_iter_builder, bool add_range_tombstone_iter) {
+    MergeIteratorBuilder* merge_iter_builder, bool add_range_tombstone_iter,
+    SequenceNumber read_seq, const MultiScanArgs* scan_opts,
+    const Comparator* user_comparator) {
   for (auto& m : memlist_) {
+    const bool should_probe_scan_intersection =
+        scan_opts != nullptr && scan_opts->HasBoundedScanRanges() &&
+        m->NumRangeDeletion() == 0;
+    if (should_probe_scan_intersection &&
+        !MultiScanIntersectsMemTable(m, options, seqno_to_time_mapping,
+                                     prefix_extractor, scan_opts,
+                                     user_comparator)) {
+      continue;
+    }
     auto mem_iter =
         m->NewIterator(options, seqno_to_time_mapping,
                        merge_iter_builder->GetArena(), prefix_extractor,
                        /*for_flush=*/false);
+    ScopedArenaPtr<InternalIterator> mem_iter_guard(mem_iter);
     if (!add_range_tombstone_iter || options.ignore_range_deletions) {
-      merge_iter_builder->AddIterator(mem_iter);
+      merge_iter_builder->AddIterator(mem_iter_guard.release());
     } else {
-      // Except for snapshot read, using kMaxSequenceNumber is OK because these
-      // are immutable memtables.
-      SequenceNumber read_seq = options.snapshot != nullptr
-                                    ? options.snapshot->GetSequenceNumber()
-                                    : kMaxSequenceNumber;
+      const SequenceNumber range_del_read_seq =
+          read_seq != kMaxSequenceNumber || options.snapshot == nullptr
+              ? read_seq
+              : options.snapshot->GetSequenceNumber();
       std::unique_ptr<TruncatedRangeDelIterator> mem_tombstone_iter;
-      auto range_del_iter = m->NewRangeTombstoneIterator(
-          options, read_seq, true /* immutale_memtable */);
-      if (range_del_iter == nullptr || range_del_iter->empty()) {
-        delete range_del_iter;
-      } else {
+      std::unique_ptr<FragmentedRangeTombstoneIterator> range_del_iter(
+          m->NewRangeTombstoneIterator(options, range_del_read_seq,
+                                       true /* immutale_memtable */));
+      if (range_del_iter != nullptr && !range_del_iter->empty()) {
         mem_tombstone_iter = std::make_unique<TruncatedRangeDelIterator>(
-            std::unique_ptr<FragmentedRangeTombstoneIterator>(range_del_iter),
-            &m->GetInternalKeyComparator(), nullptr /* smallest */,
-            nullptr /* largest */);
+            std::move(range_del_iter), &m->GetInternalKeyComparator(),
+            nullptr /* smallest */, nullptr /* largest */);
       }
       merge_iter_builder->AddPointAndTombstoneIterator(
-          mem_iter, std::move(mem_tombstone_iter));
+          mem_iter_guard.release(), std::move(mem_tombstone_iter));
     }
   }
 }

db/memtable_list.h

@@ -34,6 +34,21 @@ class MemTableList;
 
 struct FlushJobInfo;
 
+// Returns true when at least one bounded scan range overlaps the user-key
+// range. Unbounded scan options conservatively overlap.
+bool MultiScanOverlapsUserKeyRange(const MultiScanArgs* scan_opts,
+                                   const Comparator* user_comparator,
+                                   const Slice& smallest_user_key,
+                                   const Slice& largest_user_key);
+
+// Returns true when the memtable may contain keys or range deletions relevant
+// to the scan ranges. Iterator/status errors conservatively overlap.
+bool MultiScanIntersectsMemTable(
+    ReadOnlyMemTable* memtable, const ReadOptions& read_options,
+    UnownedPtr<const SeqnoToTimeMapping> seqno_to_time_mapping,
+    const SliceTransform* prefix_extractor, const MultiScanArgs* scan_opts,
+    const Comparator* user_comparator);
+
 // keeps a list of immutable memtables (ReadOnlyMemtable*) in a vector.
 // The list is immutable if refcount is bigger than one. It is used as
 // a state for Get() and iterator code paths.
@@ -122,11 +137,17 @@ class MemTableListVersion {
                     std::vector<InternalIterator*>* iterator_list,
                     Arena* arena);
 
+  // read_seq controls range tombstone visibility. It may be captured before
+  // lazy iterator initialization. scan_opts can prune immutable memtables that
+  // cannot intersect the requested scan ranges.
   void AddIterators(const ReadOptions& options,
                     UnownedPtr<const SeqnoToTimeMapping> seqno_to_time_mapping,
                     const SliceTransform* prefix_extractor,
                     MergeIteratorBuilder* merge_iter_builder,
-                    bool add_range_tombstone_iter);
+                    bool add_range_tombstone_iter,
+                    SequenceNumber read_seq = kMaxSequenceNumber,
+                    const MultiScanArgs* scan_opts = nullptr,
+                    const Comparator* user_comparator = nullptr);
 
   uint64_t GetTotalNumEntries() const;
 

db/range_tombstone_fragmenter.cc

@@ -497,6 +497,9 @@ FragmentedRangeTombstoneIterator::SplitBySnapshot(
       splits;
   SequenceNumber lower = 0;
   SequenceNumber upper;
+  // Safe to dereference icmp_ below: SplitBySnapshot is construction-time work
+  // (called synchronously from the range-del aggregator while the reader that
+  // owns *icmp_ is alive). See the icmp_ note in the header.
   for (size_t i = 0; i <= snapshots.size(); i++) {
     if (i >= snapshots.size()) {
       upper = kMaxSequenceNumber;

db/range_tombstone_fragmenter.h

@@ -199,7 +199,9 @@ class FragmentedRangeTombstoneIterator : public InternalIterator {
 
   RangeTombstone Tombstone() const {
     assert(Valid());
-    if (icmp_->user_comparator()->timestamp_size()) {
+    // Use ucmp_ (long-lived; see the icmp_ note below) rather than
+    // icmp_->user_comparator(). They are equal for all constructors.
+    if (ucmp_->timestamp_size()) {
       return RangeTombstone(start_key(), end_key(), seq(), timestamp());
     }
     return RangeTombstone(start_key(), end_key(), seq());
@@ -316,6 +318,12 @@ class FragmentedRangeTombstoneIterator : public InternalIterator {
 
   const RangeTombstoneStackStartComparator tombstone_start_cmp_;
   const RangeTombstoneStackEndComparator tombstone_end_cmp_;
+  // icmp_ is a borrowed pointer and is not guaranteed to outlive this iterator:
+  // it may be built from a comparator with a shorter lifetime than an iterator
+  // some owner keeps alive. Only dereference it during construction-time work
+  // (e.g. SplitBySnapshot), when the referent is known alive. Post-construction
+  // navigation must use ucmp_ (the long-lived user comparator) and tombstones_;
+  // do not add new icmp_ dereferences in Next()/Seek()/Tombstone().
   const InternalKeyComparator* icmp_;
   const Comparator* ucmp_;
   std::shared_ptr<FragmentedRangeTombstoneList> tombstones_ref_;

db/table_cache.cc

@@ -98,7 +98,7 @@ Status TableCache::GetTableReader(
     const MutableCFOptions& mutable_cf_options, bool skip_filters, int level,
     bool prefetch_index_and_filter_in_cache,
     size_t max_file_size_for_l0_meta_pin, Temperature file_temperature,
-    std::string* file_open_metadata) {
+    std::string* file_open_metadata, bool avoid_shared_metadata_cache) {
   std::string fname = TableFileName(
       ioptions_.cf_paths, file_meta.fd.GetNumber(), file_meta.fd.GetPathId());
   std::unique_ptr<FSRandomAccessFile> file;
@@ -190,7 +190,8 @@ Status TableCache::GetTableReader(
             block_cache_tracer_, max_file_size_for_l0_meta_pin, db_session_id_,
             file_meta.fd.GetNumber(), expected_unique_id,
             file_meta.fd.largest_seqno, file_meta.tail_size,
-            file_meta.user_defined_timestamps_persisted),
+            file_meta.user_defined_timestamps_persisted,
+            avoid_shared_metadata_cache),
         std::move(file_reader), file_meta.fd.GetFileSize(), table_reader,
         prefetch_index_and_filter_in_cache);
     TEST_SYNC_POINT("TableCache::GetTableReader:0");
@@ -213,11 +214,53 @@ Status TableCache::FindTable(
     const bool no_io, HistogramImpl* file_read_hist, bool skip_filters,
     int level, bool prefetch_index_and_filter_in_cache,
     size_t max_file_size_for_l0_meta_pin, Temperature file_temperature,
-    bool pin_table_handle, std::string* file_open_metadata) {
+    bool pin_table_handle, std::string* file_open_metadata,
+    std::unique_ptr<TableReader>* fresh_table_reader_owner,
+    const TableCacheOpenOptions& open_options) {
   assert(out_table_reader != nullptr && *out_table_reader == nullptr);
   assert(handle != nullptr && *handle == nullptr);
+  // open_ephemeral_table_reader requests a fresh reader;
+  // fresh_table_reader_owner is where we return it. The two must agree.
+  assert(open_options.open_ephemeral_table_reader ==
+         (fresh_table_reader_owner != nullptr));
   PERF_TIMER_GUARD_WITH_CLOCK(find_table_nanos, ioptions_.clock);
 
+  // Bypass path: open a fresh TableReader, skipping the pinned-reader fast path
+  // and the shared cache. The caller takes ownership via
+  // fresh_table_reader_owner. no_io is not allowed here since opening a new
+  // reader always needs I/O.
+  if (fresh_table_reader_owner != nullptr) {
+    assert(!no_io);
+    if (no_io) {
+      // Defensive in release builds: the caller violated the contract.
+      return Status::Incomplete(
+          "fresh TableReader requested but no_io is set; cannot open file");
+    }
+    std::unique_ptr<TableReader> table_reader;
+    const bool effective_skip_filters =
+        skip_filters || open_options.skip_filters;
+    TEST_SYNC_POINT_CALLBACK("TableCache::FindTable:FreshTableReader",
+                             const_cast<TableCacheOpenOptions*>(&open_options));
+    Status s = GetTableReader(
+        ro, file_options, internal_comparator, file_meta,
+        false /* sequential mode */, file_read_hist, &table_reader,
+        mutable_cf_options, effective_skip_filters, level,
+        prefetch_index_and_filter_in_cache &&
+            !open_options.avoid_shared_metadata_cache,
+        max_file_size_for_l0_meta_pin, file_temperature, file_open_metadata,
+        open_options.avoid_shared_metadata_cache);
+    if (!s.ok()) {
+      assert(table_reader == nullptr);
+      RecordTick(ioptions_.stats, NO_FILE_ERRORS);
+      IGNORE_STATUS_IF_ERROR(s);
+      return s;
+    }
+    *out_table_reader = table_reader.get();
+    *fresh_table_reader_owner = std::move(table_reader);
+    *handle = nullptr;
+    return s;
+  }
+
   // Fast path: if table reader is already pinned, return it directly without a
   // cache lookup.
   auto pinned_reader = file_meta.fd.pinned_reader.Get();
@@ -313,15 +356,23 @@ InternalIterator* TableCache::NewIterator(
     const InternalKey* largest_compaction_key, bool allow_unprepared_value,
     const SequenceNumber* read_seqno,
     std::unique_ptr<TruncatedRangeDelIterator>* range_del_iter,
-    bool maybe_pin_table_handle, std::string* file_open_metadata) {
+    bool maybe_pin_table_handle, std::string* file_open_metadata,
+    const TableCacheOpenOptions& open_options) {
   PERF_TIMER_GUARD(new_table_iterator_nanos);
 
   Status s;
   TableReader* table_reader = nullptr;
   TypedHandle* handle = nullptr;
+  assert(!open_options.open_ephemeral_table_reader ||
+         table_reader_ptr == nullptr);
+  // Holds ownership of a freshly-opened TableReader when the caller asked us
+  // to bypass the shared cache. When non-empty, the iterator we hand back must
+  // arrange to free it on destruction.
+  std::unique_ptr<TableReader> ephemeral_reader;
   if (table_reader_ptr != nullptr) {
     *table_reader_ptr = nullptr;
   }
+  const bool effective_skip_filters = skip_filters || open_options.skip_filters;
   bool for_compaction = caller == TableReaderCaller::kCompaction;
   TEST_SYNC_POINT_CALLBACK("TableCache::NewIterator::BeforeFindTable",
                            const_cast<FileDescriptor*>(&file_meta.fd));
@@ -329,9 +380,13 @@ InternalIterator* TableCache::NewIterator(
       options, file_options, icomparator, file_meta, &handle,
       mutable_cf_options, &table_reader,
       options.read_tier == kBlockCacheTier /* no_io */, file_read_hist,
-      skip_filters, level, true /* prefetch_index_and_filter_in_cache */,
-      max_file_size_for_l0_meta_pin, file_meta.temperature,
-      maybe_pin_table_handle && should_pin_table_handles_, file_open_metadata);
+      effective_skip_filters, level,
+      /*prefetch_index_and_filter_in_cache=*/
+      !open_options.avoid_shared_metadata_cache, max_file_size_for_l0_meta_pin,
+      file_meta.temperature,
+      maybe_pin_table_handle && should_pin_table_handles_, file_open_metadata,
+      open_options.open_ephemeral_table_reader ? &ephemeral_reader : nullptr,
+      open_options);
   InternalIterator* result = nullptr;
   if (s.ok()) {
     if (options.table_filter &&
@@ -340,13 +395,17 @@ InternalIterator* TableCache::NewIterator(
     } else {
       result = table_reader->NewIterator(
           options, mutable_cf_options.prefix_extractor.get(), arena,
-          skip_filters, caller, file_options.compaction_readahead_size,
-          allow_unprepared_value);
+          effective_skip_filters, caller,
+          file_options.compaction_readahead_size, allow_unprepared_value);
     }
     if (handle != nullptr) {
       cache_.RegisterReleaseAsCleanup(handle, *result);
       handle = nullptr;  // prevent from releasing below
     }
+    // Don't hand ephemeral_reader to result's cleanup yet: range-del
+    // processing below can set s to non-OK, in which case result is replaced
+    // by an error iterator at function end. Transfer ownership only once we
+    // know s stays OK; otherwise ephemeral_reader's destructor frees it.
 
     if (for_compaction) {
       table_reader->SetupForCompaction();
@@ -398,8 +457,37 @@ InternalIterator* TableCache::NewIterator(
   if (handle != nullptr) {
     cache_.Release(handle);
   }
+  // Range-del processing is done and s is final. Hand the ephemeral reader's
+  // lifetime to the returned iterator; if s is non-OK, leave it for
+  // ephemeral_reader's destructor. RegisterCleanup gets the raw pointer before
+  // release(), so if its allocation throws the reader is still owned by
+  // ephemeral_reader and freed on unwind.
+  if (s.ok() && ephemeral_reader && result != nullptr) {
+    TableReader* raw = ephemeral_reader.get();
+    result->RegisterCleanup(
+        [](void* arg1, void* /*arg2*/) {
+          delete static_cast<TableReader*>(arg1);
+        },
+        raw, nullptr);
+    // release() returns raw, which the cleanup now owns; drop the unique_ptr's
+    // ownership so the reader isn't freed twice.
+    [[maybe_unused]] TableReader* released = ephemeral_reader.release();
+    assert(released == raw);
+  }
   if (!s.ok()) {
+    // Today result is always null here: it is only set when s was OK, and the
+    // only later change to s, new_range_del_iter->status(), is always OK for a
+    // FragmentedRangeTombstoneIterator. The assert documents that; the cleanup
+    // still disposes of any stray result before the error iterator replaces it.
     assert(result == nullptr);
+    if (result != nullptr) {
+      if (arena != nullptr) {
+        result->~InternalIterator();
+      } else {
+        delete result;
+      }
+      result = nullptr;
+    }
     result = NewErrorInternalIterator<Slice>(s, arena);
   }
   return result;

db/table_cache.h

@@ -35,6 +35,18 @@ struct FileDescriptor;
 class GetContext;
 class HistogramImpl;
 
+struct TableCacheOpenOptions {
+  // Open a new TableReader owned by the returned iterator instead of reusing
+  // the pinned reader or shared TableCache entry.
+  bool open_ephemeral_table_reader = false;
+
+  // Avoid inserting open-time metadata blocks into the shared block cache.
+  bool avoid_shared_metadata_cache = false;
+
+  // Disable loading/accessing filter blocks for this open/iterator.
+  bool skip_filters = false;
+};
+
 // Manages caching for TableReader objects for a column family. The actual
 // cache is allocated separately and passed to the constructor. TableCache
 // wraps around the underlying SST file readers by providing Get(),
@@ -73,7 +85,9 @@ class TableCache {
   // underlying the returned iterator, or nullptr if no Table object underlies
   // the returned iterator.  The returned "*table_reader_ptr" object is owned
   // by the cache and should not be deleted, and is valid for as long as the
-  // returned iterator is live.
+  // returned iterator is live. `table_reader_ptr` must be nullptr when
+  // `open_options.open_ephemeral_table_reader` is set because that reader is
+  // owned by iterator cleanup rather than by the cache.
   // If !options.ignore_range_deletions, and range_del_iter is non-nullptr,
   // then range_del_iter is set to a TruncatedRangeDelIterator for range
   // tombstones in the SST file corresponding to the specified file number. The
@@ -82,12 +96,16 @@ class TableCache {
   // @param options Must outlive the returned iterator.
   // @param range_del_agg If non-nullptr, adds range deletions to the
   //    aggregator. If an error occurs, returns it in a NewErrorInternalIterator
-  // @param for_compaction If true, a new TableReader may be allocated (but
-  //                       not cached), depending on the CF options
+  // @param caller Identifies the high-level caller for table-reader behavior
+  //               such as compaction preparation.
   // @param skip_filters Disables loading/accessing the filter block
   // @param level The level this table is at, -1 for "not set / don't know"
   // @param range_del_read_seqno If non-nullptr, will be used to create
   // *range_del_iter.
+  // @param open_options Optional table-open policy overrides. The caller must
+  //                       not combine `open_ephemeral_table_reader` with
+  //                       `ReadOptions::read_tier = kBlockCacheTier` (no_io);
+  //                       opening an ephemeral reader inherently requires I/O.
   InternalIterator* NewIterator(
       const ReadOptions& options, const FileOptions& toptions,
       const InternalKeyComparator& internal_comparator,
@@ -103,7 +121,8 @@ class TableCache {
       bool maybe_pin_table_handle = false,
       // If non-null, and the table reader is newly opened (not cached),
       // retrieves file open metadata via GetFileOpenMetadata().
-      std::string* file_open_metadata = nullptr);
+      std::string* file_open_metadata = nullptr,
+      const TableCacheOpenOptions& open_options = TableCacheOpenOptions());
 
   // If a seek to internal key "k" in specified file finds an entry,
   // call get_context->SaveValue() repeatedly until
@@ -190,18 +209,30 @@ class TableCache {
   // @param pin_table_handle If true, pins the table reader on file_meta so
   //              future lookups bypass the cache. *handle is set to nullptr
   //              on return in this case.
-  Status FindTable(const ReadOptions& ro, const FileOptions& toptions,
-                   const InternalKeyComparator& internal_comparator,
-                   const FileMetaData& file_meta, TypedHandle**,
-                   const MutableCFOptions& mutable_cf_options,
-                   TableReader** table_reader, const bool no_io = false,
-                   HistogramImpl* file_read_hist = nullptr,
-                   bool skip_filters = false, int level = -1,
-                   bool prefetch_index_and_filter_in_cache = true,
-                   size_t max_file_size_for_l0_meta_pin = 0,
-                   Temperature file_temperature = Temperature::kUnknown,
-                   bool pin_table_handle = false,
-                   std::string* file_open_metadata = nullptr);
+  // @param fresh_table_reader_owner If non-null, FindTable always opens a new
+  //              TableReader (skipping the pinned-reader fast path and the
+  //              shared cache) and writes ownership into this unique_ptr.
+  //              `*handle` will be nullptr on return and `*table_reader` will
+  //              point to the freshly allocated reader. Callers that pass this
+  //              are responsible for keeping the unique_ptr alive for the
+  //              lifetime of any iterators built on top.
+  // @param open_options Optional table-open policy overrides (e.g.
+  //              avoid_shared_metadata_cache, skip_filters).
+  //              `open_options.open_ephemeral_table_reader` must be set if and
+  //              only if `fresh_table_reader_owner` is non-null.
+  Status FindTable(
+      const ReadOptions& ro, const FileOptions& toptions,
+      const InternalKeyComparator& internal_comparator,
+      const FileMetaData& file_meta, TypedHandle**,
+      const MutableCFOptions& mutable_cf_options, TableReader** table_reader,
+      const bool no_io = false, HistogramImpl* file_read_hist = nullptr,
+      bool skip_filters = false, int level = -1,
+      bool prefetch_index_and_filter_in_cache = true,
+      size_t max_file_size_for_l0_meta_pin = 0,
+      Temperature file_temperature = Temperature::kUnknown,
+      bool pin_table_handle = false, std::string* file_open_metadata = nullptr,
+      std::unique_ptr<TableReader>* fresh_table_reader_owner = nullptr,
+      const TableCacheOpenOptions& open_options = TableCacheOpenOptions());
 
   // Get the table properties of a given table.
   // @no_io: indicates if we should load table to the cache if it is not present
@@ -286,7 +317,8 @@ class TableCache {
                         bool prefetch_index_and_filter_in_cache = true,
                         size_t max_file_size_for_l0_meta_pin = 0,
                         Temperature file_temperature = Temperature::kUnknown,
-                        std::string* file_open_metadata = nullptr);
+                        std::string* file_open_metadata = nullptr,
+                        bool avoid_shared_metadata_cache = false);
 
   // Update the max_covering_tombstone_seq in the GetContext for each key based
   // on the range deletions in the table

db/version_edit.h

@@ -978,6 +978,11 @@ class VersionEdit {
 
   bool IsColumnFamilyDrop() const { return is_column_family_drop_; }
 
+  void MarkForegroundOperation() { is_foreground_operation_ = true; }
+  bool IsForegroundOperation() const {
+    return is_foreground_operation_ || IsColumnFamilyManipulation();
+  }
+
   void MarkNoManifestWriteDummy() { is_no_manifest_write_dummy_ = true; }
   bool IsNoManifestWriteDummy() const { return is_no_manifest_write_dummy_; }
 
@@ -1111,6 +1116,7 @@ class VersionEdit {
   // it also includes column family name.
   bool is_column_family_drop_ = false;
   bool is_column_family_add_ = false;
+  bool is_foreground_operation_ = false;
   std::string column_family_name_;
 
   uint32_t remaining_entries_ = 0;

db/version_set.cc

@@ -18,7 +18,9 @@
 #include <set>
 #include <stdexcept>
 #include <string>
+#include <type_traits>
 #include <unordered_map>
+#include <utility>
 #include <vector>
 
 #include "db/blob/blob_fetcher.h"
@@ -69,7 +71,6 @@
 #include "table/multiget_context.h"
 #include "table/plain/plain_table_factory.h"
 #include "table/table_reader.h"
-#include "table/two_level_iterator.h"
 #include "table/unique_id_impl.h"
 #include "test_util/sync_point.h"
 #include "util/cast_util.h"
@@ -110,6 +111,181 @@ int FindFileInRange(const InternalKeyComparator& icmp,
   return static_cast<int>(std::lower_bound(b + left, b + right, key, cmp) - b);
 }
 
+InternalKey MultiScanInternalKey(const Slice& user_key,
+                                 const Comparator* user_comparator) {
+  const size_t timestamp_size = user_comparator->timestamp_size();
+  if (timestamp_size == 0) {
+    return InternalKey(user_key, kMaxSequenceNumber, kValueTypeForSeek);
+  }
+
+  std::string key_with_ts;
+  AppendKeyWithMaxTimestamp(&key_with_ts, user_key, timestamp_size);
+  return InternalKey(key_with_ts, kMaxSequenceNumber, kValueTypeForSeek);
+}
+
+bool MultiScanRangeOverlapsFile(const UserComparatorWrapper& user_comparator,
+                                const ScanOptions& scan_opts,
+                                const FdWithKeyRange& file) {
+  assert(scan_opts.range.start.has_value());
+  assert(scan_opts.range.limit.has_value());
+
+  if (user_comparator.CompareWithoutTimestamp(
+          scan_opts.range.limit.value(), /*a_has_ts=*/false,
+          ExtractUserKey(file.smallest_key), /*b_has_ts=*/true) <= 0) {
+    return false;
+  }
+
+  if (user_comparator.CompareWithoutTimestamp(
+          scan_opts.range.start.value(), /*a_has_ts=*/false,
+          ExtractUserKey(file.largest_key), /*b_has_ts=*/true) > 0) {
+    return false;
+  }
+
+  return true;
+}
+
+void AddMultiScanFileIndex(std::vector<size_t>* file_indexes,
+                           std::vector<char>* selected, size_t file_index) {
+  if ((*selected)[file_index]) {
+    return;
+  }
+  (*selected)[file_index] = true;
+  file_indexes->push_back(file_index);
+}
+
+template <typename OnOverlappingFile>
+bool NotifyMultiScanOverlappingFile(OnOverlappingFile& on_overlapping_file,
+                                    size_t file_index,
+                                    const ScanOptions& scan_range) {
+  if constexpr (std::is_same_v<std::invoke_result_t<OnOverlappingFile, size_t,
+                                                    const ScanOptions&>,
+                               bool>) {
+    return on_overlapping_file(file_index, scan_range);
+  } else {
+    on_overlapping_file(file_index, scan_range);
+    return true;
+  }
+}
+
+template <typename OnOverlappingFile>
+void ForEachMultiScanOverlappingFile(const InternalKeyComparator& icmp,
+                                     const UserComparatorWrapper& ucmp,
+                                     const LevelFilesBrief& file_level,
+                                     int level, const MultiScanArgs& scan_opts,
+                                     OnOverlappingFile on_overlapping_file) {
+  assert(scan_opts.HasBoundedScanRanges());
+  if (file_level.num_files == 0) {
+    return;
+  }
+
+  if (level == 0) {
+    for (size_t i = 0; i < file_level.num_files; ++i) {
+      for (const ScanOptions& scan_range : scan_opts.GetScanRanges()) {
+        if (MultiScanRangeOverlapsFile(ucmp, scan_range, file_level.files[i])) {
+          if (!NotifyMultiScanOverlappingFile(on_overlapping_file, i,
+                                              scan_range)) {
+            return;
+          }
+        }
+
+        if (ucmp.CompareWithoutTimestamp(
+                scan_range.range.start.value(), /*a_has_ts=*/false,
+                ExtractUserKey(file_level.files[i].largest_key),
+                /*b_has_ts=*/true) > 0) {
+          break;
+        }
+      }
+    }
+    return;
+  }
+
+  for (const ScanOptions& scan_range : scan_opts.GetScanRanges()) {
+    const InternalKey start_key = MultiScanInternalKey(
+        scan_range.range.start.value(), ucmp.user_comparator());
+    size_t first_file = FindFile(icmp, file_level, start_key.Encode());
+    if (first_file >= file_level.num_files) {
+      continue;
+    }
+
+    const InternalKey limit_key = MultiScanInternalKey(
+        scan_range.range.limit.value(), ucmp.user_comparator());
+    size_t last_file = FindFile(icmp, file_level, limit_key.Encode());
+    if (last_file >= file_level.num_files) {
+      last_file = file_level.num_files - 1;
+    }
+
+    for (size_t i = first_file; i <= last_file; ++i) {
+      if (MultiScanRangeOverlapsFile(ucmp, scan_range, file_level.files[i])) {
+        if (!NotifyMultiScanOverlappingFile(on_overlapping_file, i,
+                                            scan_range)) {
+          return;
+        }
+      }
+    }
+  }
+}
+
+std::vector<size_t> GetMultiScanOverlappingFiles(
+    const InternalKeyComparator& icmp, const UserComparatorWrapper& ucmp,
+    const LevelFilesBrief& file_level, int level,
+    const MultiScanArgs& scan_opts,
+    size_t max_file_indexes = std::numeric_limits<size_t>::max()) {
+  std::vector<size_t> file_indexes;
+  if (!scan_opts.HasBoundedScanRanges() || file_level.num_files == 0) {
+    return file_indexes;
+  }
+  file_indexes.reserve(
+      std::min(static_cast<size_t>(file_level.num_files), max_file_indexes));
+
+  std::vector<char> selected(file_level.num_files, false);
+  ForEachMultiScanOverlappingFile(
+      icmp, ucmp, file_level, level, scan_opts,
+      [&](size_t file_index, const ScanOptions& /*scan_range*/) {
+        if (file_indexes.size() >= max_file_indexes) {
+          return false;
+        }
+        AddMultiScanFileIndex(&file_indexes, &selected, file_index);
+        return file_indexes.size() < max_file_indexes;
+      });
+
+  return file_indexes;
+}
+
+void AddTableIteratorForLevel(
+    ColumnFamilyData* cfd, const ReadOptions& read_options,
+    const FileOptions& soptions, const MutableCFOptions& mutable_cf_options,
+    MergeIteratorBuilder* merge_iter_builder, Arena* arena,
+    const LevelFilesBrief& file_level, int level, size_t file_index,
+    bool skip_filters, size_t max_file_size_for_l0_meta_pin,
+    bool allow_unprepared_value, SequenceNumber read_seq) {
+  std::unique_ptr<TruncatedRangeDelIterator> tombstone_iter = nullptr;
+  const auto& file = file_level.files[file_index];
+  auto table_iter = cfd->table_cache()->NewIterator(
+      read_options, soptions, cfd->internal_comparator(), *file.file_metadata,
+      /*range_del_agg=*/nullptr, mutable_cf_options, nullptr,
+      cfd->internal_stats()->GetFileReadHist(level),
+      TableReaderCaller::kUserIterator, arena, skip_filters, level,
+      max_file_size_for_l0_meta_pin,
+      /*smallest_compaction_key=*/nullptr,
+      /*largest_compaction_key=*/nullptr, allow_unprepared_value, &read_seq,
+      &tombstone_iter,
+      /*maybe_pin_table_handle=*/true);
+#ifndef NDEBUG
+  TEST_SYNC_POINT_CALLBACK("Version::AddIteratorsForLevel:AddedFile",
+                           file.file_metadata);
+  std::pair<bool, bool> iterator_type(true /* is_block_based_table_iterator */,
+                                      false /* is_level_iterator */);
+  TEST_SYNC_POINT_CALLBACK("Version::AddIteratorsForLevel:IteratorType",
+                           &iterator_type);
+#endif
+  if (read_options.ignore_range_deletions) {
+    merge_iter_builder->AddIterator(table_iter);
+  } else {
+    merge_iter_builder->AddPointAndTombstoneIterator(table_iter,
+                                                     std::move(tombstone_iter));
+  }
+}
+
 Status OverlapWithIterator(const Comparator* ucmp,
                            const Slice& smallest_user_key,
                            const Slice& largest_user_key,
@@ -966,6 +1142,15 @@ bool SomeFileOverlapsRange(const InternalKeyComparator& icmp,
 
 namespace {
 
+TableCacheOpenOptions GetCompactionTableCacheOpenOptions(
+    bool open_ephemeral_table_reader) {
+  TableCacheOpenOptions options;
+  options.open_ephemeral_table_reader = open_ephemeral_table_reader;
+  options.avoid_shared_metadata_cache = open_ephemeral_table_reader;
+  options.skip_filters = open_ephemeral_table_reader;
+  return options;
+}
+
 class LevelIterator final : public InternalIterator {
  public:
   // NOTE: many of the const& parameters are saved in this object (so
@@ -981,7 +1166,9 @@ class LevelIterator final : public InternalIterator {
       bool allow_unprepared_value = false,
       std::unique_ptr<TruncatedRangeDelIterator>*** range_tombstone_iter_ptr_ =
           nullptr,
-      Statistics* db_statistics = nullptr, SystemClock* clock = nullptr)
+      SequenceNumber read_seq = kMaxSequenceNumber,
+      Statistics* db_statistics = nullptr, SystemClock* clock = nullptr,
+      bool open_ephemeral_table_reader = false)
       : table_cache_(table_cache),
         read_options_(read_options),
         file_options_(file_options),
@@ -997,9 +1184,11 @@ class LevelIterator final : public InternalIterator {
         pinned_iters_mgr_(nullptr),
         compaction_boundaries_(compaction_boundaries),
         range_tombstone_iter_(nullptr),
-        read_seq_(read_options.snapshot
-                      ? read_options.snapshot->GetSequenceNumber()
-                      : kMaxSequenceNumber),
+        range_tombstone_iter_required_(range_tombstone_iter_ptr_ != nullptr),
+        read_seq_(read_seq != kMaxSequenceNumber ||
+                          read_options.snapshot == nullptr
+                      ? read_seq
+                      : read_options.snapshot->GetSequenceNumber()),
         level_(level),
         skip_filters_(skip_filters),
         allow_unprepared_value_(allow_unprepared_value),
@@ -1007,7 +1196,9 @@ class LevelIterator final : public InternalIterator {
         to_return_sentinel_(false),
         scan_opts_(nullptr),
         db_statistics_(db_statistics),
-        clock_(clock) {
+        clock_(clock),
+        table_cache_open_options_(
+            GetCompactionTableCacheOpenOptions(open_ephemeral_table_reader)) {
     // Empty level is not supported.
     assert(flevel_ != nullptr && flevel_->num_files > 0);
     if (range_tombstone_iter_ptr_) {
@@ -1146,69 +1337,18 @@ class LevelIterator final : public InternalIterator {
     assert(so->GetComparator() == user_comparator_.user_comparator());
 
     file_to_scan_opts_ = std::make_unique<ScanOptionsMap>();
-    for (size_t k = 0; k < scan_opts_->size(); k++) {
-      const ScanOptions& opt = scan_opts_->GetScanRanges().at(k);
-      auto start = opt.range.start;
-      auto end = opt.range.limit;
-
-      if (!start.has_value()) {
-        continue;
-      }
-
-      // We can capture this case in the future, but for now lets skip this.
-      if (!end.has_value()) {
-        continue;
-      }
-
-      const size_t timestamp_size =
-          user_comparator_.user_comparator()->timestamp_size();
-      InternalKey istart, iend;
-      if (timestamp_size == 0) {
-        istart =
-            InternalKey(start.value(), kMaxSequenceNumber, kValueTypeForSeek);
-        // end key is exclusive for multiscan
-        iend = InternalKey(end.value(), kMaxSequenceNumber, kValueTypeForSeek);
-      } else {
-        std::string start_key_with_ts, end_key_with_ts;
-        AppendKeyWithMaxTimestamp(&start_key_with_ts, start.value(),
-                                  timestamp_size);
-        AppendKeyWithMaxTimestamp(&end_key_with_ts, end.value(),
-                                  timestamp_size);
-        istart = InternalKey(start_key_with_ts, kMaxSequenceNumber,
-                             kValueTypeForSeek);
-        // end key is exclusive for multiscan
-        iend =
-            InternalKey(end_key_with_ts, kMaxSequenceNumber, kValueTypeForSeek);
-      }
-
-      // TODO: This needs to be optimized, right now we iterate twice, which
-      // we dont need to. We can do this in N rather than 2N.
-      size_t fstart = FindFile(icomparator_, *flevel_, istart.Encode());
-      size_t fend = FindFile(icomparator_, *flevel_, iend.Encode());
-
-      // We need to check the relevant cases
-      // Cases:
-      // 1. [  S        E  ]
-      // 2. [  S  ]  [  E  ]
-      // 3. [  S  ] ...... [  E  ]
-      for (auto i = fstart; i <= fend; i++) {
-        if (i < flevel_->num_files) {
-          // FindFile only compares against the largest_key, so we need this
-          // additional check to ensure the scan range overlaps the file
-          if (icomparator_.InternalKeyComparator::Compare(
-                  iend.Encode(), flevel_->files[i].smallest_key) < 0) {
-            continue;
-          }
-          auto const metadata = flevel_->files[i].file_metadata;
+    ForEachMultiScanOverlappingFile(
+        icomparator_, user_comparator_, *flevel_, level_, *scan_opts_,
+        [&](size_t file_index, const ScanOptions& scan_range) {
+          auto const metadata = flevel_->files[file_index].file_metadata;
           if (metadata->FileIsStandAloneRangeTombstone()) {
             // Skip stand alone range deletion files.
-            continue;
+            return;
           }
-          auto& args = GetMultiScanArgForFile(i);
-          args.insert(start.value(), end.value(), opt.property_bag);
-        }
-      }
-    }
+          auto& args = GetMultiScanArgForFile(file_index);
+          args.insert(scan_range.range.start.value(),
+                      scan_range.range.limit.value(), scan_range.property_bag);
+        });
 
     StopWatch timer(clock_, db_statistics_, MULTISCAN_PREPARE_ITERATORS);
 
@@ -1221,7 +1361,8 @@ class LevelIterator final : public InternalIterator {
                           file_to_arg.first));
     }
 
-    if (so->use_async_io) {
+    if (so->use_async_io &&
+        (!range_tombstone_iter_required_ || range_tombstone_iter_ != nullptr)) {
       auto before = file_index_;
       // Pre-create and prepare only relevant file iterators
       for (auto& file_to_arg : *file_to_scan_opts_) {
@@ -1314,7 +1455,9 @@ class LevelIterator final : public InternalIterator {
         /*max_file_size_for_l0_meta_pin=*/0, smallest_compaction_key,
         largest_compaction_key, allow_unprepared_value_, &read_seq_,
         range_tombstone_iter_,
-        /*maybe_pin_table_handle=*/true);
+        /*maybe_pin_table_handle=*/
+        !table_cache_open_options_.open_ephemeral_table_reader,
+        /*file_open_metadata=*/nullptr, table_cache_open_options_);
   }
 
   // Check if current file being fully within iterate_lower_bound.
@@ -1371,6 +1514,7 @@ class LevelIterator final : public InternalIterator {
   //
   // *range_tombstone_iter_ points to range tombstones of the current SST file
   std::unique_ptr<TruncatedRangeDelIterator>* range_tombstone_iter_;
+  bool range_tombstone_iter_required_;
 
   // The sentinel key to be returned
   Slice sentinel_;
@@ -1391,6 +1535,7 @@ class LevelIterator final : public InternalIterator {
 
   Statistics* db_statistics_ = nullptr;
   SystemClock* clock_ = nullptr;
+  TableCacheOpenOptions table_cache_open_options_;
 
   // Our stored scan_opts for each prefix
   std::unique_ptr<ScanOptionsMap> file_to_scan_opts_ = nullptr;
@@ -1680,22 +1825,20 @@ bool LevelIterator::SkipEmptyFileForward() {
       if (scan_opts_ && FileHasMultiScanArg(file_index_)) {
         const ScanOptions& opts =
             GetMultiScanArgForFile(file_index_).GetScanRanges().front();
-        if (opts.range.start.has_value()) {
-          InternalKey target;
-          const size_t ts_size =
-              user_comparator_.user_comparator()->timestamp_size();
-          if (ts_size == 0) {
-            target = InternalKey(opts.range.start.value(), kMaxSequenceNumber,
-                                 kValueTypeForSeek);
-          } else {
-            std::string seek_key;
-            AppendKeyWithMaxTimestamp(&seek_key, opts.range.start.value(),
-                                      ts_size);
-            target =
-                InternalKey(seek_key, kMaxSequenceNumber, kValueTypeForSeek);
-          }
-          file_iter_.Seek(target.Encode());
+        assert(opts.range.start.has_value());
+        InternalKey target;
+        const size_t ts_size =
+            user_comparator_.user_comparator()->timestamp_size();
+        if (ts_size == 0) {
+          target = InternalKey(opts.range.start.value(), kMaxSequenceNumber,
+                               kValueTypeForSeek);
+        } else {
+          std::string seek_key;
+          AppendKeyWithMaxTimestamp(&seek_key, opts.range.start.value(),
+                                    ts_size);
+          target = InternalKey(seek_key, kMaxSequenceNumber, kValueTypeForSeek);
         }
+        file_iter_.Seek(target.Encode());
       } else {
         file_iter_.SeekToFirst();
       }
@@ -1742,14 +1885,8 @@ void LevelIterator::SkipEmptyFileBackward() {
 
 #ifndef NDEBUG
 bool LevelIterator::OverlapRange(const ScanOptions& opts, size_t file_index) {
-  return (user_comparator_.CompareWithoutTimestamp(
-              opts.range.start.value(), /*a_has_ts=*/false,
-              ExtractUserKey(flevel_->files[file_index].largest_key),
-              /*b_has_ts=*/true) <= 0 &&
-          user_comparator_.CompareWithoutTimestamp(
-              opts.range.limit.value(), /*a_has_ts=*/false,
-              ExtractUserKey(flevel_->files[file_index].smallest_key),
-              /*b_has_ts=*/true) > 0);
+  return MultiScanRangeOverlapsFile(user_comparator_, opts,
+                                    flevel_->files[file_index]);
 }
 #endif
 
@@ -2271,7 +2408,9 @@ InternalIterator* Version::TEST_GetLevelIterator(
       mutable_cf_options_, cfd_->internal_stats()->GetFileReadHist(level),
       TableReaderCaller::kUserIterator, IsFilterSkipped(level), level,
       nullptr /* range_del_agg */, nullptr /* compaction_boundaries */,
-      allow_unprepared_value, &tombstone_iter_ptr, db_statistics_, clock_);
+      allow_unprepared_value,
+      read_options.ignore_range_deletions ? nullptr : &tombstone_iter_ptr,
+      kMaxSequenceNumber, db_statistics_, clock_);
   if (read_options.ignore_range_deletions) {
     merge_iter_builder->AddIterator(level_iter);
   } else {
@@ -2342,19 +2481,22 @@ double VersionStorageInfo::GetEstimatedCompressionRatioAtLevel(
 void Version::AddIterators(const ReadOptions& read_options,
                            const FileOptions& soptions,
                            MergeIteratorBuilder* merge_iter_builder,
-                           bool allow_unprepared_value) {
+                           bool allow_unprepared_value, SequenceNumber read_seq,
+                           const MultiScanArgs* scan_opts) {
   assert(storage_info_.finalized_);
 
   for (int level = 0; level < storage_info_.num_non_empty_levels(); level++) {
     AddIteratorsForLevel(read_options, soptions, merge_iter_builder, level,
-                         allow_unprepared_value);
+                         allow_unprepared_value, read_seq, scan_opts);
   }
 }
 
 void Version::AddIteratorsForLevel(const ReadOptions& read_options,
                                    const FileOptions& soptions,
                                    MergeIteratorBuilder* merge_iter_builder,
-                                   int level, bool allow_unprepared_value) {
+                                   int level, bool allow_unprepared_value,
+                                   SequenceNumber read_seq,
+                                   const MultiScanArgs* scan_opts) {
   assert(storage_info_.finalized_);
   if (level >= storage_info_.num_non_empty_levels()) {
     // This is an empty level
@@ -2365,37 +2507,95 @@ void Version::AddIteratorsForLevel(const ReadOptions& read_options,
   }
 
   auto* arena = merge_iter_builder->GetArena();
+  const LevelFilesBrief& file_level = storage_info_.LevelFilesBrief(level);
+  const UserComparatorWrapper user_comparator(
+      cfd_->internal_comparator().user_comparator());
+
+  if (scan_opts != nullptr) {
+    const std::vector<size_t> file_indexes = GetMultiScanOverlappingFiles(
+        cfd_->internal_comparator(), user_comparator, file_level, level,
+        *scan_opts, level == 0 ? std::numeric_limits<size_t>::max() : 2);
+    if (file_indexes.empty()) {
+      return;
+    }
+
+    if (level == 0) {
+      const bool should_sample = should_sample_file_read();
+      for (size_t file_index : file_indexes) {
+        AddTableIteratorForLevel(
+            cfd_, read_options, soptions, mutable_cf_options_,
+            merge_iter_builder, arena, file_level, level, file_index,
+            /*skip_filters=*/false, max_file_size_for_l0_meta_pin_,
+            allow_unprepared_value, read_seq);
+        if (should_sample) {
+          sample_file_read_inc(file_level.files[file_index].file_metadata);
+        }
+      }
+      return;
+    }
+
+    if (file_indexes.size() == 1) {
+      AddTableIteratorForLevel(cfd_, read_options, soptions,
+                               mutable_cf_options_, merge_iter_builder, arena,
+                               file_level, level, file_indexes.front(),
+                               IsFilterSkipped(level),
+                               /*max_file_size_for_l0_meta_pin=*/0,
+                               allow_unprepared_value, read_seq);
+      return;
+    }
+
+#ifndef NDEBUG
+    for (size_t file_index = 0; file_index < file_level.num_files;
+         ++file_index) {
+      TEST_SYNC_POINT_CALLBACK("Version::AddIteratorsForLevel:AddedFile",
+                               file_level.files[file_index].file_metadata);
+    }
+#endif
+    auto* mem = arena->AllocateAligned(sizeof(LevelIterator));
+    std::unique_ptr<TruncatedRangeDelIterator>** tombstone_iter_ptr = nullptr;
+    auto level_iter = new (mem) LevelIterator(
+        cfd_->table_cache(), read_options, soptions,
+        cfd_->internal_comparator(), &file_level, mutable_cf_options_,
+        cfd_->internal_stats()->GetFileReadHist(level),
+        TableReaderCaller::kUserIterator, IsFilterSkipped(level), level,
+        /*range_del_agg=*/nullptr,
+        /*compaction_boundaries=*/nullptr, allow_unprepared_value,
+        read_options.ignore_range_deletions ? nullptr : &tombstone_iter_ptr,
+        read_seq, db_statistics_, clock_);
+#ifndef NDEBUG
+    std::pair<bool, bool> iterator_type(
+        false /* is_block_based_table_iterator */,
+        true /* is_level_iterator */);
+    TEST_SYNC_POINT_CALLBACK("Version::AddIteratorsForLevel:IteratorType",
+                             &iterator_type);
+#endif
+    if (read_options.ignore_range_deletions) {
+      merge_iter_builder->AddIterator(level_iter);
+    } else {
+      assert(tombstone_iter_ptr);
+      merge_iter_builder->AddPointAndTombstoneIterator(
+          level_iter, nullptr /* tombstone_iter */, tombstone_iter_ptr);
+    }
+    return;
+  }
+
   if (level == 0) {
     // Merge all level zero files together since they may overlap
-    std::unique_ptr<TruncatedRangeDelIterator> tombstone_iter = nullptr;
-    for (size_t i = 0; i < storage_info_.LevelFilesBrief(0).num_files; i++) {
-      const auto& file = storage_info_.LevelFilesBrief(0).files[i];
-      auto table_iter = cfd_->table_cache()->NewIterator(
-          read_options, soptions, cfd_->internal_comparator(),
-          *file.file_metadata, /*range_del_agg=*/nullptr, mutable_cf_options_,
-          nullptr, cfd_->internal_stats()->GetFileReadHist(0),
-          TableReaderCaller::kUserIterator, arena,
-          /*skip_filters=*/false, /*level=*/0, max_file_size_for_l0_meta_pin_,
-          /*smallest_compaction_key=*/nullptr,
-          /*largest_compaction_key=*/nullptr, allow_unprepared_value,
-          /*range_del_read_seqno=*/nullptr, &tombstone_iter,
-          /*maybe_pin_table_handle=*/true);
-      if (read_options.ignore_range_deletions) {
-        merge_iter_builder->AddIterator(table_iter);
-      } else {
-        merge_iter_builder->AddPointAndTombstoneIterator(
-            table_iter, std::move(tombstone_iter));
+    const bool should_sample = should_sample_file_read();
+    for (size_t i = 0; i < file_level.num_files; i++) {
+      AddTableIteratorForLevel(
+          cfd_, read_options, soptions, mutable_cf_options_, merge_iter_builder,
+          arena, file_level, level, i,
+          /*skip_filters=*/false, max_file_size_for_l0_meta_pin_,
+          allow_unprepared_value, read_seq);
+      if (should_sample) {
+        // Count once for every L0 file. This is done per iterator creation
+        // rather than Seek(), while files in other levels are sampled on
+        // seek/next/prev.
+        sample_file_read_inc(file_level.files[i].file_metadata);
       }
     }
-    if (should_sample_file_read()) {
-      // Count ones for every L0 files. This is done per iterator creation
-      // rather than Seek(), while files in other levels are sampled on
-      // seek/next/prev.
-      for (FileMetaData* meta : storage_info_.LevelFiles(0)) {
-        sample_file_read_inc(meta);
-      }
-    }
-  } else if (storage_info_.LevelFilesBrief(level).num_files > 0) {
+  } else if (file_level.num_files > 0) {
     // For levels > 0, we can use a concatenating iterator that sequentially
     // walks through the non-overlapping files in the level, opening them
     // lazily.
@@ -2403,12 +2603,13 @@ void Version::AddIteratorsForLevel(const ReadOptions& read_options,
     std::unique_ptr<TruncatedRangeDelIterator>** tombstone_iter_ptr = nullptr;
     auto level_iter = new (mem) LevelIterator(
         cfd_->table_cache(), read_options, soptions,
-        cfd_->internal_comparator(), &storage_info_.LevelFilesBrief(level),
-        mutable_cf_options_, cfd_->internal_stats()->GetFileReadHist(level),
+        cfd_->internal_comparator(), &file_level, mutable_cf_options_,
+        cfd_->internal_stats()->GetFileReadHist(level),
         TableReaderCaller::kUserIterator, IsFilterSkipped(level), level,
         /*range_del_agg=*/nullptr,
         /*compaction_boundaries=*/nullptr, allow_unprepared_value,
-        &tombstone_iter_ptr, db_statistics_, clock_);
+        read_options.ignore_range_deletions ? nullptr : &tombstone_iter_ptr,
+        read_seq, db_statistics_, clock_);
     if (read_options.ignore_range_deletions) {
       merge_iter_builder->AddIterator(level_iter);
     } else {
@@ -2464,7 +2665,8 @@ Status Version::OverlapWithLevelIterator(const ReadOptions& read_options,
         cfd_->internal_comparator(), &storage_info_.LevelFilesBrief(level),
         mutable_cf_options_, cfd_->internal_stats()->GetFileReadHist(level),
         TableReaderCaller::kUserIterator, IsFilterSkipped(level), level,
-        &range_del_agg, nullptr, false, nullptr, db_statistics_, clock_));
+        &range_del_agg, nullptr, false, nullptr, kMaxSequenceNumber,
+        db_statistics_, clock_));
     status = OverlapWithIterator(ucmp, smallest_user_key, largest_user_key,
                                  iter.get(), overlap);
   }
@@ -5520,16 +5722,19 @@ struct VersionSet::ManifestWriter {
   ColumnFamilyData* cfd;
   const autovector<VersionEdit*>& edit_list;
   const std::function<void(const Status&)> manifest_write_callback;
+  int max_file_opening_threads;
 
   explicit ManifestWriter(
       InstrumentedMutex* mu, ColumnFamilyData* _cfd,
       const autovector<VersionEdit*>& e,
-      const std::function<void(const Status&)>& manifest_wcb)
+      const std::function<void(const Status&)>& manifest_wcb,
+      int _max_file_opening_threads = 1)
       : done(false),
         cv(mu),
         cfd(_cfd),
         edit_list(e),
-        manifest_write_callback(manifest_wcb) {}
+        manifest_write_callback(manifest_wcb),
+        max_file_opening_threads(_max_file_opening_threads) {}
   ~ManifestWriter() { status.PermitUncheckedError(); }
 
   bool IsAllWalEdits() const {
@@ -5904,6 +6109,10 @@ Status VersionSet::ProcessManifestWrites(
   ManifestWriter& first_writer = writers.front();
   ManifestWriter* last_writer = &first_writer;
 
+  // Supports opening table readers with multiple threads, mostly useful for
+  // batched external sst file ingestion.
+  int batch_max_file_opening_threads = 1;
+
   assert(!manifest_writers_.empty());
   assert(manifest_writers_.front() == &first_writer);
 
@@ -5940,6 +6149,9 @@ Status VersionSet::ProcessManifestWrites(
     for (;;) {
       assert(!(*it)->edit_list.front()->IsColumnFamilyManipulation());
       last_writer = *it;
+      batch_max_file_opening_threads =
+          std::max(batch_max_file_opening_threads,
+                   last_writer->max_file_opening_threads);
       assert(last_writer != nullptr);
       assert(last_writer->cfd != nullptr);
       if (last_writer->cfd->IsDropped()) {
@@ -6111,9 +6323,32 @@ Status VersionSet::ProcessManifestWrites(
 
   uint64_t prev_manifest_file_size = manifest_file_size_;
   assert(pending_manifest_file_number_ == 0);
+  bool has_foreground_operation = false;
+  for (const VersionEdit* e : batch_edits) {
+    if (e->IsForegroundOperation()) {
+      has_foreground_operation = true;
+      break;
+    }
+  }
+  // For MANIFEST write batches with any foreground operation (external file
+  // ingestion/import, DeleteFilesInRange(s), and column family manipulations
+  // like CreateColumnFamily and DropColumnFamily), relax the size limit by 25%
+  // to reduce the likelihood of a user operation blocking on MANIFEST rotation.
+  // Background-only batches (flush/compaction) still rotate at the normal
+  // threshold.
+  // TODO/future: for workloads like atomic-replace ingestion-only, with zero
+  // or few flushes and compactions, it might be nice to trigger background
+  // manifest rotation if we are beyond the soft limit. But the vast majority
+  // of workloads should have plenty of background manifest ops to avoid
+  // foreground rotation.
+  uint64_t enforced_limit = tuned_max_manifest_file_size_;
+  if (has_foreground_operation) {
+    uint64_t new_limit = enforced_limit + enforced_limit / 4;
+    // don't keep in case of overflow
+    enforced_limit = std::max(enforced_limit, new_limit);
+  }
   if (!skip_manifest_write &&
-      (!descriptor_log_ ||
-       prev_manifest_file_size >= tuned_max_manifest_file_size_)) {
+      (!descriptor_log_ || prev_manifest_file_size >= enforced_limit)) {
     TEST_SYNC_POINT("VersionSet::ProcessManifestWrites:BeforeNewManifest");
     new_descriptor_log = true;
   } else {
@@ -6175,7 +6410,7 @@ Status VersionSet::ProcessManifestWrites(
                builder_guards.size() == versions.size());
         ColumnFamilyData* cfd = versions[i]->cfd_;
         s = builder_guards[i]->version_builder()->LoadTableHandlers(
-            cfd->internal_stats(), 1 /* max_threads */,
+            cfd->internal_stats(), batch_max_file_opening_threads,
             true /* prefetch_index_and_filter_in_cache */,
             false /* is_initial_load */, versions[i]->GetMutableCFOptions(),
             MaxFileSizeForL0MetaPin(versions[i]->GetMutableCFOptions()),
@@ -6538,7 +6773,7 @@ Status VersionSet::LogAndApply(
     InstrumentedMutex* mu, FSDirectory* dir_contains_current_file,
     bool new_descriptor_log, const ColumnFamilyOptions* new_cf_options,
     const std::vector<std::function<void(const Status&)>>& manifest_wcbs,
-    const std::function<Status()>& pre_cb) {
+    const std::function<Status()>& pre_cb, int max_file_opening_threads) {
   mu->AssertHeld();
   int num_edits = 0;
   for (const auto& elist : edit_lists) {
@@ -6570,7 +6805,8 @@ Status VersionSet::LogAndApply(
   for (int i = 0; i < num_cfds; ++i) {
     const auto wcb =
         manifest_wcbs.empty() ? [](const Status&) {} : manifest_wcbs[i];
-    writers.emplace_back(mu, column_family_datas[i], edit_lists[i], wcb);
+    writers.emplace_back(mu, column_family_datas[i], edit_lists[i], wcb,
+                         max_file_opening_threads);
     manifest_writers_.push_back(&writers[i]);
   }
   assert(!writers.empty());
@@ -7836,7 +8072,7 @@ InternalIterator* VersionSet::MakeInputIterator(
     RangeDelAggregator* range_del_agg,
     const FileOptions& file_options_compactions,
     const std::optional<const Slice>& start,
-    const std::optional<const Slice>& end) {
+    const std::optional<const Slice>& end, bool open_ephemeral_table_reader) {
   auto cfd = c->column_family_data();
   // Level-0 files have to be merged together.  For other levels,
   // we will make a concatenating iterator per level.
@@ -7855,6 +8091,8 @@ InternalIterator* VersionSet::MakeInputIterator(
       range_tombstones;
   size_t num = 0;
   [[maybe_unused]] size_t num_input_files = 0;
+  const TableCacheOpenOptions table_cache_open_options =
+      GetCompactionTableCacheOpenOptions(open_ephemeral_table_reader);
   for (size_t which = 0; which < c->num_input_levels(); which++) {
     const LevelFilesBrief* flevel = c->input_levels(which);
     num_input_files += flevel->num_files;
@@ -7891,7 +8129,9 @@ InternalIterator* VersionSet::MakeInputIterator(
               /*largest_compaction_key=*/nullptr,
               /*allow_unprepared_value=*/false,
               /*range_del_read_seqno=*/nullptr,
-              /*range_del_iter=*/&range_tombstone_iter);
+              /*range_del_iter=*/&range_tombstone_iter,
+              /*maybe_pin_table_handle=*/false,
+              /*file_open_metadata=*/nullptr, table_cache_open_options);
           range_tombstones.emplace_back(std::move(range_tombstone_iter),
                                         nullptr);
         }
@@ -7906,7 +8146,8 @@ InternalIterator* VersionSet::MakeInputIterator(
             TableReaderCaller::kCompaction, /*skip_filters=*/false,
             /*level=*/static_cast<int>(c->level(which)), range_del_agg,
             c->boundaries(which), false, &tombstone_iter_ptr,
-            db_options_->statistics.get(), clock_);
+            kMaxSequenceNumber, db_options_->statistics.get(), clock_,
+            open_ephemeral_table_reader);
         range_tombstones.emplace_back(nullptr, tombstone_iter_ptr);
       }
     }

db/version_set.h

@@ -85,6 +85,7 @@ class MergeIteratorBuilder;
 class SystemClock;
 class ManifestTailer;
 class FilePickerMultiGet;
+class MultiScanArgs;
 
 // VersionEdit is always supposed to be valid and it is used to point at
 // entries in Manifest. Ideally it should not be used as a container to
@@ -916,17 +917,27 @@ class Version {
   // yield the contents of this Version when merged together.
   // @param read_options Must outlive any iterator built by
   // `merger_iter_builder`.
+  // @param read_seq Snapshot sequence to use for range tombstone visibility.
+  // This is passed separately because lazy iterator initialization may happen
+  // after read_options.snapshot has been released by the caller.
+  // @param scan_opts Optional bounded scan ranges used to prune levels/files
+  // while building the iterator tree.
   void AddIterators(const ReadOptions& read_options,
                     const FileOptions& soptions,
                     MergeIteratorBuilder* merger_iter_builder,
-                    bool allow_unprepared_value);
+                    bool allow_unprepared_value, SequenceNumber read_seq,
+                    const MultiScanArgs* scan_opts = nullptr);
 
   // @param read_options Must outlive any iterator built by
   // `merger_iter_builder`.
+  // @param read_seq Snapshot sequence to use for range tombstone visibility.
+  // @param scan_opts Optional bounded scan ranges used to prune this level.
   void AddIteratorsForLevel(const ReadOptions& read_options,
                             const FileOptions& soptions,
                             MergeIteratorBuilder* merger_iter_builder,
-                            int level, bool allow_unprepared_value);
+                            int level, bool allow_unprepared_value,
+                            SequenceNumber read_seq,
+                            const MultiScanArgs* scan_opts = nullptr);
 
   Status OverlapWithLevelIterator(const ReadOptions&, const FileOptions&,
                                   const Slice& smallest_user_key,
@@ -1319,7 +1330,8 @@ class VersionSet {
       bool new_descriptor_log = false,
       const ColumnFamilyOptions* new_cf_options = nullptr,
       const std::vector<std::function<void(const Status&)>>& manifest_wcbs = {},
-      const std::function<Status()>& pre_cb = {});
+      const std::function<Status()>& pre_cb = {},
+      int max_file_opening_threads = 1);
 
   void WakeUpWaitingManifestWriters();
 
@@ -1558,12 +1570,16 @@ class VersionSet {
   // The caller should delete the iterator when no longer needed.
   // @param read_options Must outlive the returned iterator.
   // @param start, end indicates compaction range
+  // @param open_ephemeral_table_reader When true, the per-file iterators
+  //              bypass the shared TableCache and open fresh TableReaders
+  //              using `file_options_compactions`.
   InternalIterator* MakeInputIterator(
       const ReadOptions& read_options, const Compaction* c,
       RangeDelAggregator* range_del_agg,
       const FileOptions& file_options_compactions,
       const std::optional<const Slice>& start,
-      const std::optional<const Slice>& end);
+      const std::optional<const Slice>& end,
+      bool open_ephemeral_table_reader = false);
 
   // Add all files listed in any live version to *live_table_files and
   // *live_blob_files. Note that these lists may contain duplicates.
@@ -1950,7 +1966,8 @@ class ReactiveVersionSet : public VersionSet {
       InstrumentedMutex* /*mu*/, FSDirectory* /*dir_contains_current_file*/,
       bool /*new_descriptor_log*/, const ColumnFamilyOptions* /*new_cf_option*/,
       const std::vector<std::function<void(const Status&)>>& /*manifest_wcbs*/,
-      const std::function<Status()>& /*pre_cb*/) override {
+      const std::function<Status()>& /*pre_cb*/,
+      int /*max_file_opening_threads*/) override {
     return Status::NotSupported("not supported in reactive mode");
   }
 

db_stress_tool/db_stress_common.h

@@ -201,6 +201,7 @@ DECLARE_bool(verify_checksum);
 DECLARE_bool(mmap_read);
 DECLARE_bool(mmap_write);
 DECLARE_bool(use_direct_reads);
+DECLARE_bool(use_direct_io_for_compaction_reads);
 DECLARE_bool(use_direct_io_for_flush_and_compaction);
 DECLARE_bool(mock_direct_io);
 DECLARE_bool(statistics);
@@ -228,6 +229,8 @@ DECLARE_uint64(backup_max_size);
 DECLARE_int32(checkpoint_one_in);
 DECLARE_int32(ingest_external_file_one_in);
 DECLARE_int32(ingest_external_file_width);
+DECLARE_int32(ingest_external_file_prepare_commit_one_in);
+DECLARE_int32(ingest_external_file_use_file_info_one_in);
 DECLARE_int32(compact_files_one_in);
 DECLARE_int32(compact_range_one_in);
 DECLARE_int32(promote_l0_one_in);
@@ -303,7 +306,7 @@ DECLARE_string(default_write_temperature);
 DECLARE_string(default_temperature);
 DECLARE_uint32(verify_output_flags);
 DECLARE_bool(paranoid_memory_checks);
-DECLARE_bool(memtable_veirfy_per_key_checksum_on_seek);
+DECLARE_bool(memtable_verify_per_key_checksum_on_seek);
 DECLARE_bool(memtable_batch_lookup_optimization);
 
 // Options for transaction dbs.
@@ -349,6 +352,7 @@ DECLARE_int32(prepopulate_blob_cache);
 DECLARE_int32(approximate_size_one_in);
 DECLARE_bool(best_efforts_recovery);
 DECLARE_bool(skip_verifydb);
+DECLARE_string(verify_cpu_corruption_dir);
 DECLARE_bool(paranoid_file_checks);
 DECLARE_uint64(batch_protection_bytes_per_key);
 DECLARE_uint32(memtable_protection_bytes_per_key);
@@ -466,6 +470,7 @@ DECLARE_uint32(ingest_wbwi_one_in);
 DECLARE_bool(universal_reduce_file_locking);
 DECLARE_bool(use_multiscan);
 DECLARE_bool(multiscan_use_async_io);
+DECLARE_bool(read_scoped_block_buffer_provider);
 DECLARE_uint64(multiscan_max_prefetch_memory_bytes);
 
 // Compaction deletion trigger declarations for stress testing

db_stress_tool/db_stress_gflags.cc

@@ -130,6 +130,34 @@ DEFINE_bool(enable_pipelined_write, false, "Pipeline WAL/memtable writes");
 
 DEFINE_bool(verify_before_write, false, "Verify before write");
 
+DEFINE_string(
+    verify_cpu_corruption_dir, "",
+    "If non-empty, right after each write op (put/delete/deleterange), "
+    "flush, and compaction (compactrange/compactfiles), db_stress verifies "
+    "that op for corruption and, on the first hit, writes a result file "
+    "into this directory and marks the run as a verification failure. It "
+    "catches: (a) a corruption returned by the op itself or by the "
+    "read-back (Status::Corruption); or (b) a silent data corruption (SDC) "
+    "-- a read-back that SUCCEEDS but returns the wrong result vs the "
+    "committed expected state: a lost key (NotFound), a resurrected key "
+    "(deleted but present), or a wrong value (bytes differ) -- surfaced by "
+    "an immediate read right after the op, instead of waiting for a later "
+    "read operation or the end-of-run db verification. Running this check "
+    "right after the op gives the most immediate and accurate verification "
+    "when an external CPU-fault injector (e.g. gdb) flips a register inside "
+    "that op. REQUIRES (enforced at startup) --threads=1 and all fault "
+    "injection off (every *_fault_one_in = 0 and sync_fault_injection = "
+    "false): the full-keyspace read-back is only well-defined with a single "
+    "writer, and injected I/O faults would otherwise taint it. OUTPUT "
+    "CONTRACT (one file per worker thread): "
+    "<dir>/data_corruption.<thread_id>.json -- a JSON object with fields: "
+    "kind (one of lost|resurrected|wrong-value|detected-corruption), cf "
+    "(int), key (int), value_from_db (hex), value_from_expected (hex), "
+    "op_status (string). PERFORMANCE: the read-back scans the entire keyspace "
+    "after every op (O(max_key) Get calls per op), so this is meant for "
+    "single-op CPU-fault-injection debugging, not general stress runs. Empty "
+    "(default) = off.");
+
 DEFINE_bool(histogram, false, "Print histogram of operation timings");
 
 DEFINE_bool(destroy_db_initially, true,
@@ -732,6 +760,11 @@ DEFINE_bool(mmap_write, ROCKSDB_NAMESPACE::Options().allow_mmap_writes,
 DEFINE_bool(use_direct_reads, ROCKSDB_NAMESPACE::Options().use_direct_reads,
             "Use O_DIRECT for reading data");
 
+DEFINE_bool(use_direct_io_for_compaction_reads,
+            ROCKSDB_NAMESPACE::Options().use_direct_io_for_compaction_reads,
+            "Use O_DIRECT for compaction-input SST reads only, while keeping "
+            "user reads buffered");
+
 DEFINE_bool(use_direct_io_for_flush_and_compaction,
             ROCKSDB_NAMESPACE::Options().use_direct_io_for_flush_and_compaction,
             "Use O_DIRECT for writing data");
@@ -862,6 +895,19 @@ DEFINE_int32(ingest_external_file_one_in, 0,
 DEFINE_int32(ingest_external_file_width, 100,
              "The width of the ingested external files.");
 
+DEFINE_int32(ingest_external_file_prepare_commit_one_in, 0,
+             "If non-zero, an ingestion that would call IngestExternalFile() "
+             "instead uses the two-phase PrepareFileIngestion()/"
+             "CommitFileIngestion() API once for every N such ingestions on "
+             "average, occasionally dropping the prepared handle without "
+             "committing to exercise the rollback path. 0 disables it.");
+
+DEFINE_int32(ingest_external_file_use_file_info_one_in, 0,
+             "If non-zero, the ingestexternalfile flow reuses each file's "
+             "metadata via IngestExternalFileArg::file_infos (from "
+             "SstFileWriter::Finish) once every N ingestions on average, so "
+             "ingestion skips re-opening and scanning the files.");
+
 DEFINE_int32(compact_files_one_in, 0,
              "If non-zero, then CompactFiles() will be called once for every N "
              "operations on average.  0 indicates CompactFiles() is disabled.");
@@ -1021,7 +1067,8 @@ DEFINE_int32(iterpercent, 10,
 static const bool FLAGS_iterpercent_dummy __attribute__((__unused__)) =
     RegisterFlagValidator(&FLAGS_iterpercent, &ValidateInt32Percent);
 
-DEFINE_uint64(num_iterations, 10, "Number of iterations per MultiIterate run");
+DEFINE_uint64(num_iterations, 10,
+              "Number of iterations per iterator or MultiScan run");
 static const bool FLAGS_num_iterations_dummy __attribute__((__unused__)) =
     RegisterFlagValidator(&FLAGS_num_iterations, &ValidateUint32Range);
 
@@ -1630,9 +1677,9 @@ DEFINE_bool(paranoid_memory_checks,
             "Sets CF option paranoid_memory_checks.");
 
 DEFINE_bool(
-    memtable_veirfy_per_key_checksum_on_seek,
-    ROCKSDB_NAMESPACE::Options().memtable_veirfy_per_key_checksum_on_seek,
-    "Sets CF option memtable_veirfy_per_key_checksum_on_seek.");
+    memtable_verify_per_key_checksum_on_seek,
+    ROCKSDB_NAMESPACE::Options().memtable_verify_per_key_checksum_on_seek,
+    "Sets CF option memtable_verify_per_key_checksum_on_seek.");
 
 DEFINE_bool(memtable_batch_lookup_optimization,
             ROCKSDB_NAMESPACE::Options().memtable_batch_lookup_optimization,
@@ -1705,6 +1752,10 @@ DEFINE_bool(use_multiscan, false,
 DEFINE_bool(multiscan_use_async_io, false,
             "If set, enable async_io for MultiScan operations.");
 
+DEFINE_bool(read_scoped_block_buffer_provider, false,
+            "If set, configure ReadOptions::read_scoped_block_buffer_provider "
+            "with a stress-test provider for supported scan reads.");
+
 DEFINE_uint64(multiscan_max_prefetch_memory_bytes, 0,
               "If non-zero, sets the max_prefetch_memory_bytes on the "
               "IODispatcher used for MultiScan. This limits the total memory "

db_stress_tool/db_stress_test_base.cc

@@ -8,8 +8,13 @@
 // found in the LICENSE file. See the AUTHORS file for names of contributors.
 //
 
+#include <iomanip>
 #include <ios>
+#include <memory>
+#include <mutex>
+#include <optional>
 #include <thread>
+#include <unordered_set>
 
 #include "db_stress_tool/db_stress_compression_manager.h"
 #include "db_stress_tool/db_stress_listener.h"
@@ -32,11 +37,13 @@
 #include "rocksdb/io_dispatcher.h"
 #include "rocksdb/secondary_cache.h"
 #include "rocksdb/sst_file_manager.h"
+#include "rocksdb/table.h"
 #include "rocksdb/table_properties.h"
 #include "rocksdb/types.h"
 #include "rocksdb/utilities/object_registry.h"
 #include "rocksdb/utilities/write_batch_with_index.h"
 #include "test_util/testutil.h"
+#include "util/aligned_buffer.h"
 #include "util/cast_util.h"
 #include "util/simple_mixed_compressor.h"
 #include "utilities/backup/backup_engine_impl.h"
@@ -48,6 +55,120 @@ namespace ROCKSDB_NAMESPACE {
 
 namespace {
 
+class StressReadScopedBlockBufferProvider
+    : public ReadScopedBlockBufferProvider {
+ public:
+  StressReadScopedBlockBufferProvider() : state_(std::make_shared<State>()) {}
+  StressReadScopedBlockBufferProvider(
+      const StressReadScopedBlockBufferProvider&) = delete;
+  StressReadScopedBlockBufferProvider& operator=(
+      const StressReadScopedBlockBufferProvider&) = delete;
+  StressReadScopedBlockBufferProvider(StressReadScopedBlockBufferProvider&&) =
+      delete;
+  StressReadScopedBlockBufferProvider& operator=(
+      StressReadScopedBlockBufferProvider&&) = delete;
+
+  ~StressReadScopedBlockBufferProvider() override {
+    state_->ValidateNoLiveAllocations();
+  }
+
+  Status Allocate(size_t size, size_t alignment, Lease* out) override {
+    Check(out != nullptr, "lease output is nullptr");
+    Check(size > 0, "allocation size is zero");
+    Check(alignment > 0 && (alignment & (alignment - 1)) == 0,
+          "alignment must be a power of two");
+
+    auto* allocation = new Allocation();
+    allocation->state = state_;
+    allocation->requested_size = size;
+    allocation->requested_alignment = alignment;
+    allocation->storage.Alignment(alignment);
+    allocation->storage.AllocateNewBuffer(size);
+    allocation->data = allocation->storage.BufferStart();
+    allocation->size = allocation->storage.Capacity();
+
+    Check(allocation->data != nullptr, "provider returned null data");
+    Check(allocation->size >= size, "provider returned short allocation");
+    Check(AlignedBuffer::isAligned(allocation->data, alignment),
+          "provider returned misaligned data");
+    Check(
+        alignment == 1 || AlignedBuffer::isAligned(allocation->size, alignment),
+        "provider returned misaligned direct-I/O size");
+
+    {
+      std::lock_guard<std::mutex> lock(state_->mutex);
+      Check(state_->live_allocations.insert(allocation).second,
+            "duplicate allocation registration");
+      state_->live_bytes += allocation->size;
+      ++state_->total_allocations;
+    }
+
+    out->data = allocation->data;
+    out->size = allocation->size;
+    out->cleanup.Allocate();
+    out->cleanup->RegisterCleanup(&ReleaseAllocation, allocation, nullptr);
+    return Status::OK();
+  }
+
+ private:
+  struct Allocation;
+
+  struct State {
+    std::mutex mutex;
+    std::unordered_set<Allocation*> live_allocations;
+    size_t live_bytes = 0;
+    uint64_t total_allocations = 0;
+
+    void ValidateNoLiveAllocations() {
+      std::lock_guard<std::mutex> lock(mutex);
+      Check(live_allocations.empty(),
+            "provider destroyed with live allocations");
+      Check(live_bytes == 0, "provider destroyed with live bytes");
+    }
+  };
+
+  struct Allocation {
+    std::shared_ptr<State> state;
+    size_t requested_size = 0;
+    size_t requested_alignment = 1;
+    char* data = nullptr;
+    size_t size = 0;
+    AlignedBuffer storage;
+  };
+
+  static void Check(bool condition, const char* message) {
+    if (!condition) {
+      fprintf(stderr, "ReadScopedBlockBufferProvider invariant failed: %s\n",
+              message);
+      std::abort();
+    }
+  }
+
+  static void ReleaseAllocation(void* arg1, void* /*arg2*/) {
+    auto* allocation = static_cast<Allocation*>(arg1);
+    Check(allocation != nullptr, "cleanup received null allocation");
+    Check(allocation->state != nullptr, "cleanup received null state");
+    Check(allocation->data != nullptr, "cleanup received null data");
+    Check(allocation->size >= allocation->requested_size,
+          "cleanup received short allocation");
+    Check(AlignedBuffer::isAligned(allocation->data,
+                                   allocation->requested_alignment),
+          "cleanup received misaligned data");
+
+    {
+      std::lock_guard<std::mutex> lock(allocation->state->mutex);
+      Check(allocation->state->live_allocations.erase(allocation) == 1,
+            "cleanup for unknown or duplicate allocation");
+      Check(allocation->state->live_bytes >= allocation->size,
+            "cleanup underflowed live bytes");
+      allocation->state->live_bytes -= allocation->size;
+    }
+    delete allocation;
+  }
+
+  std::shared_ptr<State> state_;
+};
+
 std::shared_ptr<const FilterPolicy> CreateFilterPolicy() {
   if (FLAGS_bloom_bits < 0) {
     return BlockBasedTableOptions().filter_policy;
@@ -633,7 +754,7 @@ void StressTest::FinishInitDb(SharedState* shared) {
     if (!s.ok()) {
       fprintf(stderr, "Error restoring historical expected values: %s\n",
               s.ToString().c_str());
-      exit(1);
+      port::ImmediateExit(1);
     }
   }
   if (FLAGS_use_txn && !FLAGS_use_optimistic_txn) {
@@ -666,7 +787,7 @@ void StressTest::TrackExpectedState(SharedState* shared) {
     if (!s.ok()) {
       fprintf(stderr, "Error enabling history tracing: %s\n",
               s.ToString().c_str());
-      exit(1);
+      port::ImmediateExit(1);
     }
   }
 }
@@ -796,6 +917,148 @@ void StressTest::VerificationAbort(SharedState* shared, int cf, int64_t key,
   shared->SetVerificationFailure();
 }
 
+namespace {
+
+Slice ExpectedValueSlice(const ExpectedValue& expected_value, char* scratch) {
+  const size_t size =
+      GenerateValue(expected_value.GetValueBase(), scratch, kValueMaxLen);
+  return Slice(scratch, size);
+}
+
+struct DataCorruption {
+  // "lost" | "resurrected" | "wrong-value" | "detected-corruption"
+  const char* kind;
+  Slice value_from_db;
+  Slice value_from_expected;
+  std::string op_status;
+};
+
+std::optional<DataCorruption> ClassifyReadBack(
+    const Status& read_status, const std::string& db_value,
+    const ExpectedValue& expected_value, char* scratch) {
+  if (read_status.IsCorruption()) {
+    return DataCorruption{"detected-corruption", db_value, Slice(),
+                          "Get: " + read_status.ToString()};
+  }
+  // Fault injection is off (enforced at startup), so the read-back can only be
+  // OK or NotFound here. Anything else is outside this tool's contract -- stop
+  // now rather than guess a classification. ImmediateExit (not exit/abort): the
+  // DB is open with live background threads, so a normal exit would race their
+  // static teardown (see port::ImmediateExit).
+  if (!read_status.ok() && !read_status.IsNotFound()) {
+    fprintf(stderr, "verify_cpu_corruption: unexpected read-back status: %s\n",
+            read_status.ToString().c_str());
+    port::ImmediateExit(1);
+  }
+  const bool present_in_db = read_status.ok();
+  // Exists() asserts the key is not mid-write -- which holds under --threads=1:
+  // the just-run op committed before this read-back.
+  const bool expected_to_exist = expected_value.Exists();
+
+  if (!expected_to_exist && !present_in_db) {
+    return std::nullopt;  // Agree: key absent.
+  }
+  if (expected_to_exist && !present_in_db) {
+    return DataCorruption{"lost", Slice(),
+                          ExpectedValueSlice(expected_value, scratch),
+                          "Get: NotFound"};
+  }
+  if (!expected_to_exist && present_in_db) {
+    return DataCorruption{"resurrected", db_value, Slice(), "Get: OK"};
+  }
+  // Present in both: compare the values.
+  const Slice expected = ExpectedValueSlice(expected_value, scratch);
+  if (Slice(db_value) == expected) {
+    return std::nullopt;  // Agree: same value.
+  }
+  return DataCorruption{"wrong-value", db_value, expected, "Get: OK"};
+}
+
+void WriteDataCorruption(uint32_t thread_id, int cf, int64_t key,
+                         const DataCorruption& corruption) {
+  std::ostringstream oss;
+  oss << "{\"kind\":\"" << corruption.kind << "\",\"cf\":" << cf
+      << ",\"key\":" << key << ",\"value_from_db\":\""
+      << corruption.value_from_db.ToString(/* hex */ true)
+      << "\",\"value_from_expected\":\""
+      << corruption.value_from_expected.ToString(/* hex */ true)
+      << "\",\"op_status\":" << std::quoted(corruption.op_status) << "}\n";
+  const std::string path = FLAGS_verify_cpu_corruption_dir +
+                           "/data_corruption." + std::to_string(thread_id) +
+                           ".json";
+  FILE* f = fopen(path.c_str(), "w");
+  if (f == nullptr) {
+    fprintf(stderr, "Failed to open CPU corruption result file: %s\n",
+            path.c_str());
+    return;
+  }
+  fputs(oss.str().c_str(), f);
+  fclose(f);
+}
+
+}  // namespace
+
+void StressTest::MaybeVerifyCpuCorruption(ThreadState* thread,
+                                          const char* op_label,
+                                          const Status& op_status) {
+  if (FLAGS_verify_cpu_corruption_dir.empty()) {
+    return;
+  }
+  auto shared = thread->shared;
+
+  // The op itself returned Corruption: an integrity check caught the
+  // corruption.
+  if (op_status.IsCorruption()) {
+    WriteDataCorruption(thread->tid, /* cf */ -1, /* key */ -1,
+                        {"detected-corruption", Slice(), Slice(),
+                         std::string(op_label) + ": " + op_status.ToString()});
+    VerificationAbort(shared,
+                      std::string(op_label) +
+                          " detected data corruption: " + op_status.ToString());
+    return;
+  }
+
+  // Single writer (the run is pinned to --threads=1): read every key back and
+  // compare it against the committed expected state, ending the run on the
+  // first data corruption found.
+  // TODO: this full-keyspace read-back duplicates the canonical verifier
+  // VerifyDb (no_batched_ops_stress.cc). Refactor that key/value compare into a
+  // shared per-key helper both can call, so this stays in sync with the model.
+  ReadOptions read_opts;
+  read_opts.verify_checksums = true;
+  // A user-timestamp DB needs a read timestamp; supply one like the canonical
+  // verifier (no_batched_ops_stress.cc).
+  std::string read_ts_str;
+  Slice read_ts;
+  if (FLAGS_user_timestamp_size > 0) {
+    read_ts_str = GetNowNanos();
+    read_ts = read_ts_str;
+    read_opts.timestamp = &read_ts;
+  }
+  const int64_t max_key = shared->GetMaxKey();
+  std::string db_value;
+  char expected_scratch[kValueMaxLen];
+  for (int cf = 0; cf < static_cast<int>(column_families_.size()); ++cf) {
+    for (int64_t key = 0; key < max_key; ++key) {
+      const ExpectedValue expected = shared->Get(cf, key);
+      db_value.clear();
+      const Status s =
+          db_->Get(read_opts, column_families_[cf], Key(key), &db_value);
+      const std::optional<DataCorruption> corruption =
+          ClassifyReadBack(s, db_value, expected, expected_scratch);
+      if (corruption.has_value()) {
+        WriteDataCorruption(thread->tid, cf, key, *corruption);
+        VerificationAbort(
+            shared,
+            std::string(corruption->kind) + " (" + corruption->op_status + ")",
+            cf, key, corruption->value_from_db,
+            corruption->value_from_expected);
+        return;
+      }
+    }
+  }
+}
+
 std::string StressTest::DebugString(const Slice& value,
                                     const WideColumns& columns) {
   std::ostringstream oss;
@@ -1207,6 +1470,14 @@ void StressTest::OperateDb(ThreadState* thread) {
   if (FLAGS_use_trie_index && !FLAGS_use_udi_as_primary_index && udi_factory_) {
     read_opts.table_index_factory = udi_factory_.get();
   }
+  std::unique_ptr<StressReadScopedBlockBufferProvider>
+      read_scoped_block_buffer_provider;
+  if (FLAGS_read_scoped_block_buffer_provider) {
+    read_scoped_block_buffer_provider =
+        std::make_unique<StressReadScopedBlockBufferProvider>();
+    read_opts.read_scoped_block_buffer_provider =
+        read_scoped_block_buffer_provider.get();
+  }
   WriteOptions write_opts;
   if (FLAGS_rate_limit_auto_wal_flush) {
     write_opts.rate_limiter_priority = Env::IO_USER;
@@ -1417,7 +1688,8 @@ void StressTest::OperateDb(ThreadState* thread) {
       ColumnFamilyHandle* column_family = column_families_[rand_column_family];
 
       if (thread->rand.OneInOpt(FLAGS_compact_files_one_in)) {
-        TestCompactFiles(thread, column_family);
+        Status s = TestCompactFiles(thread, column_family);
+        MaybeVerifyCpuCorruption(thread, "compactfiles", s);
       }
 
       int64_t rand_key = GenerateOneKey(thread, i);
@@ -1425,7 +1697,8 @@ void StressTest::OperateDb(ThreadState* thread) {
       Slice key = keystr;
 
       if (thread->rand.OneInOpt(FLAGS_compact_range_one_in)) {
-        TestCompactRange(thread, rand_key, key, column_family);
+        Status s = TestCompactRange(thread, rand_key, key, column_family);
+        MaybeVerifyCpuCorruption(thread, "compactrange", s);
         if (thread->shared->HasVerificationFailedYet()) {
           break;
         }
@@ -1440,6 +1713,7 @@ void StressTest::OperateDb(ThreadState* thread) {
 
       if (thread->rand.OneInOpt(FLAGS_flush_one_in)) {
         Status status = TestFlush(rand_column_families);
+        MaybeVerifyCpuCorruption(thread, "flush", status);
         ProcessStatus(shared, "Flush", status);
       }
 
@@ -1665,8 +1939,11 @@ void StressTest::OperateDb(ThreadState* thread) {
         if (disable_fault_injection_during_user_write) {
           db_fault_injection_fs_->DisableAllThreadLocalErrorInjection();
         }
-        TestPut(thread, write_opts, read_opts, rand_column_families, rand_keys,
-                value);
+        Status write_status = TestPut(thread, write_opts, read_opts,
+                                      rand_column_families, rand_keys, value);
+        // Verify before re-enabling injection: the read-back must run in the
+        // no-injection window so injected I/O errors cannot taint it.
+        MaybeVerifyCpuCorruption(thread, "put", write_status);
         if (disable_fault_injection_during_user_write) {
           db_fault_injection_fs_->EnableAllThreadLocalErrorInjection();
         }
@@ -1676,7 +1953,10 @@ void StressTest::OperateDb(ThreadState* thread) {
         if (disable_fault_injection_during_user_write) {
           db_fault_injection_fs_->DisableAllThreadLocalErrorInjection();
         }
-        TestDelete(thread, write_opts, rand_column_families, rand_keys);
+        Status del_status =
+            TestDelete(thread, write_opts, rand_column_families, rand_keys);
+        // Verify before re-enabling injection (see the put case).
+        MaybeVerifyCpuCorruption(thread, "delete", del_status);
         if (disable_fault_injection_during_user_write) {
           db_fault_injection_fs_->EnableAllThreadLocalErrorInjection();
         }
@@ -1686,7 +1966,10 @@ void StressTest::OperateDb(ThreadState* thread) {
         if (disable_fault_injection_during_user_write) {
           db_fault_injection_fs_->DisableAllThreadLocalErrorInjection();
         }
-        TestDeleteRange(thread, write_opts, rand_column_families, rand_keys);
+        Status delrange_status = TestDeleteRange(
+            thread, write_opts, rand_column_families, rand_keys);
+        // Verify before re-enabling injection (see the put case).
+        MaybeVerifyCpuCorruption(thread, "deleterange", delrange_status);
         if (disable_fault_injection_during_user_write) {
           db_fault_injection_fs_->EnableAllThreadLocalErrorInjection();
         }
@@ -1961,6 +2244,16 @@ Status StressTest::TestMultiScan(ThreadState* thread,
     return true;
   };
 
+  // Sometimes stop before draining all prefetched blocks, matching applications
+  // that stop after a bounded number of results. In one mode the whole prepared
+  // MultiScan is abandoned. In the other mode only the current range is
+  // stopped, so the next range's Seek() exercises releasing skipped blocks on
+  // the same prepared iterator.
+  const bool early_exit = thread->rand.OneIn(2);
+  const bool abandon_scan_after_early_exit =
+      early_exit && thread->rand.OneIn(2);
+  bool abandon_prepared_scan = false;
+
   for (const ScanOptions& scan_opt : scan_opts.GetScanRanges()) {
     if (op_logs.size() > kOpLogsLimit) {
       // Shouldn't take too much memory for the history log. Clear it.
@@ -2021,13 +2314,25 @@ Status StressTest::TestMultiScan(ThreadState* thread,
     VerifyIterator(thread, cmp_cfh, ro, iter.get(), cmp_iter.get(), last_op,
                    key, rand_column_families, op_logs, verify_func, &diverged);
 
+    uint64_t range_iterations = 0;
     while (iter->Valid()) {
+      if (early_exit && range_iterations >= FLAGS_num_iterations) {
+        if (abandon_scan_after_early_exit) {
+          op_logs += "E";
+          abandon_prepared_scan = true;
+        } else {
+          op_logs += "R";
+        }
+        break;
+      }
+
       iter->Next();
       if (!diverged) {
         assert(cmp_iter->Valid());
         cmp_iter->Next();
       }
       op_logs += "N";
+      ++range_iterations;
 
       if (iter->Valid() && ro.allow_unprepared_value) {
         op_logs += "*";
@@ -2066,7 +2371,7 @@ Status StressTest::TestMultiScan(ThreadState* thread,
     thread->stats.AddIterations(1);
 
     op_logs += "; ";
-    if (diverged) {
+    if (diverged || abandon_prepared_scan) {
       break;
     }
   }
@@ -3355,13 +3660,13 @@ Status StressTest::TestGetPropertiesOfAllTables() const {
   return db_->GetPropertiesOfAllTables(&props);
 }
 
-void StressTest::TestCompactFiles(ThreadState* thread,
-                                  ColumnFamilyHandle* column_family) {
+Status StressTest::TestCompactFiles(ThreadState* thread,
+                                    ColumnFamilyHandle* column_family) {
   ROCKSDB_NAMESPACE::ColumnFamilyMetaData cf_meta_data;
   db_->GetColumnFamilyMetaData(column_family, &cf_meta_data);
 
   if (cf_meta_data.levels.empty()) {
-    return;
+    return Status::OK();
   }
 
   // Randomly compact up to three consecutive files from a level
@@ -3420,9 +3725,10 @@ void StressTest::TestCompactFiles(ThreadState* thread,
       } else {
         thread->stats.AddNumCompactFilesSucceed(1);
       }
-      break;
+      return s;
     }
   }
+  return Status::OK();
 }
 
 void StressTest::TestPromoteL0(ThreadState* thread,
@@ -3612,9 +3918,9 @@ Status StressTest::MaybeReleaseSnapshots(ThreadState* thread, uint64_t i) {
   return Status::OK();
 }
 
-void StressTest::TestCompactRange(ThreadState* thread, int64_t rand_key,
-                                  const Slice& start_key,
-                                  ColumnFamilyHandle* column_family) {
+Status StressTest::TestCompactRange(ThreadState* thread, int64_t rand_key,
+                                    const Slice& start_key,
+                                    ColumnFamilyHandle* column_family) {
   int64_t end_key_num;
   if (std::numeric_limits<int64_t>::max() - rand_key <
       FLAGS_compact_range_width) {
@@ -3729,6 +4035,7 @@ void StressTest::TestCompactRange(ThreadState* thread, int64_t rand_key,
       db_fault_injection_fs_->EnableAllThreadLocalErrorInjection();
     }
   }
+  return status;
 }
 
 uint32_t StressTest::GetRangeHash(ThreadState* thread, const Snapshot* snapshot,
@@ -4394,7 +4701,7 @@ void StressTest::Open(SharedState* shared, bool reopen) {
 
   if (!s.ok()) {
     fprintf(stderr, "open error: %s\n", s.ToString().c_str());
-    exit(1);
+    port::ImmediateExit(1);
   }
 
   if (db_->GetLatestSequenceNumber() < shared->GetPersistedSeqno()) {
@@ -4403,7 +4710,7 @@ void StressTest::Open(SharedState* shared, bool reopen) {
             "did not recover to the persisted "
             "sequence number %" PRIu64 " from last DB session\n",
             db_->GetLatestSequenceNumber(), shared->GetPersistedSeqno());
-    exit(1);
+    port::ImmediateExit(1);
   }
 }
 
@@ -5039,6 +5346,8 @@ void InitializeOptionsFromFlags(
   options.allow_mmap_reads = FLAGS_mmap_read;
   options.allow_mmap_writes = FLAGS_mmap_write;
   options.use_direct_reads = FLAGS_use_direct_reads;
+  options.use_direct_io_for_compaction_reads =
+      FLAGS_use_direct_io_for_compaction_reads;
   options.use_direct_io_for_flush_and_compaction =
       FLAGS_use_direct_io_for_flush_and_compaction;
   options.recycle_log_file_num =
@@ -5126,8 +5435,8 @@ void InitializeOptionsFromFlags(
   options.verify_output_flags =
       static_cast<VerifyOutputFlags>(FLAGS_verify_output_flags);
   options.paranoid_memory_checks = FLAGS_paranoid_memory_checks;
-  options.memtable_veirfy_per_key_checksum_on_seek =
-      FLAGS_memtable_veirfy_per_key_checksum_on_seek;
+  options.memtable_verify_per_key_checksum_on_seek =
+      FLAGS_memtable_verify_per_key_checksum_on_seek;
   options.memtable_batch_lookup_optimization =
       FLAGS_memtable_batch_lookup_optimization;
 

db_stress_tool/db_stress_test_base.h

@@ -252,10 +252,10 @@ class StressTest {
       const std::vector<int64_t>& rand_keys) = 0;
 
   // Issue compact range, starting with start_key, whose integer value
-  // is rand_key.
-  virtual void TestCompactRange(ThreadState* thread, int64_t rand_key,
-                                const Slice& start_key,
-                                ColumnFamilyHandle* column_family);
+  // is rand_key. Returns the CompactRange() status.
+  virtual Status TestCompactRange(ThreadState* thread, int64_t rand_key,
+                                  const Slice& start_key,
+                                  ColumnFamilyHandle* column_family);
 
   virtual void TestPromoteL0(ThreadState* thread,
                              ColumnFamilyHandle* column_family);
@@ -378,7 +378,9 @@ class StressTest {
                                 const std::vector<int>& rand_column_families,
                                 const std::vector<int64_t>& rand_keys);
 
-  void TestCompactFiles(ThreadState* thread, ColumnFamilyHandle* column_family);
+  // Returns the CompactFiles() status (OK if no compaction was performed).
+  Status TestCompactFiles(ThreadState* thread,
+                          ColumnFamilyHandle* column_family);
 
   Status TestFlush(const std::vector<int>& rand_column_families);
 
@@ -433,6 +435,13 @@ class StressTest {
   void VerificationAbort(SharedState* shared, int cf, int64_t key,
                          const Slice& value, const WideColumns& columns) const;
 
+  // Under --verify_cpu_corruption_dir (see that flag's comment for behavior and
+  // the result-file contract), verifies the just-run op (named by `op_label`,
+  // e.g. "put"/"flush"/"compactrange") for a returned/read-back corruption or a
+  // silent data corruption. A no-op when the flag is empty.
+  void MaybeVerifyCpuCorruption(ThreadState* thread, const char* op_label,
+                                const Status& op_status);
+
   static std::string DebugString(const Slice& value,
                                  const WideColumns& columns);
 

db_stress_tool/db_stress_tool.cc

@@ -176,6 +176,45 @@ int db_stress_tool(int argc, char** argv) {
     }
   }
 
+  if (!FLAGS_verify_cpu_corruption_dir.empty()) {
+    // The full-keyspace read-back is only well-defined with a single writer,
+    // and injected I/O faults would taint it -- so require --threads=1 and all
+    // fault injection off (see the flag's comment).
+    if (FLAGS_threads != 1) {
+      fprintf(stderr,
+              "Error: --verify_cpu_corruption_dir requires --threads=1\n");
+      exit(1);  // NOLINT(concurrency-mt-unsafe)
+    }
+    if (FLAGS_read_fault_one_in != 0 || FLAGS_write_fault_one_in != 0 ||
+        FLAGS_metadata_read_fault_one_in != 0 ||
+        FLAGS_metadata_write_fault_one_in != 0 ||
+        FLAGS_open_read_fault_one_in != 0 ||
+        FLAGS_open_write_fault_one_in != 0 ||
+        FLAGS_open_metadata_read_fault_one_in != 0 ||
+        FLAGS_open_metadata_write_fault_one_in != 0 ||
+        FLAGS_secondary_cache_fault_one_in != 0 || FLAGS_sync_fault_injection) {
+      fprintf(stderr,
+              "Error: --verify_cpu_corruption_dir requires all fault injection "
+              "off (every *_fault_one_in = 0 and sync_fault_injection = "
+              "false)\n");
+      exit(1);  // NOLINT(concurrency-mt-unsafe)
+    }
+    // The read-back compares against the expected-state model, which only the
+    // default (state-tracked) stress test maintains. The
+    // batched/cf-consistency/ multi-ops-txns variants set
+    // IsStateTracked()=false, so every key would look absent -- reject them
+    // rather than report false losses.
+    if (FLAGS_test_batches_snapshots || FLAGS_test_cf_consistency ||
+        FLAGS_test_multi_ops_txns) {
+      fprintf(stderr,
+              "Error: --verify_cpu_corruption_dir requires the default "
+              "state-tracked stress test; it is incompatible with "
+              "--test_batches_snapshots, --test_cf_consistency, and "
+              "--test_multi_ops_txns\n");
+      exit(1);  // NOLINT(concurrency-mt-unsafe)
+    }
+  }
+
   FLAGS_rep_factory = StringToRepFactory(FLAGS_memtablerep.c_str());
 
   // The number of background threads should be at least as much the

db_stress_tool/multi_ops_txns_stress.cc

@@ -481,7 +481,7 @@ void MultiOpsTxnsStressTest::TestIngestExternalFile(
   (void)rand_column_families;
 }
 
-void MultiOpsTxnsStressTest::TestCompactRange(
+Status MultiOpsTxnsStressTest::TestCompactRange(
     ThreadState* thread, int64_t /*rand_key*/, const Slice& /*start_key*/,
     ColumnFamilyHandle* column_family) {
   // TODO (yanqin).
@@ -489,6 +489,7 @@ void MultiOpsTxnsStressTest::TestCompactRange(
   // completes.
   (void)thread;
   (void)column_family;
+  return Status::OK();
 }
 
 Status MultiOpsTxnsStressTest::TestBackupRestore(

db_stress_tool/multi_ops_txns_stress.h

@@ -253,9 +253,9 @@ class MultiOpsTxnsStressTest : public StressTest {
                               const std::vector<int>& rand_column_families,
                               const std::vector<int64_t>& rand_keys) override;
 
-  void TestCompactRange(ThreadState* thread, int64_t rand_key,
-                        const Slice& start_key,
-                        ColumnFamilyHandle* column_family) override;
+  Status TestCompactRange(ThreadState* thread, int64_t rand_key,
+                          const Slice& start_key,
+                          ColumnFamilyHandle* column_family) override;
 
   Status TestBackupRestore(ThreadState* thread,
                            const std::vector<int>& rand_column_families,

db_stress_tool/no_batched_ops_stress.cc

@@ -14,6 +14,7 @@
 #include "rocksdb/status.h"
 #ifdef GFLAGS
 #include <cinttypes>
+#include <deque>
 #include <unordered_map>
 
 #include "db/wide/wide_columns_helper.h"
@@ -2326,60 +2327,16 @@ class NonBatchedOpsStressTest : public StressTest {
     // deletion file's compaction input optimization.
     bool test_standalone_range_deletion = thread->rand.OneInOpt(
         FLAGS_test_ingest_standalone_range_deletion_one_in);
-    std::vector<std::string> external_files;
-    const std::string sst_filename =
-        GetDbPath() + "/." + std::to_string(thread->tid) + ".sst";
-    external_files.push_back(sst_filename);
-    std::string standalone_rangedel_filename;
-    if (test_standalone_range_deletion) {
-      standalone_rangedel_filename = GetDbPath() + "/." +
-                                     std::to_string(thread->tid) +
-                                     "_standalone_rangedel.sst";
-      external_files.push_back(standalone_rangedel_filename);
-    }
+    // When true, reuse the writer's metadata via IngestExternalFileArg's
+    // file_infos so ingestion skips re-opening and scanning the file. Not
+    // combined with the standalone range deletion mode (a range-del-only file).
+    bool use_file_info =
+        !test_standalone_range_deletion &&
+        thread->rand.OneInOpt(FLAGS_ingest_external_file_use_file_info_one_in);
+
     Status s;
     std::ostringstream ingest_options_oss;
 
-    // Temporarily disable error injection for preparation
-    if (db_fault_injection_fs_) {
-      db_fault_injection_fs_->DisableThreadLocalErrorInjection(
-          FaultInjectionIOType::kMetadataRead);
-      db_fault_injection_fs_->DisableThreadLocalErrorInjection(
-          FaultInjectionIOType::kMetadataWrite);
-    }
-
-    for (const auto& filename : external_files) {
-      if (raw_env->FileExists(filename).ok()) {
-        // Maybe we terminated abnormally before, so cleanup to give this file
-        // ingestion a clean slate
-        s = raw_env->DeleteFile(filename);
-      }
-      if (!s.ok()) {
-        return;
-      }
-    }
-
-    if (db_fault_injection_fs_) {
-      db_fault_injection_fs_->EnableThreadLocalErrorInjection(
-          FaultInjectionIOType::kMetadataRead);
-      db_fault_injection_fs_->EnableThreadLocalErrorInjection(
-          FaultInjectionIOType::kMetadataWrite);
-    }
-
-    SstFileWriter sst_file_writer(EnvOptions(options_), options_);
-    SstFileWriter standalone_rangedel_sst_file_writer(EnvOptions(options_),
-                                                      options_);
-    if (s.ok()) {
-      s = sst_file_writer.Open(sst_filename);
-    }
-    if (s.ok() && test_standalone_range_deletion) {
-      s = standalone_rangedel_sst_file_writer.Open(
-          standalone_rangedel_filename);
-    }
-    if (!s.ok()) {
-      return;
-    }
-
     int64_t key_base = rand_keys[0];
     int column_family = rand_column_families[0];
     std::vector<std::unique_ptr<MutexLock>> range_locks;
@@ -2428,13 +2385,76 @@ class NonBatchedOpsStressTest : public StressTest {
       }
     }
 
-    if (s.ok() && keys.empty()) {
+    if (keys.empty()) {
+      return;
+    }
+    size_t total_keys = keys.size();
+
+    const size_t data_file_count =
+        std::min<size_t>(1 + thread->rand.Uniform(3), total_keys);
+    std::vector<std::string> external_files;
+    std::vector<std::string> data_filenames;
+    data_filenames.reserve(data_file_count);
+    for (size_t file_idx = 0; file_idx < data_file_count; ++file_idx) {
+      data_filenames.push_back(GetDbPath() + "/." +
+                               std::to_string(thread->tid) + "_" +
+                               std::to_string(file_idx) + ".sst");
+      external_files.push_back(data_filenames.back());
+    }
+    std::string standalone_rangedel_filename;
+    if (test_standalone_range_deletion) {
+      standalone_rangedel_filename = GetDbPath() + "/." +
+                                     std::to_string(thread->tid) +
+                                     "_standalone_rangedel.sst";
+      external_files.push_back(standalone_rangedel_filename);
+    }
+
+    // Temporarily disable error injection for preparation
+    if (db_fault_injection_fs_) {
+      db_fault_injection_fs_->DisableThreadLocalErrorInjection(
+          FaultInjectionIOType::kMetadataRead);
+      db_fault_injection_fs_->DisableThreadLocalErrorInjection(
+          FaultInjectionIOType::kMetadataWrite);
+    }
+
+    for (const auto& filename : external_files) {
+      if (raw_env->FileExists(filename).ok()) {
+        // Maybe we terminated abnormally before, so cleanup to give this file
+        // ingestion a clean slate
+        s = raw_env->DeleteFile(filename);
+      }
+      if (!s.ok()) {
+        return;
+      }
+    }
+
+    if (db_fault_injection_fs_) {
+      db_fault_injection_fs_->EnableThreadLocalErrorInjection(
+          FaultInjectionIOType::kMetadataRead);
+      db_fault_injection_fs_->EnableThreadLocalErrorInjection(
+          FaultInjectionIOType::kMetadataWrite);
+    }
+
+    std::vector<ExternalSstFileInfo> file_infos(data_file_count);
+    std::deque<SstFileWriter> sst_file_writers;
+    for (size_t file_idx = 0; s.ok() && file_idx < data_file_count;
+         ++file_idx) {
+      sst_file_writers.emplace_back(EnvOptions(options_), options_);
+      s = sst_file_writers.back().Open(data_filenames[file_idx]);
+    }
+    SstFileWriter standalone_rangedel_sst_file_writer(EnvOptions(options_),
+                                                      options_);
+    if (s.ok() && test_standalone_range_deletion) {
+      s = standalone_rangedel_sst_file_writer.Open(
+          standalone_rangedel_filename);
+    }
+    if (!s.ok()) {
       return;
     }
 
     // set pending state on expected values, create and ingest files.
-    size_t total_keys = keys.size();
     for (size_t i = 0; s.ok() && i < total_keys; i++) {
+      auto& sst_file_writer = sst_file_writers[i % sst_file_writers.size()];
       int64_t key = keys.at(i);
       char value[100];
       auto key_str = Key(key);
@@ -2462,7 +2482,9 @@ class NonBatchedOpsStressTest : public StressTest {
       }
     }
     if (s.ok() && !keys.empty()) {
-      s = sst_file_writer.Finish();
+      for (size_t i = 0; s.ok() && i < sst_file_writers.size(); i++) {
+        s = sst_file_writers[i].Finish(&file_infos[i]);
+      }
     }
 
     if (s.ok() && total_keys != 0 && test_standalone_range_deletion) {
@@ -2480,6 +2502,7 @@ class NonBatchedOpsStressTest : public StressTest {
         s = standalone_rangedel_sst_file_writer.Finish();
       }
     }
+    bool dropped_without_commit = false;
     if (s.ok()) {
       IngestExternalFileOptions ingest_options;
       ingest_options.move_files = thread->rand.OneInOpt(2);
@@ -2487,24 +2510,91 @@ class NonBatchedOpsStressTest : public StressTest {
       ingest_options.verify_checksums_readahead_size =
           thread->rand.OneInOpt(2) ? 1024 * 1024 : 0;
       ingest_options.fill_cache = thread->rand.OneInOpt(4);
+      ingest_options.file_opening_threads = 1 + thread->rand.Uniform(4);
+      const bool use_prepare_commit = thread->rand.OneInOpt(
+          FLAGS_ingest_external_file_prepare_commit_one_in);
+      const bool use_separate_prepare_calls = use_prepare_commit &&
+                                              external_files.size() > 1 &&
+                                              thread->rand.OneInOpt(2);
       ingest_options_oss << "move_files: " << ingest_options.move_files
                          << ", verify_checksums_before_ingest: "
                          << ingest_options.verify_checksums_before_ingest
                          << ", verify_checksums_readahead_size: "
                          << ingest_options.verify_checksums_readahead_size
                          << ", fill_cache: " << ingest_options.fill_cache
+                         << ", file_opening_threads: "
+                         << ingest_options.file_opening_threads
+                         << ", ingest_external_file_data_file_count: "
+                         << data_file_count
+                         << ", num_external_files: " << external_files.size()
                          << ", test_standalone_range_deletion: "
-                         << test_standalone_range_deletion;
-      s = db_->IngestExternalFile(column_families_[column_family],
-                                  external_files, ingest_options);
+                         << test_standalone_range_deletion
+                         << ", use_prepare_commit: " << use_prepare_commit
+                         << ", use_separate_prepare_calls: "
+                         << use_separate_prepare_calls
+                         << ", use_file_info: " << use_file_info;
+      IngestExternalFileArg arg;
+      arg.column_family = column_families_[column_family];
+      arg.external_files = external_files;
+      arg.options = ingest_options;
+      if (use_file_info) {
+        for (const auto& file_info : file_infos) {
+          arg.file_infos.push_back(file_info.prepared_file_info.get());
+        }
+      }
+      if (use_prepare_commit) {
+        std::vector<std::unique_ptr<FileIngestionHandle>> handles;
+        handles.reserve(use_separate_prepare_calls ? external_files.size() : 1);
+        if (use_separate_prepare_calls) {
+          for (const auto& external_file : external_files) {
+            IngestExternalFileArg file_arg;
+            file_arg.column_family = column_families_[column_family];
+            file_arg.external_files = {external_file};
+            file_arg.options = ingest_options;
+            std::unique_ptr<FileIngestionHandle> handle;
+            s = db_->PrepareFileIngestion({file_arg}, &handle);
+            if (!s.ok()) {
+              break;
+            }
+            handles.push_back(std::move(handle));
+          }
+        } else {
+          std::unique_ptr<FileIngestionHandle> handle;
+          s = db_->PrepareFileIngestion({arg}, &handle);
+          if (s.ok()) {
+            handles.push_back(std::move(handle));
+          }
+        }
+        if (s.ok()) {
+          // Occasionally cancel instead of committing, covering both rollback
+          // paths.
+          if (thread->rand.OneInOpt(4)) {
+            if (thread->rand.OneInOpt(2)) {
+              for (auto& handle : handles) {
+                Status abort_status = handle->Abort();
+                if (!abort_status.ok() && s.ok()) {
+                  s = abort_status;
+                }
+              }
+            } else {
+              handles.clear();  // RAII rollback via destructors
+            }
+            dropped_without_commit = true;
+          } else {
+            s = db_->CommitFileIngestionHandles(std::move(handles));
+          }
+        }
+      } else {
+        s = db_->IngestExternalFiles({arg});
+      }
     }
-    if (!s.ok()) {
+    if (!s.ok() || dropped_without_commit) {
       for (PendingExpectedValue& pending_expected_value :
            pending_expected_values) {
         pending_expected_value.Rollback();
       }
 
-      if (!IsErrorInjectedAndRetryable(s)) {
+      if (!s.ok() && !IsErrorInjectedAndRetryable(s)) {
         fprintf(stderr,
                 "file ingestion error: %s under specified "
                 "IngestExternalFileOptions: %s (Empty string or "

env/composite_env_wrapper.h

@@ -142,6 +142,13 @@ class CompositeEnv : public Env {
     return file_system_->LinkFile(s, t, io_opts, &dbg);
   }
 
+  Status SyncFile(const std::string& fname, const EnvOptions& env_options,
+                  bool use_fsync) override {
+    IODebugContext dbg;
+    return file_system_->SyncFile(fname, FileOptions(env_options), IOOptions(),
+                                  use_fsync, &dbg);
+  }
+
   Status NumFileLinks(const std::string& fname, uint64_t* count) override {
     IOOptions io_opts;
     IODebugContext dbg;

env/env.cc

@@ -26,6 +26,7 @@
 #include "rocksdb/utilities/customizable_util.h"
 #include "rocksdb/utilities/object_registry.h"
 #include "rocksdb/utilities/options_type.h"
+#include "test_util/sync_point.h"
 #include "util/autovector.h"
 #include "util/string_util.h"
 
@@ -530,6 +531,13 @@ class LegacyFileSystemWrapper : public FileSystem {
     return status_to_io_status(target_->LinkFile(s, t));
   }
 
+  IOStatus SyncFile(const std::string& fname, const FileOptions& file_options,
+                    const IOOptions& /*io_options*/, bool use_fsync,
+                    IODebugContext* /*dbg*/) override {
+    return status_to_io_status(
+        target_->SyncFile(fname, file_options, use_fsync));
+  }
+
   IOStatus NumFileLinks(const std::string& fname, const IOOptions& /*options*/,
                         uint64_t* count, IODebugContext* /*dbg*/) override {
     return status_to_io_status(target_->NumFileLinks(fname, count));
@@ -796,6 +804,23 @@ Status Env::ReuseWritableFile(const std::string& fname,
   return NewWritableFile(fname, result, options);
 }
 
+Status Env::SyncFile(const std::string& fname, const EnvOptions& options,
+                     bool use_fsync) {
+  std::unique_ptr<WritableFile> file_to_sync;
+  Status status = ReopenWritableFile(fname, &file_to_sync, options);
+  TEST_SYNC_POINT_CALLBACK("Env::SyncFile:Open", &status);
+  if (status.ok()) {
+    status = use_fsync ? file_to_sync->Fsync() : file_to_sync->Sync();
+    Status close_status = file_to_sync->Close();
+    if (status.ok()) {
+      status = close_status;
+    } else {
+      close_status.PermitUncheckedError();
+    }
+  }
+  return status;
+}
+
 Status Env::GetChildrenFileAttributes(const std::string& dir,
                                       std::vector<FileAttributes>* result) {
   assert(result != nullptr);

env/env_encryption.cc

@@ -814,6 +814,16 @@ class EncryptedFileSystemImpl : public EncryptedFileSystem {
     return status;
   }
 
+  IOStatus SyncFile(const std::string& fname, const FileOptions& file_opts,
+                    const IOOptions& io_opts, bool use_fsync,
+                    IODebugContext* dbg) override {
+    // SyncFile does not read or write file contents, so it can delegate
+    // directly to the underlying filesystem without constructing an encrypted
+    // writable wrapper.
+    return FileSystemWrapper::SyncFile(fname, file_opts, io_opts, use_fsync,
+                                       dbg);
+  }
+
  private:
   std::shared_ptr<EncryptionProvider> provider_;
 };

env/env_test.cc

@@ -295,6 +295,46 @@ class EnvPosixTestWithParam
     }
   }
 
+  // ReserveThreads() returns the number of threads observed waiting at that
+  // instant. When this test runs in parallel with many other CPU-heavy tests,
+  // worker-thread scheduling can vary enough that the sync points prove key
+  // transitions were reached before waiting-thread accounting has fully
+  // settled. Use this helper only for positive exact-reservation checks; it
+  // releases any partial reservation before retrying so the retry does not
+  // perturb test state.
+  testing::AssertionResult ReserveThreadsEventually(int expected, int requested,
+                                                    Env::Priority priority,
+                                                    int wait_micros) {
+    constexpr int kRetryMicros = 1000;
+    int last_reserved = -1;
+    for (int waited_micros = 0; waited_micros <= wait_micros;
+         waited_micros += kRetryMicros) {
+      int reserved = env_->ReserveThreads(requested, priority);
+      if (reserved == expected) {
+        return testing::AssertionSuccess();
+      }
+      if (reserved > 0) {
+        int released = env_->ReleaseThreads(reserved, priority);
+        if (released != reserved) {
+          return testing::AssertionFailure()
+                 << "ReserveThreads(" << requested << ") returned " << reserved
+                 << ", but ReleaseThreads(" << reserved << ") released "
+                 << released;
+        }
+      }
+      if (reserved > expected) {
+        return testing::AssertionFailure()
+               << "ReserveThreads(" << requested << ") returned " << reserved
+               << ", more than expected " << expected;
+      }
+      last_reserved = reserved;
+      Env::Default()->SleepForMicroseconds(kRetryMicros);
+    }
+    return testing::AssertionFailure()
+           << "ReserveThreads(" << requested << ") returned " << last_reserved
+           << " after waiting " << wait_micros << "us, expected " << expected;
+  }
+
   ~EnvPosixTestWithParam() override { WaitThreadPoolsEmpty(); }
 };
 
@@ -1022,7 +1062,7 @@ TEST_P(EnvPosixTestWithParam, ReserveThreads) {
   TEST_SYNC_POINT("EnvTest::ReserveThreads:2");
   TEST_SYNC_POINT("EnvTest::ReserveThreads:3");
   // Reserve 2 threads
-  ASSERT_EQ(2, env_->ReserveThreads(2, Env::Priority::HIGH));
+  ASSERT_TRUE(ReserveThreadsEventually(2, 2, Env::Priority::HIGH, kWaitMicros));
 
   // Schedule 3 tasks. Task 0 running (in this context, doing
   // SleepingBackgroundTask); Task 1, 2 waiting; 3 reserved threads.
@@ -1051,7 +1091,7 @@ TEST_P(EnvPosixTestWithParam, ReserveThreads) {
   // Add sync point to ensure the 4th thread starts
   TEST_SYNC_POINT("EnvTest::ReserveThreads:4");
   // As the thread pool is expanded, we can reserve one more thread
-  ASSERT_EQ(1, env_->ReserveThreads(3, Env::Priority::HIGH));
+  ASSERT_TRUE(ReserveThreadsEventually(1, 3, Env::Priority::HIGH, kWaitMicros));
   // No more threads can be reserved
   ASSERT_EQ(0, env_->ReserveThreads(3, Env::Priority::HIGH));
 
@@ -1070,7 +1110,7 @@ TEST_P(EnvPosixTestWithParam, ReserveThreads) {
   // Add sync point to ensure the number of waiting threads increases
   TEST_SYNC_POINT("EnvTest::ReserveThreads:5");
   // 1 more thread can be reserved
-  ASSERT_EQ(1, env_->ReserveThreads(3, Env::Priority::HIGH));
+  ASSERT_TRUE(ReserveThreadsEventually(1, 3, Env::Priority::HIGH, kWaitMicros));
   // 2 reserved threads now
 
   // Currently, two threads are blocked since the number of waiting
@@ -4125,10 +4165,12 @@ TEST_F(TestAsyncRead, ReadAsyncQueueFull) {
     std::unique_ptr<FSRandomAccessFile> file;
     ASSERT_OK(fs->NewRandomAccessFile(fname, FileOptions(), &file, nullptr));
 
-    // Force io_uring_get_sqe to appear to return null via SyncPoint.
+    // Force the queue-full path without consuming an SQ slot. Overwriting the
+    // SQE pointer after io_uring_get_sqe() would leave a stale submission in
+    // the ring and pollute later tests using the same thread-local io_uring.
     SyncPoint::GetInstance()->SetCallBack(
-        "PosixRandomAccessFile::ReadAsync:io_uring_get_sqe",
-        [](void* arg) { *static_cast<io_uring_sqe**>(arg) = nullptr; });
+        "PosixRandomAccessFile::ReadAsync:skip_io_uring_get_sqe",
+        [](void* arg) { *static_cast<bool*>(arg) = true; });
     SyncPoint::GetInstance()->EnableProcessing();
 
     IOOptions opts;
@@ -4613,6 +4655,344 @@ TEST_F(EnvTest, WriteStringToFileClosesFile) {
   ASSERT_OK(counted_fs->DeleteFile(fname, IOOptions(), nullptr));
 }
 
+enum class SyncFileTestResult {
+  kOk,
+  kNotSupported,
+  kSyncError,
+  kFsyncError,
+  kCloseError,
+};
+
+Status MakeSyncFileTestStatus(SyncFileTestResult result) {
+  switch (result) {
+    case SyncFileTestResult::kOk:
+      return Status::OK();
+    case SyncFileTestResult::kNotSupported:
+      return Status::NotSupported("injected reopen failure");
+    case SyncFileTestResult::kSyncError:
+      return Status::IOError("injected sync failure");
+    case SyncFileTestResult::kFsyncError:
+      return Status::IOError("injected fsync failure");
+    case SyncFileTestResult::kCloseError:
+      return Status::IOError("injected close failure");
+  }
+  assert(false);
+  return Status::Corruption("unexpected sync file test result");
+}
+
+IOStatus MakeSyncFileTestIOStatus(SyncFileTestResult result) {
+  switch (result) {
+    case SyncFileTestResult::kOk:
+      return IOStatus::OK();
+    case SyncFileTestResult::kNotSupported:
+      return IOStatus::NotSupported("injected reopen failure");
+    case SyncFileTestResult::kSyncError:
+      return IOStatus::IOError("injected sync failure");
+    case SyncFileTestResult::kFsyncError:
+      return IOStatus::IOError("injected fsync failure");
+    case SyncFileTestResult::kCloseError:
+      return IOStatus::IOError("injected close failure");
+  }
+  assert(false);
+  return IOStatus::Corruption("unexpected sync file test result");
+}
+
+struct SyncFileTestState {
+  SyncFileTestResult reopen_result = SyncFileTestResult::kOk;
+  SyncFileTestResult sync_result = SyncFileTestResult::kOk;
+  SyncFileTestResult fsync_result = SyncFileTestResult::kOk;
+  SyncFileTestResult close_result = SyncFileTestResult::kOk;
+  int reopen_count = 0;
+  int sync_count = 0;
+  int fsync_count = 0;
+  int close_count = 0;
+};
+
+class SyncFileTestWritableFile : public WritableFileWrapper {
+ public:
+  SyncFileTestWritableFile(std::unique_ptr<WritableFile>&& target,
+                           SyncFileTestState* state)
+      : WritableFileWrapper(target.get()),
+        target_guard_(std::move(target)),
+        state_(state) {}
+
+  Status Sync() override {
+    ++state_->sync_count;
+    return MakeSyncFileTestStatus(state_->sync_result);
+  }
+
+  Status Fsync() override {
+    ++state_->fsync_count;
+    return MakeSyncFileTestStatus(state_->fsync_result);
+  }
+
+  Status Close() override {
+    ++state_->close_count;
+    if (state_->close_result == SyncFileTestResult::kOk) {
+      return WritableFileWrapper::Close();
+    }
+    Status status = WritableFileWrapper::Close();
+    status.PermitUncheckedError();
+    return MakeSyncFileTestStatus(state_->close_result);
+  }
+
+ private:
+  std::unique_ptr<WritableFile> target_guard_;
+  SyncFileTestState* state_;
+};
+
+class SyncFileTestEnv : public EnvWrapper {
+ public:
+  SyncFileTestEnv(Env* target, SyncFileTestState* state)
+      : EnvWrapper(target), state_(state) {}
+
+  Status ReopenWritableFile(const std::string& fname,
+                            std::unique_ptr<WritableFile>* result,
+                            const EnvOptions& options) override {
+    ++state_->reopen_count;
+    Status status = MakeSyncFileTestStatus(state_->reopen_result);
+    if (!status.ok()) {
+      return status;
+    }
+    status = target()->ReopenWritableFile(fname, result, options);
+    if (status.ok()) {
+      result->reset(new SyncFileTestWritableFile(std::move(*result), state_));
+    }
+    return status;
+  }
+
+  Status SyncFile(const std::string& fname, const EnvOptions& options,
+                  bool use_fsync) override {
+    return Env::SyncFile(fname, options, use_fsync);
+  }
+
+ private:
+  SyncFileTestState* state_;
+};
+
+class SyncFileTestFSWritableFile : public FSWritableFileOwnerWrapper {
+ public:
+  SyncFileTestFSWritableFile(std::unique_ptr<FSWritableFile>&& target,
+                             SyncFileTestState* state)
+      : FSWritableFileOwnerWrapper(std::move(target)), state_(state) {}
+
+  IOStatus Sync(const IOOptions& /*options*/,
+                IODebugContext* /*dbg*/) override {
+    ++state_->sync_count;
+    return MakeSyncFileTestIOStatus(state_->sync_result);
+  }
+
+  IOStatus Fsync(const IOOptions& /*options*/,
+                 IODebugContext* /*dbg*/) override {
+    ++state_->fsync_count;
+    return MakeSyncFileTestIOStatus(state_->fsync_result);
+  }
+
+  IOStatus Close(const IOOptions& options, IODebugContext* dbg) override {
+    ++state_->close_count;
+    if (state_->close_result == SyncFileTestResult::kOk) {
+      return FSWritableFileOwnerWrapper::Close(options, dbg);
+    }
+    IOStatus status = FSWritableFileOwnerWrapper::Close(options, dbg);
+    status.PermitUncheckedError();
+    return MakeSyncFileTestIOStatus(state_->close_result);
+  }
+
+ private:
+  SyncFileTestState* state_;
+};
+
+class SyncFileTestFileSystem : public FileSystemWrapper {
+ public:
+  SyncFileTestFileSystem(const std::shared_ptr<FileSystem>& target,
+                         SyncFileTestState* state)
+      : FileSystemWrapper(target), state_(state) {}
+
+  const char* Name() const override { return "SyncFileTestFileSystem"; }
+
+  IOStatus ReopenWritableFile(const std::string& fname,
+                              const FileOptions& file_opts,
+                              std::unique_ptr<FSWritableFile>* result,
+                              IODebugContext* dbg) override {
+    ++state_->reopen_count;
+    IOStatus status = MakeSyncFileTestIOStatus(state_->reopen_result);
+    if (!status.ok()) {
+      return status;
+    }
+    status = target()->ReopenWritableFile(fname, file_opts, result, dbg);
+    if (status.ok()) {
+      result->reset(new SyncFileTestFSWritableFile(std::move(*result), state_));
+    }
+    return status;
+  }
+
+  IOStatus SyncFile(const std::string& fname, const FileOptions& file_opts,
+                    const IOOptions& io_opts, bool use_fsync,
+                    IODebugContext* dbg) override {
+    return FileSystem::SyncFile(fname, file_opts, io_opts, use_fsync, dbg);
+  }
+
+ private:
+  SyncFileTestState* state_;
+};
+
+TEST_F(EnvTest, EnvSyncFileDefaultUsesSyncAndFsync) {
+  const std::string fname = test::PerThreadDBPath("env_sync_file_default");
+  ASSERT_OK(WriteStringToFile(Env::Default(), "sync-file-test", fname));
+
+  SyncFileTestState state;
+  SyncFileTestEnv env(Env::Default(), &state);
+
+  ASSERT_OK(env.SyncFile(fname, EnvOptions(), /*use_fsync=*/false));
+  ASSERT_EQ(state.reopen_count, 1);
+  ASSERT_EQ(state.sync_count, 1);
+  ASSERT_EQ(state.fsync_count, 0);
+  ASSERT_EQ(state.close_count, 1);
+
+  ASSERT_OK(env.SyncFile(fname, EnvOptions(), /*use_fsync=*/true));
+  ASSERT_EQ(state.reopen_count, 2);
+  ASSERT_EQ(state.sync_count, 1);
+  ASSERT_EQ(state.fsync_count, 1);
+  ASSERT_EQ(state.close_count, 2);
+
+  ASSERT_OK(Env::Default()->DeleteFile(fname));
+}
+
+TEST_F(EnvTest, EnvSyncFileDefaultReturnsReopenError) {
+  SyncFileTestState state;
+  state.reopen_result = SyncFileTestResult::kNotSupported;
+  SyncFileTestEnv env(Env::Default(), &state);
+
+  const Status status =
+      env.SyncFile("unused", EnvOptions(), /*use_fsync=*/false);
+  ASSERT_TRUE(status.IsNotSupported()) << status.ToString();
+  ASSERT_EQ(state.reopen_count, 1);
+  ASSERT_EQ(state.sync_count, 0);
+  ASSERT_EQ(state.fsync_count, 0);
+  ASSERT_EQ(state.close_count, 0);
+}
+
+TEST_F(EnvTest, EnvSyncFileDefaultReturnsCloseErrorAfterSuccessfulSync) {
+  const std::string fname = test::PerThreadDBPath("env_sync_file_close_error");
+  ASSERT_OK(WriteStringToFile(Env::Default(), "sync-file-test", fname));
+
+  SyncFileTestState state;
+  state.close_result = SyncFileTestResult::kCloseError;
+  SyncFileTestEnv env(Env::Default(), &state);
+
+  const Status status = env.SyncFile(fname, EnvOptions(), /*use_fsync=*/false);
+  ASSERT_TRUE(status.IsIOError()) << status.ToString();
+  ASSERT_NE(status.ToString().find("close failure"), std::string::npos)
+      << status.ToString();
+  ASSERT_EQ(state.reopen_count, 1);
+  ASSERT_EQ(state.sync_count, 1);
+  ASSERT_EQ(state.close_count, 1);
+
+  ASSERT_OK(Env::Default()->DeleteFile(fname));
+}
+
+TEST_F(EnvTest, EnvSyncFileDefaultReturnsSyncErrorBeforeCloseError) {
+  const std::string fname = test::PerThreadDBPath("env_sync_file_sync_error");
+  ASSERT_OK(WriteStringToFile(Env::Default(), "sync-file-test", fname));
+
+  SyncFileTestState state;
+  state.sync_result = SyncFileTestResult::kSyncError;
+  state.close_result = SyncFileTestResult::kCloseError;
+  SyncFileTestEnv env(Env::Default(), &state);
+
+  const Status status = env.SyncFile(fname, EnvOptions(), /*use_fsync=*/false);
+  ASSERT_TRUE(status.IsIOError()) << status.ToString();
+  ASSERT_NE(status.ToString().find("sync failure"), std::string::npos)
+      << status.ToString();
+  ASSERT_EQ(state.reopen_count, 1);
+  ASSERT_EQ(state.sync_count, 1);
+  ASSERT_EQ(state.close_count, 1);
+
+  ASSERT_OK(Env::Default()->DeleteFile(fname));
+}
+
+TEST_F(EnvTest, FileSystemSyncFileDefaultUsesSyncAndFsync) {
+  const std::string fname = test::PerThreadDBPath("fs_sync_file_default");
+  ASSERT_OK(
+      WriteStringToFile(FileSystem::Default().get(), "sync-file-test", fname));
+
+  SyncFileTestState state;
+  SyncFileTestFileSystem fs(FileSystem::Default(), &state);
+
+  ASSERT_OK(fs.SyncFile(fname, FileOptions(), IOOptions(), /*use_fsync=*/false,
+                        nullptr));
+  ASSERT_EQ(state.reopen_count, 1);
+  ASSERT_EQ(state.sync_count, 1);
+  ASSERT_EQ(state.fsync_count, 0);
+  ASSERT_EQ(state.close_count, 1);
+
+  ASSERT_OK(fs.SyncFile(fname, FileOptions(), IOOptions(), /*use_fsync=*/true,
+                        nullptr));
+  ASSERT_EQ(state.reopen_count, 2);
+  ASSERT_EQ(state.sync_count, 1);
+  ASSERT_EQ(state.fsync_count, 1);
+  ASSERT_EQ(state.close_count, 2);
+
+  ASSERT_OK(FileSystem::Default()->DeleteFile(fname, IOOptions(), nullptr));
+}
+
+TEST_F(EnvTest, FileSystemSyncFileDefaultReturnsReopenError) {
+  SyncFileTestState state;
+  state.reopen_result = SyncFileTestResult::kNotSupported;
+  SyncFileTestFileSystem fs(FileSystem::Default(), &state);
+
+  const IOStatus status = fs.SyncFile("unused", FileOptions(), IOOptions(),
+                                      /*use_fsync=*/false, nullptr);
+  ASSERT_TRUE(status.IsNotSupported()) << status.ToString();
+  ASSERT_EQ(state.reopen_count, 1);
+  ASSERT_EQ(state.sync_count, 0);
+  ASSERT_EQ(state.fsync_count, 0);
+  ASSERT_EQ(state.close_count, 0);
+}
+
+TEST_F(EnvTest, FileSystemSyncFileDefaultReturnsCloseErrorAfterSuccessfulSync) {
+  const std::string fname = test::PerThreadDBPath("fs_sync_file_close_error");
+  ASSERT_OK(
+      WriteStringToFile(FileSystem::Default().get(), "sync-file-test", fname));
+
+  SyncFileTestState state;
+  state.close_result = SyncFileTestResult::kCloseError;
+  SyncFileTestFileSystem fs(FileSystem::Default(), &state);
+
+  const IOStatus status = fs.SyncFile(fname, FileOptions(), IOOptions(),
+                                      /*use_fsync=*/false, nullptr);
+  ASSERT_TRUE(status.IsIOError()) << status.ToString();
+  ASSERT_NE(status.ToString().find("close failure"), std::string::npos)
+      << status.ToString();
+  ASSERT_EQ(state.reopen_count, 1);
+  ASSERT_EQ(state.sync_count, 1);
+  ASSERT_EQ(state.close_count, 1);
+
+  ASSERT_OK(FileSystem::Default()->DeleteFile(fname, IOOptions(), nullptr));
+}
+
+TEST_F(EnvTest, FileSystemSyncFileDefaultReturnsSyncErrorBeforeCloseError) {
+  const std::string fname = test::PerThreadDBPath("fs_sync_file_sync_error");
+  ASSERT_OK(
+      WriteStringToFile(FileSystem::Default().get(), "sync-file-test", fname));
+
+  SyncFileTestState state;
+  state.sync_result = SyncFileTestResult::kSyncError;
+  state.close_result = SyncFileTestResult::kCloseError;
+  SyncFileTestFileSystem fs(FileSystem::Default(), &state);
+
+  const IOStatus status = fs.SyncFile(fname, FileOptions(), IOOptions(),
+                                      /*use_fsync=*/false, nullptr);
+  ASSERT_TRUE(status.IsIOError()) << status.ToString();
+  ASSERT_NE(status.ToString().find("sync failure"), std::string::npos)
+      << status.ToString();
+  ASSERT_EQ(state.reopen_count, 1);
+  ASSERT_EQ(state.sync_count, 1);
+  ASSERT_EQ(state.close_count, 1);
+
+  ASSERT_OK(FileSystem::Default()->DeleteFile(fname, IOOptions(), nullptr));
+}
+
 // Writable file wrapper that injects a Close() failure.
 // Uses FSWritableFileOwnerWrapper to properly take ownership of the wrapped
 // file.

env/file_system.cc

@@ -16,6 +16,7 @@
 #include "rocksdb/utilities/customizable_util.h"
 #include "rocksdb/utilities/object_registry.h"
 #include "rocksdb/utilities/options_type.h"
+#include "test_util/sync_point.h"
 #include "util/string_util.h"
 #include "utilities/counted_fs.h"
 #include "utilities/env_timed.h"
@@ -107,6 +108,26 @@ IOStatus FileSystem::ReuseWritableFile(const std::string& fname,
   return NewWritableFile(fname, opts, result, dbg);
 }
 
+IOStatus FileSystem::SyncFile(const std::string& fname,
+                              const FileOptions& file_opts,
+                              const IOOptions& io_opts, bool use_fsync,
+                              IODebugContext* dbg) {
+  std::unique_ptr<FSWritableFile> file_to_sync;
+  IOStatus status = ReopenWritableFile(fname, file_opts, &file_to_sync, dbg);
+  TEST_SYNC_POINT_CALLBACK("FileSystem::SyncFile:Open", &status);
+  if (status.ok()) {
+    status = use_fsync ? file_to_sync->Fsync(io_opts, dbg)
+                       : file_to_sync->Sync(io_opts, dbg);
+    IOStatus close_status = file_to_sync->Close(io_opts, dbg);
+    if (status.ok()) {
+      status = close_status;
+    } else {
+      close_status.PermitUncheckedError();
+    }
+  }
+  return status;
+}
+
 IOStatus FileSystem::NewLogger(const std::string& fname,
                                const IOOptions& io_opts,
                                std::shared_ptr<Logger>* result,

env/fs_posix.cc

@@ -950,15 +950,14 @@ class PosixFileSystem : public FileSystem {
   FileOptions OptimizeForCompactionTableRead(
       const FileOptions& file_options,
       const ImmutableDBOptions& db_options) const override {
-    FileOptions fo = FileOptions(file_options);
+    FileOptions fo =
+        FileSystem::OptimizeForCompactionTableRead(file_options, db_options);
 #ifdef OS_LINUX
     // To fix https://github.com/facebook/rocksdb/issues/12038
-    if (!file_options.use_direct_reads &&
-        file_options.compaction_readahead_size > 0) {
+    if (!fo.use_direct_reads && fo.compaction_readahead_size > 0) {
       size_t system_limit =
           GetCompactionReadaheadSizeSystemLimit(db_options.db_paths);
-      if (system_limit > 0 &&
-          file_options.compaction_readahead_size > system_limit) {
+      if (system_limit > 0 && fo.compaction_readahead_size > system_limit) {
         fo.compaction_readahead_size = system_limit;
       }
     }

env/fs_readonly.h

@@ -89,6 +89,12 @@ class ReadOnlyFileSystem : public FileSystemWrapper {
                     IODebugContext* /*dbg*/) override {
     return FailReadOnly();
   }
+  IOStatus SyncFile(const std::string& /*fname*/,
+                    const FileOptions& /*file_opts*/,
+                    const IOOptions& /*io_opts*/, bool /*use_fsync*/,
+                    IODebugContext* /*dbg*/) override {
+    return FailReadOnly();
+  }
   IOStatus LockFile(const std::string& /*fname*/, const IOOptions& /*options*/,
                     FileLock** /*lock*/, IODebugContext* /*dbg*/) override {
     return FailReadOnly();

env/fs_remap.cc

@@ -309,6 +309,18 @@ IOStatus RemapFileSystem::LinkFile(const std::string& src,
                                      dbg);
 }
 
+IOStatus RemapFileSystem::SyncFile(const std::string& fname,
+                                   const FileOptions& file_opts,
+                                   const IOOptions& io_opts, bool use_fsync,
+                                   IODebugContext* dbg) {
+  auto status_and_enc_path = EncodePathWithNewBasename(fname);
+  if (!status_and_enc_path.first.ok()) {
+    return status_and_enc_path.first;
+  }
+  return FileSystemWrapper::SyncFile(status_and_enc_path.second, file_opts,
+                                     io_opts, use_fsync, dbg);
+}
+
 IOStatus RemapFileSystem::LockFile(const std::string& fname,
                                    const IOOptions& options, FileLock** lock,
                                    IODebugContext* dbg) {

env/fs_remap.h

@@ -125,6 +125,10 @@ class RemapFileSystem : public FileSystemWrapper {
   IOStatus LinkFile(const std::string& src, const std::string& dest,
                     const IOOptions& options, IODebugContext* dbg) override;
 
+  IOStatus SyncFile(const std::string& fname, const FileOptions& file_opts,
+                    const IOOptions& io_opts, bool use_fsync,
+                    IODebugContext* dbg) override;
+
   IOStatus LockFile(const std::string& fname, const IOOptions& options,
                     FileLock** lock, IODebugContext* dbg) override;
 

env/io_posix.cc

@@ -1116,10 +1116,15 @@ IOStatus PosixRandomAccessFile::ReadAsync(
   posix_handle->iov.iov_len = req.len;
 
   // Step 3: io_uring_sqe_set_data
-  struct io_uring_sqe* sqe;
-  sqe = io_uring_get_sqe(iu);
-  TEST_SYNC_POINT_CALLBACK("PosixRandomAccessFile::ReadAsync:io_uring_get_sqe",
-                           &sqe);
+  bool skip_get_sqe = false;
+  TEST_SYNC_POINT_CALLBACK(
+      "PosixRandomAccessFile::ReadAsync:skip_io_uring_get_sqe", &skip_get_sqe);
+  struct io_uring_sqe* sqe = nullptr;
+  if (!skip_get_sqe) {
+    sqe = io_uring_get_sqe(iu);
+    TEST_SYNC_POINT_CALLBACK(
+        "PosixRandomAccessFile::ReadAsync:io_uring_get_sqe", &sqe);
+  }
   if (sqe == nullptr) {
     // Submission queue is full, so outstanding completions have not been
     // reaped yet. Submission never succeeded, so clean up the local handle and

env/mock_env.cc

@@ -957,6 +957,13 @@ IOStatus MockFileSystem::LinkFile(const std::string& src,
   return IOStatus::OK();
 }
 
+IOStatus MockFileSystem::SyncFile(const std::string& /*fname*/,
+                                  const FileOptions& /*file_opts*/,
+                                  const IOOptions& /*io_opts*/,
+                                  bool /*use_fsync*/, IODebugContext* /*dbg*/) {
+  return IOStatus::OK();
+}
+
 IOStatus MockFileSystem::NewLogger(const std::string& fname,
                                    const IOOptions& io_opts,
                                    std::shared_ptr<Logger>* result,

env/mock_env.h

@@ -86,6 +86,10 @@ class MockFileSystem : public FileSystem {
   IOStatus LinkFile(const std::string& /*src*/, const std::string& /*target*/,
                     const IOOptions& /*options*/,
                     IODebugContext* /*dbg*/) override;
+  IOStatus SyncFile(const std::string& /*fname*/,
+                    const FileOptions& /*file_opts*/,
+                    const IOOptions& /*io_opts*/, bool /*use_fsync*/,
+                    IODebugContext* /*dbg*/) override;
   IOStatus LockFile(const std::string& fname, const IOOptions& options,
                     FileLock** lock, IODebugContext* dbg) override;
   IOStatus UnlockFile(FileLock* lock, const IOOptions& options,

file/file_prefetch_buffer.cc

@@ -103,7 +103,7 @@ Status FilePrefetchBuffer::Read(BufferInfo* buf, const IOOptions& opts,
   } else {
     to_buf = buf->buffer_.BufferStart() + aligned_useful_len;
     s = reader->Read(opts, start_offset + aligned_useful_len, read_len, &result,
-                     to_buf, /*aligned_buf=*/nullptr);
+                     to_buf);
   }
 
 #ifndef NDEBUG
@@ -165,7 +165,7 @@ Status FilePrefetchBuffer::ReadAsync(BufferInfo* buf, const IOOptions& opts,
     // Fall back to synchronous read so the buffer is populated inline
     // and callers proceed transparently.
     s = reader->Read(opts, start_offset, read_len, &result,
-                     buf->buffer_.BufferStart(), /*aligned_buf=*/nullptr);
+                     buf->buffer_.BufferStart());
     if (s.ok()) {
       buf->buffer_.Size(buf->CurrentSize() + result.size());
       if (usage_ == FilePrefetchBufferUsage::kUserScanPrefetch) {

file/file_prefetch_buffer.h

@@ -527,7 +527,9 @@ class FilePrefetchBuffer {
     read_req.offset = offset;
     read_req.len = n;
     read_req.scratch = nullptr;
-    IOStatus s = reader->MultiRead(opts, &read_req, 1, nullptr);
+    AlignedBuffer direct_io_buffer;
+    AlignedBufferAllocationContext direct_io_context{&direct_io_buffer};
+    IOStatus s = reader->MultiRead(opts, &read_req, 1, &direct_io_context);
     if (!s.ok()) {
       return s;
     }

file/random_access_file_reader.cc

@@ -16,6 +16,7 @@
 #include "monitoring/histogram.h"
 #include "monitoring/iostats_context_imp.h"
 #include "port/port.h"
+#include "rocksdb/io_status.h"
 #include "table/format.h"
 #include "test_util/sync_point.h"
 #include "util/random.h"
@@ -128,11 +129,17 @@ IOStatus RandomAccessFileReader::Create(
   return io_s;
 }
 
-IOStatus RandomAccessFileReader::Read(const IOOptions& opts, uint64_t offset,
-                                      size_t n, Slice* result, char* scratch,
-                                      AlignedBuf* aligned_buf,
-                                      IODebugContext* dbg) const {
-  (void)aligned_buf;
+IOStatus RandomAccessFileReader::Read(
+    const IOOptions& opts, uint64_t offset, size_t n, Slice* result,
+    char* scratch, AlignedBufferAllocationContext* direct_io_buffer_context,
+    IODebugContext* dbg) const {
+  AlignedBuffer* direct_io_buffer = direct_io_buffer_context != nullptr
+                                        ? direct_io_buffer_context->buffer
+                                        : nullptr;
+  const AlignedBuffer::Allocator* direct_io_allocator =
+      direct_io_buffer_context != nullptr ? direct_io_buffer_context->allocator
+                                          : nullptr;
+  assert(direct_io_buffer_context == nullptr || direct_io_buffer != nullptr);
   const Env::IOPriority rate_limiter_priority = opts.rate_limiter_priority;
 
   TEST_SYNC_POINT_CALLBACK("RandomAccessFileReader::Read", nullptr);
@@ -151,7 +158,7 @@ IOStatus RandomAccessFileReader::Read(const IOOptions& opts, uint64_t offset,
   uint64_t elapsed = 0;
   size_t alignment = file_->GetRequiredBufferAlignment();
   bool is_aligned = false;
-  if (scratch != nullptr) {
+  if (direct_io_buffer == nullptr && scratch != nullptr) {
     // Check if offset, length and buffer are aligned.
     is_aligned = (offset & (alignment - 1)) == 0 &&
                  (n & (alignment - 1)) == 0 &&
@@ -172,17 +179,25 @@ IOStatus RandomAccessFileReader::Read(const IOOptions& opts, uint64_t offset,
       size_t read_size =
           Roundup(static_cast<size_t>(offset + n), alignment) - aligned_offset;
       AlignedBuffer buf;
-      buf.Alignment(alignment);
-      buf.AllocateNewBuffer(read_size);
-      while (buf.CurrentSize() < read_size) {
+      AlignedBuffer* aligned_read_buffer =
+          direct_io_buffer != nullptr ? direct_io_buffer : &buf;
+      aligned_read_buffer->Alignment(alignment);
+      Status allocate_status = aligned_read_buffer->AllocateNewBuffer(
+          read_size, direct_io_allocator);
+      if (!allocate_status.ok()) {
+        io_s = status_to_io_status(std::move(allocate_status));
+      }
+      char* aligned_scratch = aligned_read_buffer->BufferStart();
+      size_t current_size = 0;
+      while (io_s.ok() && current_size < read_size) {
         size_t allowed;
         if (rate_limiter_priority != Env::IO_TOTAL &&
             rate_limiter_ != nullptr) {
           allowed = rate_limiter_->RequestToken(
-              buf.Capacity() - buf.CurrentSize(), buf.Alignment(),
-              rate_limiter_priority, stats_, RateLimiter::OpType::kRead);
+              read_size - current_size, alignment, rate_limiter_priority,
+              stats_, RateLimiter::OpType::kRead);
         } else {
-          assert(buf.CurrentSize() == 0);
+          assert(current_size == 0);
           allowed = read_size;
         }
         Slice tmp;
@@ -191,7 +206,7 @@ IOStatus RandomAccessFileReader::Read(const IOOptions& opts, uint64_t offset,
         uint64_t orig_offset = 0;
         if (ShouldNotifyListeners()) {
           start_ts = FileOperationInfo::StartNow();
-          orig_offset = aligned_offset + buf.CurrentSize();
+          orig_offset = aligned_offset + current_size;
         }
 
         {
@@ -201,8 +216,8 @@ IOStatus RandomAccessFileReader::Read(const IOOptions& opts, uint64_t offset,
           // one iteration of this loop, so we don't need to check and adjust
           // the opts.timeout before calling file_->Read
           assert(!opts.timeout.count() || allowed == read_size);
-          io_s = file_->Read(aligned_offset + buf.CurrentSize(), allowed, opts,
-                             &tmp, buf.Destination(), dbg);
+          io_s = file_->Read(aligned_offset + current_size, allowed, opts, &tmp,
+                             aligned_scratch + current_size, dbg);
         }
         if (ShouldNotifyListeners()) {
           auto finish_ts = FileOperationInfo::FinishNow();
@@ -214,19 +229,22 @@ IOStatus RandomAccessFileReader::Read(const IOOptions& opts, uint64_t offset,
           }
         }
 
-        buf.Size(buf.CurrentSize() + tmp.size());
+        current_size += tmp.size();
+        if (direct_io_buffer == nullptr) {
+          buf.Size(current_size);
+        }
         if (!io_s.ok() || tmp.size() < allowed) {
           break;
         }
       }
       size_t res_len = 0;
-      if (io_s.ok() && offset_advance < buf.CurrentSize()) {
-        res_len = std::min(buf.CurrentSize() - offset_advance, n);
-        if (aligned_buf == nullptr) {
-          buf.Read(scratch, offset_advance, res_len);
+      if (io_s.ok() && offset_advance < current_size) {
+        res_len = std::min(current_size - offset_advance, n);
+        if (direct_io_buffer != nullptr) {
+          scratch = aligned_scratch + offset_advance;
         } else {
-          scratch = buf.BufferStart() + offset_advance;
-          *aligned_buf = buf.Release();
+          assert(scratch != nullptr);
+          buf.Read(scratch, offset_advance, res_len);
         }
       }
       *result = Slice(scratch, res_len);
@@ -335,13 +353,18 @@ bool TryMerge(FSReadRequest* dest, const FSReadRequest& src) {
   return true;
 }
 
-IOStatus RandomAccessFileReader::MultiRead(const IOOptions& opts,
-                                           FSReadRequest* read_reqs,
-                                           size_t num_reqs,
-                                           AlignedBuf* aligned_buf,
-                                           IODebugContext* dbg) const {
-  (void)aligned_buf;  // suppress warning of unused variable in LITE mode
+IOStatus RandomAccessFileReader::MultiRead(
+    const IOOptions& opts, FSReadRequest* read_reqs, size_t num_reqs,
+    AlignedBufferAllocationContext* direct_io_buffer_context,
+    IODebugContext* dbg) const {
   assert(num_reqs > 0);
+  AlignedBuffer* direct_io_buffer = direct_io_buffer_context != nullptr
+                                        ? direct_io_buffer_context->buffer
+                                        : nullptr;
+  const AlignedBuffer::Allocator* direct_io_allocator =
+      direct_io_buffer_context != nullptr ? direct_io_buffer_context->allocator
+                                          : nullptr;
+  assert(direct_io_buffer != nullptr);
 
 #ifndef NDEBUG
   for (size_t i = 0; i < num_reqs - 1; ++i) {
@@ -403,16 +426,20 @@ IOStatus RandomAccessFileReader::MultiRead(const IOOptions& opts,
       for (const auto& r : aligned_reqs) {
         total_len += r.len;
       }
-      AlignedBuffer buf;
-      buf.Alignment(alignment);
-      buf.AllocateNewBuffer(total_len);
-      char* scratch = buf.BufferStart();
-      for (auto& r : aligned_reqs) {
-        r.scratch = scratch;
-        scratch += r.len;
+      direct_io_buffer->Alignment(alignment);
+      Status allocate_status =
+          direct_io_buffer->AllocateNewBuffer(total_len, direct_io_allocator);
+      if (!allocate_status.ok()) {
+        io_s = status_to_io_status(std::move(allocate_status));
+      }
+      if (io_s.ok()) {
+        char* scratch = direct_io_buffer->BufferStart();
+        for (auto& r : aligned_reqs) {
+          r.scratch = scratch;
+          scratch += r.len;
+        }
       }
 
-      *aligned_buf = buf.Release();
       fs_reqs = aligned_reqs.data();
       num_fs_reqs = aligned_reqs.size();
     }
@@ -422,7 +449,7 @@ IOStatus RandomAccessFileReader::MultiRead(const IOOptions& opts,
       start_ts = FileOperationInfo::StartNow();
     }
 
-    {
+    if (io_s.ok()) {
       IOSTATS_CPU_TIMER_GUARD(cpu_read_nanos, clock_);
       if (rate_limiter_priority != Env::IO_TOTAL && rate_limiter_ != nullptr) {
         // TODO: ideally we should call `RateLimiter::RequestToken()` for
@@ -455,7 +482,7 @@ IOStatus RandomAccessFileReader::MultiRead(const IOOptions& opts,
       RecordInHistogram(stats_, MULTIGET_IO_BATCH_SIZE, num_fs_reqs);
     }
 
-    if (use_direct_io()) {
+    if (use_direct_io() && io_s.ok()) {
       // Populate results in the unaligned read requests.
       size_t aligned_i = 0;
       for (size_t i = 0; i < num_reqs; i++) {
@@ -481,19 +508,20 @@ IOStatus RandomAccessFileReader::MultiRead(const IOOptions& opts,
       }
     }
 
+    const bool overall_io_ok = io_s.ok();
     for (size_t i = 0; i < num_reqs; ++i) {
+      const IOStatus& req_status = overall_io_ok ? read_reqs[i].status : io_s;
+      const size_t result_size = overall_io_ok ? read_reqs[i].result.size() : 0;
       if (ShouldNotifyListeners()) {
         auto finish_ts = FileOperationInfo::FinishNow();
-        NotifyOnFileReadFinish(read_reqs[i].offset, read_reqs[i].result.size(),
-                               start_ts, finish_ts, read_reqs[i].status);
+        NotifyOnFileReadFinish(read_reqs[i].offset, result_size, start_ts,
+                               finish_ts, req_status);
       }
-      if (!read_reqs[i].status.ok()) {
-        NotifyOnIOError(read_reqs[i].status, FileOperationType::kRead,
-                        file_name(), read_reqs[i].result.size(),
-                        read_reqs[i].offset);
+      if (!req_status.ok()) {
+        NotifyOnIOError(req_status, FileOperationType::kRead, file_name(),
+                        result_size, read_reqs[i].offset);
       }
-      RecordIOStats(stats_, file_temperature_, is_last_level_,
-                    read_reqs[i].result.size());
+      RecordIOStats(stats_, file_temperature_, is_last_level_, result_size);
     }
     SetPerfLevel(prev_perf_level);
   }
@@ -517,16 +545,26 @@ IOStatus RandomAccessFileReader::PrepareIOOptions(const ReadOptions& ro,
 
 // Notes for when direct_io is enabled:
 // Unless req.offset, req.len, req.scratch are all already aligned,
-// RandomAccessFileReader will creats aligned requests and aligned buffer for
-// the request. User should only provide either req.scratch or aligned_buf. If
-// only req.scratch is provided, result will be copied from allocated aligned
-// buffer to req.scratch. If only alignd_buf is provided, it will be set to
-// the ailgned buf allocated by RandomAccessFileReader and saves a copy.
+// RandomAccessFileReader creates an aligned request and aligned buffer for the
+// request. If direct_io_buffer_context is provided, its buffer owns the aligned
+// backing storage and its optional allocator is used only for this allocation.
+// Otherwise, callers should provide either req.scratch or aligned_buf. If only
+// req.scratch is provided, the result is copied from the allocated aligned
+// buffer to req.scratch. If only aligned_buf is provided, it is set to the
+// aligned buffer allocated by RandomAccessFileReader and saves a copy.
 IOStatus RandomAccessFileReader::ReadAsync(
     FSReadRequest& req, const IOOptions& opts,
     std::function<void(FSReadRequest&, void*)> cb, void* cb_arg,
     void** io_handle, IOHandleDeleter* del_fn, AlignedBuf* aligned_buf,
-    IODebugContext* dbg) {
+    IODebugContext* dbg,
+    AlignedBufferAllocationContext* direct_io_buffer_context) {
+  AlignedBuffer* direct_io_buffer = direct_io_buffer_context != nullptr
+                                        ? direct_io_buffer_context->buffer
+                                        : nullptr;
+  const AlignedBuffer::Allocator* direct_io_allocator =
+      direct_io_buffer_context != nullptr ? direct_io_buffer_context->allocator
+                                          : nullptr;
+  assert(direct_io_buffer_context == nullptr || direct_io_buffer != nullptr);
   IOStatus s;
   TEST_SYNC_POINT_CALLBACK("RandomAccessFileReader::ReadAsync:InjectStatus",
                            &s);
@@ -546,7 +584,8 @@ IOStatus RandomAccessFileReader::ReadAsync(
   }
 
   size_t alignment = file_->GetRequiredBufferAlignment();
-  bool is_aligned = (req.offset & (alignment - 1)) == 0 &&
+  bool is_aligned = direct_io_buffer == nullptr &&
+                    (req.offset & (alignment - 1)) == 0 &&
                     (req.len & (alignment - 1)) == 0 &&
                     (uintptr_t(req.scratch) & (alignment - 1)) == 0;
   read_async_info->is_aligned_ = is_aligned;
@@ -556,21 +595,27 @@ IOStatus RandomAccessFileReader::ReadAsync(
     FSReadRequest aligned_req = Align(req, alignment);
     aligned_req.status.PermitUncheckedError();
 
-    // Allocate aligned buffer.
-    read_async_info->buf_.Alignment(alignment);
-    read_async_info->buf_.AllocateNewBuffer(aligned_req.len);
-
-    // Set rem fields in aligned FSReadRequest.
-    aligned_req.scratch = read_async_info->buf_.BufferStart();
+    AlignedBuffer* aligned_read_buffer =
+        direct_io_buffer != nullptr ? direct_io_buffer : &read_async_info->buf_;
+    aligned_read_buffer->Alignment(alignment);
+    Status allocate_status = aligned_read_buffer->AllocateNewBuffer(
+        aligned_req.len, direct_io_allocator);
+    if (!allocate_status.ok()) {
+      delete read_async_info;
+      return status_to_io_status(std::move(allocate_status));
+    }
+    aligned_req.scratch = aligned_read_buffer->BufferStart();
 
     // Set user provided fields to populate back in callback.
     read_async_info->user_scratch_ = req.scratch;
     read_async_info->user_aligned_buf_ = aligned_buf;
+    read_async_info->direct_io_buffer_ = direct_io_buffer;
     read_async_info->user_len_ = req.len;
     read_async_info->user_offset_ = req.offset;
     read_async_info->user_result_ = req.result;
 
-    assert(read_async_info->buf_.CurrentSize() == 0);
+    assert(direct_io_buffer != nullptr ||
+           read_async_info->buf_.CurrentSize() == 0);
 
     StopWatch sw(clock_, stats_, hist_type_,
                  GetFileReadHistograms(stats_, opts.io_activity),
@@ -618,20 +663,25 @@ void RandomAccessFileReader::ReadAsyncCallback(FSReadRequest& req,
     user_req.result = req.result;
     user_req.status = req.status;
 
-    read_async_info->buf_.Size(read_async_info->buf_.CurrentSize() +
-                               req.result.size());
+    size_t current_size = req.result.size();
+    if (read_async_info->direct_io_buffer_ == nullptr) {
+      read_async_info->buf_.Size(read_async_info->buf_.CurrentSize() +
+                                 req.result.size());
+      current_size = read_async_info->buf_.CurrentSize();
+    }
 
     size_t offset_advance_len = static_cast<size_t>(
         /*offset_passed_by_user=*/read_async_info->user_offset_ -
         /*aligned_offset=*/req.offset);
 
     size_t res_len = 0;
-    if (req.status.ok() &&
-        offset_advance_len < read_async_info->buf_.CurrentSize()) {
-      res_len =
-          std::min(read_async_info->buf_.CurrentSize() - offset_advance_len,
-                   read_async_info->user_len_);
-      if (read_async_info->user_aligned_buf_ == nullptr) {
+    if (req.status.ok() && offset_advance_len < current_size) {
+      res_len = std::min(current_size - offset_advance_len,
+                         read_async_info->user_len_);
+      if (read_async_info->direct_io_buffer_ != nullptr) {
+        user_req.scratch = read_async_info->direct_io_buffer_->BufferStart() +
+                           offset_advance_len;
+      } else if (read_async_info->user_aligned_buf_ == nullptr) {
         // Copy the data into user's scratch.
 // Clang analyzer assumes that it will take use_direct_io() == false in
 // ReadAsync and use_direct_io() == true in Callback which cannot be true.

file/random_access_file_reader.h

@@ -9,6 +9,7 @@
 
 #pragma once
 #include <atomic>
+#include <functional>
 #include <sstream>
 #include <string>
 
@@ -27,6 +28,15 @@ class SystemClock;
 
 using AlignedBuf = FSAllocationPtr;
 
+struct AlignedBufferAllocationContext {
+  // `allocator` is intentionally not owned. RandomAccessFileReader invokes it
+  // only while allocating `buffer`, before Read/ReadAsync/MultiRead returns.
+  // For ReadAsync, `buffer` is still referenced by the completion callback and
+  // must outlive the async operation.
+  AlignedBuffer* buffer = nullptr;
+  const AlignedBuffer::Allocator* allocator = nullptr;
+};
+
 // Align the request r according to alignment and return the aligned result.
 FSReadRequest Align(const FSReadRequest& r, size_t alignment);
 
@@ -97,6 +107,7 @@ class RandomAccessFileReader {
           start_time_(start_time),
           user_scratch_(nullptr),
           user_aligned_buf_(nullptr),
+          direct_io_buffer_(nullptr),
           user_offset_(0),
           user_len_(0),
           is_aligned_(false) {}
@@ -108,6 +119,10 @@ class RandomAccessFileReader {
     // Below fields stores the parameters passed by caller in case of direct_io.
     char* user_scratch_;
     AlignedBuf* user_aligned_buf_;
+    // Raw pointer to caller-owned direct-I/O storage used when forming the
+    // user-visible result in ReadAsyncCallback. The caller must keep it alive
+    // until the async operation completes or is cancelled.
+    AlignedBuffer* direct_io_buffer_;
     uint64_t user_offset_;
     size_t user_len_;
     Slice user_result_;
@@ -157,23 +172,28 @@ class RandomAccessFileReader {
   // 1. if using mmap, result is stored in a buffer other than scratch;
   // 2. if not using mmap, result is stored in the buffer starting from scratch.
   //
-  // In direct IO mode, an aligned buffer is allocated internally.
-  // 1. If aligned_buf is null, then results are copied to the buffer
-  // starting from scratch;
-  // 2. Otherwise, scratch is not used and can be null, the aligned_buf owns
-  // the internally allocated buffer on return, and the result refers to a
-  // region in aligned_buf.
-  IOStatus Read(const IOOptions& opts, uint64_t offset, size_t n, Slice* result,
-                char* scratch, AlignedBuf* aligned_buf,
-                IODebugContext* dbg = nullptr) const;
+  // In direct IO mode, if direct_io_buffer_context is provided then it
+  // allocates the aligned buffer and the result refers to a region in
+  // direct_io_buffer_context->buffer. Otherwise, results are returned in
+  // scratch; unaligned reads use an internal aligned buffer and copy the
+  // requested subrange to scratch.
+  IOStatus Read(
+      const IOOptions& opts, uint64_t offset, size_t n, Slice* result,
+      char* scratch,
+      AlignedBufferAllocationContext* direct_io_buffer_context = nullptr,
+      IODebugContext* dbg = nullptr) const;
 
   // REQUIRES:
   // num_reqs > 0, reqs do not overlap, and offsets in reqs are increasing.
-  // In non-direct IO mode, aligned_buf should be null;
-  // In direct IO mode, aligned_buf stores the aligned buffer allocated inside
-  // MultiRead, the result Slices in reqs refer to aligned_buf.
+  // MultiRead uses direct_io_buffer_context to allocate the aligned buffer in
+  // direct IO mode. The result Slices in reqs refer to
+  // direct_io_buffer_context->buffer, so callers must keep it alive while those
+  // Slices are used. Callers should pass a default-constructed AlignedBuffer
+  // when default heap-backed allocation is sufficient. direct_io_buffer_context
+  // is ignored in non-direct IO mode.
   IOStatus MultiRead(const IOOptions& opts, FSReadRequest* reqs,
-                     size_t num_reqs, AlignedBuf* aligned_buf,
+                     size_t num_reqs,
+                     AlignedBufferAllocationContext* direct_io_buffer_context,
                      IODebugContext* dbg = nullptr) const;
 
   IOStatus Prefetch(const IOOptions& opts, uint64_t offset, size_t n,
@@ -190,10 +210,12 @@ class RandomAccessFileReader {
   IOStatus PrepareIOOptions(const ReadOptions& ro, IOOptions& opts,
                             IODebugContext* dbg = nullptr) const;
 
-  IOStatus ReadAsync(FSReadRequest& req, const IOOptions& opts,
-                     std::function<void(FSReadRequest&, void*)> cb,
-                     void* cb_arg, void** io_handle, IOHandleDeleter* del_fn,
-                     AlignedBuf* aligned_buf, IODebugContext* dbg = nullptr);
+  IOStatus ReadAsync(
+      FSReadRequest& req, const IOOptions& opts,
+      std::function<void(FSReadRequest&, void*)> cb, void* cb_arg,
+      void** io_handle, IOHandleDeleter* del_fn, AlignedBuf* aligned_buf,
+      IODebugContext* dbg = nullptr,
+      AlignedBufferAllocationContext* direct_io_buffer_context = nullptr);
 
   void ReadAsyncCallback(FSReadRequest& req, void* cb_arg);
 };

file/random_access_file_reader_test.cc

@@ -81,15 +81,91 @@ TEST_F(RandomAccessFileReaderTest, ReadDirectIO) {
   size_t offset = page_size / 2;
   size_t len = page_size / 3;
   Slice result;
-  AlignedBuf buf;
   for (Env::IOPriority rate_limiter_priority : {Env::IO_LOW, Env::IO_TOTAL}) {
     IOOptions io_opts;
     io_opts.rate_limiter_priority = rate_limiter_priority;
-    ASSERT_OK(r->Read(io_opts, offset, len, &result, nullptr, &buf));
+    AlignedBuffer direct_io_buffer;
+    AlignedBufferAllocationContext direct_io_context{&direct_io_buffer};
+    ASSERT_OK(r->Read(io_opts, offset, len, &result, nullptr,
+                      &direct_io_context,
+                      /*dbg=*/nullptr));
     ASSERT_EQ(result.ToString(), content.substr(offset, len));
   }
 }
 
+TEST_F(RandomAccessFileReaderTest, ReadDirectIOCopiesToScratch) {
+  std::string fname = "read-direct-io-copies-to-scratch";
+  Random rand(0);
+  std::string content = rand.RandomString(kDefaultPageSize);
+  Write(fname, content);
+
+  FileOptions opts;
+  opts.use_direct_reads = true;
+  std::unique_ptr<RandomAccessFileReader> r;
+  Read(fname, opts, &r);
+  ASSERT_TRUE(r->use_direct_io());
+
+  const size_t page_size = r->file()->GetRequiredBufferAlignment();
+  size_t offset = page_size / 2;
+  size_t len = page_size / 3;
+  std::string scratch(len, '\0');
+  Slice result;
+  ASSERT_OK(r->Read(IOOptions(), offset, len, &result, scratch.data(),
+                    /*direct_io_buffer=*/nullptr, /*dbg=*/nullptr));
+  ASSERT_EQ(result.data(), scratch.data());
+  ASSERT_EQ(result.ToString(), content.substr(offset, len));
+}
+
+TEST_F(RandomAccessFileReaderTest, ReadDirectIOUsesExternalBuffer) {
+  std::string fname = "read-direct-io-external-buffer";
+  Random rand(0);
+  std::string content = rand.RandomString(kDefaultPageSize);
+  Write(fname, content);
+
+  FileOptions opts;
+  opts.use_direct_reads = true;
+  std::unique_ptr<RandomAccessFileReader> r;
+  Read(fname, opts, &r);
+  ASSERT_TRUE(r->use_direct_io());
+
+  const size_t page_size = r->file()->GetRequiredBufferAlignment();
+  const size_t offset = page_size / 4;
+  const size_t len = page_size / 2;
+
+  AlignedBuffer external_storage;
+  int allocations = 0;
+  size_t requested_size = 0;
+  size_t requested_alignment = 0;
+  AlignedBuffer::Allocator allocator =
+      [&](size_t size, size_t alignment,
+          AlignedBuffer::ExternalAllocation* out) {
+        ++allocations;
+        requested_size = size;
+        requested_alignment = alignment;
+        external_storage.Alignment(alignment);
+        external_storage.AllocateNewBuffer(size);
+        out->data = external_storage.BufferStart();
+        out->size = external_storage.Capacity();
+        out->owner =
+            FSAllocationPtr(external_storage.BufferStart(), [](void*) {});
+        return Status::OK();
+      };
+
+  AlignedBuffer direct_io_buffer;
+  AlignedBufferAllocationContext direct_io_context{&direct_io_buffer,
+                                                   &allocator};
+  Slice result;
+  ASSERT_OK(r->Read(IOOptions(), offset, len, &result, /*scratch=*/nullptr,
+                    &direct_io_context, /*dbg=*/nullptr));
+  ASSERT_EQ(result.ToString(), content.substr(offset, len));
+
+  ASSERT_EQ(allocations, 1);
+  ASSERT_EQ(requested_alignment, page_size);
+  ASSERT_EQ(requested_size, page_size);
+  ASSERT_EQ(direct_io_buffer.BufferStart(), external_storage.BufferStart());
+  ASSERT_EQ(result.data(), external_storage.BufferStart() + offset);
+}
+
 TEST_F(RandomAccessFileReaderTest, MultiReadDirectIO) {
   std::vector<FSReadRequest> aligned_reqs;
   ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->SetCallBack(
@@ -146,10 +222,11 @@ TEST_F(RandomAccessFileReaderTest, MultiReadDirectIO) {
     std::vector<FSReadRequest> reqs;
     reqs.push_back(std::move(r0));
     reqs.push_back(std::move(r1));
-    AlignedBuf aligned_buf;
+    AlignedBuffer direct_io_buffer;
+    AlignedBufferAllocationContext direct_io_context{&direct_io_buffer};
     IODebugContext dbg;
-    ASSERT_OK(r->MultiRead(IOOptions(), reqs.data(), reqs.size(), &aligned_buf,
-                           &dbg));
+    ASSERT_OK(r->MultiRead(IOOptions(), reqs.data(), reqs.size(),
+                           &direct_io_context, &dbg));
 
     AssertResult(content, reqs);
 
@@ -192,10 +269,11 @@ TEST_F(RandomAccessFileReaderTest, MultiReadDirectIO) {
     reqs.push_back(std::move(r0));
     reqs.push_back(std::move(r1));
     reqs.push_back(std::move(r2));
-    AlignedBuf aligned_buf;
+    AlignedBuffer direct_io_buffer;
+    AlignedBufferAllocationContext direct_io_context{&direct_io_buffer};
     IODebugContext dbg;
-    ASSERT_OK(r->MultiRead(IOOptions(), reqs.data(), reqs.size(), &aligned_buf,
-                           &dbg));
+    ASSERT_OK(r->MultiRead(IOOptions(), reqs.data(), reqs.size(),
+                           &direct_io_context, &dbg));
 
     AssertResult(content, reqs);
 
@@ -238,10 +316,11 @@ TEST_F(RandomAccessFileReaderTest, MultiReadDirectIO) {
     reqs.push_back(std::move(r0));
     reqs.push_back(std::move(r1));
     reqs.push_back(std::move(r2));
-    AlignedBuf aligned_buf;
+    AlignedBuffer direct_io_buffer;
+    AlignedBufferAllocationContext direct_io_context{&direct_io_buffer};
     IODebugContext dbg;
-    ASSERT_OK(r->MultiRead(IOOptions(), reqs.data(), reqs.size(), &aligned_buf,
-                           &dbg));
+    ASSERT_OK(r->MultiRead(IOOptions(), reqs.data(), reqs.size(),
+                           &direct_io_context, &dbg));
 
     AssertResult(content, reqs);
 
@@ -276,10 +355,11 @@ TEST_F(RandomAccessFileReaderTest, MultiReadDirectIO) {
     std::vector<FSReadRequest> reqs;
     reqs.push_back(std::move(r0));
     reqs.push_back(std::move(r1));
-    AlignedBuf aligned_buf;
+    AlignedBuffer direct_io_buffer;
+    AlignedBufferAllocationContext direct_io_context{&direct_io_buffer};
     IODebugContext dbg;
-    ASSERT_OK(r->MultiRead(IOOptions(), reqs.data(), reqs.size(), &aligned_buf,
-                           &dbg));
+    ASSERT_OK(r->MultiRead(IOOptions(), reqs.data(), reqs.size(),
+                           &direct_io_context, &dbg));
 
     AssertResult(content, reqs);
 
@@ -299,6 +379,71 @@ TEST_F(RandomAccessFileReaderTest, MultiReadDirectIO) {
   ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->ClearAllCallBacks();
 }
 
+TEST_F(RandomAccessFileReaderTest, MultiReadDirectIOUsesExternalBuffer) {
+  std::string fname = "multi-read-direct-io-external-buffer";
+  Random rand(0);
+  std::string content = rand.RandomString(3 * kDefaultPageSize);
+  Write(fname, content);
+
+  FileOptions opts;
+  opts.use_direct_reads = true;
+  std::unique_ptr<RandomAccessFileReader> r;
+  Read(fname, opts, &r);
+  ASSERT_TRUE(r->use_direct_io());
+
+  const size_t page_size = r->file()->GetRequiredBufferAlignment();
+  FSReadRequest r0;
+  r0.offset = page_size / 4;
+  r0.len = page_size / 2;
+  r0.scratch = nullptr;
+
+  FSReadRequest r1;
+  r1.offset = 2 * page_size + page_size / 4;
+  r1.len = page_size / 2;
+  r1.scratch = nullptr;
+
+  std::vector<FSReadRequest> reqs;
+  reqs.push_back(std::move(r0));
+  reqs.push_back(std::move(r1));
+
+  AlignedBuffer external_storage;
+  int allocations = 0;
+  size_t requested_size = 0;
+  size_t requested_alignment = 0;
+  AlignedBuffer::Allocator allocator =
+      [&](size_t size, size_t alignment,
+          AlignedBuffer::ExternalAllocation* out) {
+        ++allocations;
+        requested_size = size;
+        requested_alignment = alignment;
+        external_storage.Alignment(alignment);
+        external_storage.AllocateNewBuffer(size);
+        out->data = external_storage.BufferStart();
+        out->size = external_storage.Capacity();
+        out->owner =
+            FSAllocationPtr(external_storage.BufferStart(), [](void*) {});
+        return Status::OK();
+      };
+
+  AlignedBuffer direct_io_buffer;
+  AlignedBufferAllocationContext direct_io_context{&direct_io_buffer,
+                                                   &allocator};
+  ASSERT_OK(r->MultiRead(IOOptions(), reqs.data(), reqs.size(),
+                         &direct_io_context, /*dbg=*/nullptr));
+  AssertResult(content, reqs);
+
+  ASSERT_EQ(allocations, 1);
+  ASSERT_EQ(requested_alignment, page_size);
+  ASSERT_EQ(requested_size, 2 * page_size);
+  ASSERT_EQ(direct_io_buffer.BufferStart(), external_storage.BufferStart());
+  const char* storage_begin = external_storage.BufferStart();
+  const char* storage_end = storage_begin + external_storage.Capacity();
+  for (const auto& req : reqs) {
+    ASSERT_GE(req.result.data(), storage_begin);
+    ASSERT_LE(req.result.data() + req.result.size(), storage_end);
+  }
+}
+
 TEST(FSReadRequest, Align) {
   FSReadRequest r;
   r.offset = 2000;

file/read_write_util.cc

@@ -9,6 +9,7 @@
 
 #include "file/read_write_util.h"
 
+#include <cassert>
 #include <sstream>
 
 #include "test_util/sync_point.h"
@@ -25,6 +26,29 @@ IOStatus NewWritableFile(FileSystem* fs, const std::string& fname,
   return s;
 }
 
+IOStatus GetFileSizeFromOpenFileOrPath(
+    FSRandomAccessFile* file, FileSystem* fs, const std::string& fname,
+    uint64_t* file_size, IODebugContext* dbg, FileSizeFallback fallback,
+    BeforePathFileSizeFallback before_path_fallback) {
+  assert(file != nullptr);
+  assert(fs != nullptr);
+  assert(file_size != nullptr);
+
+  IOStatus s = file->GetFileSize(file_size);
+  if (s.ok()) {
+    return s;
+  }
+
+  if (fallback == FileSizeFallback::kAnyOpenFileError || s.IsNotSupported()) {
+    if (before_path_fallback != nullptr) {
+      before_path_fallback();
+    }
+    return fs->GetFileSize(fname, IOOptions(), file_size, dbg);
+  }
+
+  return s;
+}
+
 #ifndef NDEBUG
 bool IsFileSectorAligned(const size_t off, size_t sector_size) {
   return off % sector_size == 0;

file/read_write_util.h

@@ -25,6 +25,18 @@ IOStatus NewWritableFile(FileSystem* fs, const std::string& fname,
                          std::unique_ptr<FSWritableFile>* result,
                          const FileOptions& options);
 
+enum class FileSizeFallback {
+  kNotSupportedOnly,
+  kAnyOpenFileError,
+};
+
+using BeforePathFileSizeFallback = void (*)();
+
+IOStatus GetFileSizeFromOpenFileOrPath(
+    FSRandomAccessFile* file, FileSystem* fs, const std::string& fname,
+    uint64_t* file_size, IODebugContext* dbg, FileSizeFallback fallback,
+    BeforePathFileSizeFallback before_path_fallback = nullptr);
+
 #ifndef NDEBUG
 bool IsFileSectorAligned(const size_t off, size_t sector_size);
 #endif  // NDEBUG

include/rocksdb/advanced_options.h

@@ -1364,7 +1364,7 @@ struct AdvancedColumnFamilyOptions {
   // operation.
   // This option depends on memtable_protection_bytes_per_key to be non zero.
   // If memtable_protection_bytes_per_key is zero, no validation is performed.
-  bool memtable_veirfy_per_key_checksum_on_seek = false;
+  bool memtable_verify_per_key_checksum_on_seek = false;
 
   // When an iterator scans this number of invisible entries (tombstones or
   // hidden puts) from the active memtable during a single iterator operation,

include/rocksdb/c.h

@@ -273,6 +273,12 @@ extern ROCKSDB_LIBRARY_API uint32_t rocksdb_backup_engine_info_number_files(
 extern ROCKSDB_LIBRARY_API void rocksdb_backup_engine_info_destroy(
     const rocksdb_backup_engine_info_t* info);
 
+/* Cancels any in-progress backup. This is a one-way operation: once called,
+   all future CreateNewBackup requests on this engine will fail with
+   Status::Incomplete. Open a new BackupEngine instance to resume backups. */
+extern ROCKSDB_LIBRARY_API void rocksdb_backup_engine_stop_backup(
+    rocksdb_backup_engine_t* be);
+
 extern ROCKSDB_LIBRARY_API void rocksdb_backup_engine_close(
     rocksdb_backup_engine_t* be);
 
@@ -379,6 +385,18 @@ extern ROCKSDB_LIBRARY_API uint64_t
 rocksdb_backup_engine_options_get_restore_rate_limit(
     rocksdb_backup_engine_options_t* options);
 
+/* Sets a RateLimiter for backup transfers. When non-null, overrides the
+   uint64_t backup_rate_limit value. */
+extern ROCKSDB_LIBRARY_API void
+rocksdb_backup_engine_options_set_backup_rate_limiter(
+    rocksdb_backup_engine_options_t* options, rocksdb_ratelimiter_t* limiter);
+
+/* Sets a RateLimiter for restore transfers. When non-null, overrides the
+   uint64_t restore_rate_limit value. */
+extern ROCKSDB_LIBRARY_API void
+rocksdb_backup_engine_options_set_restore_rate_limiter(
+    rocksdb_backup_engine_options_t* options, rocksdb_ratelimiter_t* limiter);
+
 extern ROCKSDB_LIBRARY_API void
 rocksdb_backup_engine_options_set_max_background_operations(
     rocksdb_backup_engine_options_t* options, int val);
@@ -1434,6 +1452,10 @@ extern ROCKSDB_LIBRARY_API void rocksdb_set_options(rocksdb_t* db, int count,
                                                     const char* const values[],
                                                     char** errptr);
 
+extern ROCKSDB_LIBRARY_API void rocksdb_set_db_options(
+    rocksdb_t* db, int count, const char* const keys[],
+    const char* const values[], char** errptr);
+
 extern ROCKSDB_LIBRARY_API void rocksdb_set_options_cf(
     rocksdb_t* db, rocksdb_column_family_handle_t* handle, int count,
     const char* const keys[], const char* const values[], char** errptr);
@@ -1890,6 +1912,11 @@ rocksdb_options_set_use_direct_io_for_flush_and_compaction(rocksdb_options_t*,
                                                            unsigned char);
 extern ROCKSDB_LIBRARY_API unsigned char
 rocksdb_options_get_use_direct_io_for_flush_and_compaction(rocksdb_options_t*);
+extern ROCKSDB_LIBRARY_API void
+rocksdb_options_set_use_direct_io_for_compaction_reads(rocksdb_options_t*,
+                                                       unsigned char);
+extern ROCKSDB_LIBRARY_API unsigned char
+rocksdb_options_get_use_direct_io_for_compaction_reads(rocksdb_options_t*);
 extern ROCKSDB_LIBRARY_API void rocksdb_options_set_is_fd_close_on_exec(
     rocksdb_options_t*, unsigned char);
 extern ROCKSDB_LIBRARY_API unsigned char
@@ -3446,6 +3473,14 @@ extern ROCKSDB_LIBRARY_API void
 rocksdb_transactiondb_options_set_default_lock_timeout(
     rocksdb_transactiondb_options_t* opt, int64_t default_lock_timeout);
 
+enum {
+  rocksdb_txndb_write_policy_write_committed = 0,
+  rocksdb_txndb_write_policy_write_prepared = 1,
+  rocksdb_txndb_write_policy_write_unprepared = 2
+};
+extern ROCKSDB_LIBRARY_API void rocksdb_transactiondb_options_set_write_policy(
+    rocksdb_transactiondb_options_t* opt, int write_policy);
+
 extern ROCKSDB_LIBRARY_API rocksdb_transaction_options_t*
 rocksdb_transaction_options_create(void);
 

include/rocksdb/cleanable.h

@@ -107,6 +107,7 @@ class SharedCleanablePtr {
   Cleanable* operator->();
   // Get as raw pointer to Cleanable
   Cleanable* get();
+  const Cleanable* get() const;
 
   // Creates a (virtual) copy of this SharedCleanablePtr and registers its
   // destruction with target, so that the cleanups registered with the

include/rocksdb/compression_type.h

@@ -22,6 +22,8 @@ enum CompressionType : unsigned char {
   kSnappyCompression = 0x01,
   kZlibCompression = 0x02,
   kBZip2Compression = 0x03,
+  // NOTE: LZ4 and LZ4HC should be considered variants of the same compression
+  // type. See CompressionOptions::level
   kLZ4Compression = 0x04,
   kLZ4HCCompression = 0x05,
   kXpressCompression = 0x06,
@@ -188,10 +190,26 @@ struct CompressionOptions {
   // `kDefaultCompressionLevel` values will either favor speed over
   // compression ratio or have no effect.
   //
-  // In LZ4 specifically, the absolute value of a negative `level` internally
-  // configures the `acceleration` parameter. For example, set `level=-10` for
-  // `acceleration=10`. This negation is necessary to ensure decreasing `level`
-  // values favor speed over compression ratio.
+  // LZ4 and LZ4HC share one on-disk format, so they expose a single monotonic
+  // `level` axis: `level <= 0` selects LZ4 fast with `acceleration = -level`
+  // (e.g. -10 means acceleration 10, 0 is clamped to minimum acceleration 1),
+  // and `level >= 1` selects LZ4HC at that level (1..12). The configured type
+  // (kLZ4Compression vs kLZ4HCCompression) only sets the default `level` and
+  // the compression type byte recorded per block. Defaults are acceleration 1
+  // (level -1) for LZ4 and level 9 for LZ4HC. Levels outside the
+  // algorithm-recognized ranges are clamped to the nearest effective value:
+  // below -65537 acts like -65537 (max acceleration) and above 12 like 12 (max
+  // compression effort).
+  //
+  // For ZSTD (the other generally recommended compression alongside LZ4),
+  // `level` follows zstd's own scale, where higher values trade more CPU for a
+  // better compression ratio. The standard range is 1 (fastest) through 19,
+  // with 20..22 being "ultra" levels that require substantially more memory to
+  // compress (and decompress). zstd also supports negative levels (-1 and
+  // below) that favor speed over ratio, analogous to LZ4 acceleration.
+  // RocksDB's default (kDefaultCompressionLevel) maps to zstd's default of 3.
+  // (zstd itself treats a `level` of 0 as "use the default", but prefer
+  // kDefaultCompressionLevel to request that.)
   int level = kDefaultCompressionLevel;
 
   // zlib only: strategy parameter. See https://www.zlib.net/manual.html

include/rocksdb/convenience.h

@@ -432,9 +432,33 @@ Status GetOptionsFromString(const ConfigOptions& config_options,
                             const Options& base_options,
                             const std::string& opts_str, Options* new_options);
 
+// StringToMap parses a serialized options string into a map. Each
+// resulting map value is in a self-contained form (it can be embedded
+// directly in a `key=value;` context -- e.g. SetOptions -- without further
+// escaping). Specifically: nested braced values from the input are
+// preserved with their outer braces. Permissive: accepts both braced and
+// unbraced forms; values not requiring braces are returned as-is.
+// Example:
+//   "filter_policy={id=ribbonfilter:10;bloom_before_level=-1};block_size=4096"
+// produces:
+//   {filter_policy -> "{id=ribbonfilter:10;bloom_before_level=-1}",
+//    block_size    -> "4096"}
 Status StringToMap(const std::string& opts_str,
                    std::unordered_map<std::string, std::string>* opts_map);
 
+// MapToString is the inverse of StringToMap: a naive `key=value;` join.
+// Each map value must already be in self-contained form (as returned by
+// StringToMap) -- i.e. simple text or a single balanced `{...}` block.
+// Values from StringToMap satisfy this property, so the round-trip
+//
+//   StringToMap(MapToString(StringToMap(s))) == StringToMap(s)
+//
+// holds. Callers building a map by hand are responsible for ensuring
+// values are self-contained; raw values containing `;` or starting with
+// `{` without matching `}` won't round-trip.
+Status MapToString(const std::unordered_map<std::string, std::string>& opts_map,
+                   std::string* opts_str);
+
 // Request stopping background work, if wait is true wait until it's done
 void CancelAllBackgroundWork(DB* db, bool wait = false);
 

include/rocksdb/db.h

@@ -91,6 +91,32 @@ class ColumnFamilyHandle {
   virtual const Comparator* GetComparator() const = 0;
 };
 
+// Opaque handle to a prepared-but-not-committed external file ingestion,
+// produced by DB::PrepareFileIngestion(). Pass it to
+// DB::CommitFileIngestionHandle(s)() to make the prepared files visible, or
+// call Abort() (or simply destroy the handle) to roll the ingestion back: that
+// deletes the staged files and releases the reserved internal file numbers.
+//
+// A handle MUST NOT outlive the DB that produced it.
+class FileIngestionHandle {
+ public:
+  virtual ~FileIngestionHandle() = default;
+
+  // Cancel this prepared ingestion: delete the staged files and release the
+  // reserved file numbers, leaving the DB unchanged. The handle is spent
+  // afterwards. Equivalent to simply destroying the handle, but explicit and
+  // returns a Status.
+  virtual Status Abort() = 0;
+
+  FileIngestionHandle(const FileIngestionHandle&) = delete;
+  FileIngestionHandle& operator=(const FileIngestionHandle&) = delete;
+  FileIngestionHandle(FileIngestionHandle&&) = delete;
+  FileIngestionHandle& operator=(FileIngestionHandle&&) = delete;
+
+ protected:
+  FileIngestionHandle() = default;
+};
+
 static const int kMajorVersion = ROCKSDB_MAJOR;
 static const int kMinorVersion = ROCKSDB_MINOR;
 
@@ -452,6 +478,8 @@ class DB {
   // Set the database entry for "key" in the column family specified by
   // "column_family" to the wide-column entity defined by "columns". If the key
   // already exists in the column family, it will be overwritten.
+  // `columns` is a non-owning view. The backing storage for each column name
+  // and value must remain valid until this method returns.
   //
   // Returns OK on success, and a non-OK status on error.
   virtual Status PutEntity(const WriteOptions& options,
@@ -1924,6 +1952,14 @@ class DB {
       const LiveFilesStorageInfoOptions& opts,
       std::vector<LiveFileStorageInfo>* files) = 0;
 
+  // Prepares a live DB-generated table file for use with IngestExternalFiles().
+  // See `PreparedFileInfo` for details. The path must exactly match a live
+  // table file owned by this DB. This may open the table file and populate this
+  // DB's table cache if the table reader is not already cached.
+  virtual Status GetPreparedFileInfoForExternalSstIngestion(
+      const std::string& /*file_path*/,
+      std::shared_ptr<const PreparedFileInfo>* /*file_info*/) = 0;
+
   // Obtains the LSM-tree meta data of the specified column family of the DB,
   // including metadata for each live table (SST) file in that column family.
   virtual void GetColumnFamilyMetaData(ColumnFamilyHandle* /*column_family*/,
@@ -2057,6 +2093,58 @@ class DB {
   virtual Status IngestExternalFiles(
       const std::vector<IngestExternalFileArg>& args) = 0;
 
+  // Two-phase external file ingestion. PrepareFileIngestion() performs all of
+  // the work that does not require the DB mutex (validating the args, reserving
+  // internal file numbers, reading each file's metadata, and
+  // linking/copying/fsyncing the files into the DB), returning an opaque
+  // FileIngestionHandle. DB::CommitFileIngestionHandle(s)() then commits the
+  // external file into the DB.
+  //
+  // The main advantage here is that the application has more flexibility to
+  // "Prepare" the file in advance, which makes the actual Commit phase much
+  // shorter than a typical DB::IngestExternalFiles call.
+  //
+  // `args` has the same requirements as IngestExternalFiles(). On success
+  // *handle holds the prepared ingestion; on failure *handle is reset and no DB
+  // state is changed. PrepareFileIngestion() is safe to call concurrently with
+  // other DB operations and from a background thread.
+  virtual Status PrepareFileIngestion(
+      const std::vector<IngestExternalFileArg>& args,
+      std::unique_ptr<FileIngestionHandle>* handle) = 0;
+
+  // Single-column-family convenience overload of PrepareFileIngestion().
+  virtual Status PrepareFileIngestion(
+      ColumnFamilyHandle* column_family,
+      const std::vector<std::string>& external_files,
+      const IngestExternalFileOptions& options,
+      std::unique_ptr<FileIngestionHandle>* handle) {
+    IngestExternalFileArg arg;
+    arg.column_family = column_family;
+    arg.external_files = external_files;
+    arg.options = options;
+    return PrepareFileIngestion({arg}, handle);
+  }
+
+  // Commits one or more prepared ingestions (from PrepareFileIngestion())
+  // atomically: all of their files become visible together under a single DB
+  // mutex acquisition and a single atomic MANIFEST write, or none do. Consumes
+  // the handles. Multiple handles MAY target the same column family; that
+  // column family's files are committed together as if all of them had been
+  // passed to a single IngestExternalFiles() call, in the order the handles
+  // appear in `handles` (so for overlapping keys a later handle's data wins).
+  // Handles that target the same column family must have been prepared with the
+  // same IngestExternalFileOptions. On failure all handles are rolled back.
+  virtual Status CommitFileIngestionHandles(
+      std::vector<std::unique_ptr<FileIngestionHandle>> handles) = 0;
+
+  // Single-handle convenience for CommitFileIngestionHandles().
+  virtual Status CommitFileIngestionHandle(
+      std::unique_ptr<FileIngestionHandle> handle) {
+    std::vector<std::unique_ptr<FileIngestionHandle>> handles;
+    handles.push_back(std::move(handle));
+    return CommitFileIngestionHandles(std::move(handles));
+  }
+
   // CreateColumnFamilyWithImport() will create a new column family with
   // column_family_name and import external SST files specified in `metadata`
   // into this column family.

include/rocksdb/env.h

@@ -385,6 +385,14 @@ class Env : public Customizable {
     return Status::NotSupported("LinkFile is not supported for this Env");
   }
 
+  // Syncs file data and, when requested, file metadata for `fname`.
+  //
+  // See FileSystem::SyncFile() for the corresponding path-level sync contract.
+  // The default implementation reopens the file as writable, calls Sync() or
+  // Fsync(), then closes it.
+  virtual Status SyncFile(const std::string& fname, const EnvOptions& options,
+                          bool use_fsync);
+
   virtual Status NumFileLinks(const std::string& /*fname*/,
                               uint64_t* /*count*/) {
     return Status::NotSupported(
@@ -1694,6 +1702,11 @@ class EnvWrapper : public Env {
     return target_.env->LinkFile(s, t);
   }
 
+  Status SyncFile(const std::string& fname, const EnvOptions& options,
+                  bool use_fsync) override {
+    return target_.env->SyncFile(fname, options, use_fsync);
+  }
+
   Status NumFileLinks(const std::string& fname, uint64_t* count) override {
     return target_.env->NumFileLinks(fname, count);
   }

include/rocksdb/file_system.h

@@ -119,7 +119,8 @@ struct IOOptions {
   // custom contract between a FileSystem user and the provider. This is only
   // useful in cases where a RocksDB user directly uses the FileSystem or file
   // object for their own purposes, and wants to pass extra options to APIs
-  // such as NewRandomAccessFile and NewWritableFile.
+  // such as NewRandomAccessFile and NewWritableFile. Prefer typed fields on
+  // IOOptions/FileOptions when a standardized semantic exists.
   std::unordered_map<std::string, std::string> property_bag;
 
   // Force directory fsync, some file systems like btrfs may skip directory
@@ -176,9 +177,37 @@ struct DirFsyncOptions {
   explicit DirFsyncOptions(FsyncReason fsync_reason);
 };
 
-// File scope options that control how a file is opened/created and accessed
-// while its open. We may add more options here in the future such as
-// redundancy level, media to use etc.
+// File-scope contracts that describe how RocksDB will open/create and access a
+// file. A FileSystem that detects a contract violation must return a non-OK
+// status or preserve its normal valid semantics; it must not expose undefined
+// or corrupt data.
+enum class FileOpenContract : uint8_t {
+  kDefault = 0,
+  // RocksDB will not call ReopenWritableFile for this file to overwrite,
+  // append, or truncate data. SyncFile is a separate durability API.
+  kNoReopenForWrite = 1 << 0,
+  // RocksDB will not open readers while this file is open for write.
+  kNoReadersWhileOpenForWrite = 1 << 1,
+};
+
+constexpr FileOpenContract operator|(FileOpenContract lhs,
+                                     FileOpenContract rhs) {
+  return static_cast<FileOpenContract>(static_cast<uint8_t>(lhs) |
+                                       static_cast<uint8_t>(rhs));
+}
+
+inline FileOpenContract& operator|=(FileOpenContract& lhs,
+                                    FileOpenContract rhs) {
+  lhs = lhs | rhs;
+  return lhs;
+}
+
+constexpr bool HasFileOpenContract(FileOpenContract contracts,
+                                   FileOpenContract contract) {
+  return (static_cast<uint8_t>(contracts) & static_cast<uint8_t>(contract)) ==
+         static_cast<uint8_t>(contract);
+}
+
 struct FileOptions : EnvOptions {
   // Embedded IOOptions to control the parameters for any IOs that need
   // to be issued for the file open/creation
@@ -191,6 +220,9 @@ struct FileOptions : EnvOptions {
   // coding.
   Temperature temperature = Temperature::kUnknown;
 
+  // File-open contract. kDefault uses the provider's standard semantics.
+  FileOpenContract open_contract = FileOpenContract::kDefault;
+
   // The checksum type that is used to calculate the checksum value for
   // handoff during file writes.
   ChecksumType handoff_checksum_type;
@@ -236,6 +268,7 @@ struct FileOptions : EnvOptions {
       : EnvOptions(opts),
         io_options(opts.io_options),
         temperature(opts.temperature),
+        open_contract(opts.open_contract),
         handoff_checksum_type(opts.handoff_checksum_type),
         write_hint(opts.write_hint),
         file_checksum(opts.file_checksum),
@@ -625,6 +658,22 @@ class FileSystem : public Customizable {
         "LinkFile is not supported for this FileSystem");
   }
 
+  // Syncs file data and, when requested, file metadata for `fname`.
+  //
+  // The default implementation reopens the file as writable, calls Sync() or
+  // Fsync(), then closes it. Filesystems that already make file contents
+  // durable on file flush/close can override this as a no-op to avoid the
+  // reopen overhead.
+  //
+  // RocksDB code that needs to sync a named file should use this API instead of
+  // hand-rolling ReopenWritableFile()+Sync()/Fsync(). This lets filesystems
+  // reject ReopenWritableFile() for post-close data writes while still
+  // providing a cleaner path-level sync implementation.
+  virtual IOStatus SyncFile(const std::string& fname,
+                            const FileOptions& file_opts,
+                            const IOOptions& io_opts, bool use_fsync,
+                            IODebugContext* dbg);
+
   virtual IOStatus NumFileLinks(const std::string& /*fname*/,
                                 const IOOptions& /*options*/,
                                 uint64_t* /*count*/, IODebugContext* /*dbg*/) {
@@ -1642,6 +1691,12 @@ class FileSystemWrapper : public FileSystem {
     return target_->LinkFile(s, t, options, dbg);
   }
 
+  IOStatus SyncFile(const std::string& fname, const FileOptions& file_opts,
+                    const IOOptions& io_opts, bool use_fsync,
+                    IODebugContext* dbg) override {
+    return target_->SyncFile(fname, file_opts, io_opts, use_fsync, dbg);
+  }
+
   IOStatus NumFileLinks(const std::string& fname, const IOOptions& options,
                         uint64_t* count, IODebugContext* dbg) override {
     return target_->NumFileLinks(fname, options, count, dbg);