Blob direct write v1: write-path blob separation with partitioned files (reduced scope) (#14535)

Summary:
This PR introduces **blob direct write v1**, a reduced-scope write-path optimization where large values (>= `min_blob_size`) are written directly to blob files during `Put()` and replaced in the memtable with compact `BlobIndex` references. This avoids holding full values in memory until flush time.

### Motivation

In the existing BlobDB architecture, values are written to the WAL and memtable in their full form and separated into blob files only at flush time. This means:
- Large values are held in memory twice (raw in memtable + blob file at flush)
- Blob I/O is serialized through a single flush thread per column family

Blob direct write addresses both: values leave the write path as small `BlobIndex` references, and multiple **partitions** (configurable via `blob_direct_write_partitions`) allow concurrent blob writes with independent locks.

### Design (v1 — single-writer, WAL-disabled, reduced scope)

The v1 design intentionally keeps scope narrow for correctness and reviewability:

- **Single writer thread assumption**: no concurrent writes to the same partition file. One logical writer serializes the batch.
- **WAL-disabled**: direct-write blob files are only registered in MANIFEST at flush time. WAL replay cannot recover unregistered blob references, so WAL is disabled for this v1.
- **Flush-on-write**: each `AddRecord` call flushes to the OS immediately.
- **FIFO generation batching**: each memtable switch creates one generation batch. Direct-write files for that memtable are sealed and registered atomically when the batch is flushed to MANIFEST.
- **Round-robin partitions**: blob writes are distributed across `blob_direct_write_partitions` files using an atomic counter.

### New components

| Component | Description |
|---|---|
| `BlobFilePartitionManager` | Owns N partition files per CF. Manages open/seal/register lifecycle tied to memtable generations. |
| `BlobWriteBatchTransformer` | A `WriteBatch::Handler` that rewrites qualifying `Put` values as `BlobIndex` entries before the batch enters the write group. |

### Write path integration

1. `DBImpl::WriteImpl` calls `BlobWriteBatchTransformer::TransformBatch` before entering the writer group (for default write path), or before joining the batch group (for pipelined/unordered write).
2. Values >= `min_blob_size` are written to a partition file; the key is stored with a `BlobIndex` in the transformed batch. A rollback guard marks blob bytes as initial garbage if the write fails.
3. On `SwitchMemtable`, `RotateCurrentGeneration` moves active partitions into the next immutable batch.
4. `FlushMemTableToOutputFile` / `AtomicFlushMemTablesToOutputFiles` call `PrepareFlushAdditions` to seal partition files and collect `BlobFileAddition` + `BlobFileGarbage` entries registered to MANIFEST alongside the flush.
5. Shutdown paths (`CancelAllBackgroundWork`, `WaitForCompact` with `close_db=true`) force-flush all CFs with active direct-write managers to ensure blob files are registered before close.

### Read path

- **Get/MultiGet**: `MaybeResolveBlobForWritePath` resolves `BlobIndex` references found in memtable or immutable memtable via `BlobFilePartitionManager::ResolveBlobDirectWriteIndex`, which first checks manifest-visible state and falls back to direct blob-file reads via `BlobFileCache`.
- **Iterator**: `DBIter::BlobReader` is extended with a `BlobFilePartitionManager*` to resolve direct-write blob indexes during iteration. The unified `ResolveBlobDirectWriteIndex` path handles both manifest-visible and not-yet-flushed files.

### New options

| Option | Default | Description |
|---|---|---|
| `enable_blob_direct_write` | `false` | Enable write-path blob separation for this CF. Requires `enable_blob_files = true`. Not dynamically changeable. |
| `blob_direct_write_partitions` | `1` | Number of parallel partition files per CF. Not dynamically changeable. |

### Feature incompatibilities (reduced v1 scope)

The following features are *not supported* when `enable_blob_direct_write = true`, and are enforced both in `db_stress_tool` validation and `db_crashtest.py` sanitization:

**Write model constraints:**
- `threads` must be 1 (single writer assumption)
- `allow_concurrent_memtable_write` = 0
- `enable_pipelined_write` = 0 (transformation done before batch group, but pipelined path supported with pre-transform)
- `two_write_queues` = 0
- `unordered_write` = 0 (transformation done before batch group, but unordered path supported with pre-transform)

**WAL and recovery:**
- `disable_wal` = 1 (required — WAL replay of unregistered blob files is out of v1 scope)
- `best_efforts_recovery` = 0
- `reopen` = 0 (no crash-restart with WAL replay)
- All WAL-related stress features disabled: `manual_wal_flush_one_in`, `sync_wal_one_in`, `lock_wal_one_in`, `get_sorted_wal_files_one_in`, `get_current_wal_file_one_in`, `track_and_verify_wals`, `rate_limit_auto_wal_flush`, `recycle_log_file_num`

**Blob GC and dynamic options:**
- `use_blob_db` = 0 (stacked BlobDB not supported)
- `allow_setting_blob_options_dynamically` = 0
- `enable_blob_garbage_collection` = 0
- `blob_compaction_readahead_size` = 0
- `blob_file_starting_level` = 0

**Unsupported value types and APIs:**
- Merge (`use_merge`, `use_full_merge_v1`) — merge values pass through untransformed
- Entity APIs (`use_put_entity_one_in`, `use_get_entity`, `use_multi_get_entity`, `use_attribute_group`)
- `use_timed_put_one_in`
- User-defined timestamps (`user_timestamp_size`, `persist_user_defined_timestamps`, `create_timestamped_snapshot_one_in`)
- Transactions (`use_txn`, `use_optimistic_txn`, `test_multi_ops_txns`, `commit_bypass_memtable_one_in`) — though `WriteCommittedTxn::CommitInternal` falls back from bypass-memtable to normal path when BDW is active
- `IngestWriteBatchWithIndex` returns `NotSupported`
- `inplace_update_support` = 0

**Fault injection:**
- All write/read/metadata fault injection disabled (`sync_fault_injection`, `write_fault_one_in`, `metadata_write_fault_one_in`, `read_fault_one_in`, `metadata_read_fault_one_in`, `open_*_fault_one_in`)

**Infrastructure/snapshot APIs:**
- `remote_compaction_worker_threads` = 0
- `test_secondary` = 0
- `backup_one_in` = 0
- `checkpoint_one_in` = 0
- `get_live_files_apis_one_in` = 0
- `ingest_external_file_one_in` = 0
- `ingest_wbwi_one_in` = 0

### Tests

- `db/blob/db_blob_basic_test.cc`: ~660 lines of new direct-write unit tests covering basic put/get, multi-partition, flush/compaction, recovery, and error injection.
- `db/blob/blob_file_cache_test.cc`: ~96 lines of new tests for direct-write blob file cache behavior.
- `db/write_batch_test.cc`: ~96 lines of tests for WriteBatch with blob index entries.
- `utilities/transactions/transaction_test.cc`: verifies transaction commit path falls back correctly with direct write enabled.
- `db_stress_tool/`: full stress test support with `--enable_blob_direct_write` and `--blob_direct_write_partitions` flags, integrated into `db_crashtest.py` with 10% random selection alongside regular blob params.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14535

Test Plan:
```
make -j128 db_blob_basic_test && ./db_blob_basic_test
make -j128 blob_file_cache_test && ./blob_file_cache_test
make -j128 write_batch_test && ./write_batch_test
make -j128 transaction_test && ./transaction_test
make -j128 check
```

Stress test:
```
python3 tools/db_crashtest.py blackbox --enable_blob_direct_write=1 \
  --enable_blob_files=1 --blob_direct_write_partitions=4 \
  --disable_wal=1 --threads=1
```

Reviewed By: pdillinger

Differential Revision: D98766843

Pulled By: xingbowang

fbshipit-source-id: 1577653826913a59d05680a87bce5534ac5a5e69

BUCK

@@ -32,12 +32,14 @@ cpp_library_wrapper(name="rocksdb_lib", srcs=[
         "db/blob/blob_file_cache.cc",
         "db/blob/blob_file_garbage.cc",
         "db/blob/blob_file_meta.cc",
+        "db/blob/blob_file_partition_manager.cc",
         "db/blob/blob_file_reader.cc",
         "db/blob/blob_garbage_meter.cc",
         "db/blob/blob_log_format.cc",
         "db/blob/blob_log_sequential_reader.cc",
         "db/blob/blob_log_writer.cc",
         "db/blob/blob_source.cc",
+        "db/blob/blob_write_batch_transformer.cc",
         "db/blob/prefetch_buffer_collection.cc",
         "db/builder.cc",
         "db/c.cc",
@@ -4805,6 +4807,12 @@ cpp_unittest_wrapper(name="db_blob_corruption_test",
             extra_compiler_flags=[])
 
 
+cpp_unittest_wrapper(name="db_blob_direct_write_test",
+            srcs=["db/blob/db_blob_direct_write_test.cc"],
+            deps=[":rocksdb_test_lib"],
+            extra_compiler_flags=[])
+
+
 cpp_unittest_wrapper(name="db_blob_index_test",
             srcs=["db/blob/db_blob_index_test.cc"],
             deps=[":rocksdb_test_lib"],

CMakeLists.txt

@@ -707,6 +707,7 @@ set(SOURCES
         db/blob/blob_file_addition.cc
         db/blob/blob_file_builder.cc
         db/blob/blob_file_cache.cc
+        db/blob/blob_file_partition_manager.cc
         db/blob/blob_file_garbage.cc
         db/blob/blob_file_meta.cc
         db/blob/blob_file_reader.cc
@@ -715,6 +716,7 @@ set(SOURCES
         db/blob/blob_log_sequential_reader.cc
         db/blob/blob_log_writer.cc
         db/blob/blob_source.cc
+        db/blob/blob_write_batch_transformer.cc
         db/blob/prefetch_buffer_collection.cc
         db/builder.cc
         db/c.cc
@@ -1387,6 +1389,7 @@ if(WITH_TESTS)
         db/blob/blob_garbage_meter_test.cc
         db/blob/blob_source_test.cc
         db/blob/db_blob_basic_test.cc
+        db/blob/db_blob_direct_write_test.cc
         db/blob/db_blob_compaction_test.cc
         db/blob/db_blob_corruption_test.cc
         db/blob/db_blob_index_test.cc

Makefile

@@ -638,6 +638,7 @@ PARALLEL_TEST = $(filter-out $(NON_PARALLEL_TEST), $(ROCKSDBTESTS_SUBSET))
 TESTS_PLATFORM_DEPENDENT := \
 	db_basic_test \
 	db_blob_basic_test \
+	db_blob_direct_write_test \
 	db_encryption_test \
 	external_sst_file_basic_test \
 	auto_roll_logger_test \
@@ -1447,6 +1448,9 @@ db_basic_test: $(OBJ_DIR)/db/db_basic_test.o $(TEST_LIBRARY) $(LIBRARY)
 db_blob_basic_test: $(OBJ_DIR)/db/blob/db_blob_basic_test.o $(TEST_LIBRARY) $(LIBRARY)
 	$(AM_LINK)
 
+db_blob_direct_write_test: $(OBJ_DIR)/db/blob/db_blob_direct_write_test.o $(TEST_LIBRARY) $(LIBRARY)
+	$(AM_LINK)
+
 db_blob_compaction_test: $(OBJ_DIR)/db/blob/db_blob_compaction_test.o $(TEST_LIBRARY) $(LIBRARY)
 	$(AM_LINK)
 

db/blob/blob_file_cache.cc

@@ -9,6 +9,7 @@
 #include <memory>
 
 #include "db/blob/blob_file_reader.h"
+#include "logging/logging.h"
 #include "options/cf_options.h"
 #include "rocksdb/cache.h"
 #include "rocksdb/slice.h"
@@ -38,7 +39,8 @@ BlobFileCache::BlobFileCache(Cache* cache,
 
 Status BlobFileCache::GetBlobFileReader(
     const ReadOptions& read_options, uint64_t blob_file_number,
-    CacheHandleGuard<BlobFileReader>* blob_file_reader) {
+    CacheHandleGuard<BlobFileReader>* blob_file_reader,
+    bool allow_footer_skip_retry) {
   assert(blob_file_reader);
   assert(blob_file_reader->IsEmpty());
 
@@ -73,10 +75,22 @@ Status BlobFileCache::GetBlobFileReader(
 
   {
     assert(file_options_);
-    const Status s = BlobFileReader::Create(
+    Status s = BlobFileReader::Create(
         *immutable_options_, read_options, *file_options_, column_family_id_,
-        blob_file_read_hist_, blob_file_number, io_tracer_, &reader);
+        blob_file_read_hist_, blob_file_number, io_tracer_,
+        /*skip_footer_validation=*/false, &reader);
+    if (!s.ok() && s.IsCorruption() && allow_footer_skip_retry) {
+      reader.reset();
+      s = BlobFileReader::Create(
+          *immutable_options_, read_options, *file_options_, column_family_id_,
+          blob_file_read_hist_, blob_file_number, io_tracer_,
+          /*skip_footer_validation=*/true, &reader);
+    }
     if (!s.ok()) {
+      ROCKS_LOG_WARN(immutable_options_->logger,
+                     "BlobFileCache open failed for blob file %" PRIu64
+                     " in CF %u: %s",
+                     blob_file_number, column_family_id_, s.ToString().c_str());
       RecordTick(statistics, NO_FILE_ERRORS);
       return s;
     }
@@ -99,12 +113,120 @@ Status BlobFileCache::GetBlobFileReader(
   return Status::OK();
 }
 
+Status BlobFileCache::OpenBlobFileReaderUncached(
+    const ReadOptions& read_options, uint64_t blob_file_number,
+    std::unique_ptr<BlobFileReader>* blob_file_reader,
+    bool allow_footer_skip_retry) {
+  assert(blob_file_reader);
+  assert(!*blob_file_reader);
+
+  Statistics* const statistics = immutable_options_->stats;
+  RecordTick(statistics, NO_FILE_OPENS);
+
+  Status s = BlobFileReader::Create(
+      *immutable_options_, read_options, *file_options_, column_family_id_,
+      blob_file_read_hist_, blob_file_number, io_tracer_,
+      /*skip_footer_validation=*/false, blob_file_reader);
+  if (!s.ok() && s.IsCorruption() && allow_footer_skip_retry) {
+    blob_file_reader->reset();
+    s = BlobFileReader::Create(
+        *immutable_options_, read_options, *file_options_, column_family_id_,
+        blob_file_read_hist_, blob_file_number, io_tracer_,
+        /*skip_footer_validation=*/true, blob_file_reader);
+  }
+  if (!s.ok()) {
+    RecordTick(statistics, NO_FILE_ERRORS);
+  }
+  return s;
+}
+
+Status BlobFileCache::InsertBlobFileReader(
+    uint64_t blob_file_number,
+    std::unique_ptr<BlobFileReader>* blob_file_reader,
+    CacheHandleGuard<BlobFileReader>* cached_blob_file_reader) {
+  assert(blob_file_reader);
+  assert(*blob_file_reader);
+  assert(cached_blob_file_reader);
+  assert(cached_blob_file_reader->IsEmpty());
+
+  const Slice key = GetSliceForKey(&blob_file_number);
+  // Serialize refreshes for the same blob file. The cache API does not expose
+  // a conditional replace, so refresh is intentionally modeled as
+  // Lookup/optional Erase/Insert under this per-key mutex.
+  MutexLock lock(&mutex_.Get(key));
+
+  TypedHandle* handle = cache_.Lookup(key);
+  if (handle) {
+    *cached_blob_file_reader = cache_.Guard(handle);
+    blob_file_reader->reset();
+    return Status::OK();
+  }
+
+  constexpr size_t charge = 1;
+  Status s = cache_.Insert(key, blob_file_reader->get(), charge, &handle);
+  if (!s.ok()) {
+    RecordTick(immutable_options_->stats, NO_FILE_ERRORS);
+    return s;
+  }
+
+  // Ownership transferred to the cache.
+  [[maybe_unused]] BlobFileReader* released_reader =
+      blob_file_reader->release();
+  *cached_blob_file_reader = cache_.Guard(handle);
+  return Status::OK();
+}
+
+Status BlobFileCache::RefreshBlobFileReader(
+    uint64_t blob_file_number,
+    std::unique_ptr<BlobFileReader>* blob_file_reader,
+    CacheHandleGuard<BlobFileReader>* cached_blob_file_reader) {
+  assert(blob_file_reader);
+  assert(*blob_file_reader);
+  assert(cached_blob_file_reader);
+  assert(cached_blob_file_reader->IsEmpty());
+
+  const Slice key = GetSliceForKey(&blob_file_number);
+  MutexLock lock(&mutex_.Get(key));
+
+  TypedHandle* handle = cache_.Lookup(key);
+  if (handle) {
+    BlobFileReader* const cached_reader = cache_.Value(handle);
+    assert(cached_reader != nullptr);
+
+    // Active direct-write blob files can grow between refresh attempts. Keep
+    // whichever reader observed the larger on-disk size so an older refresh
+    // cannot overwrite a newer one that another thread already installed.
+    if (cached_reader->GetFileSize() >= (*blob_file_reader)->GetFileSize()) {
+      *cached_blob_file_reader = cache_.Guard(handle);
+      blob_file_reader->reset();
+      return Status::OK();
+    }
+
+    cache_.Release(handle);
+    cache_.get()->Erase(key);
+  }
+
+  constexpr size_t charge = 1;
+  Status s = cache_.Insert(key, blob_file_reader->get(), charge, &handle);
+  if (!s.ok()) {
+    RecordTick(immutable_options_->stats, NO_FILE_ERRORS);
+    return s;
+  }
+
+  // Ownership transferred to the cache.
+  [[maybe_unused]] BlobFileReader* released_reader =
+      blob_file_reader->release();
+  *cached_blob_file_reader = cache_.Guard(handle);
+  return Status::OK();
+}
+
 void BlobFileCache::Evict(uint64_t blob_file_number) {
   // NOTE: sharing same Cache with table_cache
   const Slice key = GetSliceForKey(&blob_file_number);
 
   assert(cache_);
 
+  MutexLock lock(&mutex_.Get(key));
   cache_.get()->Erase(key);
 }
 

db/blob/blob_file_cache.h

@@ -32,9 +32,35 @@ class BlobFileCache {
   BlobFileCache(const BlobFileCache&) = delete;
   BlobFileCache& operator=(const BlobFileCache&) = delete;
 
+  // Returns a cached reader for `blob_file_number`, opening and caching it on
+  // miss. If `allow_footer_skip_retry` is true, a footer-validation corruption
+  // retries once without requiring a footer.
   Status GetBlobFileReader(const ReadOptions& read_options,
                            uint64_t blob_file_number,
-                           CacheHandleGuard<BlobFileReader>* blob_file_reader);
+                           CacheHandleGuard<BlobFileReader>* blob_file_reader,
+                           bool allow_footer_skip_retry = false);
+
+  // Opens a blob file reader without inserting it into the cache.
+  Status OpenBlobFileReaderUncached(
+      const ReadOptions& read_options, uint64_t blob_file_number,
+      std::unique_ptr<BlobFileReader>* blob_file_reader,
+      bool allow_footer_skip_retry = false);
+
+  // Inserts a freshly opened uncached reader unless another thread already
+  // cached the same blob file.
+  Status InsertBlobFileReader(
+      uint64_t blob_file_number,
+      std::unique_ptr<BlobFileReader>* blob_file_reader,
+      CacheHandleGuard<BlobFileReader>* cached_blob_file_reader);
+
+  // Installs a refresh-opened reader into the cache. If another thread has
+  // already cached a reader opened on at least as large a file size, keep that
+  // reader instead so a racing refresh cannot reintroduce an older active-file
+  // view after the blob file grows.
+  Status RefreshBlobFileReader(
+      uint64_t blob_file_number,
+      std::unique_ptr<BlobFileReader>* blob_file_reader,
+      CacheHandleGuard<BlobFileReader>* cached_blob_file_reader);
 
   // Called when a blob file is obsolete to ensure it is removed from the cache
   // to avoid effectively leaking the open file and assicated memory

db/blob/blob_file_cache_test.cc

@@ -192,6 +192,102 @@ TEST_F(BlobFileCacheTest, GetBlobFileReader_Race) {
   SyncPoint::GetInstance()->ClearAllCallBacks();
 }
 
+TEST_F(BlobFileCacheTest, RefreshBlobFileReaderPrefersLargestObservedFileSize) {
+  Options options;
+  options.env = mock_env_.get();
+  options.statistics = CreateDBStatistics();
+  options.cf_paths.emplace_back(
+      test::PerThreadDBPath(
+          mock_env_.get(),
+          "BlobFileCacheTest_"
+          "RefreshBlobFileReaderPrefersLargestObservedFileSize"),
+      0);
+  options.enable_blob_files = true;
+
+  constexpr uint32_t column_family_id = 1;
+  ImmutableOptions immutable_options(options);
+  constexpr uint64_t blob_file_number = 123;
+
+  const std::string blob_file_path =
+      BlobFileName(immutable_options.cf_paths.front().path, blob_file_number);
+
+  std::unique_ptr<FSWritableFile> file;
+  ASSERT_OK(NewWritableFile(immutable_options.fs.get(), blob_file_path, &file,
+                            FileOptions()));
+
+  std::unique_ptr<WritableFileWriter> file_writer(new WritableFileWriter(
+      std::move(file), blob_file_path, FileOptions(), immutable_options.clock));
+
+  constexpr Statistics* statistics = nullptr;
+  constexpr bool use_fsync = false;
+  constexpr bool do_flush = false;
+  BlobLogWriter blob_log_writer(std::move(file_writer), immutable_options.clock,
+                                statistics, blob_file_number, use_fsync,
+                                do_flush);
+
+  constexpr bool has_ttl = false;
+  constexpr ExpirationRange expiration_range;
+  BlobLogHeader header(column_family_id, kNoCompression, has_ttl,
+                       expiration_range);
+  ASSERT_OK(blob_log_writer.WriteHeader(WriteOptions(), header));
+
+  uint64_t key_offset = 0;
+  uint64_t blob_offset = 0;
+  ASSERT_OK(blob_log_writer.AddRecord(WriteOptions(), "key0", "blob0",
+                                      &key_offset, &blob_offset));
+  ASSERT_OK(blob_log_writer.Sync(WriteOptions()));
+
+  constexpr size_t capacity = 10;
+  std::shared_ptr<Cache> backing_cache = NewLRUCache(capacity);
+
+  FileOptions file_options;
+  constexpr HistogramImpl* blob_file_read_hist = nullptr;
+  BlobFileCache blob_file_cache(backing_cache.get(), &immutable_options,
+                                &file_options, column_family_id,
+                                blob_file_read_hist, nullptr /*IOTracer*/);
+
+  const ReadOptions read_options;
+  std::unique_ptr<BlobFileReader> stale_reader;
+  ASSERT_OK(blob_file_cache.OpenBlobFileReaderUncached(
+      read_options, blob_file_number, &stale_reader,
+      /*allow_footer_skip_retry=*/true));
+
+  std::unique_ptr<BlobFileReader> initial_cached_reader;
+  ASSERT_OK(blob_file_cache.OpenBlobFileReaderUncached(
+      read_options, blob_file_number, &initial_cached_reader,
+      /*allow_footer_skip_retry=*/true));
+
+  CacheHandleGuard<BlobFileReader> cached_reader;
+  ASSERT_OK(blob_file_cache.RefreshBlobFileReader(
+      blob_file_number, &initial_cached_reader, &cached_reader));
+  ASSERT_NE(cached_reader.GetValue(), nullptr);
+  const uint64_t initial_file_size = cached_reader.GetValue()->GetFileSize();
+
+  ASSERT_OK(blob_log_writer.AddRecord(WriteOptions(), "key1", "blob1",
+                                      &key_offset, &blob_offset));
+  ASSERT_OK(blob_log_writer.Sync(WriteOptions()));
+
+  std::unique_ptr<BlobFileReader> fresh_reader;
+  ASSERT_OK(blob_file_cache.OpenBlobFileReaderUncached(
+      read_options, blob_file_number, &fresh_reader,
+      /*allow_footer_skip_retry=*/true));
+  ASSERT_GT(fresh_reader->GetFileSize(), initial_file_size);
+
+  CacheHandleGuard<BlobFileReader> refreshed_reader;
+  ASSERT_OK(blob_file_cache.RefreshBlobFileReader(
+      blob_file_number, &fresh_reader, &refreshed_reader));
+  ASSERT_NE(refreshed_reader.GetValue(), nullptr);
+  ASSERT_GT(refreshed_reader.GetValue()->GetFileSize(), initial_file_size);
+  BlobFileReader* const largest_reader = refreshed_reader.GetValue();
+
+  CacheHandleGuard<BlobFileReader> preserved_reader;
+  ASSERT_OK(blob_file_cache.RefreshBlobFileReader(
+      blob_file_number, &stale_reader, &preserved_reader));
+  ASSERT_NE(preserved_reader.GetValue(), nullptr);
+  ASSERT_EQ(preserved_reader.GetValue(), largest_reader);
+  ASSERT_EQ(stale_reader.get(), nullptr);
+}
+
 TEST_F(BlobFileCacheTest, GetBlobFileReader_IOError) {
   Options options;
   options.env = mock_env_.get();

db/blob/blob_file_partition_manager.cc

@@ -0,0 +1,791 @@
+//  Copyright (c) Meta Platforms, Inc. and affiliates.
+//  This source code is licensed under both the GPLv2 (found in the
+//  COPYING file in the root directory) and Apache 2.0 License
+//  (found in the LICENSE.Apache file in the root directory).
+
+#include "db/blob/blob_file_partition_manager.h"
+
+#include <array>
+#include <cinttypes>
+#include <memory>
+#include <utility>
+
+#include "cache/cache_key.h"
+#include "cache/typed_cache.h"
+#include "db/blob/blob_contents.h"
+#include "db/blob/blob_file_cache.h"
+#include "db/blob/blob_file_completion_callback.h"
+#include "db/blob/blob_file_reader.h"
+#include "db/blob/blob_index.h"
+#include "db/blob/blob_log_writer.h"
+#include "db/version_set.h"
+#include "file/filename.h"
+#include "file/read_write_util.h"
+#include "file/writable_file_writer.h"
+#include "logging/logging.h"
+#include "util/aligned_buffer.h"
+#include "util/compression.h"
+#include "util/mutexlock.h"
+
+namespace ROCKSDB_NAMESPACE {
+
+namespace {
+
+struct DirectWriteCompressionState {
+  // `working_area` must be released before its owning compressor.
+  std::unique_ptr<Compressor> compressor;
+  Compressor::ManagedWorkingArea working_area;
+};
+
+DirectWriteCompressionState& GetDirectWriteCompressionState(
+    CompressionType compression) {
+  assert(compression <= kLastBuiltinCompression);
+
+  static thread_local std::array<DirectWriteCompressionState,
+                                 static_cast<size_t>(kLastBuiltinCompression) +
+                                     1>
+      compression_states;
+
+  auto& compression_state =
+      compression_states[static_cast<size_t>(compression)];
+  if (compression != kNoCompression &&
+      compression_state.compressor == nullptr) {
+    // BDW v1 mirrors BlobFileBuilder by using the built-in compressor with the
+    // default CompressionOptions only. Cache it per thread so repeated writes
+    // do not allocate a new compressor and working area every time.
+    compression_state.compressor =
+        GetBuiltinV2CompressionManager()->GetCompressor(CompressionOptions{},
+                                                        compression);
+    if (compression_state.compressor != nullptr) {
+      compression_state.working_area =
+          compression_state.compressor->ObtainWorkingArea();
+    }
+  }
+
+  return compression_state;
+}
+
+}  // namespace
+
+BlobFilePartitionManager::BlobFilePartitionManager(
+    uint32_t num_partitions, FileNumberAllocator file_number_allocator,
+    FileSystem* fs, SystemClock* clock, Statistics* statistics,
+    const FileOptions& file_options, std::string db_path,
+    std::string column_family_name, uint64_t blob_file_size, bool use_fsync,
+    BlobFileCache* blob_file_cache, BlobFileCompletionCallback* blob_callback,
+    const std::vector<std::shared_ptr<EventListener>>& listeners,
+    FileChecksumGenFactory* file_checksum_gen_factory,
+    const FileTypeSet& checksum_handoff_file_types,
+    const std::shared_ptr<IOTracer>& io_tracer, std::string db_id,
+    std::string db_session_id, Logger* info_log)
+    : num_partitions_(num_partitions == 0 ? 1 : num_partitions),
+      file_number_allocator_(std::move(file_number_allocator)),
+      fs_(fs),
+      clock_(clock),
+      statistics_(statistics),
+      file_options_(file_options),
+      db_path_(std::move(db_path)),
+      column_family_name_(std::move(column_family_name)),
+      blob_file_size_(blob_file_size),
+      use_fsync_(use_fsync),
+      blob_file_cache_(blob_file_cache),
+      blob_callback_(blob_callback),
+      listeners_(listeners),
+      file_checksum_gen_factory_(file_checksum_gen_factory),
+      checksum_handoff_file_types_(checksum_handoff_file_types),
+      io_tracer_(io_tracer),
+      db_id_(std::move(db_id)),
+      db_session_id_(std::move(db_session_id)),
+      info_log_(info_log) {
+  partitions_.reserve(num_partitions_);
+  for (uint32_t i = 0; i < num_partitions_; ++i) {
+    partitions_.emplace_back(new Partition());
+  }
+}
+
+BlobFilePartitionManager::~BlobFilePartitionManager() {
+  if (blob_file_cache_ != nullptr) {
+    std::vector<uint64_t> tracked_file_numbers;
+    {
+      ReadLock lock(&file_partition_mutex_);
+      tracked_file_numbers.reserve(file_to_partition_.size());
+      for (const auto& entry : file_to_partition_) {
+        tracked_file_numbers.push_back(entry.first);
+      }
+    }
+
+    for (uint64_t file_number : tracked_file_numbers) {
+      // Dropping the CF or closing the DB can destroy the manager while active
+      // direct-write readers are still cached. Evict them before the CF-owned
+      // blob cache goes away so Close() does not retain obsolete footer-less
+      // readers for files that are no longer live.
+      blob_file_cache_->Evict(file_number);
+    }
+  }
+}
+
+void BlobFilePartitionManager::ResetPartitionState(Partition* partition) {
+  partition->writer.reset();
+  partition->file_number = 0;
+  partition->file_size = 0;
+  partition->blob_count = 0;
+  partition->total_blob_bytes = 0;
+  partition->garbage_blob_count = 0;
+  partition->garbage_blob_bytes = 0;
+  partition->column_family_id = 0;
+  partition->compression = kNoCompression;
+  partition->sync_required = false;
+}
+
+void BlobFilePartitionManager::AddFilePartitionMapping(uint64_t file_number,
+                                                       uint32_t partition_idx) {
+  WriteLock lock(&file_partition_mutex_);
+  file_to_partition_[file_number] = partition_idx;
+}
+
+void BlobFilePartitionManager::RemoveFilePartitionMapping(
+    uint64_t file_number) {
+  WriteLock lock(&file_partition_mutex_);
+  file_to_partition_.erase(file_number);
+}
+
+Status BlobFilePartitionManager::OpenNewBlobFile(Partition* partition,
+                                                 uint32_t column_family_id,
+                                                 CompressionType compression,
+                                                 uint32_t partition_idx) {
+  assert(partition != nullptr);
+  assert(!partition->writer);
+
+  const uint64_t blob_file_number = file_number_allocator_();
+  const std::string blob_file_path = BlobFileName(db_path_, blob_file_number);
+
+  AddFilePartitionMapping(blob_file_number, partition_idx);
+
+  if (blob_callback_ != nullptr) {
+    blob_callback_->OnBlobFileCreationStarted(
+        blob_file_path, column_family_name_,
+        /*job_id=*/0, BlobFileCreationReason::kFlush);
+  }
+
+  std::unique_ptr<FSWritableFile> file;
+  Status s = NewWritableFile(fs_, blob_file_path, &file, file_options_);
+  if (!s.ok()) {
+    RemoveFilePartitionMapping(blob_file_number);
+    return s;
+  }
+
+  const bool perform_data_verification =
+      checksum_handoff_file_types_.Contains(FileType::kBlobFile);
+  auto file_writer = std::make_unique<WritableFileWriter>(
+      std::move(file), blob_file_path, file_options_, clock_, io_tracer_,
+      statistics_, Histograms::BLOB_DB_BLOB_FILE_WRITE_MICROS, listeners_,
+      file_checksum_gen_factory_, perform_data_verification);
+
+  // This only drains WritableFileWriter's buffered bytes so readers can see
+  // each appended record promptly. Durability still comes from SyncAllOpenFiles
+  // or AppendFooter(), both of which call Sync().
+  constexpr bool kDoFlushEachRecord = true;
+  auto blob_log_writer = std::make_unique<BlobLogWriter>(
+      std::move(file_writer), clock_, statistics_, blob_file_number, use_fsync_,
+      kDoFlushEachRecord);
+
+  constexpr bool has_ttl = false;
+  constexpr ExpirationRange expiration_range{};
+  BlobLogHeader header(column_family_id, compression, has_ttl,
+                       expiration_range);
+  s = blob_log_writer->WriteHeader(WriteOptions(), header);
+  if (!s.ok()) {
+    RemoveFilePartitionMapping(blob_file_number);
+    return s;
+  }
+
+  partition->writer = std::move(blob_log_writer);
+  partition->file_number = blob_file_number;
+  partition->file_size = BlobLogHeader::kSize;
+  partition->blob_count = 0;
+  partition->total_blob_bytes = 0;
+  partition->garbage_blob_count = 0;
+  partition->garbage_blob_bytes = 0;
+  partition->column_family_id = column_family_id;
+  partition->compression = compression;
+  partition->sync_required = false;
+  return Status::OK();
+}
+
+Status BlobFilePartitionManager::SealActiveBlobFile(
+    const WriteOptions& write_options, Partition* partition,
+    SealedFile* sealed_file) {
+  assert(partition != nullptr);
+  assert(partition->writer);
+  assert(sealed_file != nullptr);
+
+  const uint64_t file_number = partition->file_number;
+  Status s =
+      FinalizeBlobFile(write_options, partition->writer.get(), file_number,
+                       partition->blob_count, partition->total_blob_bytes,
+                       &sealed_file->addition);
+  if (!s.ok()) {
+    RemoveFilePartitionMapping(file_number);
+    ResetPartitionState(partition);
+    return s;
+  }
+
+  sealed_file->garbage_blob_count = partition->garbage_blob_count;
+  sealed_file->garbage_blob_bytes = partition->garbage_blob_bytes;
+  ResetPartitionState(partition);
+  return Status::OK();
+}
+
+Status BlobFilePartitionManager::SealDeferredFile(
+    const WriteOptions& write_options, DeferredFile* deferred,
+    SealedFile* sealed_file) {
+  assert(deferred != nullptr);
+  assert(deferred->writer);
+  assert(sealed_file != nullptr);
+
+  Status s = FinalizeBlobFile(
+      write_options, deferred->writer.get(), deferred->file_number,
+      deferred->blob_count, deferred->total_blob_bytes, &sealed_file->addition);
+  if (!s.ok()) {
+    RemoveFilePartitionMapping(deferred->file_number);
+    return s;
+  }
+
+  sealed_file->garbage_blob_count = deferred->garbage_blob_count;
+  sealed_file->garbage_blob_bytes = deferred->garbage_blob_bytes;
+  deferred->writer.reset();
+  return Status::OK();
+}
+
+Status BlobFilePartitionManager::FinalizeBlobFile(
+    const WriteOptions& write_options, BlobLogWriter* writer,
+    uint64_t file_number, uint64_t blob_count, uint64_t total_blob_bytes,
+    BlobFileAddition* addition) {
+  assert(writer != nullptr);
+  assert(addition != nullptr);
+
+  BlobLogFooter footer;
+  footer.blob_count = blob_count;
+
+  std::string checksum_method;
+  std::string checksum_value;
+  Status s = writer->AppendFooter(write_options, footer, &checksum_method,
+                                  &checksum_value);
+  if (!s.ok()) {
+    return s;
+  }
+
+  if (blob_callback_ != nullptr) {
+    const std::string blob_file_path = BlobFileName(db_path_, file_number);
+    s = blob_callback_->OnBlobFileCompleted(
+        blob_file_path, column_family_name_, /*job_id=*/0, file_number,
+        BlobFileCreationReason::kFlush, s, checksum_value, checksum_method,
+        blob_count, total_blob_bytes);
+    if (!s.ok()) {
+      return s;
+    }
+  }
+
+  *addition =
+      BlobFileAddition(file_number, blob_count, total_blob_bytes,
+                       std::move(checksum_method), std::move(checksum_value));
+  return Status::OK();
+}
+
+void BlobFilePartitionManager::AddSealedFileGarbage(
+    const SealedFile& sealed_file, std::vector<BlobFileGarbage>* garbages) {
+  assert(garbages != nullptr);
+
+  if (sealed_file.garbage_blob_count == 0) {
+    assert(sealed_file.garbage_blob_bytes == 0);
+    return;
+  }
+
+  garbages->emplace_back(sealed_file.addition.GetBlobFileNumber(),
+                         sealed_file.garbage_blob_count,
+                         sealed_file.garbage_blob_bytes);
+}
+
+bool BlobFilePartitionManager::MarkPartitionGarbage(Partition* partition,
+                                                    uint64_t file_number,
+                                                    uint64_t blob_count,
+                                                    uint64_t blob_bytes) {
+  assert(partition != nullptr);
+
+  if (!partition->writer || partition->file_number != file_number) {
+    return false;
+  }
+
+  partition->garbage_blob_count += blob_count;
+  partition->garbage_blob_bytes += blob_bytes;
+  assert(partition->garbage_blob_count <= partition->blob_count);
+  assert(partition->garbage_blob_bytes <= partition->total_blob_bytes);
+  return true;
+}
+
+bool BlobFilePartitionManager::MarkDeferredFileGarbage(DeferredFile* deferred,
+                                                       uint64_t file_number,
+                                                       uint64_t blob_count,
+                                                       uint64_t blob_bytes) {
+  assert(deferred != nullptr);
+
+  if (deferred->file_number != file_number) {
+    return false;
+  }
+
+  deferred->garbage_blob_count += blob_count;
+  deferred->garbage_blob_bytes += blob_bytes;
+  assert(deferred->garbage_blob_count <= deferred->blob_count);
+  assert(deferred->garbage_blob_bytes <= deferred->total_blob_bytes);
+  return true;
+}
+
+bool BlobFilePartitionManager::MarkSealedFileGarbage(
+    std::vector<SealedFile>* sealed_files, uint64_t file_number,
+    uint64_t blob_count, uint64_t blob_bytes) {
+  assert(sealed_files != nullptr);
+
+  for (auto& sealed_file : *sealed_files) {
+    if (sealed_file.addition.GetBlobFileNumber() != file_number) {
+      continue;
+    }
+
+    sealed_file.garbage_blob_count += blob_count;
+    sealed_file.garbage_blob_bytes += blob_bytes;
+    assert(sealed_file.garbage_blob_count <=
+           sealed_file.addition.GetTotalBlobCount());
+    assert(sealed_file.garbage_blob_bytes <=
+           sealed_file.addition.GetTotalBlobBytes());
+    return true;
+  }
+
+  return false;
+}
+
+Status BlobFilePartitionManager::MaybePrepopulateBlobCache(
+    const BlobDirectWriteSettings* settings, const Slice& original_value,
+    uint64_t blob_file_number, uint64_t blob_offset) {
+  if (settings == nullptr || settings->blob_cache == nullptr ||
+      settings->prepopulate_blob_cache != PrepopulateBlobCache::kFlushOnly) {
+    return Status::OK();
+  }
+
+  FullTypedCacheInterface<BlobContents, BlobContentsCreator> blob_cache{
+      settings->blob_cache};
+  const OffsetableCacheKey base_cache_key(db_id_, db_session_id_,
+                                          blob_file_number);
+  const CacheKey cache_key = base_cache_key.WithOffset(blob_offset);
+  return blob_cache.InsertSaved(cache_key.AsSlice(), original_value, nullptr,
+                                Cache::Priority::BOTTOM,
+                                CacheTier::kVolatileTier);
+}
+
+Status BlobFilePartitionManager::WriteBlob(
+    const WriteOptions& write_options, uint32_t column_family_id,
+    CompressionType compression, const Slice& key, const Slice& value,
+    uint64_t* blob_file_number, uint64_t* blob_offset, uint64_t* blob_size,
+    const BlobDirectWriteSettings* settings) {
+  assert(blob_file_number != nullptr);
+  assert(blob_offset != nullptr);
+  assert(blob_size != nullptr);
+
+  // Do compression before taking the partition mutex so large-value CPU work
+  // does not serialize writers. A concurrent file rollover can still cause
+  // this compressed buffer to be discarded below, which is acceptable on this
+  // non-hot failure/configuration-change path.
+  GrowableBuffer compressed_value;
+  Slice write_value = value;
+  if (compression != kNoCompression) {
+    auto& compression_state = GetDirectWriteCompressionState(compression);
+    if (compression_state.compressor == nullptr) {
+      return Status::NotSupported(
+          "Blob direct write compression type not supported");
+    }
+    Status s = LegacyForceBuiltinCompression(*compression_state.compressor,
+                                             &compression_state.working_area,
+                                             value, &compressed_value);
+    if (!s.ok()) {
+      return s;
+    }
+    write_value = Slice(compressed_value);
+  }
+
+  const uint32_t partition_idx = static_cast<uint32_t>(
+      next_partition_.fetch_add(1, std::memory_order_relaxed) %
+      num_partitions_);
+
+  {
+    MutexLock lock(&mutex_);
+    Partition* partition = partitions_[partition_idx].get();
+
+    auto seal_current_file = [&]() -> Status {
+      if (!partition->writer) {
+        return Status::OK();
+      }
+      SealedFile sealed_file;
+      Status s = SealActiveBlobFile(write_options, partition, &sealed_file);
+      if (s.ok()) {
+        current_generation_sealed_files_.push_back(std::move(sealed_file));
+      }
+      return s;
+    };
+
+    const uint64_t record_size =
+        BlobLogRecord::kHeaderSize + key.size() + write_value.size();
+    const uint64_t future_file_size =
+        partition->file_size + record_size + BlobLogFooter::kSize;
+
+    if (partition->writer &&
+        (partition->column_family_id != column_family_id ||
+         partition->compression != compression ||
+         (partition->blob_count > 0 && future_file_size > blob_file_size_))) {
+      Status s = seal_current_file();
+      if (!s.ok()) {
+        return s;
+      }
+    }
+
+    if (!partition->writer) {
+      Status s = OpenNewBlobFile(partition, column_family_id, compression,
+                                 partition_idx);
+      if (!s.ok()) {
+        return s;
+      }
+    }
+
+    uint64_t key_offset = 0;
+    Status s = partition->writer->AddRecord(write_options, key, write_value,
+                                            &key_offset, blob_offset);
+    if (!s.ok()) {
+      return s;
+    }
+
+    partition->sync_required = true;
+    partition->blob_count += 1;
+    partition->total_blob_bytes += record_size;
+    partition->file_size = BlobLogHeader::kSize + partition->total_blob_bytes;
+
+    *blob_file_number = partition->file_number;
+    *blob_size = write_value.size();
+  }
+
+  Status prepopulate_s = MaybePrepopulateBlobCache(
+      settings, value, *blob_file_number, *blob_offset);
+  if (!prepopulate_s.ok() && info_log_ != nullptr) {
+    ROCKS_LOG_WARN(info_log_,
+                   "Failed to pre-populate direct-write blob cache entry: %s",
+                   prepopulate_s.ToString().c_str());
+  }
+
+  return Status::OK();
+}
+
+void BlobFilePartitionManager::RotateCurrentGeneration() {
+  MutexLock lock(&mutex_);
+
+  GenerationBatch batch;
+  batch.sealed_files = std::move(current_generation_sealed_files_);
+  current_generation_sealed_files_.clear();
+
+  for (auto& partition : partitions_) {
+    if (!partition->writer) {
+      continue;
+    }
+
+    DeferredFile deferred;
+    deferred.writer = std::move(partition->writer);
+    deferred.file_number = partition->file_number;
+    deferred.blob_count = partition->blob_count;
+    deferred.total_blob_bytes = partition->total_blob_bytes;
+    deferred.garbage_blob_count = partition->garbage_blob_count;
+    deferred.garbage_blob_bytes = partition->garbage_blob_bytes;
+    ResetPartitionState(partition.get());
+    batch.deferred_files.emplace_back(std::move(deferred));
+  }
+
+  pending_generations_.emplace_back(std::move(batch));
+}
+
+Status BlobFilePartitionManager::PrepareFlushAdditions(
+    const WriteOptions& write_options, size_t num_generations,
+    std::vector<BlobFileAddition>* additions,
+    std::vector<BlobFileGarbage>* garbages,
+    std::vector<std::vector<uint64_t>>* generation_blob_file_numbers) {
+  assert(additions != nullptr);
+  assert(garbages != nullptr);
+  additions->clear();
+  garbages->clear();
+  if (generation_blob_file_numbers != nullptr) {
+    generation_blob_file_numbers->clear();
+  }
+
+  MutexLock lock(&mutex_);
+  if (num_generations > pending_generations_.size()) {
+    return Status::Corruption(
+        "Missing blob direct write generation metadata for flush");
+  }
+
+  for (size_t i = 0; i < num_generations; ++i) {
+    GenerationBatch& batch = pending_generations_[i];
+    while (!batch.deferred_files.empty()) {
+      DeferredFile deferred = std::move(batch.deferred_files.front());
+      batch.deferred_files.pop_front();
+
+      SealedFile sealed_file;
+      Status s = SealDeferredFile(write_options, &deferred, &sealed_file);
+      if (!s.ok()) {
+        return s;
+      }
+      // Keep each successfully sealed file attached to the generation
+      // immediately. If a later seal or the flush job itself fails, retry must
+      // reuse these exact on-disk files instead of finalizing replacements.
+      batch.sealed_files.push_back(std::move(sealed_file));
+    }
+
+    std::vector<uint64_t>* generation_file_numbers = nullptr;
+    if (generation_blob_file_numbers != nullptr) {
+      generation_blob_file_numbers->emplace_back();
+      generation_file_numbers = &generation_blob_file_numbers->back();
+      generation_file_numbers->reserve(batch.sealed_files.size());
+    }
+
+    for (const auto& sealed_file : batch.sealed_files) {
+      additions->push_back(sealed_file.addition);
+      AddSealedFileGarbage(sealed_file, garbages);
+      if (generation_file_numbers != nullptr) {
+        generation_file_numbers->push_back(
+            sealed_file.addition.GetBlobFileNumber());
+      }
+    }
+  }
+
+  return Status::OK();
+}
+
+Status BlobFilePartitionManager::MarkBlobWriteAsGarbage(uint64_t file_number,
+                                                        uint64_t blob_count,
+                                                        uint64_t blob_bytes) {
+  if (blob_count == 0) {
+    assert(blob_bytes == 0);
+    return Status::OK();
+  }
+
+  MutexLock lock(&mutex_);
+  for (auto& partition : partitions_) {
+    if (MarkPartitionGarbage(partition.get(), file_number, blob_count,
+                             blob_bytes)) {
+      return Status::OK();
+    }
+  }
+
+  if (MarkSealedFileGarbage(&current_generation_sealed_files_, file_number,
+                            blob_count, blob_bytes)) {
+    return Status::OK();
+  }
+
+  for (auto& batch : pending_generations_) {
+    for (auto& deferred : batch.deferred_files) {
+      if (MarkDeferredFileGarbage(&deferred, file_number, blob_count,
+                                  blob_bytes)) {
+        return Status::OK();
+      }
+    }
+    if (MarkSealedFileGarbage(&batch.sealed_files, file_number, blob_count,
+                              blob_bytes)) {
+      return Status::OK();
+    }
+  }
+
+  const std::string message =
+      "Could not match failed blob direct-write rollback for file #" +
+      std::to_string(file_number);
+  if (info_log_ != nullptr) {
+    ROCKS_LOG_ERROR(info_log_, "%s", message.c_str());
+  }
+  return Status::Corruption(message);
+}
+
+void BlobFilePartitionManager::CommitPreparedGenerations(
+    size_t num_generations) {
+  MutexLock lock(&mutex_);
+  while (num_generations-- > 0 && !pending_generations_.empty()) {
+    pending_generations_.pop_front();
+  }
+}
+
+Status BlobFilePartitionManager::SyncAllOpenFiles(
+    const WriteOptions& write_options) {
+  MutexLock lock(&mutex_);
+  for (const auto& partition : partitions_) {
+    if (!partition->writer || !partition->sync_required) {
+      continue;
+    }
+    Status s = partition->writer->Sync(write_options);
+    if (!s.ok()) {
+      return s;
+    }
+    partition->sync_required = false;
+  }
+  return Status::OK();
+}
+
+void BlobFilePartitionManager::GetActiveBlobFileNumbers(
+    UnorderedSet<uint64_t>* file_numbers) const {
+  assert(file_numbers != nullptr);
+  ReadLock lock(&file_partition_mutex_);
+  for (const auto& entry : file_to_partition_) {
+    file_numbers->insert(entry.first);
+  }
+}
+
+void BlobFilePartitionManager::GetProtectedBlobFileNumbers(
+    UnorderedSet<uint64_t>* file_numbers) const {
+  assert(file_numbers != nullptr);
+  ReadLock lock(&file_partition_mutex_);
+  for (const auto& entry : protected_blob_file_refs_) {
+    file_numbers->insert(entry.first);
+  }
+}
+
+void BlobFilePartitionManager::ProtectSealedBlobFileNumbers(
+    const std::vector<uint64_t>& file_numbers) {
+  if (file_numbers.empty()) {
+    return;
+  }
+
+  WriteLock lock(&file_partition_mutex_);
+  for (uint64_t file_number : file_numbers) {
+    ++protected_blob_file_refs_[file_number];
+  }
+}
+
+void BlobFilePartitionManager::UnprotectSealedBlobFileNumbers(
+    const std::vector<uint64_t>& file_numbers) {
+  if (file_numbers.empty()) {
+    return;
+  }
+
+  WriteLock lock(&file_partition_mutex_);
+  for (uint64_t file_number : file_numbers) {
+    auto it = protected_blob_file_refs_.find(file_number);
+    if (it == protected_blob_file_refs_.end()) {
+      if (info_log_ != nullptr) {
+        ROCKS_LOG_ERROR(info_log_,
+                        "Memtable blob protection underflow for file #%" PRIu64,
+                        file_number);
+      }
+      continue;
+    }
+
+    assert(it->second > 0);
+    --it->second;
+    if (it->second == 0) {
+      protected_blob_file_refs_.erase(it);
+      if (blob_file_cache_ != nullptr) {
+        // Once the last memtable-backed reference goes away, any cached reader
+        // for this file is either obsolete or can be reopened through the
+        // manifest-visible path. Evict it here so delayed protection does not
+        // leave an obsolete blob reader behind until DB close.
+        blob_file_cache_->Evict(file_number);
+      }
+    }
+  }
+}
+
+void BlobFilePartitionManager::RemoveFilePartitionMappings(
+    const std::vector<uint64_t>& file_numbers) {
+  if (file_numbers.empty()) {
+    return;
+  }
+
+  WriteLock lock(&file_partition_mutex_);
+  for (uint64_t file_number : file_numbers) {
+    file_to_partition_.erase(file_number);
+    if (blob_file_cache_ != nullptr) {
+      // In-flight direct-write reads may have cached a footer-less reader for
+      // this file before it was sealed. Drop it now so future manifest-visible
+      // reads reopen against the finalized on-disk size and footer state.
+      blob_file_cache_->Evict(file_number);
+    }
+  }
+}
+
+Status BlobFilePartitionManager::ResolveBlobDirectWriteIndex(
+    const ReadOptions& read_options, const Slice& user_key,
+    const BlobIndex& blob_idx, const Version* version,
+    BlobFileCache* blob_file_cache, PinnableSlice* blob_value) {
+  assert(blob_value != nullptr);
+
+  constexpr FilePrefetchBuffer* prefetch_buffer = nullptr;
+  constexpr uint64_t* bytes_read = nullptr;
+
+  if (version != nullptr) {
+    // Only fall back when the blob file is still owned exclusively by the
+    // write path and therefore absent from Version metadata. Once Version
+    // knows the file number, every result from Version::GetBlob(), including
+    // corruption, must propagate directly.
+    if (blob_idx.HasTTL() || blob_idx.IsInlined() ||
+        version->storage_info()->GetBlobFileMetaData(blob_idx.file_number()) !=
+            nullptr) {
+      return version->GetBlob(read_options, user_key, blob_idx, prefetch_buffer,
+                              blob_value, bytes_read);
+    }
+  }
+
+  Status s = version != nullptr ? Status::Corruption("Invalid blob file number")
+                                : Status::NotFound();
+
+  if (blob_file_cache == nullptr) {
+    return s;
+  }
+
+  CacheHandleGuard<BlobFileReader> reader;
+  s = blob_file_cache->GetBlobFileReader(read_options, blob_idx.file_number(),
+                                         &reader,
+                                         /*allow_footer_skip_retry=*/true);
+  if (!s.ok()) {
+    return s;
+  }
+
+  std::unique_ptr<BlobContents> blob_contents;
+  s = reader.GetValue()->GetBlob(read_options, user_key, blob_idx.offset(),
+                                 blob_idx.size(), blob_idx.compression(),
+                                 prefetch_buffer, nullptr, &blob_contents,
+                                 bytes_read);
+  if (s.ok()) {
+    blob_value->PinSelf(blob_contents->data());
+    return s;
+  }
+
+  if (!s.IsCorruption()) {
+    return s;
+  }
+
+  reader.Reset();
+  blob_file_cache->Evict(blob_idx.file_number());
+
+  std::unique_ptr<BlobFileReader> fresh_reader;
+  s = blob_file_cache->OpenBlobFileReaderUncached(
+      read_options, blob_idx.file_number(), &fresh_reader,
+      /*allow_footer_skip_retry=*/true);
+  if (!s.ok()) {
+    return s;
+  }
+
+  std::unique_ptr<BlobContents> fresh_contents;
+  s = fresh_reader->GetBlob(read_options, user_key, blob_idx.offset(),
+                            blob_idx.size(), blob_idx.compression(),
+                            prefetch_buffer, nullptr, &fresh_contents,
+                            bytes_read);
+  if (s.ok()) {
+    blob_value->PinSelf(fresh_contents->data());
+    CacheHandleGuard<BlobFileReader> ignored;
+    blob_file_cache
+        ->RefreshBlobFileReader(blob_idx.file_number(), &fresh_reader, &ignored)
+        .PermitUncheckedError();
+  }
+
+  return s;
+}
+
+}  // namespace ROCKSDB_NAMESPACE

db/blob/blob_file_partition_manager.h

@@ -0,0 +1,282 @@
+//  Copyright (c) Meta Platforms, Inc. and affiliates.
+//  This source code is licensed under both the GPLv2 (found in the
+//  COPYING file in the root directory) and Apache 2.0 License
+//  (found in the LICENSE.Apache file in the root directory).
+#pragma once
+
+#include <atomic>
+#include <cstdint>
+#include <deque>
+#include <functional>
+#include <memory>
+#include <string>
+#include <unordered_map>
+#include <vector>
+
+#include "db/blob/blob_file_addition.h"
+#include "db/blob/blob_file_garbage.h"
+#include "db/blob/blob_log_format.h"
+#include "db/blob/blob_write_batch_transformer.h"
+#include "port/port.h"
+#include "rocksdb/compression_type.h"
+#include "rocksdb/env.h"
+#include "rocksdb/file_system.h"
+#include "rocksdb/options.h"
+#include "rocksdb/rocksdb_namespace.h"
+#include "rocksdb/slice.h"
+#include "rocksdb/status.h"
+#include "util/hash_containers.h"
+namespace ROCKSDB_NAMESPACE {
+
+class BlobFileCache;
+class BlobFileCompletionCallback;
+class BlobIndex;
+class BlobLogWriter;
+class IOTracer;
+class Logger;
+class PinnableSlice;
+class Version;
+class WritableFileWriter;
+struct FileOptions;
+struct ReadOptions;
+
+// Manages per-partition blob files for write-path blob direct write.
+//
+// The v1 design keeps each memtable switch as one FIFO generation batch.
+// Direct-write files for that memtable are sealed and registered when the
+// matching flush commits. The current implementation still serializes blob
+// appends through one manager mutex; follow-up PRs can add independent
+// per-partition locks to allow concurrent blob writes without changing the
+// generation/flush contract. Follow-up PRs can also broaden compatibility as
+// the feature matures.
+class BlobFilePartitionManager {
+ public:
+  using FileNumberAllocator = std::function<uint64_t()>;
+
+  // Creates the per-column-family manager for write-path blob files.
+  BlobFilePartitionManager(
+      uint32_t num_partitions, FileNumberAllocator file_number_allocator,
+      FileSystem* fs, SystemClock* clock, Statistics* statistics,
+      const FileOptions& file_options, std::string db_path,
+      std::string column_family_name, uint64_t blob_file_size, bool use_fsync,
+      BlobFileCache* blob_file_cache, BlobFileCompletionCallback* blob_callback,
+      const std::vector<std::shared_ptr<EventListener>>& listeners,
+      FileChecksumGenFactory* file_checksum_gen_factory,
+      const FileTypeSet& checksum_handoff_file_types,
+      const std::shared_ptr<IOTracer>& io_tracer, std::string db_id,
+      std::string db_session_id, Logger* info_log);
+
+  // Evicts cached readers for manager-owned files during shutdown.
+  ~BlobFilePartitionManager();
+
+  // Appends one blob record to a partition file and returns the resulting
+  // BlobIndex location metadata.
+  Status WriteBlob(const WriteOptions& write_options, uint32_t column_family_id,
+                   CompressionType compression, const Slice& key,
+                   const Slice& value, uint64_t* blob_file_number,
+                   uint64_t* blob_offset, uint64_t* blob_size,
+                   const BlobDirectWriteSettings* settings = nullptr);
+
+  // Move the current active partition files into the next immutable
+  // memtable-generation batch. Called from SwitchMemtable() while DB mutex is
+  // held. No I/O is performed here.
+  void RotateCurrentGeneration();
+
+  // Seal the first `num_generations` queued immutable generations and return
+  // all blob additions and initial-garbage updates that must be registered
+  // with the matching flush. Generations stay queued until
+  // CommitPreparedGenerations() is called. If `generation_blob_file_numbers`
+  // is non-null, it receives the sealed blob file numbers for each prepared
+  // generation in the same FIFO order.
+  Status PrepareFlushAdditions(const WriteOptions& write_options,
+                               size_t num_generations,
+                               std::vector<BlobFileAddition>* additions,
+                               std::vector<BlobFileGarbage>* garbages,
+                               std::vector<std::vector<uint64_t>>*
+                                   generation_blob_file_numbers = nullptr);
+
+  // Remove the first `num_generations` prepared immutable generations after
+  // their blob-file additions and garbage edits were committed to MANIFEST.
+  void CommitPreparedGenerations(size_t num_generations);
+
+  // In v1 flush-on-write mode each AddRecord flushes to the OS immediately, so
+  // there is nothing extra to do for the non-sync write path.
+  Status FlushAllOpenFiles(const WriteOptions& /*write_options*/) {
+    return Status::OK();
+  }
+
+  // Sync the active open blob files before a sync WAL write.
+  Status SyncAllOpenFiles(const WriteOptions& write_options);
+
+  // Returns blob files still owned by the manager and therefore not yet safe
+  // to consider obsolete.
+  void GetActiveBlobFileNumbers(UnorderedSet<uint64_t>* file_numbers) const;
+
+  // Returns sealed blob files that are still reachable through live memtables
+  // or old SuperVersions and therefore must not be purged yet.
+  void GetProtectedBlobFileNumbers(UnorderedSet<uint64_t>* file_numbers) const;
+
+  // Increments / decrements memtable-held protection on sealed blob files.
+  void ProtectSealedBlobFileNumbers(const std::vector<uint64_t>& file_numbers);
+  void UnprotectSealedBlobFileNumbers(
+      const std::vector<uint64_t>& file_numbers);
+
+  // Stops protecting file numbers that are now registered in MANIFEST.
+  void RemoveFilePartitionMappings(const std::vector<uint64_t>& file_numbers);
+
+  // Marks blob records from a failed transformed write as initial garbage for
+  // the target blob file while keeping the physical bytes in place.
+  // Returns Corruption if the manager can no longer match the blob file.
+  Status MarkBlobWriteAsGarbage(uint64_t file_number, uint64_t blob_count,
+                                uint64_t blob_bytes);
+
+  // Resolves a direct-write BlobIndex by consulting manifest-visible state
+  // first, then falling back to direct blob-file reads only when the target
+  // blob file is still write-path-owned and therefore not yet tracked by
+  // Version. Existing manifest-visible read results, including I/O failures,
+  // are returned directly rather than masked by fallback logic.
+  static Status ResolveBlobDirectWriteIndex(const ReadOptions& read_options,
+                                            const Slice& user_key,
+                                            const BlobIndex& blob_idx,
+                                            const Version* version,
+                                            BlobFileCache* blob_file_cache,
+                                            PinnableSlice* blob_value);
+
+ private:
+  struct Partition {
+    // Active writer for this partition, or null when the partition is idle.
+    std::unique_ptr<BlobLogWriter> writer;
+    // Metadata tracked while the active file remains open.
+    uint64_t file_number = 0;
+    uint64_t file_size = 0;
+    uint64_t blob_count = 0;
+    uint64_t total_blob_bytes = 0;
+    // Failed transformed writes already appended to this physical file.
+    uint64_t garbage_blob_count = 0;
+    uint64_t garbage_blob_bytes = 0;
+    uint32_t column_family_id = 0;
+    CompressionType compression = kNoCompression;
+    bool sync_required = false;
+  };
+
+  struct DeferredFile {
+    // Blob writer moved out of an active partition at memtable rotation.
+    std::unique_ptr<BlobLogWriter> writer;
+    // Metadata preserved until flush preparation seals the file.
+    uint64_t file_number = 0;
+    uint64_t blob_count = 0;
+    uint64_t total_blob_bytes = 0;
+    // Failed transformed writes already appended before the file is sealed.
+    uint64_t garbage_blob_count = 0;
+    uint64_t garbage_blob_bytes = 0;
+  };
+
+  struct SealedFile {
+    // Blob-file metadata ready to be added to MANIFEST.
+    BlobFileAddition addition;
+    // Initial unreachable bytes that should be registered as garbage
+    // alongside the file addition.
+    uint64_t garbage_blob_count = 0;
+    uint64_t garbage_blob_bytes = 0;
+  };
+
+  struct GenerationBatch {
+    // Files that were still open when the memtable became immutable.
+    std::deque<DeferredFile> deferred_files;
+    // Files already sealed for this memtable generation.
+    std::vector<SealedFile> sealed_files;
+  };
+
+  // Clears all active-file bookkeeping from a partition after sealing.
+  void ResetPartitionState(Partition* partition);
+  // Marks a blob file as manager-owned until it is committed or abandoned.
+  void AddFilePartitionMapping(uint64_t file_number, uint32_t partition_idx);
+  // Stops tracking a blob file after open/seal failure.
+  void RemoveFilePartitionMapping(uint64_t file_number);
+
+  // Opens a new active blob file for one partition.
+  Status OpenNewBlobFile(Partition* partition, uint32_t column_family_id,
+                         CompressionType compression, uint32_t partition_idx);
+  // Finalizes one active partition file and returns the MANIFEST addition.
+  Status SealActiveBlobFile(const WriteOptions& write_options,
+                            Partition* partition, SealedFile* sealed_file);
+  // Finalizes one deferred file that was carried over to flush preparation.
+  Status SealDeferredFile(const WriteOptions& write_options,
+                          DeferredFile* deferred, SealedFile* sealed_file);
+  // Appends the footer, reports file completion, and builds the resulting
+  // BlobFileAddition for one sealed blob file.
+  Status FinalizeBlobFile(const WriteOptions& write_options,
+                          BlobLogWriter* writer, uint64_t file_number,
+                          uint64_t blob_count, uint64_t total_blob_bytes,
+                          BlobFileAddition* addition);
+  // Appends one initial-garbage record to the flush output if any failed
+  // transformed writes targeted the sealed blob file.
+  static void AddSealedFileGarbage(const SealedFile& sealed_file,
+                                   std::vector<BlobFileGarbage>* garbages);
+  // Returns true if the active open file matches `file_number` and consumes
+  // the failed-write garbage accounting.
+  static bool MarkPartitionGarbage(Partition* partition, uint64_t file_number,
+                                   uint64_t blob_count, uint64_t blob_bytes);
+  // Returns true if the deferred flush-preparation file matches `file_number`
+  // and consumes the failed-write garbage accounting.
+  static bool MarkDeferredFileGarbage(DeferredFile* deferred,
+                                      uint64_t file_number, uint64_t blob_count,
+                                      uint64_t blob_bytes);
+  // Returns true if one already-sealed file matches `file_number` and
+  // consumes the failed-write garbage accounting.
+  static bool MarkSealedFileGarbage(std::vector<SealedFile>* sealed_files,
+                                    uint64_t file_number, uint64_t blob_count,
+                                    uint64_t blob_bytes);
+  // Optionally seeds the blob cache with the original uncompressed value.
+  Status MaybePrepopulateBlobCache(const BlobDirectWriteSettings* settings,
+                                   const Slice& original_value,
+                                   uint64_t blob_file_number,
+                                   uint64_t blob_offset);
+
+  // Configured partition fanout and round-robin write selector.
+  const uint32_t num_partitions_;
+  std::atomic<uint64_t> next_partition_{0};
+  // Shared services needed to create, track, and cache blob files.
+  FileNumberAllocator file_number_allocator_;
+  FileSystem* fs_;
+  SystemClock* clock_;
+  Statistics* statistics_;
+  FileOptions file_options_;
+  // Column-family context used for file naming and callbacks.
+  std::string db_path_;
+  std::string column_family_name_;
+  // File creation policy and shared blob-file infrastructure.
+  uint64_t blob_file_size_;
+  bool use_fsync_;
+  BlobFileCache* blob_file_cache_;
+  BlobFileCompletionCallback* blob_callback_;
+  std::vector<std::shared_ptr<EventListener>> listeners_;
+  FileChecksumGenFactory* file_checksum_gen_factory_;
+  FileTypeSet checksum_handoff_file_types_;
+  std::shared_ptr<IOTracer> io_tracer_;
+  // DB identity used when constructing cache keys.
+  std::string db_id_;
+  std::string db_session_id_;
+  Logger* info_log_;
+
+  // One active writer slot per configured partition.
+  std::vector<std::unique_ptr<Partition>> partitions_;
+
+  // Protects partition state and generation queues.
+  mutable port::Mutex mutex_;
+  // Files already sealed while the current mutable memtable was active.
+  std::vector<SealedFile> current_generation_sealed_files_;
+  // FIFO immutable memtable generations waiting to be flushed.
+  std::deque<GenerationBatch> pending_generations_;
+
+  // Tracks blob files still owned by active or deferred write-path state until
+  // MANIFEST commit publishes them.
+  std::unordered_map<uint64_t, uint32_t> file_to_partition_;
+  // Sealed direct-write files that remain reachable from live memtables, such
+  // as old SuperVersions serving lazy iterator reads after flush commit.
+  std::unordered_map<uint64_t, uint32_t> protected_blob_file_refs_;
+  // Protects file_to_partition_ and protected_blob_file_refs_.
+  mutable port::RWMutex file_partition_mutex_;
+};
+
+}  // namespace ROCKSDB_NAMESPACE

db/blob/blob_file_reader.cc

@@ -29,7 +29,7 @@ Status BlobFileReader::Create(
     const ImmutableOptions& immutable_options, const ReadOptions& read_options,
     const FileOptions& file_options, uint32_t column_family_id,
     HistogramImpl* blob_file_read_hist, uint64_t blob_file_number,
-    const std::shared_ptr<IOTracer>& io_tracer,
+    const std::shared_ptr<IOTracer>& io_tracer, bool skip_footer_validation,
     std::unique_ptr<BlobFileReader>* blob_file_reader) {
   assert(blob_file_reader);
   assert(!*blob_file_reader);
@@ -40,7 +40,8 @@ Status BlobFileReader::Create(
   {
     const Status s =
         OpenFile(immutable_options, file_options, blob_file_read_hist,
-                 blob_file_number, io_tracer, &file_size, &file_reader);
+                 blob_file_number, io_tracer, &file_size, &file_reader,
+                 /*skip_footer_size_check=*/skip_footer_validation);
     if (!s.ok()) {
       return s;
     }
@@ -61,7 +62,7 @@ Status BlobFileReader::Create(
     }
   }
 
-  {
+  if (!skip_footer_validation) {
     const Status s =
         ReadFooter(file_reader.get(), read_options, file_size, statistics);
     if (!s.ok()) {
@@ -76,9 +77,10 @@ Status BlobFileReader::Create(
         compression_type);
   }
 
-  blob_file_reader->reset(new BlobFileReader(
-      std::move(file_reader), file_size, compression_type,
-      std::move(decompressor), immutable_options.clock, statistics));
+  blob_file_reader->reset(
+      new BlobFileReader(std::move(file_reader), file_size, compression_type,
+                         std::move(decompressor), immutable_options.clock,
+                         statistics, !skip_footer_validation));
 
   return Status::OK();
 }
@@ -87,7 +89,8 @@ Status BlobFileReader::OpenFile(
     const ImmutableOptions& immutable_options, const FileOptions& file_opts,
     HistogramImpl* blob_file_read_hist, uint64_t blob_file_number,
     const std::shared_ptr<IOTracer>& io_tracer, uint64_t* file_size,
-    std::unique_ptr<RandomAccessFileReader>* file_reader) {
+    std::unique_ptr<RandomAccessFileReader>* file_reader,
+    bool skip_footer_size_check) {
   assert(file_size);
   assert(file_reader);
 
@@ -112,17 +115,26 @@ Status BlobFileReader::OpenFile(
     }
   }
 
-  if (*file_size < BlobLogHeader::kSize + BlobLogFooter::kSize) {
+  if (!skip_footer_size_check &&
+      *file_size < BlobLogHeader::kSize + BlobLogFooter::kSize) {
+    return Status::Corruption("Malformed blob file");
+  }
+  if (skip_footer_size_check && *file_size < BlobLogHeader::kSize) {
     return Status::Corruption("Malformed blob file");
   }
 
   std::unique_ptr<FSRandomAccessFile> file;
+  FileOptions reader_file_opts = file_opts;
+
+  if (skip_footer_size_check && reader_file_opts.use_direct_reads) {
+    reader_file_opts.use_direct_reads = false;
+  }
 
   {
     TEST_SYNC_POINT("BlobFileReader::OpenFile:NewRandomAccessFile");
 
     const Status s =
-        fs->NewRandomAccessFile(blob_file_path, file_opts, &file, dbg);
+        fs->NewRandomAccessFile(blob_file_path, reader_file_opts, &file, dbg);
     if (!s.ok()) {
       return s;
     }
@@ -291,13 +303,14 @@ BlobFileReader::BlobFileReader(
     std::unique_ptr<RandomAccessFileReader>&& file_reader, uint64_t file_size,
     CompressionType compression_type,
     std::shared_ptr<Decompressor> decompressor, SystemClock* clock,
-    Statistics* statistics)
+    Statistics* statistics, bool has_footer)
     : file_reader_(std::move(file_reader)),
       file_size_(file_size),
       compression_type_(compression_type),
       decompressor_(std::move(decompressor)),
       clock_(clock),
-      statistics_(statistics) {
+      statistics_(statistics),
+      has_footer_(has_footer) {
   assert(file_reader_);
 }
 
@@ -312,7 +325,8 @@ Status BlobFileReader::GetBlob(
 
   const uint64_t key_size = user_key.size();
 
-  if (!IsValidBlobOffset(offset, key_size, value_size, file_size_)) {
+  if (!IsValidBlobOffset(offset, key_size, value_size, file_size_,
+                         has_footer_)) {
     return Status::Corruption("Invalid blob offset");
   }
 
@@ -428,7 +442,8 @@ void BlobFileReader::MultiGetBlob(
     const uint64_t offset = req->offset;
     const uint64_t value_size = req->len;
 
-    if (!IsValidBlobOffset(offset, key_size, value_size, file_size_)) {
+    if (!IsValidBlobOffset(offset, key_size, value_size, file_size_,
+                           has_footer_)) {
       *req->status = Status::Corruption("Invalid blob offset");
       continue;
     }
@@ -454,6 +469,13 @@ void BlobFileReader::MultiGetBlob(
 
   RecordTick(statistics_, BLOB_DB_BLOB_FILE_BYTES_READ, total_len);
 
+  if (read_reqs.empty()) {
+    if (bytes_read) {
+      *bytes_read = 0;
+    }
+    return;
+  }
+
   Buffer buf;
   AlignedBuf aligned_buf;
 

db/blob/blob_file_reader.h

@@ -36,7 +36,20 @@ class BlobFileReader {
                        HistogramImpl* blob_file_read_hist,
                        uint64_t blob_file_number,
                        const std::shared_ptr<IOTracer>& io_tracer,
-                       std::unique_ptr<BlobFileReader>* reader);
+                       std::unique_ptr<BlobFileReader>* reader) {
+    return Create(immutable_options, read_options, file_options,
+                  column_family_id, blob_file_read_hist, blob_file_number,
+                  io_tracer, /*skip_footer_validation=*/false, reader);
+  }
+
+  // Allows opening in-flight direct-write blob files by optionally skipping
+  // footer validation.
+  static Status Create(
+      const ImmutableOptions& immutable_options,
+      const ReadOptions& read_options, const FileOptions& file_options,
+      uint32_t column_family_id, HistogramImpl* blob_file_read_hist,
+      uint64_t blob_file_number, const std::shared_ptr<IOTracer>& io_tracer,
+      bool skip_footer_validation, std::unique_ptr<BlobFileReader>* reader);
 
   BlobFileReader(const BlobFileReader&) = delete;
   BlobFileReader& operator=(const BlobFileReader&) = delete;
@@ -63,18 +76,22 @@ class BlobFileReader {
   uint64_t GetFileSize() const { return file_size_; }
 
  private:
+  // `has_footer` tracks whether offset validation should reserve footer space.
   BlobFileReader(std::unique_ptr<RandomAccessFileReader>&& file_reader,
                  uint64_t file_size, CompressionType compression_type,
                  std::shared_ptr<Decompressor> decompressor, SystemClock* clock,
-                 Statistics* statistics);
+                 Statistics* statistics, bool has_footer);
 
+  // `skip_footer_size_check` is used for direct-write files that are still
+  // missing their footer at open time.
   static Status OpenFile(const ImmutableOptions& immutable_options,
                          const FileOptions& file_opts,
                          HistogramImpl* blob_file_read_hist,
                          uint64_t blob_file_number,
                          const std::shared_ptr<IOTracer>& io_tracer,
                          uint64_t* file_size,
-                         std::unique_ptr<RandomAccessFileReader>* file_reader);
+                         std::unique_ptr<RandomAccessFileReader>* file_reader,
+                         bool skip_footer_size_check);
 
   static Status ReadHeader(const RandomAccessFileReader* file_reader,
                            const ReadOptions& read_options,
@@ -110,6 +127,8 @@ class BlobFileReader {
   std::shared_ptr<Decompressor> decompressor_;
   SystemClock* clock_;
   Statistics* statistics_;
+  // False when the reader was opened before the blob file footer was written.
+  bool has_footer_;
 };
 
 }  // namespace ROCKSDB_NAMESPACE

db/blob/blob_log_format.h

@@ -148,13 +148,26 @@ struct BlobLogRecord {
 
 // Checks whether a blob offset is potentially valid or not.
 inline bool IsValidBlobOffset(uint64_t value_offset, uint64_t key_size,
-                              uint64_t value_size, uint64_t file_size) {
-  if (value_offset <
-      BlobLogHeader::kSize + BlobLogRecord::kHeaderSize + key_size) {
+                              uint64_t value_size, uint64_t file_size,
+                              bool has_footer = true) {
+  constexpr uint64_t kMinPrefix =
+      BlobLogHeader::kSize + BlobLogRecord::kHeaderSize;
+  if (value_offset < kMinPrefix) {
+    return false;
+  }
+  if (value_offset - kMinPrefix < key_size) {
     return false;
   }
 
-  if (value_offset + value_size + BlobLogFooter::kSize > file_size) {
+  const uint64_t footer_size = has_footer ? BlobLogFooter::kSize : 0;
+  if (file_size < footer_size) {
+    return false;
+  }
+  const uint64_t max_value_end = file_size - footer_size;
+  if (value_size > max_value_end) {
+    return false;
+  }
+  if (value_offset > max_value_end - value_size) {
     return false;
   }
 

db/blob/blob_source.cc

@@ -20,6 +20,21 @@
 
 namespace ROCKSDB_NAMESPACE {
 
+namespace {
+
+Status AppendBlobRefreshRetryFailure(const Status& stale_status,
+                                     const Status& retry_status) {
+  assert(stale_status.IsCorruption());
+  assert(!retry_status.ok());
+  if (retry_status.IsCorruption()) {
+    return retry_status;
+  }
+  return Status::CopyAppendMessage(
+      stale_status, "; refresh retry failed: ", retry_status.ToString());
+}
+
+}  // namespace
+
 BlobSource::BlobSource(const ImmutableOptions& immutable_options,
                        const MutableCFOptions& mutable_cf_options,
                        const std::string& db_id,
@@ -231,6 +246,38 @@ Status BlobSource::GetBlob(const ReadOptions& read_options,
     s = blob_file_reader.GetValue()->GetBlob(
         read_options, user_key, offset, value_size, compression_type,
         prefetch_buffer, allocator, &blob_contents, &read_size);
+    if (s.IsCorruption()) {
+      const Status stale_status = s;
+      blob_file_reader.Reset();
+      blob_file_cache_->Evict(file_number);
+
+      std::unique_ptr<BlobFileReader> fresh_reader;
+      s = blob_file_cache_->OpenBlobFileReaderUncached(
+          read_options, file_number, &fresh_reader,
+          /*allow_footer_skip_retry=*/false);
+      if (!s.ok()) {
+        return AppendBlobRefreshRetryFailure(stale_status, s);
+      }
+
+      if (compression_type != fresh_reader->GetCompressionType()) {
+        return Status::Corruption(
+            "Compression type mismatch when reading blob");
+      }
+
+      blob_contents.reset();
+      read_size = 0;
+      s = fresh_reader->GetBlob(read_options, user_key, offset, value_size,
+                                compression_type, prefetch_buffer, allocator,
+                                &blob_contents, &read_size);
+      if (!s.ok()) {
+        s = AppendBlobRefreshRetryFailure(stale_status, s);
+      } else {
+        CacheHandleGuard<BlobFileReader> ignored_reader;
+        blob_file_cache_
+            ->RefreshBlobFileReader(file_number, &fresh_reader, &ignored_reader)
+            .PermitUncheckedError();
+      }
+    }
     if (!s.ok()) {
       return s;
     }
@@ -397,6 +444,89 @@ void BlobSource::MultiGetBlobFromOneFile(const ReadOptions& read_options,
     blob_file_reader.GetValue()->MultiGetBlob(read_options, allocator,
                                               _blob_reqs, &_bytes_read);
 
+    bool needs_reader_refresh = false;
+    for (const auto& blob_req : _blob_reqs) {
+      BlobReadRequest* const req = blob_req.first;
+      assert(req != nullptr);
+      assert(req->status != nullptr);
+      if (req->status->IsCorruption()) {
+        needs_reader_refresh = true;
+        break;
+      }
+    }
+
+    if (needs_reader_refresh) {
+      blob_file_reader.Reset();
+      blob_file_cache_->Evict(file_number);
+
+      std::unique_ptr<BlobFileReader> fresh_reader;
+      s = blob_file_cache_->OpenBlobFileReaderUncached(
+          read_options, file_number, &fresh_reader,
+          /*allow_footer_skip_retry=*/false);
+      if (!s.ok()) {
+        for (const auto& blob_req : _blob_reqs) {
+          BlobReadRequest* const req = blob_req.first;
+          assert(req != nullptr);
+          assert(req->status != nullptr);
+          if (req->status->IsCorruption()) {
+            *req->status = AppendBlobRefreshRetryFailure(*req->status, s);
+          }
+        }
+        return;
+      }
+
+      autovector<std::pair<BlobReadRequest*, std::unique_ptr<BlobContents>>>
+          retry_blob_reqs;
+      autovector<Status> stale_statuses;
+      for (auto& blob_req : _blob_reqs) {
+        BlobReadRequest* const req = blob_req.first;
+        assert(req != nullptr);
+        assert(req->status != nullptr);
+        if (!req->status->IsCorruption()) {
+          continue;
+        }
+
+        stale_statuses.emplace_back(*req->status);
+        *req->status = Status::OK();
+        blob_req.second.reset();
+        retry_blob_reqs.emplace_back(req, std::unique_ptr<BlobContents>());
+      }
+
+      uint64_t refreshed_bytes_read = 0;
+      fresh_reader->MultiGetBlob(read_options, allocator, retry_blob_reqs,
+                                 &refreshed_bytes_read);
+      _bytes_read += refreshed_bytes_read;
+
+      bool install_fresh_reader = false;
+      for (size_t i = 0; i < retry_blob_reqs.size(); ++i) {
+        auto& retried_blob_req = retry_blob_reqs[i];
+        BlobReadRequest* const retried_req = retried_blob_req.first;
+        assert(retried_req != nullptr);
+        if (retried_req->status->ok()) {
+          install_fresh_reader = true;
+        } else {
+          *retried_req->status = AppendBlobRefreshRetryFailure(
+              stale_statuses[i], *retried_req->status);
+        }
+
+        for (auto& blob_req : _blob_reqs) {
+          if (blob_req.first != retried_req) {
+            continue;
+          }
+
+          blob_req.second = std::move(retried_blob_req.second);
+          break;
+        }
+      }
+
+      if (install_fresh_reader) {
+        CacheHandleGuard<BlobFileReader> ignored_reader;
+        blob_file_cache_
+            ->RefreshBlobFileReader(file_number, &fresh_reader, &ignored_reader)
+            .PermitUncheckedError();
+      }
+    }
+
     if (blob_cache_ && read_options.fill_cache) {
       // If filling cache is allowed and a cache is configured, try to put
       // the blob(s) to the cache.

db/blob/blob_source_test.cc

@@ -1044,6 +1044,156 @@ TEST_F(BlobSourceTest, MultiGetBlobsFromCache) {
   }
 }
 
+TEST_F(BlobSourceTest, GetBlobPreservesCorruptionDetailsWhenRefreshOpenFails) {
+  options_.cf_paths.emplace_back(
+      test::PerThreadDBPath(env_,
+                            "BlobSourceTest_GetBlobPreservesCorruptionDetails"),
+      0);
+
+  DestroyAndReopen(options_);
+
+  ImmutableOptions immutable_options(options_);
+  MutableCFOptions mutable_cf_options(options_);
+
+  constexpr uint32_t column_family_id = 1;
+  constexpr bool has_ttl = false;
+  constexpr ExpirationRange expiration_range;
+  constexpr uint64_t blob_file_number = 1;
+
+  const std::string key_str = "key0";
+  const std::string blob_str = "blob0";
+  std::vector<Slice> keys{Slice(key_str)};
+  std::vector<Slice> blobs{Slice(blob_str)};
+  std::vector<uint64_t> blob_offsets(1);
+  std::vector<uint64_t> blob_sizes(1);
+
+  const uint64_t file_size = BlobLogHeader::kSize + BlobLogRecord::kHeaderSize +
+                             key_str.size() + blob_str.size() +
+                             BlobLogFooter::kSize;
+
+  WriteBlobFile(immutable_options, column_family_id, has_ttl, expiration_range,
+                expiration_range, blob_file_number, keys, blobs, kNoCompression,
+                blob_offsets, blob_sizes);
+
+  constexpr size_t capacity = 1024;
+  std::shared_ptr<Cache> backing_cache = NewLRUCache(capacity);
+
+  FileOptions file_options;
+  std::unique_ptr<BlobFileCache> blob_file_cache =
+      std::make_unique<BlobFileCache>(
+          backing_cache.get(), &immutable_options, &file_options,
+          column_family_id, nullptr /*HistogramImpl*/, nullptr /*IOTracer*/);
+
+  BlobSource blob_source(immutable_options, mutable_cf_options, db_id_,
+                         db_session_id_, blob_file_cache.get());
+
+  ReadOptions read_options;
+  read_options.verify_checksums = true;
+  constexpr FilePrefetchBuffer* prefetch_buffer = nullptr;
+
+  PinnableSlice value;
+  uint64_t bytes_read = 0;
+  ASSERT_OK(blob_source.GetBlob(
+      read_options, keys[0], blob_file_number, blob_offsets[0], file_size,
+      blob_sizes[0], kNoCompression, prefetch_buffer, &value, &bytes_read));
+  ASSERT_EQ(value, blobs[0]);
+
+  const std::string blob_file_path =
+      BlobFileName(immutable_options.cf_paths.front().path, blob_file_number);
+  ASSERT_OK(env_->DeleteFile(blob_file_path));
+
+  PinnableSlice invalid_value;
+  bytes_read = 0;
+  Status s =
+      blob_source.GetBlob(read_options, keys[0], blob_file_number, file_size,
+                          file_size, blob_sizes[0], kNoCompression,
+                          prefetch_buffer, &invalid_value, &bytes_read);
+  ASSERT_TRUE(s.IsCorruption());
+  ASSERT_EQ(bytes_read, 0);
+
+  const std::string status_str = s.ToString();
+  ASSERT_NE(status_str.find("Invalid blob offset"), std::string::npos);
+  ASSERT_NE(status_str.find("refresh retry failed"), std::string::npos);
+  ASSERT_NE(status_str.find("IO error"), std::string::npos);
+}
+
+TEST_F(BlobSourceTest,
+       MultiGetBlobPreservesCorruptionDetailsWhenRefreshOpenFails) {
+  options_.cf_paths.emplace_back(
+      test::PerThreadDBPath(
+          env_, "BlobSourceTest_MultiGetBlobPreservesCorruptionDetails"),
+      0);
+
+  DestroyAndReopen(options_);
+
+  ImmutableOptions immutable_options(options_);
+  MutableCFOptions mutable_cf_options(options_);
+
+  constexpr uint32_t column_family_id = 1;
+  constexpr bool has_ttl = false;
+  constexpr ExpirationRange expiration_range;
+  constexpr uint64_t blob_file_number = 1;
+
+  const std::string key_str = "key0";
+  const std::string blob_str = "blob0";
+  std::vector<Slice> keys{Slice(key_str)};
+  std::vector<Slice> blobs{Slice(blob_str)};
+  std::vector<uint64_t> blob_offsets(1);
+  std::vector<uint64_t> blob_sizes(1);
+
+  const uint64_t file_size = BlobLogHeader::kSize + BlobLogRecord::kHeaderSize +
+                             key_str.size() + blob_str.size() +
+                             BlobLogFooter::kSize;
+
+  WriteBlobFile(immutable_options, column_family_id, has_ttl, expiration_range,
+                expiration_range, blob_file_number, keys, blobs, kNoCompression,
+                blob_offsets, blob_sizes);
+
+  constexpr size_t capacity = 1024;
+  std::shared_ptr<Cache> backing_cache = NewLRUCache(capacity);
+
+  FileOptions file_options;
+  std::unique_ptr<BlobFileCache> blob_file_cache =
+      std::make_unique<BlobFileCache>(
+          backing_cache.get(), &immutable_options, &file_options,
+          column_family_id, nullptr /*HistogramImpl*/, nullptr /*IOTracer*/);
+
+  BlobSource blob_source(immutable_options, mutable_cf_options, db_id_,
+                         db_session_id_, blob_file_cache.get());
+
+  ReadOptions read_options;
+  read_options.verify_checksums = true;
+  constexpr FilePrefetchBuffer* prefetch_buffer = nullptr;
+
+  PinnableSlice value;
+  uint64_t bytes_read = 0;
+  ASSERT_OK(blob_source.GetBlob(
+      read_options, keys[0], blob_file_number, blob_offsets[0], file_size,
+      blob_sizes[0], kNoCompression, prefetch_buffer, &value, &bytes_read));
+  ASSERT_EQ(value, blobs[0]);
+
+  const std::string blob_file_path =
+      BlobFileName(immutable_options.cf_paths.front().path, blob_file_number);
+  ASSERT_OK(env_->DeleteFile(blob_file_path));
+
+  std::array<Status, 1> statuses;
+  std::array<PinnableSlice, 1> values;
+  autovector<BlobReadRequest> blob_reqs;
+  blob_reqs.emplace_back(keys[0], file_size, blob_sizes[0], kNoCompression,
+                         &values[0], &statuses[0]);
+
+  bytes_read = 0;
+  blob_source.MultiGetBlobFromOneFile(read_options, blob_file_number, file_size,
+                                      blob_reqs, &bytes_read);
+  ASSERT_TRUE(statuses[0].IsCorruption());
+  ASSERT_EQ(bytes_read, 0);
+
+  const std::string status_str = statuses[0].ToString();
+  ASSERT_NE(status_str.find("Invalid blob offset"), std::string::npos);
+  ASSERT_NE(status_str.find("refresh retry failed"), std::string::npos);
+  ASSERT_NE(status_str.find("IO error"), std::string::npos);
+}
+
 class BlobSecondaryCacheTest : public DBTestBase {
  protected:
  public:

db/blob/blob_write_batch_transformer.cc

@@ -0,0 +1,188 @@
+//  Copyright (c) Meta Platforms, Inc. and affiliates.
+//  This source code is licensed under both the GPLv2 (found in the
+//  COPYING file in the root directory) and Apache 2.0 License
+//  (found in the LICENSE.Apache file in the root directory).
+
+//  Copyright (c) 2011-present, Facebook, Inc.  All rights reserved.
+//  This source code is licensed under both the GPLv2 (found in the
+//  COPYING file in the root directory) and Apache 2.0 License
+//  (found in the LICENSE.Apache file in the root directory).
+
+#include "db/blob/blob_write_batch_transformer.h"
+
+#include "db/blob/blob_file_partition_manager.h"
+#include "db/blob/blob_index.h"
+#include "db/blob/blob_log_format.h"
+#include "db/write_batch_internal.h"
+
+namespace ROCKSDB_NAMESPACE {
+
+BlobWriteBatchTransformer::BlobWriteBatchTransformer(
+    const BlobPartitionManagerProvider& partition_mgr_provider,
+    WriteBatch* output_batch,
+    const BlobDirectWriteSettingsProvider& settings_provider,
+    const WriteOptions& write_options)
+    : partition_mgr_provider_(partition_mgr_provider),
+      output_batch_(output_batch),
+      settings_provider_(settings_provider),
+      write_options_(write_options) {
+  assert(partition_mgr_provider_);
+  assert(output_batch_);
+  assert(settings_provider_);
+}
+
+Status BlobWriteBatchTransformer::TransformBatch(
+    const WriteOptions& write_options, WriteBatch* input_batch,
+    WriteBatch* output_batch,
+    const BlobPartitionManagerProvider& partition_mgr_provider,
+    const BlobDirectWriteSettingsProvider& settings_provider, bool* transformed,
+    std::vector<BlobFilePartitionManager*>* used_managers,
+    std::vector<RollbackInfo>* rollback_infos) {
+  assert(input_batch);
+  assert(output_batch);
+  assert(transformed);
+
+  output_batch->Clear();
+  *transformed = false;
+
+  BlobWriteBatchTransformer transformer(partition_mgr_provider, output_batch,
+                                        settings_provider, write_options);
+
+  Status s = input_batch->Iterate(&transformer);
+  *transformed = transformer.HasTransformed();
+
+  if (used_managers) {
+    used_managers->assign(transformer.used_managers_.begin(),
+                          transformer.used_managers_.end());
+  }
+
+  if (rollback_infos) {
+    *rollback_infos = std::move(transformer.rollback_infos_);
+  }
+
+  return s;
+}
+
+Status BlobWriteBatchTransformer::PutCF(uint32_t column_family_id,
+                                        const Slice& key, const Slice& value) {
+  // Use cached settings/manager for the same CF to avoid per-entry lookup.
+  if (column_family_id != cached_cf_id_) {
+    cached_settings_ = settings_provider_(column_family_id);
+    cached_partition_mgr_ = partition_mgr_provider_(column_family_id);
+    cached_cf_id_ = column_family_id;
+  }
+  const auto& settings = cached_settings_;
+
+  if (!cached_partition_mgr_ || !settings.enable_blob_direct_write ||
+      value.size() < settings.min_blob_size) {
+    return WriteBatchInternal::Put(output_batch_, column_family_id, key, value);
+  }
+
+  uint64_t blob_file_number = 0;
+  uint64_t blob_offset = 0;
+  uint64_t blob_size = 0;
+
+  Status s = cached_partition_mgr_->WriteBlob(
+      write_options_, column_family_id, settings.compression_type, key, value,
+      &blob_file_number, &blob_offset, &blob_size, &settings);
+  if (!s.ok()) {
+    return s;
+  }
+
+  used_managers_.insert(cached_partition_mgr_);
+
+  // Track the exact file so stale transformed attempts can rollback
+  // per-file rather than smearing bytes across all partitions at seal time.
+  const uint64_t record_bytes =
+      BlobLogRecord::kHeaderSize + key.size() + blob_size;
+  rollback_infos_.push_back(
+      {cached_partition_mgr_, blob_file_number, /*count=*/1, record_bytes});
+
+  BlobIndex::EncodeBlob(&blob_index_buf_, blob_file_number, blob_offset,
+                        blob_size, settings.compression_type);
+
+  has_transformed_ = true;
+  return WriteBatchInternal::PutBlobIndex(output_batch_, column_family_id, key,
+                                          blob_index_buf_);
+}
+
+Status BlobWriteBatchTransformer::TimedPutCF(uint32_t column_family_id,
+                                             const Slice& key,
+                                             const Slice& value,
+                                             uint64_t write_time) {
+  // TimedPut: pass through without blob separation for now.
+  return WriteBatchInternal::TimedPut(output_batch_, column_family_id, key,
+                                      value, write_time);
+}
+
+Status BlobWriteBatchTransformer::PutEntityCF(uint32_t column_family_id,
+                                              const Slice& key,
+                                              const Slice& entity) {
+  // Wide-column/entity writes stay serialized as-is. BDW v1 only rewrites
+  // plain Put values into BlobIndex entries.
+  return WriteBatchInternal::PutEntitySerialized(output_batch_,
+                                                 column_family_id, key, entity);
+}
+
+Status BlobWriteBatchTransformer::DeleteCF(uint32_t column_family_id,
+                                           const Slice& key) {
+  return WriteBatchInternal::Delete(output_batch_, column_family_id, key);
+}
+
+Status BlobWriteBatchTransformer::SingleDeleteCF(uint32_t column_family_id,
+                                                 const Slice& key) {
+  return WriteBatchInternal::SingleDelete(output_batch_, column_family_id, key);
+}
+
+Status BlobWriteBatchTransformer::DeleteRangeCF(uint32_t column_family_id,
+                                                const Slice& begin_key,
+                                                const Slice& end_key) {
+  return WriteBatchInternal::DeleteRange(output_batch_, column_family_id,
+                                         begin_key, end_key);
+}
+
+Status BlobWriteBatchTransformer::MergeCF(uint32_t column_family_id,
+                                          const Slice& key,
+                                          const Slice& value) {
+  return WriteBatchInternal::Merge(output_batch_, column_family_id, key, value);
+}
+
+Status BlobWriteBatchTransformer::PutBlobIndexCF(uint32_t column_family_id,
+                                                 const Slice& key,
+                                                 const Slice& value) {
+  // Already a blob index — pass through unchanged.
+  return WriteBatchInternal::PutBlobIndex(output_batch_, column_family_id, key,
+                                          value);
+}
+
+void BlobWriteBatchTransformer::LogData(const Slice& blob) {
+  output_batch_->PutLogData(blob).PermitUncheckedError();
+}
+
+Status BlobWriteBatchTransformer::MarkBeginPrepare(bool unprepared) {
+  return WriteBatchInternal::InsertBeginPrepare(
+      output_batch_, !unprepared /* write_after_commit */, unprepared);
+}
+
+Status BlobWriteBatchTransformer::MarkEndPrepare(const Slice& xid) {
+  return WriteBatchInternal::InsertEndPrepare(output_batch_, xid);
+}
+
+Status BlobWriteBatchTransformer::MarkCommit(const Slice& xid) {
+  return WriteBatchInternal::MarkCommit(output_batch_, xid);
+}
+
+Status BlobWriteBatchTransformer::MarkCommitWithTimestamp(const Slice& xid,
+                                                          const Slice& ts) {
+  return WriteBatchInternal::MarkCommitWithTimestamp(output_batch_, xid, ts);
+}
+
+Status BlobWriteBatchTransformer::MarkRollback(const Slice& xid) {
+  return WriteBatchInternal::MarkRollback(output_batch_, xid);
+}
+
+Status BlobWriteBatchTransformer::MarkNoop(bool /*empty_batch*/) {
+  return WriteBatchInternal::InsertNoop(output_batch_);
+}
+
+}  // namespace ROCKSDB_NAMESPACE

db/blob/blob_write_batch_transformer.h

@@ -0,0 +1,155 @@
+//  Copyright (c) Meta Platforms, Inc. and affiliates.
+//  This source code is licensed under both the GPLv2 (found in the
+//  COPYING file in the root directory) and Apache 2.0 License
+//  (found in the LICENSE.Apache file in the root directory).
+#pragma once
+
+#include <cstdint>
+#include <functional>
+#include <memory>
+#include <string>
+#include <unordered_map>
+#include <vector>
+
+#include "rocksdb/advanced_options.h"
+#include "rocksdb/compression_type.h"
+#include "rocksdb/options.h"
+#include "rocksdb/rocksdb_namespace.h"
+#include "rocksdb/slice.h"
+#include "rocksdb/status.h"
+#include "rocksdb/write_batch.h"
+#include "util/hash_containers.h"
+
+namespace ROCKSDB_NAMESPACE {
+
+class BlobFilePartitionManager;
+class Cache;
+
+// Callback to look up per-CF blob settings.
+struct BlobDirectWriteSettings {
+  // Whether this column family should attempt direct-write blob separation.
+  bool enable_blob_direct_write = false;
+  // Minimum value size eligible for direct-write blob separation.
+  uint64_t min_blob_size = 0;
+  // Compression to use for newly written blob records.
+  CompressionType compression_type = kNoCompression;
+  // Raw pointer — the Cache is owned by ColumnFamilyOptions and outlives all
+  // settings snapshots. Using raw avoids 2 atomic ref-count ops per Put().
+  Cache* blob_cache = nullptr;
+  // Blob-cache prepopulation policy for direct-write records.
+  PrepopulateBlobCache prepopulate_blob_cache = PrepopulateBlobCache::kDisable;
+};
+
+using BlobDirectWriteSettingsProvider =
+    std::function<BlobDirectWriteSettings(uint32_t column_family_id)>;
+
+// Callback to look up per-CF partition manager.
+using BlobPartitionManagerProvider =
+    std::function<BlobFilePartitionManager*(uint32_t column_family_id)>;
+
+// Transforms a WriteBatch by writing large values directly to blob files
+// and replacing them with BlobIndex entries. Non-qualifying entries
+// (small values, deletes, merges, etc.) are passed through unchanged.
+class BlobWriteBatchTransformer : public WriteBatch::Handler {
+ public:
+  struct RollbackInfo {
+    // Manager that owns the file updated during transform.
+    BlobFilePartitionManager* partition_mgr = nullptr;
+    // Blob file number written during transform.
+    uint64_t file_number = 0;
+    // Number of blob records appended to that file.
+    uint64_t count = 0;
+    // Number of bytes appended to that file.
+    uint64_t bytes = 0;
+  };
+
+  // Constructs a per-batch transformer. Most callers should use
+  // TransformBatch() instead of driving the handler directly.
+  BlobWriteBatchTransformer(
+      const BlobPartitionManagerProvider& partition_mgr_provider,
+      WriteBatch* output_batch,
+      const BlobDirectWriteSettingsProvider& settings_provider,
+      const WriteOptions& write_options);
+
+  // Transform a WriteBatch. If no values qualify for blob separation,
+  // output_batch will be empty and the caller should use the original batch.
+  // If any values are separated, output_batch contains the full transformed
+  // batch. used_managers (if non-null) receives the set of partition managers
+  // that had data written to them, so the caller can flush/sync them.
+  // rollback_infos (if non-null) receives the exact file/count/byte writes so
+  // a failed transformed attempt can be accounted as initial garbage. Partial
+  // results are returned even if this method fails after some blob writes.
+  static Status TransformBatch(
+      const WriteOptions& write_options, WriteBatch* input_batch,
+      WriteBatch* output_batch,
+      const BlobPartitionManagerProvider& partition_mgr_provider,
+      const BlobDirectWriteSettingsProvider& settings_provider,
+      bool* transformed,
+      std::vector<BlobFilePartitionManager*>* used_managers = nullptr,
+      std::vector<RollbackInfo>* rollback_infos = nullptr);
+
+  // WriteBatch::Handler overrides
+  Status PutCF(uint32_t column_family_id, const Slice& key,
+               const Slice& value) override;
+
+  Status TimedPutCF(uint32_t column_family_id, const Slice& key,
+                    const Slice& value, uint64_t write_time) override;
+
+  Status PutEntityCF(uint32_t column_family_id, const Slice& key,
+                     const Slice& entity) override;
+
+  Status DeleteCF(uint32_t column_family_id, const Slice& key) override;
+
+  Status SingleDeleteCF(uint32_t column_family_id, const Slice& key) override;
+
+  Status DeleteRangeCF(uint32_t column_family_id, const Slice& begin_key,
+                       const Slice& end_key) override;
+
+  Status MergeCF(uint32_t column_family_id, const Slice& key,
+                 const Slice& value) override;
+
+  Status PutBlobIndexCF(uint32_t column_family_id, const Slice& key,
+                        const Slice& value) override;
+
+  void LogData(const Slice& blob) override;
+
+  Status MarkBeginPrepare(bool unprepared = false) override;
+  Status MarkEndPrepare(const Slice& xid) override;
+  Status MarkCommit(const Slice& xid) override;
+  Status MarkCommitWithTimestamp(const Slice& xid, const Slice& ts) override;
+  Status MarkRollback(const Slice& xid) override;
+  Status MarkNoop(bool empty_batch) override;
+
+  // Returns true once at least one value has been rewritten to a BlobIndex.
+  bool HasTransformed() const { return has_transformed_; }
+
+ private:
+  // Callback to look up the partition manager for a given column family ID.
+  BlobPartitionManagerProvider partition_mgr_provider_;
+  // Output batch that receives transformed entries (BlobIndex for qualifying
+  // values, passthrough for everything else).
+  WriteBatch* output_batch_;
+  // Callback to look up blob direct write settings for a given CF ID.
+  BlobDirectWriteSettingsProvider settings_provider_;
+  // Write options from the caller, forwarded to WriteBlob calls.
+  const WriteOptions& write_options_;
+  // True once at least one value has been separated into a blob file.
+  bool has_transformed_ = false;
+  // Reusable buffer for encoding BlobIndex entries (avoids per-Put alloc).
+  std::string blob_index_buf_;
+  // Per-batch cache of the last CF's settings and manager, avoiding
+  // redundant provider lookups when consecutive entries share the same CF.
+  uint32_t cached_cf_id_ = UINT32_MAX;
+  // Cached settings for `cached_cf_id_`.
+  BlobDirectWriteSettings cached_settings_;
+  // Cached partition manager for `cached_cf_id_`.
+  BlobFilePartitionManager* cached_partition_mgr_ = nullptr;
+  // Set of partition managers that received data during this batch,
+  // returned to the caller so it can flush/sync them.
+  UnorderedSet<BlobFilePartitionManager*> used_managers_;
+  // Exact blob writes performed during this batch. We only aggregate these
+  // entries if rollback is needed so the normal path keeps minimal overhead.
+  std::vector<RollbackInfo> rollback_infos_;
+};
+
+}  // namespace ROCKSDB_NAMESPACE

db/blob/db_blob_basic_test.cc

@@ -3,17 +3,29 @@
 //  COPYING file in the root directory) and Apache 2.0 License
 //  (found in the LICENSE.Apache file in the root directory).
 
+#include <algorithm>
 #include <array>
+#include <set>
 #include <sstream>
 #include <string>
 
 #include "cache/compressed_secondary_cache.h"
+#include "db/blob/blob_file_partition_manager.h"
 #include "db/blob/blob_index.h"
 #include "db/blob/blob_log_format.h"
+#include "db/blob/blob_log_sequential_reader.h"
+#include "db/column_family.h"
 #include "db/db_test_util.h"
 #include "db/db_with_timestamp_test_util.h"
+#include "file/filename.h"
+#include "file/random_access_file_reader.h"
 #include "port/stack_trace.h"
+#include "rocksdb/convenience.h"
+#include "rocksdb/trace_reader_writer.h"
+#include "rocksdb/trace_record.h"
+#include "rocksdb/utilities/replayer.h"
 #include "test_util/sync_point.h"
+#include "util/compression.h"
 #include "utilities/fault_injection_env.h"
 
 namespace ROCKSDB_NAMESPACE {

db/column_family.cc

@@ -18,6 +18,7 @@
 #include <vector>
 
 #include "db/blob/blob_file_cache.h"
+#include "db/blob/blob_file_partition_manager.h"
 #include "db/blob/blob_source.h"
 #include "db/compaction/compaction_picker.h"
 #include "db/compaction/compaction_picker_fifo.h"
@@ -496,6 +497,15 @@ ColumnFamilyOptions SanitizeCfOptions(const ImmutableDBOptions& db_options,
       result.memtable_avg_op_scan_flush_trigger = 0;
     }
   }
+  if (result.enable_blob_direct_write && !result.enable_blob_files) {
+    ROCKS_LOG_WARN(db_options.info_log.get(),
+                   "enable_blob_direct_write requires enable_blob_files=true. "
+                   "Disabling blob direct write.");
+    result.enable_blob_direct_write = false;
+  }
+  if (result.blob_direct_write_partitions == 0) {
+    result.blob_direct_write_partitions = 1;
+  }
   return result;
 }
 
@@ -783,6 +793,11 @@ ColumnFamilyData::~ColumnFamilyData() {
   }
 }
 
+void ColumnFamilyData::SetBlobPartitionManager(
+    std::shared_ptr<BlobFilePartitionManager> mgr) {
+  blob_partition_manager_ = std::move(mgr);
+}
+
 bool ColumnFamilyData::UnrefAndTryDelete() {
   int old_refs = refs_.fetch_sub(1);
   assert(old_refs > 0);
@@ -1524,6 +1539,32 @@ Status ColumnFamilyData::ValidateOptions(
 
   const auto* ucmp = cf_options.comparator;
   assert(ucmp);
+  if (cf_options.enable_blob_direct_write) {
+    if (db_options.enable_pipelined_write) {
+      return Status::NotSupported(
+          "Blob direct write v1 does not support pipelined writes.");
+    }
+    if (db_options.allow_concurrent_memtable_write) {
+      return Status::NotSupported(
+          "Blob direct write v1 does not support concurrent memtable writes.");
+    }
+    if (db_options.unordered_write) {
+      return Status::NotSupported(
+          "Blob direct write v1 does not support unordered writes.");
+    }
+    if (db_options.two_write_queues) {
+      return Status::NotSupported(
+          "Blob direct write v1 does not support two write queues.");
+    }
+    if (cf_options.experimental_mempurge_threshold > 0.0) {
+      return Status::NotSupported(
+          "Blob direct write does not support MemPurge.");
+    }
+    if (ucmp->timestamp_size() > 0) {
+      return Status::NotSupported(
+          "Blob direct write does not support user-defined timestamps.");
+    }
+  }
   if (ucmp->timestamp_size() > 0 &&
       !cf_options.persist_user_defined_timestamps) {
     if (db_options.atomic_flush) {

db/column_family.h

@@ -49,6 +49,7 @@ class InstrumentedMutex;
 class InstrumentedMutexLock;
 struct SuperVersionContext;
 class BlobFileCache;
+class BlobFilePartitionManager;
 class BlobSource;
 
 extern const double kIncSlowdownRatio;
@@ -415,6 +416,19 @@ class ColumnFamilyData {
   TableCache* table_cache() const { return table_cache_.get(); }
   BlobFileCache* blob_file_cache() const { return blob_file_cache_.get(); }
   BlobSource* blob_source() const { return blob_source_.get(); }
+  // Returns the write-path blob partition manager for this CF, or null if BDW
+  // is disabled.
+  BlobFilePartitionManager* blob_partition_manager() const {
+    return blob_partition_manager_.get();
+  }
+  // Returns a shared ownership handle for cold paths that must keep the
+  // manager alive beyond the lifetime of this ColumnFamilyData.
+  std::shared_ptr<BlobFilePartitionManager> blob_partition_manager_handle()
+      const {
+    return blob_partition_manager_;
+  }
+  // Installs the write-path blob partition manager for this CF.
+  void SetBlobPartitionManager(std::shared_ptr<BlobFilePartitionManager> mgr);
 
   // See documentation in compaction_picker.h
   // REQUIRES: DB mutex held
@@ -648,6 +662,8 @@ class ColumnFamilyData {
   std::unique_ptr<TableCache> table_cache_;
   std::unique_ptr<BlobFileCache> blob_file_cache_;
   std::unique_ptr<BlobSource> blob_source_;
+  // Per-CF manager for write-path blob direct-write files.
+  std::shared_ptr<BlobFilePartitionManager> blob_partition_manager_;
 
   std::unique_ptr<InternalStats> internal_stats_;
 

db/db_filesnapshot.cc

@@ -205,7 +205,6 @@ Status DBImpl::GetLiveFilesStorageInfo(
 
   // NOTE: This implementation was largely migrated from Checkpoint.
 
-  Status s;
   VectorWalPtr live_wal_files;
   bool flush_memtable = true;
   if (!immutable_db_options_.allow_2pc) {
@@ -217,12 +216,12 @@ Status DBImpl::GetLiveFilesStorageInfo(
       // Don't take archived log size into account when calculating wal
       // size for flush, and don't need to verify consistency with manifest
       // here & now.
-      s = wal_manager_.GetSortedWalFiles(live_wal_files,
+      Status wal_s = wal_manager_.GetSortedWalFiles(live_wal_files,
                                                     /* need_seqnos */ false,
                                                     /*include_archived*/ false);
 
-      if (!s.ok()) {
-        return s;
+      if (!wal_s.ok()) {
+        return wal_s;
       }
 
       // Don't flush column families if total log size is smaller than
@@ -243,9 +242,24 @@ Status DBImpl::GetLiveFilesStorageInfo(
   // This is a modified version of GetLiveFiles, to get access to more
   // metadata.
   mutex_.Lock();
+  bool wal_locked = false;
+  const bool needs_blob_direct_write_flush =
+      HasInFlightBlobDirectWriteFilesWithLockHeld();
+  if (needs_blob_direct_write_flush && !flush_memtable) {
+    mutex_.Unlock();
+    return Status::NotSupported(
+        "Blob direct write requires flushing active blob files before "
+        "capturing live files. Retry with flush enabled.");
+  }
   if (flush_memtable) {
-    bool wal_locked = lock_wal_count_ > 0;
+    wal_locked = lock_wal_count_ > 0;
     if (wal_locked) {
+      if (needs_blob_direct_write_flush) {
+        mutex_.Unlock();
+        return Status::NotSupported(
+            "Blob direct write requires flushing active blob files before "
+            "capturing live files. Retry with WAL unlocked.");
+      }
       ROCKS_LOG_INFO(immutable_db_options_.info_log,
                      "Can't FlushForGetLiveFiles while WAL is locked");
     } else {
@@ -393,18 +407,16 @@ Status DBImpl::GetLiveFilesStorageInfo(
   TEST_SYNC_POINT("CheckpointImpl::CreateCheckpoint:SavedLiveFiles1");
   TEST_SYNC_POINT("CheckpointImpl::CreateCheckpoint:SavedLiveFiles2");
 
-  if (s.ok()) {
   // FlushWAL is required to ensure we can physically copy everything
   // logically written to the WAL. (Sync not strictly required for
   // active WAL to be copied rather than hard linked, even when
   // Checkpoint guarantees that the copied-to file is sync-ed. Plus we can't
   // help track_and_verify_wals_in_manifest after manifest_size is
   // already determined.)
-    s = FlushWAL(/*sync=*/false);
+  Status s = FlushWAL(/*sync=*/false);
   if (s.IsNotSupported()) {  // read-only DB or similar
     s = Status::OK();
   }
-  }
 
   TEST_SYNC_POINT("CheckpointImpl::CreateCustomCheckpoint:AfterGetLive1");
   TEST_SYNC_POINT("CheckpointImpl::CreateCustomCheckpoint:AfterGetLive2");

db/db_impl/db_impl.cc

@@ -23,11 +23,14 @@
 #include <stdexcept>
 #include <string>
 #include <unordered_map>
+#include <unordered_set>
 #include <utility>
 #include <vector>
 
 #include "db/arena_wrapped_db_iter.h"
 #include "db/attribute_group_iterator_impl.h"
+#include "db/blob/blob_file_partition_manager.h"
+#include "db/blob/blob_index.h"
 #include "db/builder.h"
 #include "db/coalescing_iterator.h"
 #include "db/compaction/compaction_job.h"
@@ -485,11 +488,37 @@ void DBImpl::CancelAllBackgroundWork(bool wait) {
   s.PermitUncheckedError();
 
   InstrumentedMutexLock l(&mutex_);
-  if (!shutting_down_.load(std::memory_order_acquire) &&
+  // Blob direct-write relies on clean shutdown flushes to register live blob
+  // files in MANIFEST before close. WAL recovery of unregistered files is out
+  // of scope in v1.
+  const bool force_flush_for_blob_direct_write =
+      HasAnyBlobDirectWriteColumnFamilyWithLockHeld();
+  const bool flush_for_unpersisted_data =
       has_unpersisted_data_.load(std::memory_order_relaxed) &&
-      !mutable_db_options_.avoid_flush_during_shutdown) {
+      !mutable_db_options_.avoid_flush_during_shutdown;
+  const bool already_shutting_down =
+      shutting_down_.load(std::memory_order_acquire);
+  if (already_shutting_down && force_flush_for_blob_direct_write &&
+      HasInFlightBlobDirectWriteFilesWithLockHeld()) {
+    // Flush jobs treat `shutting_down_` as a rollback condition, so once the
+    // shutdown marker is already published we cannot safely promise a final
+    // BDW registration flush anymore. Make that loss of crash-safety explicit.
+    ROCKS_LOG_WARN(immutable_db_options_.info_log,
+                   "Shutdown already in progress before blob direct-write "
+                   "close flush; active blob files may remain unregistered.");
+  }
+  if (!already_shutting_down &&
+      (flush_for_unpersisted_data || force_flush_for_blob_direct_write)) {
     s = DBImpl::FlushAllColumnFamilies(FlushOptions(), FlushReason::kShutDown);
-    s.PermitUncheckedError();  //**TODO: What to do on error?
+    if (!s.ok()) {
+      ROCKS_LOG_WARN(
+          immutable_db_options_.info_log, "Shutdown flush failed%s: %s",
+          force_flush_for_blob_direct_write
+              ? "; active blob direct-write files may remain unregistered"
+              : "",
+          s.ToString().c_str());
+      s.PermitUncheckedError();
+    }
   }
 
   // Cancel awaiting remote compactions
@@ -523,6 +552,88 @@ void DBImpl::UntrackDataFiles() {
                       /*track=*/false);
 }
 
+bool DBImpl::HasAnyBlobDirectWriteColumnFamily() {
+  return blob_direct_write_cf_count_.load(std::memory_order_relaxed) > 0;
+}
+
+bool DBImpl::HasAnyBlobDirectWriteColumnFamilyWithLockHeld() {
+  mutex_.AssertHeld();
+  return blob_direct_write_cf_count_.load(std::memory_order_relaxed) > 0;
+}
+
+bool DBImpl::HasInFlightBlobDirectWriteFilesWithLockHeld() {
+  mutex_.AssertHeld();
+  if (!HasAnyBlobDirectWriteColumnFamilyWithLockHeld()) {
+    return false;
+  }
+
+  UnorderedSet<uint64_t> active_blob_direct_write_files;
+  for (auto* cfd : *versions_->GetColumnFamilySet()) {
+    if (cfd->IsDropped()) {
+      continue;
+    }
+
+    auto* mgr = cfd->blob_partition_manager();
+    if (mgr == nullptr) {
+      continue;
+    }
+
+    mgr->GetActiveBlobFileNumbers(&active_blob_direct_write_files);
+    if (!active_blob_direct_write_files.empty()) {
+      return true;
+    }
+  }
+
+  return false;
+}
+
+void DBImpl::MaybeInitBlobDirectWriteColumnFamily(
+    ColumnFamilyData* cfd, const ColumnFamilyOptions& cf_options,
+    const std::string& column_family_name) {
+  assert(cfd != nullptr);
+
+  if (!cf_options.enable_blob_files || !cf_options.enable_blob_direct_write) {
+    return;
+  }
+  if (cfd->blob_partition_manager() != nullptr) {
+    // DB::Open(..., create_missing_column_families=true, ...) can create the
+    // missing CF during open and then revisit the same handle in the post-open
+    // initialization loop below. Treat BDW initialization as idempotent for
+    // that path so we do not double-register the same CF.
+    return;
+  }
+
+  auto mgr = std::make_shared<BlobFilePartitionManager>(
+      cf_options.blob_direct_write_partitions,
+      [vs = versions_.get()]() { return vs->NewFileNumber(); }, fs_.get(),
+      immutable_db_options_.clock, stats_, file_options_, dbname_,
+      column_family_name, cf_options.blob_file_size,
+      immutable_db_options_.use_fsync, cfd->blob_file_cache(), &blob_callback_,
+      immutable_db_options_.listeners,
+      immutable_db_options_.file_checksum_gen_factory.get(),
+      immutable_db_options_.checksum_handoff_file_types, io_tracer_, db_id_,
+      db_session_id_, immutable_db_options_.info_log.get());
+
+  cfd->SetBlobPartitionManager(std::move(mgr));
+  RegisterBlobDirectWriteColumnFamily();
+}
+
+void DBImpl::RegisterBlobDirectWriteColumnFamily() {
+  blob_direct_write_cf_count_.fetch_add(1, std::memory_order_relaxed);
+}
+
+void DBImpl::UnregisterBlobDirectWriteColumnFamily() {
+  uint32_t current =
+      blob_direct_write_cf_count_.load(std::memory_order_relaxed);
+  while (current != 0) {
+    if (blob_direct_write_cf_count_.compare_exchange_weak(
+            current, current - 1, std::memory_order_relaxed)) {
+      return;
+    }
+  }
+  assert(false);
+}
+
 Status DBImpl::CloseHelper() {
   // Guarantee that there is no background error recovery in progress before
   // continuing with the shutdown
@@ -1360,6 +1471,7 @@ Status DBImpl::SetOptions(
       for (const auto& cfd_opts : column_family_datas) {
         InstallSuperVersionForConfigChange(cfd_opts.first, &sv_context);
       }
+
       persist_options_status =
           WriteOptionsFile(write_options, true /*db_mutex_already_held*/);
       bg_cv_.SignalAll();
@@ -2480,6 +2592,106 @@ bool DBImpl::ShouldReferenceSuperVersion(const MergeContext& merge_context) {
              merge_context.GetOperands().size();
 }
 
+static Slice GetBlobLookupUserKey(const Slice& user_key,
+                                  const std::string* timestamp,
+                                  std::string* user_key_with_ts) {
+  if (timestamp == nullptr || timestamp->empty()) {
+    return user_key;
+  }
+
+  assert(user_key_with_ts != nullptr);
+  user_key_with_ts->assign(user_key.data(), user_key.size());
+  user_key_with_ts->append(timestamp->data(), timestamp->size());
+  return Slice(*user_key_with_ts);
+}
+
+static bool MaybeResolveDirectWriteBlobIndex(
+    const ReadOptions& read_options, const Slice& key,
+    bool resolve_direct_write_blob_index, const Version* current,
+    ColumnFamilyData* cfd, PinnableSlice* value, PinnableWideColumns* columns,
+    Status* s, bool* is_blob_index, bool* value_found = nullptr) {
+  if (!s->ok() || !*is_blob_index || !resolve_direct_write_blob_index ||
+      (!value && !columns)) {
+    return false;
+  }
+
+  // Once a blob file is registered in Version metadata, the normal integrated
+  // blob read path is already agnostic to whether the file came from direct
+  // write or flush/compaction. This helper only covers the pre-flush gap where
+  // a memtable can already reference a blob file that is not yet
+  // manifest-visible.
+  if (read_options.read_tier == kBlockCacheTier) {
+    if (value != nullptr) {
+      value->Reset();
+    } else {
+      assert(columns != nullptr);
+      columns->Reset();
+    }
+    if (value_found != nullptr) {
+      *value_found = false;
+    }
+    *s = Status::Incomplete();
+    *is_blob_index = false;
+    return true;
+  }
+
+  Slice blob_index_slice;
+  if (value != nullptr) {
+    if (value->size() > 0) {
+      blob_index_slice = Slice(value->data(), value->size());
+    } else {
+      blob_index_slice = Slice(*(value->GetSelf()));
+    }
+  } else {
+    assert(columns != nullptr);
+    assert(!columns->columns().empty());
+    blob_index_slice = columns->columns().front().value();
+  }
+
+  BlobIndex blob_idx;
+  *s = blob_idx.DecodeFrom(blob_index_slice);
+  if (s->ok()) {
+    if (blob_idx.HasTTL()) {
+      *s =
+          Status::Corruption("Unexpected TTL blob index for blob direct write");
+    } else {
+      PinnableSlice resolved_value;
+      PinnableSlice* target = value ? value : &resolved_value;
+      if (value != nullptr) {
+        value->Reset();
+      }
+      *s = BlobFilePartitionManager::ResolveBlobDirectWriteIndex(
+          read_options, key, blob_idx, current, cfd->blob_file_cache(), target);
+      if (s->ok() && columns != nullptr) {
+        columns->SetPlainValue(std::move(*target));
+      }
+    }
+  }
+
+  *is_blob_index = false;
+  return true;
+}
+
+static void MaybeResolveDirectWriteBlobIndexFromMemtable(
+    const ReadOptions& read_options, const Slice& user_key,
+    const std::string* timestamp, std::string* blob_lookup_key_storage,
+    bool resolve_direct_write_blob_index, const Version* current,
+    ColumnFamilyData* cfd, PinnableSlice* value, PinnableWideColumns* columns,
+    Status* s, bool* is_blob_index, bool* value_found = nullptr) {
+  if (resolve_direct_write_blob_index) {
+    const bool blob_resolved = MaybeResolveDirectWriteBlobIndex(
+        read_options,
+        GetBlobLookupUserKey(user_key, timestamp, blob_lookup_key_storage),
+        resolve_direct_write_blob_index, current, cfd, value, columns, s,
+        is_blob_index, value_found);
+    if (!blob_resolved && value != nullptr) {
+      value->PinSelf();
+    }
+  } else if (value != nullptr) {
+    value->PinSelf();
+  }
+}
+
 Status DBImpl::GetImpl(const ReadOptions& read_options, const Slice& key,
                        GetImplOptions& get_impl_options) {
   assert(get_impl_options.value != nullptr ||
@@ -2616,40 +2828,55 @@ Status DBImpl::GetImpl(const ReadOptions& read_options, const Slice& key,
   bool skip_memtable = (read_options.read_tier == kPersistedTier &&
                         has_unpersisted_data_.load(std::memory_order_relaxed));
   bool done = false;
-  std::string* timestamp =
-      ucmp->timestamp_size() > 0 ? get_impl_options.timestamp : nullptr;
+  bool is_blob_index = false;
+  bool* is_blob_ptr = get_impl_options.is_blob_index;
+  auto* partition_mgr = cfd->blob_partition_manager();
+  std::string timestamp_storage;
+  std::string* timestamp = nullptr;
+  if (ucmp->timestamp_size() > 0) {
+    timestamp = get_impl_options.timestamp != nullptr
+                    ? get_impl_options.timestamp
+                    : (partition_mgr != nullptr ? &timestamp_storage : nullptr);
+  }
+  if (partition_mgr != nullptr && !is_blob_ptr && get_impl_options.get_value) {
+    is_blob_ptr = &is_blob_index;
+  }
+  const bool resolve_direct_write_blob_index =
+      partition_mgr != nullptr && (is_blob_ptr == &is_blob_index);
+  std::string blob_lookup_key_storage;
   if (!skip_memtable) {
     // Get value associated with key
     if (get_impl_options.get_value) {
-      if (sv->mem->Get(
+      if (sv->mem->Get(lkey,
+                       get_impl_options.value
+                           ? get_impl_options.value->GetSelf()
+                           : nullptr,
+                       get_impl_options.columns, timestamp, &s, &merge_context,
+                       &max_covering_tombstone_seq, read_options,
+                       false /* immutable_memtable */,
+                       get_impl_options.callback, is_blob_ptr)) {
+        done = true;
+        MaybeResolveDirectWriteBlobIndexFromMemtable(
+            read_options, key, timestamp, &blob_lookup_key_storage,
+            resolve_direct_write_blob_index, sv->current, cfd,
+            get_impl_options.value, get_impl_options.columns, &s,
+            &is_blob_index, get_impl_options.value_found);
+
+        RecordTick(stats_, MEMTABLE_HIT);
+      } else if ((s.ok() || s.IsMergeInProgress()) &&
+                 sv->imm->Get(
                      lkey,
                      get_impl_options.value ? get_impl_options.value->GetSelf()
                                             : nullptr,
                      get_impl_options.columns, timestamp, &s, &merge_context,
                      &max_covering_tombstone_seq, read_options,
-              false /* immutable_memtable */, get_impl_options.callback,
-              get_impl_options.is_blob_index)) {
+                     get_impl_options.callback, is_blob_ptr)) {
         done = true;
-
-        if (get_impl_options.value) {
-          get_impl_options.value->PinSelf();
-        }
-
-        RecordTick(stats_, MEMTABLE_HIT);
-      } else if ((s.ok() || s.IsMergeInProgress()) &&
-                 sv->imm->Get(lkey,
-                              get_impl_options.value
-                                  ? get_impl_options.value->GetSelf()
-                                  : nullptr,
-                              get_impl_options.columns, timestamp, &s,
-                              &merge_context, &max_covering_tombstone_seq,
-                              read_options, get_impl_options.callback,
-                              get_impl_options.is_blob_index)) {
-        done = true;
-
-        if (get_impl_options.value) {
-          get_impl_options.value->PinSelf();
-        }
+        MaybeResolveDirectWriteBlobIndexFromMemtable(
+            read_options, key, timestamp, &blob_lookup_key_storage,
+            resolve_direct_write_blob_index, sv->current, cfd,
+            get_impl_options.value, get_impl_options.columns, &s,
+            &is_blob_index, get_impl_options.value_found);
 
         RecordTick(stats_, MEMTABLE_HIT);
       }
@@ -2689,8 +2916,16 @@ Status DBImpl::GetImpl(const ReadOptions& read_options, const Slice& key,
         get_impl_options.get_value ? get_impl_options.value_found : nullptr,
         nullptr, nullptr,
         get_impl_options.get_value ? get_impl_options.callback : nullptr,
-        get_impl_options.get_value ? get_impl_options.is_blob_index : nullptr,
+        get_impl_options.get_value ? is_blob_ptr : nullptr,
         get_impl_options.get_value);
+    if (get_impl_options.get_value && resolve_direct_write_blob_index) {
+      MaybeResolveDirectWriteBlobIndex(
+          read_options,
+          GetBlobLookupUserKey(key, timestamp, &blob_lookup_key_storage),
+          resolve_direct_write_blob_index, sv->current, cfd,
+          get_impl_options.value, get_impl_options.columns, &s, &is_blob_index,
+          get_impl_options.value_found);
+    }
     RecordTick(stats_, MEMTABLE_MISS);
   }
 
@@ -3269,9 +3504,13 @@ void DBImpl::MultiGetWithCallbackImpl(
     const ReadOptions& read_options, ColumnFamilyHandle* column_family,
     ReadCallback* callback,
     autovector<KeyContext*, MultiGetContext::MAX_BATCH_SIZE>* sorted_keys) {
+  assert(sorted_keys != nullptr);
   std::array<ColumnFamilySuperVersionPair, 1> cf_sv_pairs;
   cf_sv_pairs[0] = ColumnFamilySuperVersionPair(column_family, nullptr);
   size_t num_keys = sorted_keys->size();
+  if (num_keys == 0) {
+    return;
+  }
   SequenceNumber consistent_seqnum = kMaxSequenceNumber;
   bool sv_from_thread_local = false;
   Status s = MultiCFSnapshot<std::array<ColumnFamilySuperVersionPair, 1>>(
@@ -3345,6 +3584,13 @@ Status DBImpl::MultiGetImpl(
 
   assert(sorted_keys);
   assert(start_key + num_keys <= sorted_keys->size());
+  if (num_keys == 0) {
+    return Status::OK();
+  }
+  auto* cfh = static_cast_with_check<ColumnFamilyHandleImpl>(
+      (*sorted_keys)[start_key]->column_family);
+  ColumnFamilyData* cfd = cfh->cfd();
+  auto* partition_mgr = cfd->blob_partition_manager();
   // Clear the timestamps for returning results so that we can distinguish
   // between tombstone or key that has never been written
   for (size_t i = start_key; i < start_key + num_keys; ++i) {
@@ -3429,6 +3675,16 @@ Status DBImpl::MultiGetImpl(
     assert(key);
     assert(key->s);
 
+    if (partition_mgr != nullptr && key->s->ok() && key->is_blob_index) {
+      std::string blob_lookup_key_storage;
+      MaybeResolveDirectWriteBlobIndex(
+          read_options,
+          GetBlobLookupUserKey(*key->key, key->timestamp,
+                               &blob_lookup_key_storage),
+          /*resolve_direct_write_blob_index=*/true, super_version->current, cfd,
+          key->value, key->columns, key->s, &key->is_blob_index);
+    }
+
     if (key->s->ok()) {
       const auto& merge_threshold = read_options.merge_operand_count_threshold;
       if (merge_threshold.has_value() &&
@@ -3818,6 +4074,7 @@ Status DBImpl::CreateColumnFamilyImpl(const ReadOptions& read_options,
       auto* cfd =
           versions_->GetColumnFamilySet()->GetColumnFamily(column_family_name);
       assert(cfd != nullptr);
+      MaybeInitBlobDirectWriteColumnFamily(cfd, cf_options, column_family_name);
       InstallSuperVersionForConfigChange(cfd, &sv_context);
 
       if (!cfd->mem()->IsSnapshotSupported()) {
@@ -3915,6 +4172,9 @@ Status DBImpl::DropColumnFamilyImpl(ColumnFamilyHandle* column_family) {
       s = versions_->LogAndApply(cfd, read_options, write_options, &edit,
                                  &mutex_, directories_.GetDbDir());
       write_thread_.ExitUnbatched(&w);
+      if (s.ok() && cfd->blob_partition_manager() != nullptr) {
+        UnregisterBlobDirectWriteColumnFamily();
+      }
     }
     if (s.ok()) {
       auto& moptions = cfd->GetLatestMutableCFOptions();
@@ -4134,9 +4394,10 @@ ArenaWrappedDBIter* DBImpl::NewIteratorImpl(
   // Laying out the iterators in the order of being accessed makes it more
   // likely that any iterator pointer is close to the iterator it points to so
   // that they are likely to be in the same cache line and/or page.
-  return NewArenaWrappedDbIterator(
-      env_, read_options, cfh, sv, snapshot, read_callback, this,
-      expose_blob_index, allow_refresh, /*allow_mark_memtable_for_flush=*/true);
+  return NewArenaWrappedDbIterator(env_, read_options, cfh, sv, snapshot,
+                                   read_callback, this, expose_blob_index,
+                                   allow_refresh,
+                                   /*allow_mark_memtable_for_flush=*/true);
 }
 
 std::unique_ptr<Iterator> DBImpl::NewCoalescingIterator(
@@ -4262,7 +4523,8 @@ Status DBImpl::NewIterators(
           cf_sv_pair.super_version->mutable_cf_options,
           cf_sv_pair.cfd->user_comparator(), iter,
           cf_sv_pair.super_version->current, kMaxSequenceNumber,
-          nullptr /*read_callback*/, /*active_mem=*/nullptr, cf_sv_pair.cfh));
+          nullptr /*read_callback*/, /*active_mem=*/nullptr, cf_sv_pair.cfh,
+          /*expose_blob_index=*/false, /*arena=*/nullptr));
     }
   } else {
     for (const auto& cf_sv_pair : cf_sv_pairs) {

db/db_impl/db_impl.h

@@ -260,6 +260,10 @@ class DBImpl : public DB {
       const WriteOptions& options,
       std::shared_ptr<WriteBatchWithIndex> wbwi) override;
 
+  // Returns true if any live column family currently has blob direct write
+  // enabled.
+  bool HasAnyBlobDirectWriteColumnFamily();
+
   using DB::Get;
   Status Get(const ReadOptions& _read_options,
              ColumnFamilyHandle* column_family, const Slice& key,
@@ -1591,7 +1595,26 @@ class DBImpl : public DB {
                    PostMemTableCallback* post_memtable_callback = nullptr,
                    std::shared_ptr<WriteBatchWithIndex> wbwi = nullptr);
 
+  // Per-WriteImpl state that keeps BDW column families pinned through
+  // referenced SuperVersions until the transformed write either commits or
+  // rolls back.
+  struct BlobDirectWriteContext;
+  // Rewrites a write batch for blob direct write when the current DB and batch
+  // shape are compatible, recording touched managers and rollback metadata in
+  // `blob_direct_write_ctx`.
+  Status MaybeTransformBatchForBlobDirectWrite(
+      const WriteOptions& write_options, WriteBatch** batch,
+      bool should_write_to_memtable, bool lookup_from_write_thread,
+      WriteBatch* transformed_storage,
+      BlobDirectWriteContext* blob_direct_write_ctx);
+  // Flushes or syncs all blob direct-write managers touched by the current
+  // transformed write before the write can proceed.
+  Status SyncBlobDirectWriteManagers(
+      const WriteOptions& write_options,
+      const BlobDirectWriteContext& blob_direct_write_ctx);
+
   Status PipelinedWriteImpl(const WriteOptions& options, WriteBatch* updates,
+                            WriteBatch* trace_batch,
                             WriteCallback* callback = nullptr,
                             UserWriteCallback* user_write_cb = nullptr,
                             uint64_t* wal_used = nullptr, uint64_t log_ref = 0,
@@ -1620,7 +1643,7 @@ class DBImpl : public DB {
   // marks start of a new sub-batch.
   Status WriteImplWALOnly(
       WriteThread* write_thread, const WriteOptions& options,
-      WriteBatch* updates, WriteCallback* callback,
+      WriteBatch* updates, WriteBatch* trace_batch, WriteCallback* callback,
       UserWriteCallback* user_write_cb, uint64_t* wal_used,
       const uint64_t log_ref, uint64_t* seq_used, const size_t sub_batch_cnt,
       PreReleaseCallback* pre_release_callback, const AssignOrder assign_order,
@@ -1778,6 +1801,21 @@ class DBImpl : public DB {
   friend class CompactionServiceTest_PreservedOptionsRemoteCompaction_Test;
 #endif
 
+  // Same as HasAnyBlobDirectWriteColumnFamily(), but requires `mutex_` held.
+  bool HasAnyBlobDirectWriteColumnFamilyWithLockHeld();
+  // Returns true if any BDW column family still owns blob files that have not
+  // yet been made visible through MANIFEST. Requires `mutex_` held.
+  bool HasInFlightBlobDirectWriteFilesWithLockHeld();
+  // Creates and attaches the per-CF blob direct-write partition manager when
+  // the column family is opened with the feature enabled.
+  void MaybeInitBlobDirectWriteColumnFamily(
+      ColumnFamilyData* cfd, const ColumnFamilyOptions& cf_options,
+      const std::string& column_family_name);
+  // Increments the DB-level count of live blob direct-write column families.
+  void RegisterBlobDirectWriteColumnFamily();
+  // Decrements the DB-level count of live blob direct-write column families.
+  void UnregisterBlobDirectWriteColumnFamily();
+
   struct CompactionState;
   struct PrepickedCompaction;
   struct PurgeFileInfo;
@@ -3138,6 +3176,11 @@ class DBImpl : public DB {
   // Used when disableWAL is true.
   std::atomic<bool> has_unpersisted_data_{false};
 
+  // Number of live column families with an active blob direct write partition
+  // manager. This keeps the read/write fast paths at O(1) when the feature is
+  // completely disabled for the DB.
+  std::atomic<uint32_t> blob_direct_write_cf_count_{0};
+
   // if an attempt was made to flush all column families that
   // the oldest log depends on but uncommitted data in the oldest
   // log prevents the log from being released.

db/db_impl/db_impl_compaction_flush.cc

@@ -8,7 +8,9 @@
 // found in the LICENSE file. See the AUTHORS file for names of contributors.
 #include <cinttypes>
 #include <deque>
+#include <unordered_map>
 
+#include "db/blob/blob_file_partition_manager.h"
 #include "db/builder.h"
 #include "db/db_impl/db_impl.h"
 #include "db/error_handler.h"
@@ -279,12 +281,49 @@ Status DBImpl::FlushMemTableToOutputFile(
                      flush_reason);
 
   bool switched_to_mempurge = false;
+  size_t prepared_blob_generations = 0;
+  std::vector<uint64_t> sealed_blob_numbers;
   // Within flush_job.Run, rocksdb may call event listener to notify
   // file creation and deletion.
   //
   // Note that flush_job.Run will unlock and lock the db_mutex,
   // and EventListener callback will be called when the db_mutex
   // is unlocked by the current thread.
+  if (s.ok()) {
+    auto mgr_handle = cfd->blob_partition_manager_handle();
+    auto* mgr = mgr_handle.get();
+    if (mgr != nullptr) {
+      prepared_blob_generations = flush_job.GetMemTables().size();
+      std::vector<BlobFileAddition> write_path_additions;
+      std::vector<BlobFileGarbage> write_path_garbages;
+      std::vector<std::vector<uint64_t>> generation_blob_file_numbers;
+      mutex_.Unlock();
+      s = mgr->PrepareFlushAdditions(
+          write_options, prepared_blob_generations, &write_path_additions,
+          &write_path_garbages, &generation_blob_file_numbers);
+      mutex_.Lock();
+      if (!s.ok()) {
+        prepared_blob_generations = 0;
+      } else if (!write_path_additions.empty()) {
+        const auto& memtables = flush_job.GetMemTables();
+        assert(generation_blob_file_numbers.size() == memtables.size());
+        for (size_t i = 0; i < generation_blob_file_numbers.size(); ++i) {
+          // Old SuperVersions can keep these memtables alive after the flush
+          // commits, so keep their sealed direct-write blob files protected
+          // from obsolete-file purge until the memtable is finally released.
+          memtables[i]->ProtectSealedBlobFiles(mgr_handle,
+                                               generation_blob_file_numbers[i]);
+        }
+        for (const auto& addition : write_path_additions) {
+          sealed_blob_numbers.push_back(addition.GetBlobFileNumber());
+        }
+        flush_job.AddExternalBlobFileAdditions(std::move(write_path_additions));
+        flush_job.AddExternalBlobFileGarbages(std::move(write_path_garbages));
+      } else if (!write_path_garbages.empty()) {
+        flush_job.AddExternalBlobFileGarbages(std::move(write_path_garbages));
+      }
+    }
+  }
   if (s.ok()) {
     s = flush_job.Run(&logs_with_prep_tracker_, &file_meta,
                       &switched_to_mempurge, &skip_set_bg_error,
@@ -292,10 +331,33 @@ Status DBImpl::FlushMemTableToOutputFile(
     need_cancel = false;
   }
 
+  if (cfd->blob_partition_manager() != nullptr &&
+      prepared_blob_generations > 0) {
+    auto unconsumed_additions = flush_job.TakeExternalBlobFileAdditions();
+    auto unconsumed_garbages = flush_job.TakeExternalBlobFileGarbages();
+    if (!s.ok() || !unconsumed_additions.empty() ||
+        !unconsumed_garbages.empty()) {
+      // Skip committing this attempt, but keep the prepared generations queued
+      // in pending_generations_. The same immutable memtables still reference
+      // these sealed blob files, so a later flush retry must publish the exact
+      // same additions rather than abandoning them here.
+      prepared_blob_generations = 0;
+      sealed_blob_numbers.clear();
+    }
+  }
+
   if (!s.ok() && need_cancel) {
     flush_job.Cancel();
   }
 
+  if (s.ok() && prepared_blob_generations > 0) {
+    auto* mgr = cfd->blob_partition_manager();
+    mgr->CommitPreparedGenerations(prepared_blob_generations);
+    if (!sealed_blob_numbers.empty()) {
+      mgr->RemoveFilePartitionMappings(sealed_blob_numbers);
+    }
+  }
+
   if (s.ok()) {
     InstallSuperVersionAndScheduleWork(cfd, superversion_context);
     if (made_progress) {
@@ -535,11 +597,13 @@ Status DBImpl::AtomicFlushMemTablesToOutputFiles(
   // <bool /* executed */, Status /* status code */>
   autovector<std::pair<bool, Status>> exec_status;
   std::vector<bool> pick_status;
+  std::vector<size_t> prepared_blob_generations(num_cfs, 0);
   for (int i = 0; i != num_cfs; ++i) {
     // Initially all jobs are not executed, with status OK.
     exec_status.emplace_back(false, Status::OK());
     pick_status.push_back(false);
   }
+  std::unordered_map<int, std::vector<uint64_t>> sealed_blob_numbers_by_cf;
 
   bool flush_for_recovery =
       bg_flush_args[0].flush_reason_ == FlushReason::kErrorRecovery ||
@@ -563,6 +627,46 @@ Status DBImpl::AtomicFlushMemTablesToOutputFiles(
     }
   }
 
+  if (s.ok()) {
+    for (int i = 0; i != num_cfs; ++i) {
+      auto mgr_handle = cfds[i]->blob_partition_manager_handle();
+      auto* mgr = mgr_handle.get();
+      if (mgr == nullptr) {
+        continue;
+      }
+
+      prepared_blob_generations[i] = jobs[i]->GetMemTables().size();
+      std::vector<BlobFileAddition> write_path_additions;
+      std::vector<BlobFileGarbage> write_path_garbages;
+      std::vector<std::vector<uint64_t>> generation_blob_file_numbers;
+      mutex_.Unlock();
+      s = mgr->PrepareFlushAdditions(
+          write_options, prepared_blob_generations[i], &write_path_additions,
+          &write_path_garbages, &generation_blob_file_numbers);
+      mutex_.Lock();
+      if (!s.ok()) {
+        prepared_blob_generations[i] = 0;
+        break;
+      }
+      if (!write_path_additions.empty()) {
+        const auto& memtables = jobs[i]->GetMemTables();
+        assert(generation_blob_file_numbers.size() == memtables.size());
+        for (size_t j = 0; j < generation_blob_file_numbers.size(); ++j) {
+          memtables[j]->ProtectSealedBlobFiles(mgr_handle,
+                                               generation_blob_file_numbers[j]);
+        }
+        auto& sealed_blob_numbers = sealed_blob_numbers_by_cf[i];
+        for (const auto& addition : write_path_additions) {
+          sealed_blob_numbers.push_back(addition.GetBlobFileNumber());
+        }
+        jobs[i]->AddExternalBlobFileAdditions(std::move(write_path_additions));
+      }
+      if (!write_path_garbages.empty()) {
+        jobs[i]->AddExternalBlobFileGarbages(std::move(write_path_garbages));
+      }
+    }
+  }
+
   if (s.ok()) {
     assert(switched_to_mempurge.size() ==
            static_cast<long unsigned int>(num_cfs));
@@ -768,6 +872,30 @@ Status DBImpl::AtomicFlushMemTablesToOutputFiles(
         directories_.GetDbDir(), log_buffer);
   }
 
+  for (int i = 0; i != num_cfs; ++i) {
+    auto* mgr = cfds[i]->blob_partition_manager();
+    if (mgr == nullptr || prepared_blob_generations[i] == 0) {
+      continue;
+    }
+
+    auto unconsumed_additions = jobs[i]->TakeExternalBlobFileAdditions();
+    auto unconsumed_garbages = jobs[i]->TakeExternalBlobFileGarbages();
+    if (!s.ok() || !unconsumed_additions.empty() ||
+        !unconsumed_garbages.empty()) {
+      // Same retry semantics as the single-CF flush path above: do not
+      // consume these prepared generations yet, because the rolled-back
+      // memtables still need to reuse the same sealed blob files on retry.
+      prepared_blob_generations[i] = 0;
+      continue;
+    }
+
+    mgr->CommitPreparedGenerations(prepared_blob_generations[i]);
+    auto it = sealed_blob_numbers_by_cf.find(i);
+    if (it != sealed_blob_numbers_by_cf.end() && !it->second.empty()) {
+      mgr->RemoveFilePartitionMappings(it->second);
+    }
+  }
+
   if (s.ok()) {
     assert(num_cfs ==
            static_cast<int>(job_context->superversion_contexts.size()));
@@ -5025,6 +5153,10 @@ void DBImpl::InitSnapshotContext(JobContext* job_context) {
 Status DBImpl::WaitForCompact(
     const WaitForCompactOptions& wait_for_compact_options) {
   InstrumentedMutexLock l(&mutex_);
+  // `close_db=true` needs the same direct-write cleanup guarantee as DB close:
+  // register live blob files before relying on the DB being reopenable.
+  const bool flush_for_blob_direct_write =
+      HasAnyBlobDirectWriteColumnFamilyWithLockHeld();
   if (wait_for_compact_options.flush) {
     Status s = DBImpl::FlushAllColumnFamilies(FlushOptions(),
                                               FlushReason::kManualFlush);
@@ -5032,8 +5164,9 @@ Status DBImpl::WaitForCompact(
       return s;
     }
   } else if (wait_for_compact_options.close_db &&
-             has_unpersisted_data_.load(std::memory_order_relaxed) &&
-             !mutable_db_options_.avoid_flush_during_shutdown) {
+             (flush_for_blob_direct_write ||
+              (has_unpersisted_data_.load(std::memory_order_relaxed) &&
+               !mutable_db_options_.avoid_flush_during_shutdown))) {
     Status s =
         DBImpl::FlushAllColumnFamilies(FlushOptions(), FlushReason::kShutDown);
     if (!s.ok()) {

db/db_impl/db_impl_debug.cc

@@ -8,14 +8,17 @@
 // found in the LICENSE file. See the AUTHORS file for names of contributors.
 #ifndef NDEBUG
 #include <iostream>
+#include <unordered_set>
 
 #include "db/blob/blob_file_cache.h"
+#include "db/blob/blob_file_partition_manager.h"
 #include "db/column_family.h"
 #include "db/db_impl/db_impl.h"
 #include "db/error_handler.h"
 #include "db/periodic_task_scheduler.h"
 #include "monitoring/thread_status_updater.h"
 #include "util/cast_util.h"
+#include "util/hash_containers.h"
 
 namespace ROCKSDB_NAMESPACE {
 uint64_t DBImpl::TEST_GetLevel0TotalSize() {
@@ -357,11 +360,15 @@ void DBImpl::TEST_VerifyNoObsoleteFilesCached(
   }
 
   // Live and "quarantined" files are allowed to be open in table cache
-  std::set<uint64_t> live_and_quar_files;
+  UnorderedSet<uint64_t> live_and_quar_files;
   for (auto cfd : *versions_->GetColumnFamilySet()) {
     if (cfd->IsDropped()) {
       continue;
     }
+    if (auto* mgr = cfd->blob_partition_manager(); mgr != nullptr) {
+      mgr->GetActiveBlobFileNumbers(&live_and_quar_files);
+      mgr->GetProtectedBlobFileNumbers(&live_and_quar_files);
+    }
     // Iterate over live versions
     Version* current = cfd->current();
     Version* ver = current;

db/db_impl/db_impl_files.cc

@@ -6,10 +6,13 @@
 // Copyright (c) 2011 The LevelDB Authors. All rights reserved.
 // Use of this source code is governed by a BSD-style license that can be
 // found in the LICENSE file. See the AUTHORS file for names of contributors.
+#include <array>
 #include <cinttypes>
 #include <set>
 #include <unordered_set>
 
+#include "db/blob/blob_file_partition_manager.h"
+#include "db/blob/blob_log_format.h"
 #include "db/db_impl/db_impl.h"
 #include "db/event_helpers.h"
 #include "db/memtable_list.h"
@@ -17,6 +20,7 @@
 #include "file/filename.h"
 #include "file/sst_file_manager_impl.h"
 #include "logging/logging.h"
+#include "monitoring/thread_status_util.h"
 #include "port/port.h"
 #include "rocksdb/options.h"
 #include "util/autovector.h"
@@ -24,6 +28,91 @@
 
 namespace ROCKSDB_NAMESPACE {
 
+namespace {
+
+Env::IOActivity GetCurrentThreadIOActivityForMetadataRead() {
+  switch (ThreadStatusUtil::GetThreadOperation()) {
+    case ThreadStatus::OperationType::OP_FLUSH:
+      return Env::IOActivity::kFlush;
+    case ThreadStatus::OperationType::OP_COMPACTION:
+      return Env::IOActivity::kCompaction;
+    case ThreadStatus::OperationType::OP_DBOPEN:
+      return Env::IOActivity::kDBOpen;
+    case ThreadStatus::OperationType::OP_GET:
+      return Env::IOActivity::kGet;
+    case ThreadStatus::OperationType::OP_MULTIGET:
+      return Env::IOActivity::kMultiGet;
+    case ThreadStatus::OperationType::OP_DBITERATOR:
+      return Env::IOActivity::kDBIterator;
+    case ThreadStatus::OperationType::OP_VERIFY_DB_CHECKSUM:
+      return Env::IOActivity::kVerifyDBChecksum;
+    case ThreadStatus::OperationType::OP_VERIFY_FILE_CHECKSUMS:
+      return Env::IOActivity::kVerifyFileChecksums;
+    case ThreadStatus::OperationType::OP_GETENTITY:
+      return Env::IOActivity::kGetEntity;
+    case ThreadStatus::OperationType::OP_MULTIGETENTITY:
+      return Env::IOActivity::kMultiGetEntity;
+    case ThreadStatus::OperationType::
+        OP_GET_FILE_CHECKSUMS_FROM_CURRENT_MANIFEST:
+      return Env::IOActivity::kGetFileChecksumsFromCurrentManifest;
+    default:
+      return Env::IOActivity::kUnknown;
+  }
+}
+
+// A full-scan obsolete-file purge can observe a newly created direct-write
+// blob file before it is added to the manager's active-file set. Those
+// in-flight files are still missing their footer, so conservatively keep any
+// footer-less blob file here rather than risk deleting a live write-path file.
+bool ShouldKeepFooterlessBlobFile(FileSystem* fs,
+                                  const FileOptions& file_options,
+                                  const std::string& blob_file_path) {
+  assert(fs != nullptr);
+
+  constexpr IODebugContext* dbg = nullptr;
+  // This purge path can run from DB open, flush/compaction cleanup, or
+  // iterator-triggered obsolete-file cleanup. Tag the probe with the current
+  // thread activity so db_stress keeps validating the read against the active
+  // operation instead of seeing an unexpected kUnknown metadata read.
+  IOOptions io_options;
+  io_options.io_activity = GetCurrentThreadIOActivityForMetadataRead();
+  uint64_t file_size = 0;
+  IOStatus io_s = fs->GetFileSize(blob_file_path, io_options, &file_size, dbg);
+  if (!io_s.ok()) {
+    return !io_s.IsPathNotFound();
+  }
+
+  if (file_size < BlobLogHeader::kSize + BlobLogFooter::kSize) {
+    return true;
+  }
+
+  FileOptions read_file_options = file_options;
+  read_file_options.use_direct_reads = false;
+
+  std::unique_ptr<FSRandomAccessFile> file;
+  io_s = fs->NewRandomAccessFile(blob_file_path, read_file_options, &file, dbg);
+  if (!io_s.ok()) {
+    return !io_s.IsPathNotFound();
+  }
+
+  std::array<char, BlobLogFooter::kSize> scratch{};
+  Slice footer_slice;
+  io_s = file->Read(file_size - BlobLogFooter::kSize, BlobLogFooter::kSize,
+                    io_options, &footer_slice, scratch.data(), dbg);
+  if (!io_s.ok()) {
+    return !io_s.IsPathNotFound();
+  }
+
+  if (footer_slice.size() != BlobLogFooter::kSize) {
+    return true;
+  }
+
+  BlobLogFooter footer;
+  return !footer.DecodeFrom(footer_slice).ok();
+}
+
+}  // namespace
+
 uint64_t DBImpl::MinLogNumberToKeep() {
   return versions_->min_log_number_to_keep();
 }
@@ -156,6 +245,17 @@ void DBImpl::FindObsoleteFiles(JobContext* job_context, bool force,
   job_context->min_pending_output = MinObsoleteSstNumberToKeep();
   job_context->files_to_quarantine = error_handler_.GetFilesToQuarantine();
   job_context->min_options_file_number = MinOptionsFileNumberToKeep();
+  job_context->min_blob_file_number_to_keep =
+      versions_->current_next_file_number();
+  for (auto* cfd : *versions_->GetColumnFamilySet()) {
+    auto* mgr = cfd->blob_partition_manager();
+    if (mgr != nullptr) {
+      mgr->GetActiveBlobFileNumbers(
+          &job_context->active_blob_direct_write_files);
+      mgr->GetProtectedBlobFileNumbers(
+          &job_context->active_blob_direct_write_files);
+    }
+  }
 
   // Get obsolete files.  This function will also update the list of
   // pending files in VersionSet().
@@ -587,13 +687,25 @@ void DBImpl::PurgeObsoleteFiles(JobContext& state, bool schedule_only) {
           files_to_del.insert(number);
         }
         break;
-      case kBlobFile:
+      case kBlobFile: {
         keep = number >= state.min_pending_output ||
-               (blob_live_set.find(number) != blob_live_set.end());
+               number >= state.min_blob_file_number_to_keep ||
+               (blob_live_set.find(number) != blob_live_set.end()) ||
+               (state.active_blob_direct_write_files.find(number) !=
+                state.active_blob_direct_write_files.end());
         if (!keep) {
+          const std::string blob_file_path =
+              BlobFileName(candidate_file.file_path, number);
+          if (ShouldKeepFooterlessBlobFile(fs_.get(), file_options_,
+                                           blob_file_path)) {
+            keep = true;
+            break;
+          }
+          TableCache::Evict(table_cache_.get(), number);
           files_to_del.insert(number);
         }
         break;
+      }
       case kTempFile:
         // Any temp files that are currently being written to must
         // be recorded in pending_outputs_, which is inserted into "live".

db/db_impl/db_impl_open.cc

@@ -8,6 +8,7 @@
 // found in the LICENSE file. See the AUTHORS file for names of contributors.
 #include <cinttypes>
 
+#include "db/blob/blob_file_partition_manager.h"
 #include "db/builder.h"
 #include "db/db_impl/db_impl.h"
 #include "db/error_handler.h"
@@ -2606,6 +2607,14 @@ Status DBImpl::Open(const DBOptions& db_options, const std::string& dbname,
     }
   }
 
+  if (s.ok()) {
+    for (size_t i = 0; i < column_families.size(); ++i) {
+      const auto& cf = column_families[i];
+      auto* cfd = static_cast<ColumnFamilyHandleImpl*>((*handles)[i])->cfd();
+      impl->MaybeInitBlobDirectWriteColumnFamily(cfd, cf.options, cf.name);
+    }
+  }
+
   if (s.ok() && impl->immutable_db_options_.persist_stats_to_disk) {
     // Install SuperVersion for hidden column family
     assert(impl->persist_stats_cf_handle_);

db/db_impl/db_impl_write.cc

@@ -7,7 +7,11 @@
 // Use of this source code is governed by a BSD-style license that can be
 // found in the LICENSE file. See the AUTHORS file for names of contributors.
 #include <cinttypes>
+#include <optional>
+#include <unordered_map>
 
+#include "db/blob/blob_file_partition_manager.h"
+#include "db/blob/blob_write_batch_transformer.h"
 #include "db/db_impl/db_impl.h"
 #include "db/error_handler.h"
 #include "db/event_helpers.h"
@@ -17,8 +21,59 @@
 #include "options/options_helper.h"
 #include "test_util/sync_point.h"
 #include "util/cast_util.h"
+#include "util/hash_containers.h"
 
 namespace ROCKSDB_NAMESPACE {
+
+namespace {
+
+class BlobWriteRollbackGuard {
+ public:
+  explicit BlobWriteRollbackGuard(
+      const std::vector<BlobWriteBatchTransformer::RollbackInfo>*
+          rollback_infos,
+      Logger* info_log)
+      : rollback_infos_(rollback_infos), info_log_(info_log) {}
+  BlobWriteRollbackGuard(const BlobWriteRollbackGuard&) = delete;
+  BlobWriteRollbackGuard& operator=(const BlobWriteRollbackGuard&) = delete;
+  BlobWriteRollbackGuard(BlobWriteRollbackGuard&&) = delete;
+  BlobWriteRollbackGuard& operator=(BlobWriteRollbackGuard&&) = delete;
+
+  ~BlobWriteRollbackGuard() {
+    if (!active_ || rollback_infos_ == nullptr) {
+      return;
+    }
+
+    for (const auto& rollback_info : *rollback_infos_) {
+      if (rollback_info.partition_mgr == nullptr) {
+        continue;
+      }
+      Status rollback_s = rollback_info.partition_mgr->MarkBlobWriteAsGarbage(
+          rollback_info.file_number, rollback_info.count, rollback_info.bytes);
+      if (!rollback_s.ok()) {
+        if (info_log_ != nullptr) {
+          ROCKS_LOG_ERROR(
+              info_log_,
+              "Failed to rollback blob direct-write garbage accounting for "
+              "file #%" PRIu64 " (%" PRIu64 " blobs, %" PRIu64 " bytes): %s",
+              rollback_info.file_number, rollback_info.count,
+              rollback_info.bytes, rollback_s.ToString().c_str());
+        }
+        rollback_s.PermitUncheckedError();
+      }
+    }
+  }
+
+  void Dismiss() { active_ = false; }
+
+ private:
+  const std::vector<BlobWriteBatchTransformer::RollbackInfo>* rollback_infos_;
+  Logger* info_log_;
+  bool active_ = true;
+};
+
+}  // namespace
+
 // Convenience methods
 Status DBImpl::Put(const WriteOptions& o, ColumnFamilyHandle* column_family,
                    const Slice& key, const Slice& val) {
@@ -200,6 +255,11 @@ Status DBImpl::IngestWriteBatchWithIndex(
     return Status::NotSupported(
         "IngestWriteBatchWithIndex does not support disableWAL=true");
   }
+  if (HasAnyBlobDirectWriteColumnFamily()) {
+    return Status::NotSupported(
+        "IngestWriteBatchWithIndex is not supported with "
+        "enable_blob_direct_write");
+  }
   Status s;
   if (write_options.protection_bytes_per_key > 0) {
     s = WriteBatchInternal::UpdateProtectionInfo(
@@ -368,6 +428,199 @@ Status DBImpl::IngestWBWIAsMemtable(
   return s;
 }
 
+struct DBImpl::BlobDirectWriteContext {
+  struct Entry {
+    // Hold a referenced SuperVersion so BDW uses the same published mutable
+    // option snapshot as the rest of the DB while keeping the CFD alive.
+    SuperVersion* super_version = nullptr;
+    BlobFilePartitionManager* partition_mgr = nullptr;
+    BlobDirectWriteSettings settings;
+  };
+
+  explicit BlobDirectWriteContext(DBImpl* db_impl) : db_impl_(db_impl) {}
+  BlobDirectWriteContext(const BlobDirectWriteContext&) = delete;
+  BlobDirectWriteContext& operator=(const BlobDirectWriteContext&) = delete;
+  BlobDirectWriteContext(BlobDirectWriteContext&&) = delete;
+  BlobDirectWriteContext& operator=(BlobDirectWriteContext&&) = delete;
+
+  ~BlobDirectWriteContext() { Release(); }
+
+  BlobDirectWriteSettings GetSettings(uint32_t cf_id,
+                                      bool lookup_from_write_thread) {
+    return GetOrCreate(cf_id, lookup_from_write_thread).settings;
+  }
+
+  BlobFilePartitionManager* GetPartitionManager(uint32_t cf_id,
+                                                bool lookup_from_write_thread) {
+    return GetOrCreate(cf_id, lookup_from_write_thread).partition_mgr;
+  }
+
+  void Release() {
+    if (entries_.empty()) {
+      return;
+    }
+
+    for (auto& entry : entries_) {
+      if (entry.second.super_version != nullptr) {
+        db_impl_->CleanupSuperVersion(entry.second.super_version);
+      }
+    }
+    entries_.clear();
+  }
+
+  UnorderedSet<BlobFilePartitionManager*> touched_managers;
+  std::vector<BlobWriteBatchTransformer::RollbackInfo> rollback_infos;
+
+ private:
+  static BlobDirectWriteSettings BuildSettings(const SuperVersion* sv) {
+    assert(sv != nullptr);
+
+    BlobDirectWriteSettings settings;
+    settings.enable_blob_direct_write =
+        sv->cfd->ioptions().enable_blob_direct_write;
+    settings.min_blob_size = sv->mutable_cf_options.min_blob_size;
+    settings.compression_type = sv->mutable_cf_options.blob_compression_type;
+    settings.blob_cache = sv->cfd->ioptions().blob_cache.get();
+    settings.prepopulate_blob_cache =
+        sv->mutable_cf_options.prepopulate_blob_cache;
+    return settings;
+  }
+
+  SuperVersion* AcquireReferencedSuperVersion(uint32_t cf_id,
+                                              bool lookup_from_write_thread) {
+    if (lookup_from_write_thread) {
+      auto* cfd =
+          db_impl_->versions_->GetColumnFamilySet()->GetColumnFamily(cf_id);
+      if (cfd != nullptr) {
+        return cfd->GetReferencedSuperVersion(db_impl_);
+      }
+      return nullptr;
+    }
+
+    ColumnFamilyData* cfd = nullptr;
+    {
+      InstrumentedMutexLock lock(&db_impl_->mutex_);
+      cfd = db_impl_->versions_->GetColumnFamilySet()->GetColumnFamily(cf_id);
+      if (cfd != nullptr) {
+        // Hold the CFD long enough to acquire a referenced SuperVersion after
+        // dropping DB mutex. This reuses the existing SuperVersion publication
+        // path for mutable options rather than maintaining a separate BDW
+        // settings snapshot. A future attached-WriteBatch path could pin the
+        // needed CFDs up front and avoid this DB mutex lookup entirely.
+        cfd->Ref();
+      }
+    }
+    if (cfd == nullptr) {
+      return nullptr;
+    }
+
+    SuperVersion* sv = cfd->GetReferencedSuperVersion(db_impl_);
+    {
+      InstrumentedMutexLock lock(&db_impl_->mutex_);
+      cfd->UnrefAndTryDelete();
+    }
+    return sv;
+  }
+
+  Entry& GetOrCreate(uint32_t cf_id, bool lookup_from_write_thread) {
+    auto entry_it = entries_.find(cf_id);
+    if (entry_it != entries_.end()) {
+      return entry_it->second;
+    }
+
+    Entry entry;
+    if (SuperVersion* sv =
+            AcquireReferencedSuperVersion(cf_id, lookup_from_write_thread)) {
+      entry.super_version = sv;
+      entry.partition_mgr = sv->cfd->blob_partition_manager();
+      if (entry.partition_mgr != nullptr) {
+        entry.settings = BuildSettings(sv);
+      }
+    }
+
+    return entries_.emplace(cf_id, std::move(entry)).first->second;
+  }
+
+  DBImpl* db_impl_;
+  std::unordered_map<uint32_t, Entry> entries_;
+};
+
+Status DBImpl::MaybeTransformBatchForBlobDirectWrite(
+    const WriteOptions& write_options, WriteBatch** batch,
+    bool should_write_to_memtable, bool lookup_from_write_thread,
+    WriteBatch* transformed_storage,
+    BlobDirectWriteContext* blob_direct_write_ctx) {
+  assert(batch != nullptr);
+  assert(transformed_storage != nullptr);
+  assert(blob_direct_write_ctx != nullptr);
+
+  if (!should_write_to_memtable || *batch == nullptr || !(*batch)->HasPut() ||
+      (*batch)->HasMerge()) {
+    return Status::OK();
+  }
+
+  // Quick check: skip the (potentially expensive) batch iteration when no
+  // column family has blob direct write enabled. This also avoids Iterate()
+  // returning an error on a corrupted batch before the WAL verification path
+  // has a chance to detect it.
+  if (!HasAnyBlobDirectWriteColumnFamily()) {
+    return Status::OK();
+  }
+
+  std::vector<BlobFilePartitionManager*> batch_managers;
+  std::vector<BlobWriteBatchTransformer::RollbackInfo> batch_rollbacks;
+  bool transformed = false;
+  transformed_storage->Clear();
+
+  auto settings_provider = [&](uint32_t cf_id) -> BlobDirectWriteSettings {
+    return blob_direct_write_ctx->GetSettings(cf_id, lookup_from_write_thread);
+  };
+  auto partition_mgr_provider =
+      [&](uint32_t cf_id) -> BlobFilePartitionManager* {
+    return blob_direct_write_ctx->GetPartitionManager(cf_id,
+                                                      lookup_from_write_thread);
+  };
+  Status s = BlobWriteBatchTransformer::TransformBatch(
+      write_options, *batch, transformed_storage, partition_mgr_provider,
+      settings_provider, &transformed, &batch_managers, &batch_rollbacks);
+  if (!s.ok()) {
+    return s;
+  }
+  if (transformed) {
+    *batch = transformed_storage;
+    blob_direct_write_ctx->touched_managers.insert(batch_managers.begin(),
+                                                   batch_managers.end());
+    blob_direct_write_ctx->rollback_infos.insert(
+        blob_direct_write_ctx->rollback_infos.end(), batch_rollbacks.begin(),
+        batch_rollbacks.end());
+  }
+  return Status::OK();
+}
+
+Status DBImpl::SyncBlobDirectWriteManagers(
+    const WriteOptions& write_options,
+    const BlobDirectWriteContext& blob_direct_write_ctx) {
+  for (auto* mgr : blob_direct_write_ctx.touched_managers) {
+    Status blob_s = write_options.sync ? mgr->SyncAllOpenFiles(write_options)
+                                       : mgr->FlushAllOpenFiles(write_options);
+    if (!blob_s.ok()) {
+      return blob_s;
+    }
+  }
+
+  if (!blob_direct_write_ctx.touched_managers.empty()) {
+    // Tests can inject a post-transform failure after blob bytes are durable
+    // enough to require manifest-time garbage accounting.
+    Status blob_s;
+    TEST_SYNC_POINT_CALLBACK("DBImpl::WriteImpl:AfterBlobDirectWrite", &blob_s);
+    if (!blob_s.ok()) {
+      return blob_s;
+    }
+  }
+
+  return Status::OK();
+}
+
 Status DBImpl::WriteImpl(const WriteOptions& write_options,
                          WriteBatch* my_batch, WriteCallback* callback,
                          UserWriteCallback* user_write_cb, uint64_t* wal_used,
@@ -505,10 +758,50 @@ Status DBImpl::WriteImpl(const WriteOptions& write_options,
         seq_per_batch_ ? kDoAssignOrder : kDontAssignOrder;
     // Otherwise it is WAL-only Prepare batches in WriteCommitted policy and
     // they don't consume sequence.
-    return WriteImplWALOnly(
-        &nonmem_write_thread_, write_options, my_batch, callback, user_write_cb,
-        wal_used, log_ref, seq_used, batch_cnt, pre_release_callback,
-        assign_order, kDontPublishLastSeq, disable_memtable);
+    return WriteImplWALOnly(&nonmem_write_thread_, write_options, my_batch,
+                            my_batch, callback, user_write_cb, wal_used,
+                            log_ref, seq_used, batch_cnt, pre_release_callback,
+                            assign_order, kDontPublishLastSeq,
+                            disable_memtable);
+  }
+
+  const bool maybe_use_blob_direct_write =
+      wbwi == nullptr && HasAnyBlobDirectWriteColumnFamily();
+  std::optional<WriteBatch> transformed_batch_storage;
+  std::optional<std::vector<WriteBatch>> transformed_write_group_batches;
+  std::optional<BlobDirectWriteContext> blob_direct_write_ctx;
+  std::optional<BlobWriteRollbackGuard> blob_write_rollback_guard;
+  if (maybe_use_blob_direct_write) {
+    blob_direct_write_ctx.emplace(this);
+    blob_write_rollback_guard.emplace(&blob_direct_write_ctx->rollback_infos,
+                                      immutable_db_options_.info_log.get());
+  }
+  auto finish_write = [&](Status s) {
+    if (s.ok() && blob_write_rollback_guard.has_value()) {
+      blob_write_rollback_guard->Dismiss();
+    }
+    return s;
+  };
+  // Ordered query traces should reflect the original user batch even if the
+  // applied batch is rewritten for blob direct write.
+  WriteBatch* trace_batch = my_batch;
+
+  if (UNLIKELY(maybe_use_blob_direct_write) &&
+      (immutable_db_options_.unordered_write ||
+       immutable_db_options_.enable_pipelined_write)) {
+    transformed_batch_storage.emplace();
+    Status blob_s = MaybeTransformBatchForBlobDirectWrite(
+        write_options, &my_batch, !disable_memtable,
+        /*lookup_from_write_thread=*/false, &transformed_batch_storage.value(),
+        &blob_direct_write_ctx.value());
+    if (!blob_s.ok()) {
+      return blob_s;
+    }
+    blob_s = SyncBlobDirectWriteManagers(write_options,
+                                         blob_direct_write_ctx.value());
+    if (!blob_s.ok()) {
+      return blob_s;
+    }
   }
 
   if (immutable_db_options_.unordered_write) {
@@ -519,10 +812,11 @@ Status DBImpl::WriteImpl(const WriteOptions& write_options,
     uint64_t seq = 0;
     // Use a write thread to i) optimize for WAL write, ii) publish last
     // sequence in in increasing order, iii) call pre_release_callback serially
-    Status status = WriteImplWALOnly(
-        &write_thread_, write_options, my_batch, callback, user_write_cb,
-        wal_used, log_ref, &seq, sub_batch_cnt, pre_release_callback,
-        kDoAssignOrder, kDoPublishLastSeq, disable_memtable);
+    Status status =
+        WriteImplWALOnly(&write_thread_, write_options, my_batch, trace_batch,
+                         callback, user_write_cb, wal_used, log_ref, &seq,
+                         sub_batch_cnt, pre_release_callback, kDoAssignOrder,
+                         kDoPublishLastSeq, disable_memtable);
     TEST_SYNC_POINT("DBImpl::WriteImpl:UnorderedWriteAfterWriteWAL");
     if (!status.ok()) {
       return status;
@@ -535,19 +829,20 @@ Status DBImpl::WriteImpl(const WriteOptions& write_options,
       status = UnorderedWriteMemtable(write_options, my_batch, callback,
                                       log_ref, seq, sub_batch_cnt);
     }
-    return status;
+    return finish_write(status);
   }
 
   if (immutable_db_options_.enable_pipelined_write) {
-    return PipelinedWriteImpl(write_options, my_batch, callback, user_write_cb,
-                              wal_used, log_ref, disable_memtable, seq_used);
+    return finish_write(PipelinedWriteImpl(
+        write_options, my_batch, trace_batch, callback, user_write_cb, wal_used,
+        log_ref, disable_memtable, seq_used));
   }
 
   PERF_TIMER_GUARD(write_pre_and_post_process_time);
   WriteThread::Writer w(write_options, my_batch, callback, user_write_cb,
                         log_ref, disable_memtable, batch_cnt,
                         pre_release_callback, post_memtable_callback,
-                        /*_ingest_wbwi=*/wbwi != nullptr);
+                        /*_ingest_wbwi=*/wbwi != nullptr, trace_batch);
   StopWatch write_sw(immutable_db_options_.clock, stats_, DB_WRITE);
 
   write_thread_.JoinBatchGroup(&w);
@@ -605,7 +900,7 @@ Status DBImpl::WriteImpl(const WriteOptions& write_options,
       *seq_used = w.sequence;
     }
     // write is complete and leader has updated sequence
-    return w.FinalStatus();
+    return finish_write(w.FinalStatus());
   }
   // else we are the leader of the write batch group
   assert(w.state == WriteThread::STATE_GROUP_LEADER);
@@ -655,6 +950,38 @@ Status DBImpl::WriteImpl(const WriteOptions& write_options,
   IOStatus io_s;
   Status pre_release_cb_status;
   size_t seq_inc = 0;
+  if (status.ok()) {
+    if (UNLIKELY(maybe_use_blob_direct_write) &&
+        !immutable_db_options_.unordered_write &&
+        !immutable_db_options_.enable_pipelined_write) {
+      // PreprocessWrite() may switch memtables in order to schedule pending
+      // flushes. Defer the blob direct-write transformation until after that
+      // point so the blob file generation matches the memtable that will store
+      // the transformed blob index.
+      transformed_write_group_batches.emplace();
+      transformed_write_group_batches->reserve(write_group.size);
+      for (auto* writer : write_group) {
+        assert(writer != nullptr);
+        if (!writer->ShouldWriteToMemtable()) {
+          continue;
+        }
+        transformed_write_group_batches->emplace_back();
+        status = MaybeTransformBatchForBlobDirectWrite(
+            write_options, &writer->batch, /*should_write_to_memtable=*/true,
+            /*lookup_from_write_thread=*/true,
+            &transformed_write_group_batches->back(),
+            &blob_direct_write_ctx.value());
+        if (!status.ok()) {
+          break;
+        }
+      }
+      if (status.ok()) {
+        status = SyncBlobDirectWriteManagers(write_options,
+                                             blob_direct_write_ctx.value());
+      }
+    }
+  }
+
   if (status.ok()) {
     // Rules for when we can update the memtable concurrently
     // 1. supported by memtable
@@ -703,7 +1030,7 @@ Status DBImpl::WriteImpl(const WriteOptions& write_options,
             // is in writer->batch
             tracer_->Write(wbwi->GetWriteBatch()).PermitUncheckedError();
           } else {
-            tracer_->Write(writer->batch).PermitUncheckedError();
+            tracer_->Write(writer->trace_batch).PermitUncheckedError();
           }
         }
       }
@@ -966,11 +1293,12 @@ Status DBImpl::WriteImpl(const WriteOptions& write_options,
   if (status.ok()) {
     status = w.FinalStatus();
   }
-  return status;
+  return finish_write(status);
 }
 
 Status DBImpl::PipelinedWriteImpl(const WriteOptions& write_options,
-                                  WriteBatch* my_batch, WriteCallback* callback,
+                                  WriteBatch* my_batch, WriteBatch* trace_batch,
+                                  WriteCallback* callback,
                                   UserWriteCallback* user_write_cb,
                                   uint64_t* wal_used, uint64_t log_ref,
                                   bool disable_memtable, uint64_t* seq_used) {
@@ -981,7 +1309,9 @@ Status DBImpl::PipelinedWriteImpl(const WriteOptions& write_options,
 
   WriteThread::Writer w(write_options, my_batch, callback, user_write_cb,
                         log_ref, disable_memtable, /*_batch_cnt=*/0,
-                        /*_pre_release_callback=*/nullptr);
+                        /*_pre_release_callback=*/nullptr,
+                        /*_post_memtable_callback=*/nullptr,
+                        /*_ingest_wbwi=*/false, trace_batch);
   write_thread_.JoinBatchGroup(&w);
   TEST_SYNC_POINT("DBImplWrite::PipelinedWriteImpl:AfterJoinBatchGroup");
   if (w.state == WriteThread::STATE_GROUP_LEADER) {
@@ -1030,7 +1360,7 @@ Status DBImpl::PipelinedWriteImpl(const WriteOptions& write_options,
               continue;
             }
             // TODO: maybe handle the tracing status?
-            tracer_->Write(writer->batch).PermitUncheckedError();
+            tracer_->Write(writer->trace_batch).PermitUncheckedError();
           }
         }
       }
@@ -1225,7 +1555,7 @@ Status DBImpl::UnorderedWriteMemtable(const WriteOptions& write_options,
 // applicable in a two-queue setting.
 Status DBImpl::WriteImplWALOnly(
     WriteThread* write_thread, const WriteOptions& write_options,
-    WriteBatch* my_batch, WriteCallback* callback,
+    WriteBatch* my_batch, WriteBatch* trace_batch, WriteCallback* callback,
     UserWriteCallback* user_write_cb, uint64_t* wal_used,
     const uint64_t log_ref, uint64_t* seq_used, const size_t sub_batch_cnt,
     PreReleaseCallback* pre_release_callback, const AssignOrder assign_order,
@@ -1233,7 +1563,9 @@ Status DBImpl::WriteImplWALOnly(
   PERF_TIMER_GUARD(write_pre_and_post_process_time);
   WriteThread::Writer w(write_options, my_batch, callback, user_write_cb,
                         log_ref, disable_memtable, sub_batch_cnt,
-                        pre_release_callback);
+                        pre_release_callback,
+                        /*_post_memtable_callback=*/nullptr,
+                        /*_ingest_wbwi=*/false, trace_batch);
   StopWatch write_sw(immutable_db_options_.clock, stats_, DB_WRITE);
 
   write_thread->JoinBatchGroup(&w);
@@ -1313,7 +1645,7 @@ Status DBImpl::WriteImplWALOnly(
           continue;
         }
         // TODO: maybe handle the tracing status?
-        tracer_->Write(writer->batch).PermitUncheckedError();
+        tracer_->Write(writer->trace_batch).PermitUncheckedError();
       }
     }
   }
@@ -2722,6 +3054,10 @@ Status DBImpl::SwitchMemtable(ColumnFamilyData* cfd, WriteContext* context,
     }
   }
 
+  if (cfd->blob_partition_manager() != nullptr) {
+    cfd->blob_partition_manager()->RotateCurrentGeneration();
+  }
+
   cfd->mem()->SetNextLogNumber(cur_wal_number_);
   assert(new_mem != nullptr);
   cfd->imm()->Add(cfd->mem(), &context->memtables_to_free_);

db/db_iter.cc

@@ -12,6 +12,8 @@
 #include <limits>
 #include <string>
 
+#include "db/blob/blob_file_partition_manager.h"
+#include "db/blob/blob_index.h"
 #include "db/dbformat.h"
 #include "db/merge_context.h"
 #include "db/merge_helper.h"
@@ -53,7 +55,8 @@ DBIter::DBIter(Env* _env, const ReadOptions& read_options,
       iter_(iter),
       blob_reader_(version, read_options.read_tier,
                    read_options.verify_checksums, read_options.fill_cache,
-                   read_options.io_activity),
+                   read_options.io_activity,
+                   cfh ? cfh->cfd()->blob_file_cache() : nullptr),
       read_callback_(read_callback),
       sequence_(s),
       statistics_(ioptions.stats),
@@ -218,11 +221,12 @@ void DBIter::Next() {
   }
 }
 
-Status DBIter::BlobReader::RetrieveAndSetBlobValue(const Slice& user_key,
-                                                   const Slice& blob_index) {
+Status DBIter::BlobReader::RetrieveAndSetBlobValue(
+    const Slice& user_key, const Slice& blob_index,
+    bool allow_write_path_fallback) {
   assert(blob_value_.empty());
 
-  if (!version_) {
+  if (!version_ && (!allow_write_path_fallback || !blob_file_cache_)) {
     return Status::Corruption("Encountered unexpected blob index.");
   }
 
@@ -237,19 +241,32 @@ Status DBIter::BlobReader::RetrieveAndSetBlobValue(const Slice& user_key,
   constexpr FilePrefetchBuffer* prefetch_buffer = nullptr;
   constexpr uint64_t* bytes_read = nullptr;
 
-  const Status s = version_->GetBlob(read_options, user_key, blob_index,
+  if (!allow_write_path_fallback) {
+    assert(version_ != nullptr);
+    return version_->GetBlob(read_options, user_key, blob_index,
                              prefetch_buffer, &blob_value_, bytes_read);
+  }
 
+  BlobIndex blob_idx;
+  Status s = blob_idx.DecodeFrom(blob_index);
   if (!s.ok()) {
     return s;
   }
 
-  return Status::OK();
+  return BlobFilePartitionManager::ResolveBlobDirectWriteIndex(
+      read_options, user_key, blob_idx, version_, blob_file_cache_,
+      &blob_value_);
 }
 
 bool DBIter::SetValueAndColumnsFromBlobImpl(const Slice& user_key,
                                             const Slice& blob_index) {
-  const Status s = blob_reader_.RetrieveAndSetBlobValue(user_key, blob_index);
+  // Keep the non-BDW iterator path on the pre-existing Version::GetBlob()
+  // fast path. Only enable the direct-write fallback when this CF actually
+  // has a write-path partition manager.
+  const bool allow_write_path_fallback =
+      cfh_ != nullptr && cfh_->cfd()->blob_partition_manager() != nullptr;
+  const Status s = blob_reader_.RetrieveAndSetBlobValue(
+      user_key, blob_index, allow_write_path_fallback);
   if (!s.ok()) {
     status_ = s;
     valid_ = false;
@@ -1385,7 +1402,10 @@ bool DBIter::MergeWithBlobBaseValue(const Slice& blob_index,
     return false;
   }
 
-  const Status s = blob_reader_.RetrieveAndSetBlobValue(user_key, blob_index);
+  const bool allow_write_path_fallback =
+      cfh_ != nullptr && cfh_->cfd()->blob_partition_manager() != nullptr;
+  const Status s = blob_reader_.RetrieveAndSetBlobValue(
+      user_key, blob_index, allow_write_path_fallback);
   if (!s.ok()) {
     status_ = s;
     valid_ = false;

db/db_iter.h

@@ -21,6 +21,7 @@
 #include "util/autovector.h"
 
 namespace ROCKSDB_NAMESPACE {
+class BlobFileCache;
 class Version;
 
 // This file declares the factory functions of DBIter, in its original form
@@ -256,16 +257,18 @@ class DBIter final : public Iterator {
    public:
     BlobReader(const Version* version, ReadTier read_tier,
                bool verify_checksums, bool fill_cache,
-               Env::IOActivity io_activity)
+               Env::IOActivity io_activity, BlobFileCache* blob_file_cache)
         : version_(version),
           read_tier_(read_tier),
           verify_checksums_(verify_checksums),
           fill_cache_(fill_cache),
-          io_activity_(io_activity) {}
+          io_activity_(io_activity),
+          blob_file_cache_(blob_file_cache) {}
 
     const Slice& GetBlobValue() const { return blob_value_; }
     Status RetrieveAndSetBlobValue(const Slice& user_key,
-                                   const Slice& blob_index);
+                                   const Slice& blob_index,
+                                   bool allow_write_path_fallback);
     void ResetBlobValue() { blob_value_.Reset(); }
 
    private:
@@ -275,6 +278,9 @@ class DBIter final : public Iterator {
     bool verify_checksums_;
     bool fill_cache_;
     Env::IOActivity io_activity_;
+    // Cache used by the write-path fallback for in-flight direct-write blob
+    // files that are not yet reachable through Version.
+    BlobFileCache* blob_file_cache_;
   };
 
   // For all methods in this block:

db/db_sst_test.cc

@@ -1122,6 +1122,8 @@ TEST_F(DBSSTTest, DeleteSchedulerMultipleDBPaths) {
   begin = "Key4";
   end = "Key7";
   ASSERT_OK(db_->CompactRange(compact_options, &begin, &end));
+  ASSERT_OK(dbfull()->TEST_WaitForCompact());
+  ASSERT_OK(dbfull()->TEST_WaitForPurge());
   ASSERT_EQ("0,2", FilesPerLevel(0));
 
   sfm->WaitForEmptyTrash();
@@ -1132,6 +1134,8 @@ TEST_F(DBSSTTest, DeleteSchedulerMultipleDBPaths) {
   compact_options.bottommost_level_compaction =
       BottommostLevelCompaction::kForceOptimized;
   ASSERT_OK(db_->CompactRange(compact_options, nullptr, nullptr));
+  ASSERT_OK(dbfull()->TEST_WaitForCompact());
+  ASSERT_OK(dbfull()->TEST_WaitForPurge());
   ASSERT_EQ("0,1", FilesPerLevel(0));
 
   sfm->WaitForEmptyTrash();

db/flush_job.cc

@@ -1105,6 +1105,15 @@ Status FlushJob::WriteLevel0Table() {
                    meta_.tail_size, meta_.user_defined_timestamps_persisted,
                    meta_.min_timestamp, meta_.max_timestamp);
     edit_->SetBlobFileAdditions(std::move(blob_file_additions));
+
+    for (auto& addition : external_blob_file_additions_) {
+      edit_->AddBlobFile(std::move(addition));
+    }
+    for (auto& garbage : external_blob_file_garbages_) {
+      edit_->AddBlobFileGarbage(std::move(garbage));
+    }
+    external_blob_file_additions_.clear();
+    external_blob_file_garbages_.clear();
   }
   // Piggyback FlushJobInfo on the first first flushed memtable.
   mems_[0]->SetFlushJobInfo(GetFlushJobInfo());

db/flush_job.h

@@ -91,6 +91,37 @@ class FlushJob {
   void Cancel();
   const autovector<ReadOnlyMemTable*>& GetMemTables() const { return mems_; }
 
+  // Returns the log number recorded in the flush VersionEdit after
+  // PickMemTable() initializes `edit_`.
+  uint64_t GetLogNumber() const {
+    assert(edit_ != nullptr);
+    return edit_->GetLogNumber();
+  }
+
+  // Stashes write-path blob files so WriteLevel0Table() can add them to the
+  // same VersionEdit as the flushed SST.
+  void AddExternalBlobFileAdditions(std::vector<BlobFileAddition>&& additions) {
+    external_blob_file_additions_ = std::move(additions);
+  }
+
+  // Stashes write-path initial-garbage updates so they are committed with the
+  // same VersionEdit as the matching blob-file additions and flushed SST.
+  void AddExternalBlobFileGarbages(std::vector<BlobFileGarbage>&& garbages) {
+    external_blob_file_garbages_ = std::move(garbages);
+  }
+
+  // Transfers back any prepared blob file additions that were not consumed by
+  // the flush.
+  std::vector<BlobFileAddition> TakeExternalBlobFileAdditions() {
+    return std::move(external_blob_file_additions_);
+  }
+
+  // Transfers back any prepared blob-file garbage updates that were not
+  // consumed by the flush.
+  std::vector<BlobFileGarbage> TakeExternalBlobFileGarbages() {
+    return std::move(external_blob_file_garbages_);
+  }
+
   std::list<std::unique_ptr<FlushJobInfo>>* GetCommittedFlushJobsInfo() {
     return &committed_flush_jobs_info_;
   }
@@ -213,6 +244,11 @@ class FlushJob {
 
   const std::string full_history_ts_low_;
   BlobFileCompletionCallback* blob_callback_;
+  // Write-path blob files that should be committed with this flush.
+  std::vector<BlobFileAddition> external_blob_file_additions_;
+  // Initial garbage for write-path blob files that were partially abandoned
+  // before their owning flush committed.
+  std::vector<BlobFileGarbage> external_blob_file_garbages_;
 
   // Shared copy of DB's seqno to time mapping stored in SuperVersion. The
   // ownership is shared with this FlushJob when it's created.

db/job_context.h

@@ -9,6 +9,7 @@
 
 #pragma once
 
+#include <limits>
 #include <string>
 #include <vector>
 
@@ -16,6 +17,7 @@
 #include "db/log_writer.h"
 #include "db/version_set.h"
 #include "util/autovector.h"
+#include "util/hash_containers.h"
 
 namespace ROCKSDB_NAMESPACE {
 
@@ -212,6 +214,18 @@ struct JobContext {
   // So this data structure doesn't track log files.
   autovector<uint64_t> files_to_quarantine;
 
+  // Blob file numbers that PurgeObsoleteFiles must keep. This includes both
+  // actively written direct-write files and sealed direct-write files that are
+  // still reachable through live memtables / old SuperVersions.
+  // Collected under db_mutex_ in FindObsoleteFiles so PurgeObsoleteFiles can
+  // safely use the snapshot without taking DB mutex.
+  UnorderedSet<uint64_t> active_blob_direct_write_files;
+
+  // Snapshot of VersionSet's next file number taken before collecting active
+  // direct-write blob files. This keeps the current purge pass from racing a
+  // concurrently created blob file that was not yet part of the active set.
+  uint64_t min_blob_file_number_to_keep = std::numeric_limits<uint64_t>::max();
+
   // a list of manifest files that we need to delete
   std::vector<std::string> manifest_delete_files;
 

db/memtable.cc

@@ -15,6 +15,7 @@
 #include <memory>
 #include <optional>
 
+#include "db/blob/blob_file_partition_manager.h"
 #include "db/dbformat.h"
 #include "db/kv_checksum.h"
 #include "db/merge_context.h"
@@ -40,6 +41,7 @@
 #include "table/internal_iterator.h"
 #include "table/iterator_wrapper.h"
 #include "table/merging_iterator.h"
+#include "util/atomic.h"
 #include "util/autovector.h"
 #include "util/coding.h"
 #include "util/mutexlock.h"
@@ -76,6 +78,51 @@ ImmutableMemTableOptions::ImmutableMemTableOptions(
       memtable_batch_lookup_optimization(
           ioptions.memtable_batch_lookup_optimization) {}
 
+void ReadOnlyMemTable::ProtectSealedBlobFiles(
+    const std::shared_ptr<BlobFilePartitionManager>& blob_partition_manager,
+    const std::vector<uint64_t>& file_numbers) {
+  if (file_numbers.empty()) {
+    return;
+  }
+
+  assert(blob_partition_manager != nullptr);
+  if (protected_blob_file_manager_ == nullptr) {
+    protected_blob_file_manager_ = blob_partition_manager;
+  } else {
+    assert(protected_blob_file_manager_.get() == blob_partition_manager.get());
+  }
+
+  std::vector<uint64_t> newly_protected_file_numbers;
+  newly_protected_file_numbers.reserve(file_numbers.size());
+  for (uint64_t file_number : file_numbers) {
+    auto it = std::find(protected_blob_file_numbers_.begin(),
+                        protected_blob_file_numbers_.end(), file_number);
+    if (it != protected_blob_file_numbers_.end()) {
+      continue;
+    }
+    protected_blob_file_numbers_.push_back(file_number);
+    newly_protected_file_numbers.push_back(file_number);
+  }
+
+  if (!newly_protected_file_numbers.empty()) {
+    protected_blob_file_manager_->ProtectSealedBlobFileNumbers(
+        newly_protected_file_numbers);
+  }
+}
+
+void ReadOnlyMemTable::ReleaseProtectedSealedBlobFiles() {
+  if (protected_blob_file_manager_ == nullptr) {
+    assert(protected_blob_file_numbers_.empty());
+    return;
+  }
+
+  std::shared_ptr<BlobFilePartitionManager> blob_partition_manager =
+      std::move(protected_blob_file_manager_);
+  blob_partition_manager->UnprotectSealedBlobFileNumbers(
+      protected_blob_file_numbers_);
+  protected_blob_file_numbers_.clear();
+}
+
 MemTable::MemTable(const InternalKeyComparator& cmp,
                    const ImmutableOptions& ioptions,
                    const MutableCFOptions& mutable_cf_options,
@@ -144,26 +191,14 @@ MemTable::MemTable(const InternalKeyComparator& cmp,
   auto new_cache = std::make_shared<FragmentedRangeTombstoneListCache>();
   size_t size = cached_range_tombstone_.Size();
   for (size_t i = 0; i < size; ++i) {
-#if defined(__cpp_lib_atomic_shared_ptr)
-    std::atomic<std::shared_ptr<FragmentedRangeTombstoneListCache>>*
-        local_cache_ref_ptr = cached_range_tombstone_.AccessAtCore(i);
-    auto new_local_cache_ref = std::make_shared<
-        const std::shared_ptr<FragmentedRangeTombstoneListCache>>(new_cache);
-    std::shared_ptr<FragmentedRangeTombstoneListCache> aliased_ptr(
-        new_local_cache_ref, new_cache.get());
-    local_cache_ref_ptr->store(std::move(aliased_ptr),
-                               std::memory_order_relaxed);
-#else
     std::shared_ptr<FragmentedRangeTombstoneListCache>* local_cache_ref_ptr =
         cached_range_tombstone_.AccessAtCore(i);
     auto new_local_cache_ref = std::make_shared<
         const std::shared_ptr<FragmentedRangeTombstoneListCache>>(new_cache);
-    std::atomic_store_explicit(
-        local_cache_ref_ptr,
-        std::shared_ptr<FragmentedRangeTombstoneListCache>(new_local_cache_ref,
-                                                           new_cache.get()),
+    std::shared_ptr<FragmentedRangeTombstoneListCache> aliased_ptr(
+        new_local_cache_ref, new_cache.get());
+    AtomicSharedPtrStore(local_cache_ref_ptr, std::move(aliased_ptr),
                          std::memory_order_relaxed);
-#endif
   }
   const Comparator* ucmp = cmp.user_comparator();
   assert(ucmp);
@@ -821,13 +856,8 @@ FragmentedRangeTombstoneIterator* MemTable::NewRangeTombstoneIteratorInternal(
 
   // takes current cache
   std::shared_ptr<FragmentedRangeTombstoneListCache> cache =
-#if defined(__cpp_lib_atomic_shared_ptr)
-      cached_range_tombstone_.Access()->load(std::memory_order_relaxed)
-#else
-      std::atomic_load_explicit(cached_range_tombstone_.Access(),
-                                std::memory_order_relaxed)
-#endif
-      ;
+      AtomicSharedPtrLoad(cached_range_tombstone_.Access(),
+                          std::memory_order_relaxed);
   // construct fragmented tombstone list if necessary
   if (!cache->initialized.load(std::memory_order_acquire)) {
     cache->reader_mutex.lock();
@@ -1089,31 +1119,19 @@ Status MemTable::Add(SequenceNumber s, ValueType type,
       range_del_mutex_.lock();
     }
     for (size_t i = 0; i < size; ++i) {
-#if defined(__cpp_lib_atomic_shared_ptr)
-      std::atomic<std::shared_ptr<FragmentedRangeTombstoneListCache>>*
-          local_cache_ref_ptr = cached_range_tombstone_.AccessAtCore(i);
-      auto new_local_cache_ref = std::make_shared<
-          const std::shared_ptr<FragmentedRangeTombstoneListCache>>(new_cache);
-      std::shared_ptr<FragmentedRangeTombstoneListCache> aliased_ptr(
-          new_local_cache_ref, new_cache.get());
-      local_cache_ref_ptr->store(std::move(aliased_ptr),
-                                 std::memory_order_relaxed);
-#else
       std::shared_ptr<FragmentedRangeTombstoneListCache>* local_cache_ref_ptr =
           cached_range_tombstone_.AccessAtCore(i);
       auto new_local_cache_ref = std::make_shared<
           const std::shared_ptr<FragmentedRangeTombstoneListCache>>(new_cache);
+      std::shared_ptr<FragmentedRangeTombstoneListCache> aliased_ptr(
+          new_local_cache_ref, new_cache.get());
       // It is okay for some reader to load old cache during invalidation as
       // the new sequence number is not published yet.
       // Each core will have a shared_ptr to a shared_ptr to the cached
-      // fragmented range tombstones, so that ref count is maintianed locally
+      // fragmented range tombstones, so that ref count is maintained locally
       // per-core using the per-core shared_ptr.
-      std::atomic_store_explicit(
-          local_cache_ref_ptr,
-          std::shared_ptr<FragmentedRangeTombstoneListCache>(
-              new_local_cache_ref, new_cache.get()),
+      AtomicSharedPtrStore(local_cache_ref_ptr, std::move(aliased_ptr),
                            std::memory_order_relaxed);
-#endif
     }
 
     if (allow_concurrent) {

db/memtable.h

@@ -37,6 +37,7 @@
 
 namespace ROCKSDB_NAMESPACE {
 
+class BlobFilePartitionManager;
 struct FlushJobInfo;
 class Mutex;
 class MemTableIterator;
@@ -344,11 +345,22 @@ class ReadOnlyMemTable {
     --refs_;
     assert(refs_ >= 0);
     if (refs_ <= 0) {
+      ReleaseProtectedSealedBlobFiles();
       return this;
     }
     return nullptr;
   }
 
+  // Registers sealed direct-write blob files that this memtable can still
+  // resolve through lazy blob indexes. The protection lasts until the
+  // memtable's final Unref(). The manager handle is shared here because
+  // immutable memtables can outlive the ColumnFamilyData that created them.
+  // REQUIRES: external synchronization to prevent simultaneous operations on
+  // the same MemTable.
+  void ProtectSealedBlobFiles(
+      const std::shared_ptr<BlobFilePartitionManager>& blob_partition_manager,
+      const std::vector<uint64_t>& file_numbers);
+
   // Returns the edits area that is needed for flushing the memtable
   VersionEdit* GetEdits() { return &edit_; }
 
@@ -532,6 +544,12 @@ class ReadOnlyMemTable {
   std::unique_ptr<FlushJobInfo> flush_job_info_;
 
   RelaxedAtomic<bool> marked_for_flush_{false};
+
+ private:
+  void ReleaseProtectedSealedBlobFiles();
+
+  std::shared_ptr<BlobFilePartitionManager> protected_blob_file_manager_;
+  std::vector<uint64_t> protected_blob_file_numbers_;
 };
 
 class MemTable final : public ReadOnlyMemTable {
@@ -952,15 +970,8 @@ class MemTable final : public ReadOnlyMemTable {
 
   // makes sure there is a single range tombstone writer to invalidate cache
   std::mutex range_del_mutex_;
-#if defined(__cpp_lib_atomic_shared_ptr)
-  CoreLocalArray<
-      std::atomic<std::shared_ptr<FragmentedRangeTombstoneListCache>>>
-      cached_range_tombstone_;
-#else
   CoreLocalArray<std::shared_ptr<FragmentedRangeTombstoneListCache>>
       cached_range_tombstone_;
-
-#endif
   void UpdateEntryChecksum(const ProtectionInfoKVOS64* kv_prot_info,
                            const Slice& key, const Slice& value, ValueType type,
                            SequenceNumber s, char* checksum_ptr);

db/obsolete_files_test.cc

@@ -13,11 +13,14 @@
 #include <string>
 #include <vector>
 
+#include "db/blob/blob_log_writer.h"
 #include "db/db_impl/db_impl.h"
 #include "db/db_test_util.h"
 #include "db/version_set.h"
 #include "db/write_batch_internal.h"
 #include "file/filename.h"
+#include "file/read_write_util.h"
+#include "file/writable_file_writer.h"
 #include "port/stack_trace.h"
 #include "rocksdb/db.h"
 #include "rocksdb/env.h"
@@ -29,6 +32,44 @@
 
 namespace ROCKSDB_NAMESPACE {
 
+namespace {
+
+void WriteFooterlessBlobFile(const ImmutableOptions& immutable_options,
+                             uint64_t blob_file_number) {
+  assert(!immutable_options.cf_paths.empty());
+
+  const std::string blob_file_path =
+      BlobFileName(immutable_options.cf_paths.front().path, blob_file_number);
+
+  std::unique_ptr<FSWritableFile> file;
+  ASSERT_OK(NewWritableFile(immutable_options.fs.get(), blob_file_path, &file,
+                            FileOptions()));
+
+  std::unique_ptr<WritableFileWriter> file_writer(new WritableFileWriter(
+      std::move(file), blob_file_path, FileOptions(), immutable_options.clock));
+
+  constexpr Statistics* statistics = nullptr;
+  constexpr bool use_fsync = false;
+  constexpr bool do_flush = false;
+  BlobLogWriter blob_log_writer(std::move(file_writer), immutable_options.clock,
+                                statistics, blob_file_number, use_fsync,
+                                do_flush);
+
+  constexpr bool has_ttl = false;
+  constexpr ExpirationRange expiration_range;
+  BlobLogHeader header(/*column_family_id=*/0, kNoCompression, has_ttl,
+                       expiration_range);
+  ASSERT_OK(blob_log_writer.WriteHeader(WriteOptions(), header));
+
+  uint64_t key_offset = 0;
+  uint64_t blob_offset = 0;
+  ASSERT_OK(blob_log_writer.AddRecord(WriteOptions(), "key", "blob",
+                                      &key_offset, &blob_offset));
+  ASSERT_OK(blob_log_writer.file()->Close(IOOptions()));
+}
+
+}  // namespace
+
 class ObsoleteFilesTest : public DBTestBase {
  public:
   ObsoleteFilesTest()
@@ -269,6 +310,7 @@ TEST_F(ObsoleteFilesTest, BlobFiles) {
       BlobFileName(pending_blob_file_number), path);
 
   job_context.min_pending_output = pending_blob_file_number;
+  job_context.min_blob_file_number_to_keep = pending_blob_file_number;
 
   // Purge obsolete files and make sure we purge the old file and the first file
   // (and keep the second file and the pending file).
@@ -303,6 +345,58 @@ TEST_F(ObsoleteFilesTest, BlobFiles) {
   ASSERT_EQ(deleted_files, expected_deleted_files);
 }
 
+TEST_F(ObsoleteFilesTest, FooterlessBlobFileIsKeptDuringPurge) {
+  ReopenDB();
+
+  VersionSet* const versions = dbfull()->GetVersionSet();
+  assert(versions);
+  assert(versions->GetColumnFamilySet());
+
+  ColumnFamilyData* const cfd = versions->GetColumnFamilySet()->GetDefault();
+  assert(cfd);
+
+  const auto& cf_paths = cfd->ioptions().cf_paths;
+  assert(!cf_paths.empty());
+  const std::string& path = cf_paths.front().path;
+
+  constexpr uint64_t blob_file_number = 777;
+  const std::string blob_file_path = BlobFileName(path, blob_file_number);
+  WriteFooterlessBlobFile(cfd->ioptions(), blob_file_number);
+  ASSERT_OK(env_->FileExists(blob_file_path));
+
+  constexpr int job_id = 0;
+  JobContext job_context{job_id};
+  dbfull()->TEST_LockMutex();
+  constexpr bool force_full_scan = false;
+  dbfull()->FindObsoleteFiles(&job_context, force_full_scan);
+  dbfull()->TEST_UnlockMutex();
+
+  job_context.full_scan_candidate_files.emplace_back(
+      BlobFileName(blob_file_number), path);
+  job_context.min_pending_output = blob_file_number + 1;
+  job_context.min_blob_file_number_to_keep = blob_file_number + 1;
+
+  bool deleted = false;
+  SyncPoint::GetInstance()->SetCallBack(
+      "DBImpl::DeleteObsoleteFileImpl::BeforeDeletion", [&](void* arg) {
+        const std::string* file = static_cast<std::string*>(arg);
+        assert(file);
+        if (*file == blob_file_path) {
+          deleted = true;
+        }
+      });
+  SyncPoint::GetInstance()->EnableProcessing();
+
+  dbfull()->PurgeObsoleteFiles(job_context);
+  job_context.Clean();
+
+  SyncPoint::GetInstance()->DisableProcessing();
+  SyncPoint::GetInstance()->ClearAllCallBacks();
+
+  ASSERT_FALSE(deleted);
+  ASSERT_OK(env_->FileExists(blob_file_path));
+}
+
 TEST_F(ObsoleteFilesTest, GetSortedWalFilesHangsAfterNoopPurge) {
   // This test used to trigger a hang in `DB::GetSortedWalFiles()`, where it
   // would wait for a no-op purge that did not signal the CV upon completion.

db/version_set.cc

@@ -2637,7 +2637,7 @@ void Version::MultiGetBlob(
     BlobReadContexts& blobs_in_file = ctx.second;
     for (auto& blob : blobs_in_file) {
       const BlobIndex& blob_index = blob.blob_index;
-      const KeyContext* const key_context = blob.key_context;
+      KeyContext* const key_context = blob.key_context;
       assert(key_context);
       assert(key_context->get_context);
       assert(key_context->s);
@@ -2679,7 +2679,7 @@ void Version::MultiGetBlob(
   for (auto& ctx : blob_ctxs) {
     BlobReadContexts& blobs_in_file = ctx.second;
     for (auto& blob : blobs_in_file) {
-      const KeyContext* const key_context = blob.key_context;
+      KeyContext* const key_context = blob.key_context;
       assert(key_context);
       assert(key_context->get_context);
       assert(key_context->s);
@@ -2693,6 +2693,10 @@ void Version::MultiGetBlob(
           key_context->columns->SetPlainValue(std::move(blob.result));
           range.AddValueSize(key_context->columns->serialized_size());
         }
+        // MultiGetBlob() has already materialized the blob reference into the
+        // user-visible value, so clear this flag before later MultiGet
+        // post-processing checks whether further blob resolution is needed.
+        key_context->is_blob_index = false;
 
         if (range.GetValueSize() > read_options.value_size_soft_limit) {
           *key_context->s = Status::Aborted();

db/version_set.h

@@ -984,11 +984,11 @@ class Version {
                  uint64_t* bytes_read) const;
 
   struct BlobReadContext {
-    BlobReadContext(const BlobIndex& blob_idx, const KeyContext* key_ctx)
+    BlobReadContext(const BlobIndex& blob_idx, KeyContext* key_ctx)
         : blob_index(blob_idx), key_context(key_ctx) {}
 
     BlobIndex blob_index;
-    const KeyContext* key_context;
+    KeyContext* key_context;
     PinnableSlice result;
   };
 

db/write_batch.cc

@@ -625,6 +625,7 @@ Status WriteBatchInternal::Iterate(const WriteBatch* wb,
                (ContentFlags::DEFERRED | ContentFlags::HAS_BLOB_INDEX));
         s = handler->PutBlobIndexCF(column_family, key, value);
         if (LIKELY(s.ok())) {
+          empty_batch = false;
           found++;
         }
         break;
@@ -1093,6 +1094,23 @@ Status WriteBatchInternal::PutEntity(WriteBatch* b, uint32_t column_family_id,
     return Status::InvalidArgument("wide column entity is too large");
   }
 
+  return PutEntitySerialized(b, column_family_id, key, entity);
+}
+
+Status WriteBatchInternal::PutEntitySerialized(WriteBatch* b,
+                                               uint32_t column_family_id,
+                                               const Slice& key,
+                                               const Slice& entity) {
+  assert(b);
+
+  if (key.size() > size_t{std::numeric_limits<uint32_t>::max()}) {
+    return Status::InvalidArgument("key is too large");
+  }
+
+  if (entity.size() > size_t{std::numeric_limits<uint32_t>::max()}) {
+    return Status::InvalidArgument("wide column entity is too large");
+  }
+
   LocalSavePoint save(b);
 
   WriteBatchInternal::SetCount(b, WriteBatchInternal::Count(b) + 1);

db/write_batch_internal.h

@@ -94,6 +94,13 @@ class WriteBatchInternal {
   static Status PutEntity(WriteBatch* batch, uint32_t column_family_id,
                           const Slice& key, const WideColumns& columns);
 
+  // Appends an already-serialized wide-column entity. This is used by
+  // WriteBatch handlers that need to preserve the original entity bytes
+  // without a deserialize/re-serialize round-trip.
+  static Status PutEntitySerialized(WriteBatch* batch,
+                                    uint32_t column_family_id, const Slice& key,
+                                    const Slice& entity);
+
   static Status Delete(WriteBatch* batch, uint32_t column_family_id,
                        const SliceParts& key);
 

db/write_batch_test.cc

@@ -8,6 +8,7 @@
 // found in the LICENSE file. See the AUTHORS file for names of contributors.
 
 #include <memory>
+#include <unordered_map>
 
 #include "db/column_family.h"
 #include "db/db_test_util.h"
@@ -391,6 +392,68 @@ struct TestHandler : public WriteBatch::Handler {
     return Status::OK();
   }
 };
+
+struct ReplayUntilCountHandler : public WriteBatch::Handler {
+  explicit ReplayUntilCountHandler(uint32_t max_write_ops)
+      : max_write_ops_(max_write_ops) {}
+
+  bool Continue() override { return num_write_ops_ < max_write_ops_; }
+
+  Status PutCF(uint32_t column_family_id, const Slice& key,
+               const Slice& value) override {
+    if (buffered_writes_) {
+      return WriteBatchInternal::Put(buffered_writes_.get(), column_family_id,
+                                     key, value);
+    }
+    seen += "Put(" + key.ToString() + ", " + value.ToString() + ")";
+    ++num_write_ops_;
+    return Status::OK();
+  }
+
+  Status DeleteCF(uint32_t column_family_id, const Slice& key) override {
+    if (buffered_writes_) {
+      return WriteBatchInternal::Delete(buffered_writes_.get(),
+                                        column_family_id, key);
+    }
+    seen += "Delete(" + key.ToString() + ")";
+    ++num_write_ops_;
+    return Status::OK();
+  }
+
+  Status MarkBeginPrepare(bool /* unprepare */) override {
+    assert(!buffered_writes_);
+    buffered_writes_ = std::make_unique<WriteBatch>();
+    return Status::OK();
+  }
+
+  Status MarkEndPrepare(const Slice& xid) override {
+    assert(buffered_writes_);
+    prepared_writes_[xid.ToString()] = std::move(buffered_writes_);
+    return Status::OK();
+  }
+
+  Status MarkNoop(bool /* empty_batch */) override { return Status::OK(); }
+
+  Status MarkCommit(const Slice& xid) override {
+    auto it = prepared_writes_.find(xid.ToString());
+    if (it == prepared_writes_.end() || !it->second) {
+      return Status::Corruption("Missing prepared batch for commit");
+    }
+    Status s = it->second->Iterate(this);
+    prepared_writes_.erase(it);
+    return s;
+  }
+
+  uint32_t NumWriteOps() const { return num_write_ops_; }
+
+  std::string seen;
+
+ private:
+  const uint32_t max_write_ops_;
+  uint32_t num_write_ops_ = 0;
+  std::unordered_map<std::string, std::unique_ptr<WriteBatch>> prepared_writes_;
+  std::unique_ptr<WriteBatch> buffered_writes_;
+};
 }  // anonymous namespace
 
 TEST_F(WriteBatchTest, PutNotImplemented) {
@@ -520,6 +583,39 @@ TEST_F(WriteBatchTest, PrepareCommit) {
       handler.seen);
 }
 
+// Expected-state restore can target a sequence number in the middle of a traced
+// multi-op write batch. Verify `Continue()` stops iteration cleanly there.
+TEST_F(WriteBatchTest, ContinueStopsMidBatch) {
+  WriteBatch batch;
+  ASSERT_OK(batch.Put(Slice("k1"), Slice("v1")));
+  ASSERT_OK(batch.Delete(Slice("k2")));
+  ASSERT_OK(batch.Put(Slice("k3"), Slice("v3")));
+
+  ReplayUntilCountHandler handler(/* max_write_ops */ 2);
+  ASSERT_OK(batch.Iterate(&handler));
+  ASSERT_EQ(2u, handler.NumWriteOps());
+  ASSERT_EQ("Put(k1, v1)Delete(k2)", handler.seen);
+}
+
+// Regression test for restore replay stopping inside a committed prepared
+// batch. The handler buffers prepare contents and replays them on commit,
+// matching the expected-state restore logic.
+TEST_F(WriteBatchTest, ContinueStopsMidPreparedCommitReplay) {
+  WriteBatch batch;
+  ASSERT_OK(WriteBatchInternal::InsertNoop(&batch));
+  ASSERT_OK(batch.Put(Slice("k1"), Slice("v1")));
+  ASSERT_OK(batch.Put(Slice("k2"), Slice("v2")));
+  batch.SetSavePoint();
+  ASSERT_OK(WriteBatchInternal::MarkEndPrepare(&batch, Slice("xid1")));
+  ASSERT_EQ(Status::NotFound(), batch.RollbackToSavePoint());
+  ASSERT_OK(WriteBatchInternal::MarkCommit(&batch, Slice("xid1")));
+
+  ReplayUntilCountHandler handler(/* max_write_ops */ 1);
+  ASSERT_OK(batch.Iterate(&handler));
+  ASSERT_EQ(1u, handler.NumWriteOps());
+  ASSERT_EQ("Put(k1, v1)", handler.seen);
+}
+
 // It requires more than 30GB of memory to run the test. With single memory
 // allocation of more than 30GB.
 // Not all platform can run it. Also it runs a long time. So disable it.

db/write_thread.h

@@ -123,6 +123,9 @@ class WriteThread {
   // Information kept for every waiting writer.
   struct Writer {
     WriteBatch* batch;
+    // The logical user batch to record in query traces. This can differ from
+    // `batch` when the write path rewrites the applied batch.
+    WriteBatch* trace_batch;
     bool sync;
     bool no_slowdown;
     bool disable_wal;
@@ -152,6 +155,7 @@ class WriteThread {
 
     Writer()
         : batch(nullptr),
+          trace_batch(nullptr),
           sync(false),
           no_slowdown(false),
           disable_wal(false),
@@ -177,8 +181,9 @@ class WriteThread {
            uint64_t _log_ref, bool _disable_memtable, size_t _batch_cnt = 0,
            PreReleaseCallback* _pre_release_callback = nullptr,
            PostMemTableCallback* _post_memtable_callback = nullptr,
-           bool _ingest_wbwi = false)
+           bool _ingest_wbwi = false, WriteBatch* _trace_batch = nullptr)
         : batch(_batch),
+          trace_batch(_trace_batch != nullptr ? _trace_batch : _batch),
           // TODO: store a copy of WriteOptions instead of its separated data
           // members
           sync(write_options.sync),

db_stress_tool/db_stress_common.h

@@ -320,6 +320,8 @@ DECLARE_bool(blob_db_enable_gc);
 // Options for integrated BlobDB
 DECLARE_bool(allow_setting_blob_options_dynamically);
 DECLARE_bool(enable_blob_files);
+DECLARE_bool(enable_blob_direct_write);
+DECLARE_uint64(blob_direct_write_partitions);
 DECLARE_uint64(min_blob_size);
 DECLARE_uint64(blob_file_size);
 DECLARE_string(blob_compression_type);
@@ -497,6 +499,8 @@ inline enum RepFactory StringToRepFactory(const char* ctype) {
 extern enum RepFactory FLAGS_rep_factory;
 
 namespace ROCKSDB_NAMESPACE {
+void RegisterDbStressBdwFlagValidators();
+
 inline enum ROCKSDB_NAMESPACE::CompressionType StringToCompressionType(
     const char* ctype) {
   assert(ctype);

db_stress_tool/db_stress_gflags.cc

@@ -483,6 +483,29 @@ DEFINE_bool(
     ROCKSDB_NAMESPACE::AdvancedColumnFamilyOptions().enable_blob_files,
     "[Integrated BlobDB] Enable writing large values to separate blob files.");
 
+DEFINE_bool(
+    enable_blob_direct_write,
+    ROCKSDB_NAMESPACE::AdvancedColumnFamilyOptions().enable_blob_direct_write,
+    "[Integrated BlobDB] Enable direct-write blob file creation on "
+    "the write path.");
+
+DEFINE_uint64(blob_direct_write_partitions,
+              ROCKSDB_NAMESPACE::AdvancedColumnFamilyOptions()
+                  .blob_direct_write_partitions,
+              "[Integrated BlobDB] Number of partitions for direct-write blob "
+              "files.");
+
+namespace ROCKSDB_NAMESPACE {
+
+void RegisterDbStressBdwFlagValidators() {
+  static const bool blob_direct_write_partitions_validator_registered =
+      RegisterFlagValidator(&FLAGS_blob_direct_write_partitions,
+                            &ValidateUint32Range);
+  (void)blob_direct_write_partitions_validator_registered;
+}
+
+}  // namespace ROCKSDB_NAMESPACE
+
 DEFINE_uint64(min_blob_size,
               ROCKSDB_NAMESPACE::AdvancedColumnFamilyOptions().min_blob_size,
               "[Integrated BlobDB] The size of the smallest value to be stored "

db_stress_tool/db_stress_test_base.cc

@@ -3761,12 +3761,14 @@ void StressTest::Open(SharedState* shared, bool reopen) {
 
   fprintf(stdout,
           "Integrated BlobDB: blob files enabled %d, min blob size %" PRIu64
+          ", direct write enabled %d, direct write partitions %" PRIu32
           ", blob file size %" PRIu64
           ", blob compression type %s, blob GC enabled %d, cutoff %f, force "
           "threshold %f, blob compaction readahead size %" PRIu64
           ", blob file starting level %d\n",
           options_.enable_blob_files, options_.min_blob_size,
-          options_.blob_file_size,
+          options_.enable_blob_direct_write,
+          options_.blob_direct_write_partitions, options_.blob_file_size,
           CompressionTypeToString(options_.blob_compression_type).c_str(),
           options_.enable_blob_garbage_collection,
           options_.blob_garbage_collection_age_cutoff,
@@ -4563,6 +4565,9 @@ void InitializeOptionsFromFlags(
 
   // Integrated BlobDB
   options.enable_blob_files = FLAGS_enable_blob_files;
+  options.enable_blob_direct_write = FLAGS_enable_blob_direct_write;
+  options.blob_direct_write_partitions =
+      static_cast<uint32_t>(FLAGS_blob_direct_write_partitions);
   options.min_blob_size = FLAGS_min_blob_size;
   options.blob_file_size = FLAGS_blob_file_size;
   options.blob_compression_type =

db_stress_tool/db_stress_tool.cc

@@ -21,6 +21,8 @@
 // different behavior. See comment of the flag for details.
 
 #ifdef GFLAGS
+#include <iostream>
+
 #include "db_stress_tool/db_stress_common.h"
 #include "db_stress_tool/db_stress_driver.h"
 #include "db_stress_tool/db_stress_shared_state.h"
@@ -36,6 +38,11 @@ static std::shared_ptr<ROCKSDB_NAMESPACE::Env> legacy_env_wrapper_guard;
 static std::shared_ptr<ROCKSDB_NAMESPACE::CompositeEnvWrapper>
     dbsl_env_wrapper_guard;
 static std::shared_ptr<CompositeEnvWrapper> fault_env_guard;
+
+int ReturnFlagValidationError(const char* message) {
+  std::cerr << "Error: " << message << '\n';
+  return 1;
+}
 }  // namespace
 
 KeyGenContext key_gen_ctx;
@@ -43,6 +50,7 @@ KeyGenContext key_gen_ctx;
 int db_stress_tool(int argc, char** argv) {
   SetUsageMessage(std::string("\nUSAGE:\n") + std::string(argv[0]) +
                   " [OPTIONS]...");
+  RegisterDbStressBdwFlagValidators();
   ParseCommandLineFlags(&argc, &argv, true);
 
   SanitizeDoubleParam(&FLAGS_bloom_bits);
@@ -213,6 +221,84 @@ int db_stress_tool(int argc, char** argv) {
         "Error: nooverwritepercent must not be 100 when using merge operands");
     exit(1);
   }
+  if (FLAGS_enable_blob_direct_write) {
+    // Blob direct write is intentionally validated as a reduced-scope v1
+    // feature. We allow the WAL-disabled crash-test profile, but reject
+    // best-efforts recovery, parallel memtable/write-queue variants,
+    // transactions, remote compaction, and APIs/features that depend on
+    // active-file snapshotting or unsupported blob option transitions.
+    if (!FLAGS_enable_blob_files) {
+      return ReturnFlagValidationError(
+          "enable_blob_direct_write requires enable_blob_files");
+    }
+    if (FLAGS_allow_concurrent_memtable_write) {
+      return ReturnFlagValidationError(
+          "blob direct write stress requires "
+          "allow_concurrent_memtable_write=0");
+    }
+    if (FLAGS_enable_pipelined_write) {
+      return ReturnFlagValidationError(
+          "blob direct write stress does not support "
+          "enable_pipelined_write");
+    }
+    if (FLAGS_unordered_write) {
+      return ReturnFlagValidationError(
+          "blob direct write stress does not support unordered_write");
+    }
+    if (FLAGS_two_write_queues) {
+      return ReturnFlagValidationError(
+          "blob direct write stress does not support two_write_queues");
+    }
+    if (FLAGS_use_blob_db) {
+      return ReturnFlagValidationError(
+          "blob direct write is only supported with integrated BlobDB");
+    }
+    if (FLAGS_use_merge || FLAGS_use_full_merge_v1) {
+      return ReturnFlagValidationError(
+          "blob direct write stress does not support merge");
+    }
+    if (FLAGS_experimental_mempurge_threshold > 0.0) {
+      return ReturnFlagValidationError(
+          "blob direct write stress does not support MemPurge");
+    }
+    if (FLAGS_user_timestamp_size > 0) {
+      return ReturnFlagValidationError(
+          "blob direct write stress does not support user-defined timestamps");
+    }
+    if (FLAGS_allow_setting_blob_options_dynamically ||
+        FLAGS_enable_blob_garbage_collection) {
+      return ReturnFlagValidationError(
+          "blob direct write stress does not support dynamic blob options or "
+          "blob GC");
+    }
+    if (FLAGS_best_efforts_recovery) {
+      return ReturnFlagValidationError(
+          "blob direct write stress supports disable_wal-based crash "
+          "testing, not best-efforts recovery");
+    }
+    if (FLAGS_remote_compaction_worker_threads > 0) {
+      return ReturnFlagValidationError(
+          "blob direct write stress does not support remote compaction");
+    }
+    if (FLAGS_use_txn || FLAGS_txn_write_policy != 0 ||
+        FLAGS_use_optimistic_txn || FLAGS_test_multi_ops_txns ||
+        FLAGS_commit_bypass_memtable_one_in > 0) {
+      return ReturnFlagValidationError(
+          "blob direct write stress does not support TransactionDB modes");
+    }
+    if (FLAGS_test_secondary || FLAGS_backup_one_in > 0 ||
+        FLAGS_checkpoint_one_in > 0 || FLAGS_get_live_files_apis_one_in > 0 ||
+        FLAGS_ingest_external_file_one_in > 0) {
+      return ReturnFlagValidationError(
+          "blob direct write stress does not support secondary, backup, "
+          "checkpoint, get_live_files, or ingest_external_file modes");
+    }
+    if (FLAGS_ingest_wbwi_one_in > 0) {
+      return ReturnFlagValidationError(
+          "blob direct write stress does not support "
+          "IngestWriteBatchWithIndex");
+    }
+  }
   if (FLAGS_ingest_external_file_one_in > 0 &&
       FLAGS_nooverwritepercent == 100) {
     fprintf(

db_stress_tool/expected_state.cc

@@ -419,8 +419,8 @@ bool FileExpectedStateManager::HasHistory() {
 
 namespace {
 
-// An `ExpectedStateTraceRecordHandler` applies a configurable number of
-// write operation trace records to the configured expected state. It is used in
+// An `ExpectedStateTraceRecordHandler` applies a configurable number of traced
+// write operations to the configured expected state. It is used in
 // `FileExpectedStateManager::Restore()` to sync the expected state with the
 // DB's post-recovery state.
 class ExpectedStateTraceRecordHandler : public TraceRecord::Handler,
@@ -431,10 +431,12 @@ class ExpectedStateTraceRecordHandler : public TraceRecord::Handler,
         state_(state),
         buffered_writes_(nullptr) {}
 
-  ~ExpectedStateTraceRecordHandler() { assert(IsDone()); }
-
   // True if we have already reached the limit on write operations to apply.
-  bool IsDone() { return num_write_ops_ == max_write_ops_; }
+  bool IsDone() const { return num_write_ops_ >= max_write_ops_; }
+
+  uint64_t NumWriteOps() const { return num_write_ops_; }
+
+  bool Continue() override { return !IsDone(); }
 
   Status Handle(const WriteQueryTraceRecord& record,
                 std::unique_ptr<TraceRecordResult>* /* result */) override {
@@ -484,7 +486,7 @@ class ExpectedStateTraceRecordHandler : public TraceRecord::Handler,
     }
 
     state_->SyncPut(column_family_id, static_cast<int64_t>(key_id), value_base);
-    ++num_write_ops_;
+    NoteWriteOpApplied();
     return Status::OK();
   }
 
@@ -506,7 +508,7 @@ class ExpectedStateTraceRecordHandler : public TraceRecord::Handler,
     }
 
     state_->SyncPut(column_family_id, static_cast<int64_t>(key_id), value_base);
-    ++num_write_ops_;
+    NoteWriteOpApplied();
     return Status::OK();
   }
 
@@ -541,8 +543,7 @@ class ExpectedStateTraceRecordHandler : public TraceRecord::Handler,
         GetValueBase(WideColumnsHelper::GetDefaultColumn(columns));
 
     state_->SyncPut(column_family_id, static_cast<int64_t>(key_id), value_base);
-
-    ++num_write_ops_;
+    NoteWriteOpApplied();
 
     return Status::OK();
   }
@@ -563,7 +564,7 @@ class ExpectedStateTraceRecordHandler : public TraceRecord::Handler,
     }
 
     state_->SyncDelete(column_family_id, static_cast<int64_t>(key_id));
-    ++num_write_ops_;
+    NoteWriteOpApplied();
     return Status::OK();
   }
 
@@ -609,7 +610,7 @@ class ExpectedStateTraceRecordHandler : public TraceRecord::Handler,
     state_->SyncDeleteRange(column_family_id,
                             static_cast<int64_t>(begin_key_id),
                             static_cast<int64_t>(end_key_id));
-    ++num_write_ops_;
+    NoteWriteOpApplied();
     return Status::OK();
   }
 
@@ -627,6 +628,33 @@ class ExpectedStateTraceRecordHandler : public TraceRecord::Handler,
     return PutCF(column_family_id, key, value);
   }
 
+  Status PutBlobIndexCF(uint32_t column_family_id, const Slice& key_with_ts,
+                        const Slice& value) override {
+    Slice key =
+        StripTimestampFromUserKey(key_with_ts, FLAGS_user_timestamp_size);
+    uint64_t key_id;
+    if (!GetIntVal(key.ToString(), &key_id)) {
+      return Status::Corruption("unable to parse key", key.ToString());
+    }
+
+    bool should_buffer_write = !(buffered_writes_ == nullptr);
+    if (should_buffer_write) {
+      return WriteBatchInternal::PutBlobIndex(buffered_writes_.get(),
+                                              column_family_id, key, value);
+    }
+
+    // Blob direct-write traces record the transformed BlobIndex write rather
+    // than the original value bytes. For expected-state replay we only need the
+    // logical effect of "another put to this key", and db_stress values advance
+    // deterministically by one value_base per committed write.
+    const uint32_t value_base =
+        state_->Get(column_family_id, static_cast<int64_t>(key_id))
+            .NextValueBase();
+    state_->SyncPut(column_family_id, static_cast<int64_t>(key_id), value_base);
+    NoteWriteOpApplied();
+    return Status::OK();
+  }
+
   Status MarkBeginPrepare(bool = false) override {
     assert(!buffered_writes_);
     buffered_writes_.reset(new WriteBatch());
@@ -669,6 +697,11 @@ class ExpectedStateTraceRecordHandler : public TraceRecord::Handler,
   }
 
  private:
+  void NoteWriteOpApplied() {
+    ++num_write_ops_;
+    assert(num_write_ops_ <= max_write_ops_);
+  }
+
   uint64_t num_write_ops_ = 0;
   uint64_t max_write_ops_;
   ExpectedState* state_;
@@ -759,6 +792,12 @@ Status FileExpectedStateManager::Restore(DB* db) {
       std::unique_ptr<TraceRecordResult> res;
       s = record->Accept(handler.get(), &res);
     }
+    if (s.ok() && !handler->IsDone()) {
+      s = Status::Corruption(
+          "Trace ended before replaying all expected write ops",
+          std::to_string(handler->NumWriteOps()) + " < " +
+              std::to_string(seqno - saved_seqno_));
+    }
   }
 
   if (s.ok()) {

db_stress_tool/no_batched_ops_stress.cc

@@ -86,6 +86,13 @@ class NonBatchedOpsStressTest : public StressTest {
       if (method == VerificationMethod::kMultiGet && !FLAGS_use_multiget) {
         method = VerificationMethod::kGet;
       }
+      if (method == VerificationMethod::kGetMergeOperands &&
+          (!FLAGS_use_merge || FLAGS_enable_blob_files)) {
+        // GetMergeOperands only makes sense when merge operands are being
+        // generated, and stacked BlobDB does not support exposing raw merge
+        // operands for blob-indexed values.
+        method = VerificationMethod::kGet;
+      }
 
       if (method == VerificationMethod::kIterator) {
         std::unique_ptr<ManagedSnapshot> snapshot = nullptr;

include/rocksdb/advanced_options.h

@@ -1188,6 +1188,43 @@ struct AdvancedColumnFamilyOptions {
   // Dynamically changeable through the SetOptions() API
   PrepopulateBlobCache prepopulate_blob_cache = PrepopulateBlobCache::kDisable;
 
+  // When enabled, values >= min_blob_size are written directly to blob files
+  // during the write path and replaced in WAL and memtable with BlobIndex
+  // references.
+  //
+  // Requires enable_blob_files = true.
+  // Experimental reduced-scope v1 restrictions. These limitations keep the v1
+  // implementation intentionally small; follow-up PRs are expected to improve
+  // feature compatibility over time:
+  //  - only supports the ordered single-memtable-writer path; unordered,
+  //    pipelined, two_write_queues, and allow_concurrent_memtable_write are
+  //    not supported.
+  //  - crash recovery only supports blob files that were already made
+  //    manifest-visible by flush/SST creation; WAL replay of active
+  //    direct-write blob files is not currently supported.
+  //  - checkpoint/backup/live-files enumeration must flush pending
+  //    direct-write state first; APIs that intentionally skip the flush, or
+  //    run while WAL is locked, can return NotSupported.
+  //  - not compatible with MemPurge or user-defined timestamps.
+  //  - DB::IngestWriteBatchWithIndex() is not supported while any live column
+  //    family enables this option.
+  //  - read-only and secondary opens can read flushed/manifest-visible blob
+  //    files, but do not resolve still-active direct-write blob files.
+  //
+  // Default: false
+  //
+  // Not dynamically changeable through the SetOptions() API.
+  bool enable_blob_direct_write = false;
+
+  // Number of direct-write blob partitions for this column family.
+  // Partition selection is round-robin.
+  // Requires enable_blob_direct_write = true.
+  //
+  // Default: 1
+  //
+  // Not dynamically changeable through the SetOptions() API.
+  uint32_t blob_direct_write_partitions = 1;
+
   // Enable memtable per key-value checksum protection.
   //
   // Each entry in memtable will be suffixed by a per key-value checksum.

options/cf_options.cc

@@ -916,6 +916,14 @@ static std::unordered_map<std::string, OptionTypeInfo>
             auto* cache = static_cast<std::shared_ptr<Cache>*>(addr);
             return Cache::CreateFromString(opts, value, cache);
           }}},
+        {"enable_blob_direct_write",
+         {offsetof(struct ImmutableCFOptions, enable_blob_direct_write),
+          OptionType::kBoolean, OptionVerificationType::kNormal,
+          OptionTypeFlags::kNone}},
+        {"blob_direct_write_partitions",
+         {offsetof(struct ImmutableCFOptions, blob_direct_write_partitions),
+          OptionType::kUInt32T, OptionVerificationType::kNormal,
+          OptionTypeFlags::kNone}},
         {"persist_user_defined_timestamps",
          {offsetof(struct ImmutableCFOptions, persist_user_defined_timestamps),
           OptionType::kBoolean, OptionVerificationType::kNormal,
@@ -1067,6 +1075,8 @@ ImmutableCFOptions::ImmutableCFOptions(const ColumnFamilyOptions& cf_options)
       compaction_thread_limiter(cf_options.compaction_thread_limiter),
       sst_partitioner_factory(cf_options.sst_partitioner_factory),
       blob_cache(cf_options.blob_cache),
+      enable_blob_direct_write(cf_options.enable_blob_direct_write),
+      blob_direct_write_partitions(cf_options.blob_direct_write_partitions),
       persist_user_defined_timestamps(
           cf_options.persist_user_defined_timestamps),
       cf_allow_ingest_behind(cf_options.cf_allow_ingest_behind),

options/cf_options.h

@@ -81,6 +81,14 @@ struct ImmutableCFOptions {
 
   std::shared_ptr<Cache> blob_cache;
 
+  // Immutable snapshot of
+  // AdvancedColumnFamilyOptions::enable_blob_direct_write.
+  bool enable_blob_direct_write;
+
+  // Immutable snapshot of
+  // AdvancedColumnFamilyOptions::blob_direct_write_partitions.
+  uint32_t blob_direct_write_partitions;
+
   bool persist_user_defined_timestamps;
 
   bool cf_allow_ingest_behind;

options/options.cc

@@ -472,6 +472,11 @@ void ColumnFamilyOptions::Dump(Logger* log) const {
                    cf_allow_ingest_behind ? "true" : "false");
   ROCKS_LOG_HEADER(log, "  Options.memtable_batch_lookup_optimization: %s",
                    memtable_batch_lookup_optimization ? "true" : "false");
+  ROCKS_LOG_HEADER(log, "             Options.enable_blob_direct_write: %s",
+                   enable_blob_direct_write ? "true" : "false");
+  ROCKS_LOG_HEADER(log,
+                   "           Options.blob_direct_write_partitions: %" PRIu32,
+                   blob_direct_write_partitions);
 }  // ColumnFamilyOptions::Dump
 
 void Options::Dump(Logger* log) const {

options/options_helper.cc

@@ -351,6 +351,8 @@ void UpdateColumnFamilyOptions(const ImmutableCFOptions& ioptions,
   cf_opts->compaction_thread_limiter = ioptions.compaction_thread_limiter;
   cf_opts->sst_partitioner_factory = ioptions.sst_partitioner_factory;
   cf_opts->blob_cache = ioptions.blob_cache;
+  cf_opts->enable_blob_direct_write = ioptions.enable_blob_direct_write;
+  cf_opts->blob_direct_write_partitions = ioptions.blob_direct_write_partitions;
   cf_opts->persist_user_defined_timestamps =
       ioptions.persist_user_defined_timestamps;
   cf_opts->default_temperature = ioptions.default_temperature;

options/options_settable_test.cc

@@ -666,6 +666,7 @@ TEST_F(OptionsSettableTest, ColumnFamilyOptionsAllFieldsSettable) {
       "read_triggered_compaction_threshold=0.5;"
       "sample_for_compression=0;"
       "enable_blob_files=true;"
+      "enable_blob_direct_write=true;"
       "min_blob_size=256;"
       "blob_file_size=1000000;"
       "blob_compression_type=kBZip2Compression;"
@@ -674,6 +675,7 @@ TEST_F(OptionsSettableTest, ColumnFamilyOptionsAllFieldsSettable) {
       "blob_garbage_collection_force_threshold=0.75;"
       "blob_compaction_readahead_size=262144;"
       "blob_file_starting_level=1;"
+      "blob_direct_write_partitions=3;"
       "prepopulate_blob_cache=kDisable;"
       "bottommost_temperature=kWarm;"
       "last_level_temperature=kWarm;"
@@ -709,6 +711,8 @@ TEST_F(OptionsSettableTest, ColumnFamilyOptionsAllFieldsSettable) {
 
   // Custom verification since compaction_options_fifo was in
   // kColumnFamilyOptionsExcluded
+  ASSERT_TRUE(new_options->enable_blob_direct_write);
+  ASSERT_EQ(new_options->blob_direct_write_partitions, 3U);
   ASSERT_EQ(new_options->compaction_options_fifo.max_table_files_size, 3);
   ASSERT_EQ(new_options->compaction_options_fifo.allow_compaction, true);
   ASSERT_EQ(new_options->compaction_options_fifo.file_temperature_age_thresholds

src.mk

@@ -20,6 +20,7 @@ LIB_SOURCES =                                                   \
   db/blob/blob_file_addition.cc                                 \
   db/blob/blob_file_builder.cc                                  \
   db/blob/blob_file_cache.cc                                    \
+  db/blob/blob_file_partition_manager.cc                        \
   db/blob/blob_file_garbage.cc                                  \
   db/blob/blob_file_meta.cc                                     \
   db/blob/blob_file_reader.cc                                   \
@@ -28,6 +29,7 @@ LIB_SOURCES =                                                   \
   db/blob/blob_log_sequential_reader.cc                         \
   db/blob/blob_log_writer.cc                                    \
   db/blob/blob_source.cc                                        \
+  db/blob/blob_write_batch_transformer.cc                       \
   db/blob/prefetch_buffer_collection.cc                         \
   db/builder.cc                                                 \
   db/c.cc                                                       \
@@ -478,6 +480,7 @@ TEST_MAIN_SOURCES =                                                     \
   db/blob/blob_garbage_meter_test.cc                                    \
   db/blob/blob_source_test.cc                                           \
   db/blob/db_blob_basic_test.cc                                         \
+  db/blob/db_blob_direct_write_test.cc                                  \
   db/blob/db_blob_compaction_test.cc                                    \
   db/blob/db_blob_corruption_test.cc                                    \
   db/blob/db_blob_index_test.cc                                         \

tools/db_bench_tool.cc

@@ -1098,6 +1098,25 @@ DEFINE_bool(
     ROCKSDB_NAMESPACE::AdvancedColumnFamilyOptions().enable_blob_files,
     "[Integrated BlobDB] Enable writing large values to separate blob files.");
 
+DEFINE_bool(
+    enable_blob_direct_write,
+    ROCKSDB_NAMESPACE::AdvancedColumnFamilyOptions().enable_blob_direct_write,
+    "[Integrated BlobDB] Enable direct-write blob file creation on "
+    "the write path.");
+
+DEFINE_uint64(blob_direct_write_partitions,
+              ROCKSDB_NAMESPACE::AdvancedColumnFamilyOptions()
+                  .blob_direct_write_partitions,
+              "[Integrated BlobDB] Number of partitions for direct-write blob "
+              "files.");
+
+static void RegisterDbBenchBdwFlagValidators() {
+  static const bool blob_direct_write_partitions_validator_registered =
+      RegisterFlagValidator(&FLAGS_blob_direct_write_partitions,
+                            &ValidateUint32Range);
+  (void)blob_direct_write_partitions_validator_registered;
+}
+
 DEFINE_uint64(min_blob_size,
               ROCKSDB_NAMESPACE::AdvancedColumnFamilyOptions().min_blob_size,
               "[Integrated BlobDB] The size of the smallest value to be stored "
@@ -4996,6 +5015,9 @@ class Benchmark {
 
     // Integrated BlobDB
     options.enable_blob_files = FLAGS_enable_blob_files;
+    options.enable_blob_direct_write = FLAGS_enable_blob_direct_write;
+    options.blob_direct_write_partitions =
+        static_cast<uint32_t>(FLAGS_blob_direct_write_partitions);
     options.min_blob_size = FLAGS_min_blob_size;
     options.blob_file_size = FLAGS_blob_file_size;
     options.blob_compression_type =
@@ -9229,6 +9251,7 @@ int db_bench_tool(int argc, char** argv, ToolHooks& hooks) {
     SetVersionString(GetRocksVersionAsString(true));
     initialized = true;
   }
+  RegisterDbBenchBdwFlagValidators();
   ParseCommandLineFlags(&argc, &argv, true);
   FLAGS_compaction_style_e =
       (ROCKSDB_NAMESPACE::CompactionStyle)FLAGS_compaction_style;

tools/db_crashtest.py

@@ -166,6 +166,7 @@ default_params = {
     "expected_values_dir": lambda: setup_expected_values_dir(),
     "flush_one_in": lambda: random.choice([1000, 1000000]),
     "manual_wal_flush_one_in": lambda: random.choice([0, 1000]),
+    "sync_wal_one_in": 0,
     "file_checksum_impl": lambda: random.choice(["none", "crc32c", "xxh64", "big"]),
     "get_live_files_apis_one_in": lambda: random.choice([10000, 1000000]),
     "checkpoint_atomic_flush": lambda: random.choice([0, 1]),
@@ -200,6 +201,7 @@ default_params = {
     "optimize_filters_for_memory": lambda: random.randint(0, 1),
     "partition_filters": lambda: random.randint(0, 1),
     "partition_pinning": lambda: random.randint(0, 3),
+    "rate_limit_auto_wal_flush": 0,
     "reset_stats_one_in": lambda: random.choice([10000, 1000000]),
     "pause_background_one_in": lambda: random.choice([10000, 1000000]),
     "disable_file_deletions_one_in": lambda: random.choice([10000, 1000000]),
@@ -727,6 +729,29 @@ blob_params = {
     "remote_compaction_worker_threads": 0,
 }
 
+blob_direct_write_params = {
+    "enable_blob_files": 1,
+    "enable_blob_direct_write": 1,
+    "blob_direct_write_partitions": lambda: random.choice([1, 2, 4, 8]),
+    "allow_setting_blob_options_dynamically": 0,
+    # Keep the fixed-across-runs write mode within the reduced WAL-disabled
+    # direct-write profile.
+    "inplace_update_support": 0,
+    "min_blob_size": lambda: random.choice([8, 16, 64]),
+    "blob_file_size": lambda: random.choice([1048576, 16777216, 268435456]),
+    "blob_compression_type": lambda: random.choice(["none", "snappy", "lz4", "zstd"]),
+    "enable_blob_garbage_collection": 0,
+    "blob_garbage_collection_age_cutoff": 0.0,
+    "blob_garbage_collection_force_threshold": 1.0,
+    "blob_compaction_readahead_size": 0,
+    "blob_file_starting_level": 0,
+    "use_blob_cache": lambda: random.randint(0, 1),
+    "use_shared_block_and_blob_cache": lambda: random.randint(0, 1),
+    "blob_cache_size": lambda: random.choice([1048576, 2097152, 4194304, 8388608]),
+    "prepopulate_blob_cache": lambda: random.randint(0, 1),
+    "remote_compaction_worker_threads": 0,
+}
+
 ts_params = {
     "test_cf_consistency": 0,
     "test_batches_snapshots": 0,
@@ -869,6 +894,101 @@ def finalize_and_sanitize(src_params):
         else:
             dest_params["mock_direct_io"] = True
 
+    if dest_params.get("enable_blob_direct_write", 0) == 1:
+        # Keep blob direct write in its reduced-scope v1 profile.
+        #
+        # Supported recovery shape:
+        #   * clean shutdown / flush
+        #   * crash restart that only relies on SST + manifest-visible blob
+        #     state
+        #
+        # Unsupported here:
+        #   * WAL replay / best-efforts recovery
+        #   * broad dynamic blob option changes and blob GC
+        #   * merge / transaction variants and parallel write queue modes
+        #   * secondary / backup / checkpoint / ingest style APIs that reason
+        #     about active files directly
+        dest_params["enable_blob_files"] = 1
+        dest_params["blob_direct_write_partitions"] = max(
+            1, dest_params.get("blob_direct_write_partitions", 1)
+        )
+        # BDW can still run with multiple application threads as long as writes
+        # stay on the ordered write path. Disable the write modes that would
+        # let memtable/WAL publication diverge from the transformed blob-file
+        # write ordering.
+        dest_params["allow_concurrent_memtable_write"] = 0
+        dest_params["enable_pipelined_write"] = 0
+        dest_params["two_write_queues"] = 0
+        dest_params["unordered_write"] = 0
+        # Keep inplace updates off. The later inplace_update_support
+        # sanitization intentionally forces disable_wal=0 for its own recovery
+        # assumptions, which would silently undo the BDW crash-test profile.
+        dest_params["inplace_update_support"] = 0
+        # Direct write is implemented only for integrated BlobDB, without
+        # dynamic blob option changes or background blob GC.
+        dest_params["use_blob_db"] = 0
+        dest_params["allow_setting_blob_options_dynamically"] = 0
+        dest_params["enable_blob_garbage_collection"] = 0
+        dest_params["blob_garbage_collection_age_cutoff"] = 0.0
+        dest_params["blob_garbage_collection_force_threshold"] = 1.0
+        dest_params["blob_compaction_readahead_size"] = 0
+        dest_params["blob_file_starting_level"] = 0
+        dest_params["use_merge"] = 0
+        dest_params["use_full_merge_v1"] = 0
+        dest_params["use_put_entity_one_in"] = 0
+        dest_params["use_get_entity"] = 0
+        dest_params["use_multi_get_entity"] = 0
+        dest_params["use_timed_put_one_in"] = 0
+        dest_params["use_attribute_group"] = 0
+        # Direct write v1 only supports the plain comparator / key encoding
+        # path. User-defined timestamps and the TransactionDB-only timestamped
+        # snapshot API are outside this feature envelope.
+        dest_params["user_timestamp_size"] = 0
+        dest_params["persist_user_defined_timestamps"] = 0
+        dest_params["create_timestamped_snapshot_one_in"] = 0
+        dest_params["use_txn"] = 0
+        dest_params["txn_write_policy"] = 0
+        dest_params["use_optimistic_txn"] = 0
+        dest_params["test_multi_ops_txns"] = 0
+        dest_params["commit_bypass_memtable_one_in"] = 0
+        # Force the WAL-disabled crash-test profile. The generic disable_wal
+        # sanitization below still applies, but keep the WAL-dependent stress
+        # features explicitly off here so later sanitizers or explicit command
+        # line overrides do not silently re-enable them.
+        dest_params["disable_wal"] = 1
+        dest_params["best_efforts_recovery"] = 0
+        # Direct write v1 crash testing does not cover reopen with unlogged
+        # data, manual/sync WAL persistence, or WAL metadata/locking APIs.
+        dest_params["reopen"] = 0
+        dest_params["manual_wal_flush_one_in"] = 0
+        dest_params["sync_wal_one_in"] = 0
+        dest_params["lock_wal_one_in"] = 0
+        dest_params["get_sorted_wal_files_one_in"] = 0
+        dest_params["get_current_wal_file_one_in"] = 0
+        dest_params["track_and_verify_wals"] = 0
+        dest_params["rate_limit_auto_wal_flush"] = 0
+        dest_params["recycle_log_file_num"] = 0
+        # Write/open fault injection currently assumes WAL-based recovery or
+        # error-retry behavior that direct write v1 does not provide.
+        dest_params["sync_fault_injection"] = 0
+        dest_params["write_fault_one_in"] = 0
+        dest_params["metadata_write_fault_one_in"] = 0
+        dest_params["read_fault_one_in"] = 0
+        dest_params["metadata_read_fault_one_in"] = 0
+        dest_params["open_metadata_write_fault_one_in"] = 0
+        dest_params["open_metadata_read_fault_one_in"] = 0
+        dest_params["open_write_fault_one_in"] = 0
+        dest_params["open_read_fault_one_in"] = 0
+        # Remote compaction, secondary readers, file snapshot style APIs, and
+        # ingest APIs are outside the initial direct-write feature envelope.
+        dest_params["remote_compaction_worker_threads"] = 0
+        dest_params["test_secondary"] = 0
+        dest_params["backup_one_in"] = 0
+        dest_params["checkpoint_one_in"] = 0
+        dest_params["get_live_files_apis_one_in"] = 0
+        dest_params["ingest_external_file_one_in"] = 0
+        dest_params["ingest_wbwi_one_in"] = 0
+
     if dest_params.get("memtablerep") == "vector":
         dest_params["inplace_update_support"] = 0
 
@@ -1000,6 +1120,9 @@ def finalize_and_sanitize(src_params):
         else:
             dest_params["unordered_write"] = 0
     if dest_params.get("disable_wal", 0) == 1:
+        # WAL-disabled stress runs do not support in-process reopen. Blob
+        # direct write v1 relies on this path so crash testing stays within its
+        # SST/blob-manifest recovery envelope rather than WAL replay.
         dest_params["atomic_flush"] = 1
         dest_params["sync"] = 0
         dest_params["write_fault_one_in"] = 0
@@ -1364,16 +1487,17 @@ def gen_cmd_params(args):
     if args.test_tiered_storage:
         params.update(tiered_params)
 
-    # Best-effort recovery, tiered storage are currently incompatible with BlobDB.
-    # Test BE recovery if specified on the command line; otherwise, apply BlobDB
-    # related overrides with a 10% chance.
+    # Best-effort recovery, tiered storage are currently incompatible with
+    # BlobDB and blob direct write. Test BE recovery if specified on the
+    # command line; otherwise, apply one of the blob feature overrides with a
+    # 10% chance.
     if (
         not args.test_best_efforts_recovery
         and not args.test_tiered_storage
         and params.get("test_secondary", 0) == 0
         and random.choice([0] * 9 + [1]) == 1
     ):
-        params.update(blob_params)
+        params.update(random.choice([blob_params, blob_direct_write_params]))
 
     if "compaction_style" not in params:
         # Default to leveled compaction
@@ -1810,6 +1934,7 @@ def main():
         + list(blackbox_simple_default_params.items())
         + list(whitebox_simple_default_params.items())
         + list(blob_params.items())
+        + list(blob_direct_write_params.items())
         + list(ts_params.items())
         + list(multiops_txn_params.items())
         + list(best_efforts_recovery_params.items())

util/atomic.h

@@ -6,9 +6,27 @@
 #pragma once
 
 #include <atomic>
+#include <memory>
 
 #include "rocksdb/rocksdb_namespace.h"
 
+#if defined(__clang__)
+#define ROCKSDB_SUPPRESS_SHARED_PTR_ATOMIC_DEPRECATION_PUSH \
+  _Pragma("clang diagnostic push")                          \
+      _Pragma("clang diagnostic ignored \"-Wdeprecated-declarations\"")
+#define ROCKSDB_SUPPRESS_SHARED_PTR_ATOMIC_DEPRECATION_POP \
+  _Pragma("clang diagnostic pop")
+#elif defined(__GNUC__)
+#define ROCKSDB_SUPPRESS_SHARED_PTR_ATOMIC_DEPRECATION_PUSH \
+  _Pragma("GCC diagnostic push")                            \
+      _Pragma("GCC diagnostic ignored \"-Wdeprecated-declarations\"")
+#define ROCKSDB_SUPPRESS_SHARED_PTR_ATOMIC_DEPRECATION_POP \
+  _Pragma("GCC diagnostic pop")
+#else
+#define ROCKSDB_SUPPRESS_SHARED_PTR_ATOMIC_DEPRECATION_PUSH
+#define ROCKSDB_SUPPRESS_SHARED_PTR_ATOMIC_DEPRECATION_POP
+#endif
+
 namespace ROCKSDB_NAMESPACE {
 
 // Background:
@@ -121,4 +139,32 @@ class Atomic : public RelaxedAtomic<T> {
   }
 };
 
+// Atomic shared_ptr helper wrappers.
+//
+// The C++20 replacement is std::atomic<std::shared_ptr<T>>, but the fbcode
+// make build still uses a libstdc++ 11.x toolchain where that specialization
+// is unavailable while the older free-function API is already deprecated.
+// Centralize the workaround here so callers can keep a single code path until
+// the build toolchain can support std::atomic<std::shared_ptr<T>> directly.
+template <typename T>
+inline std::shared_ptr<T> AtomicSharedPtrLoad(const std::shared_ptr<T>* ptr,
+                                              std::memory_order order) {
+  ROCKSDB_SUPPRESS_SHARED_PTR_ATOMIC_DEPRECATION_PUSH
+  auto result = std::atomic_load_explicit(ptr, order);
+  ROCKSDB_SUPPRESS_SHARED_PTR_ATOMIC_DEPRECATION_POP
+  return result;
+}
+
+template <typename T>
+inline void AtomicSharedPtrStore(std::shared_ptr<T>* ptr,
+                                 std::shared_ptr<T> desired,
+                                 std::memory_order order) {
+  ROCKSDB_SUPPRESS_SHARED_PTR_ATOMIC_DEPRECATION_PUSH
+  std::atomic_store_explicit(ptr, std::move(desired), order);
+  ROCKSDB_SUPPRESS_SHARED_PTR_ATOMIC_DEPRECATION_POP
+}
+
 }  // namespace ROCKSDB_NAMESPACE
+
+#undef ROCKSDB_SUPPRESS_SHARED_PTR_ATOMIC_DEPRECATION_PUSH
+#undef ROCKSDB_SUPPRESS_SHARED_PTR_ATOMIC_DEPRECATION_POP

utilities/checkpoint/checkpoint_test.cc

@@ -905,6 +905,29 @@ TEST_F(CheckpointTest, CheckpointWithLockWAL) {
   ASSERT_EQ("foo_value", get_result);
 }
 
+TEST_F(CheckpointTest, CheckpointWithLockWALRejectsBlobDirectWrite) {
+  Options options = CurrentOptions();
+  options.enable_blob_files = true;
+  options.enable_blob_direct_write = true;
+  options.allow_concurrent_memtable_write = false;
+  options.min_blob_size = 32;
+  Reopen(options);
+
+  ASSERT_OK(Put("foo", std::string(128, 'b')));
+  ASSERT_OK(db_->LockWAL());
+
+  Checkpoint* checkpoint = nullptr;
+  ASSERT_OK(Checkpoint::Create(db_.get(), &checkpoint));
+  std::unique_ptr<Checkpoint> checkpoint_holder(checkpoint);
+
+  Status s = checkpoint->CreateCheckpoint(snapshot_name_);
+  ASSERT_TRUE(s.IsNotSupported()) << s.ToString();
+  ASSERT_NE(s.ToString().find("Blob direct write"), std::string::npos)
+      << s.ToString();
+
+  ASSERT_OK(db_->UnlockWAL());
+}
+
 TEST_F(CheckpointTest, CheckpointReadOnlyDBWithMultipleColumnFamilies) {
   Options options = CurrentOptions();
   CreateAndReopenWithCF({"pikachu", "eevee"}, options);
@@ -1422,6 +1445,37 @@ TEST_F(CheckpointTest, BackupWithAtomicFlushSkipsWAL) {
   test::DeleteDir(env_, backup_dir);
 }
 
+TEST_F(CheckpointTest, BackupWithoutFlushRejectsBlobDirectWrite) {
+  Options options = CurrentOptions();
+  options.enable_blob_files = true;
+  options.enable_blob_direct_write = true;
+  options.allow_concurrent_memtable_write = false;
+  options.min_blob_size = 32;
+  Reopen(options);
+
+  ASSERT_OK(Put("foo", std::string(128, 'x')));
+
+  std::string backup_dir =
+      test::PerThreadDBPath(env_, "backup_blob_direct_write_requires_flush");
+  test::DeleteDir(env_, backup_dir);
+
+  BackupEngineOptions backup_options(backup_dir);
+  backup_options.destroy_old_data = true;
+
+  BackupEngine* backup_engine_ptr = nullptr;
+  ASSERT_OK(BackupEngine::Open(env_, backup_options, &backup_engine_ptr));
+  std::unique_ptr<BackupEngine> backup_engine(backup_engine_ptr);
+
+  CreateBackupOptions create_options;
+  create_options.flush_before_backup = false;
+  Status s = backup_engine->CreateNewBackup(create_options, db_.get());
+  ASSERT_TRUE(s.IsNotSupported()) << s.ToString();
+  ASSERT_NE(s.ToString().find("Blob direct write"), std::string::npos)
+      << s.ToString();
+
+  test::DeleteDir(env_, backup_dir);
+}
+
 }  // namespace ROCKSDB_NAMESPACE
 
 int main(int argc, char** argv) {

utilities/transactions/pessimistic_transaction.cc

@@ -928,6 +928,13 @@ Status WriteCommittedTxn::CommitInternal() {
       }
     }
   }
+  // Blob direct write must go through the normal WriteBatch path in
+  // DBImpl::WriteImpl() so large values can be transformed into BlobIndex
+  // entries. The bypass-memtable optimization commits through WBWI ingestion,
+  // which skips that transformation and does not support kTypeBlobIndex.
+  if (bypass_memtable && db_impl_->HasAnyBlobDirectWriteColumnFamily()) {
+    bypass_memtable = false;
+  }
   if (!bypass_memtable) {
     // insert prepared batch into Memtable only skipping WAL.
     // Memtable will ignore BeginPrepare/EndPrepare markers

utilities/transactions/transaction_test.cc

@@ -274,6 +274,50 @@ TEST_P(TransactionTest, SuccessTest) {
   delete txn;
 }
 
+TEST_P(TransactionTest, DirectWriteCommitPath) {
+  if (options.two_write_queues || options.unordered_write) {
+    ROCKSDB_GTEST_BYPASS(
+        "Blob direct write v1 only supports the ordered single-write-queue "
+        "path");
+    return;
+  }
+
+  options.enable_blob_files = true;
+  options.enable_blob_direct_write = true;
+  options.allow_concurrent_memtable_write = false;
+  options.blob_direct_write_partitions = 2;
+  options.min_blob_size = 32;
+  options.use_direct_reads = true;
+  options.use_direct_io_for_flush_and_compaction = true;
+
+  Status s = ReOpen();
+  if (s.IsInvalidArgument()) {
+    ROCKSDB_GTEST_SKIP("This test requires direct IO support");
+    return;
+  }
+  ASSERT_OK(s);
+
+  WriteOptions write_options;
+  const std::string key = "txn_blob_key";
+  const std::string value(512, 'x');
+
+  std::unique_ptr<Transaction> txn(
+      db->BeginTransaction(write_options, TransactionOptions()));
+  ASSERT_TRUE(txn);
+  ASSERT_OK(txn->Put(key, value));
+  ASSERT_OK(txn->Commit());
+  txn.reset();
+
+  PinnableSlice result;
+  ASSERT_OK(db->Get(ReadOptions(), db->DefaultColumnFamily(), key, &result));
+  ASSERT_EQ(result, value);
+
+  ASSERT_OK(db->Flush(FlushOptions()));
+  result.Reset();
+  ASSERT_OK(db->Get(ReadOptions(), db->DefaultColumnFamily(), key, &result));
+  ASSERT_EQ(result, value);
+}
+
 // Test the basic API of the pinnable slice overload of GetForUpdate()
 TEST_P(TransactionTest, SuccessTestPinnable) {
   ASSERT_OK(db->ResetStats());

utilities/transactions/transaction_test.h

@@ -166,7 +166,7 @@ class TransactionTestBase : public ::testing::Test {
     } else {
       s = OpenWithStackableDB();
     }
-    assert(db != nullptr);
+    assert(!s.ok() || db != nullptr);
     return s;
   }