Update history and version for 10.2.3

Summary:
Remove the dependency on `allow_db_generated_files` option in `IngestExternalFile` to be set for ingesting external tables. The files are created by SstFileWriter, and we should be able to ingest them. We could make it work by having the external table implementation provide the version and global sequence number related properties, but its safer to have RocksDB generate the table properties block and store it as is in the file.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/13608

Test Plan: Add unit test to test basic ingestion and ingestion with atomic_replace_range

Reviewed By: pdillinger

Differential Revision: D74830707

Pulled By: anand1976

fbshipit-source-id: 4a9bea4a4f38f7c24c584262095c5c98cd771ddc

HISTORY.md

@@ -1,6 +1,40 @@
 # Rocksdb Change Log
 > NOTE: Entries for next release do not go here. Follow instructions in `unreleased_history/README.txt`
 
+## 10.2.3 (05/19/2025)
+* Fix a failure in IngestExternalFiles() when ingesting an external table with the allow_db_generated_files option set to false.
+
+## 10.2.2 (05/08/2025)
+### Bug Fixes
+* Pass wrapped WritableFileWriter pointer to ExternalTableBuilder so that the file checksum can be correctly calculated and returned by SstFileWriter for external table files.
+
+## 10.2.1 (04/24/2025)
+### Bug Fixes
+* Fix improper initialization of ExternalTableOptions
+
+## 10.2.0 (04/21/2025)
+### New Features
+* Provide histogram stats `COMPACTION_PREFETCH_BYTES` to measure number of bytes for RocksDB's prefetching (as opposed to file
+system's prefetch) on SST file during compaction read
+* A new API DB::GetNewestUserDefinedTimestamp is added to return the newest user defined timestamp seen in a column family
+* Introduce API `IngestWriteBatchWithIndex()` for ingesting updates into DB while bypassing memtable writes. This improves performance when writing a large write batch to the DB.
+* Add a new CF option `memtable_op_scan_flush_trigger` that triggers a flush of the memtable if an iterator's Seek()/Next() scans over a certain number of invisible entries from the memtable.
+
+### Public API Changes
+* AdvancedColumnFamilyOptions.max_write_buffer_number_to_maintain is deleted. It's deprecated since introduction of a better option max_write_buffer_size_to_maintain since RocksDB 6.5.0.
+* Deprecated API `DB::MaxMemCompactionLevel()`.
+* Deprecated `ReadOptions::ignore_range_deletions`.
+* Deprecated API `experimental::PromoteL0()`.
+* Added arbitrary string map for additional options to be overriden for remote compactions
+* The fail_if_options_file_error option in DBOptions has been removed. The behavior now is to always return failure in any API that fails to persist the OPTIONS file.
+
+### Behavior Changes
+* Make stats `PREFETCH_BYTES_USEFUL`, `PREFETCH_HITS`, `PREFETCH_BYTES` only account for prefetching during user initiated scan
+
+### Bug Fixes
+* Fix a bug in Posix file system that the FSWritableFile created via `FileSystem::ReopenWritableFile` internally does not track the correct file size.
+* Fix a bug where tail size of remote compaction output is not persisted in primary db's manifest
+
 ## 10.1.0 (03/24/2025)
 ### New Features
 * Added a new `DBOptions.calculate_sst_write_lifetime_hint_set` setting that allows to customize which compaction styles SST write lifetime hint calculation is allowed on. Today RocksDB supports only two modes `kCompactionStyleLevel` and `kCompactionStyleUniversal`.

include/rocksdb/external_table.h

@@ -125,6 +125,17 @@ class ExternalTableReader {
                         std::vector<std::string>* values,
                         std::vector<Status>* statuses) = 0;
 
+  // Allocate and return the contents of the properties block. If the builder
+  // supports PutPropertiesBlock(), then this must be supported. The
+  // properties block should be written to the table file as is (no
+  // compression or mutation of any kind), and its offset in the file
+  // should be returned in file_offset.
+  virtual Status GetPropertiesBlock(std::unique_ptr<char[]>* /*property_block*/,
+                                    uint64_t* /*size*/,
+                                    uint64_t* /*file_offset*/) {
+    return Status::NotSupported();
+  }
+
   // Return TableProperties for the file. At a minimum, the following
   // properties need to be returned -
   // comparator_name
@@ -179,6 +190,11 @@ class ExternalTableBuilder {
   // Finish().
   virtual uint64_t FileSize() const = 0;
 
+  // Write the raw properties block as is in the table file
+  virtual Status PutPropertiesBlock(const Slice& /*property_block*/) {
+    return Status::NotSupported();
+  }
+
   //  As mentioned in earlier comments, the following table properties must be
   //  returned at a minimum -
   //  comparator_name

include/rocksdb/version.h

@@ -13,7 +13,7 @@
 // minor or major version number planned for release.
 #define ROCKSDB_MAJOR 10
 #define ROCKSDB_MINOR 2
-#define ROCKSDB_PATCH 0
+#define ROCKSDB_PATCH 3
 
 // Do not use these. We made the mistake of declaring macros starting with
 // double underscore. Now we have to live with our choice. We'll deprecate these

table/external_table.cc

@@ -5,8 +5,11 @@
 
 #include "rocksdb/external_table.h"
 
+#include "logging/logging.h"
 #include "rocksdb/table.h"
+#include "table/block_based/block.h"
 #include "table/internal_iterator.h"
+#include "table/meta_blocks.h"
 #include "table/table_builder.h"
 #include "table/table_reader.h"
 
@@ -156,8 +159,9 @@ class ExternalTableIteratorAdapter : public InternalIterator {
 class ExternalTableReaderAdapter : public TableReader {
  public:
   explicit ExternalTableReaderAdapter(
+      const ImmutableOptions& ioptions,
       std::unique_ptr<ExternalTableReader>&& reader)
-      : reader_(std::move(reader)) {}
+      : ioptions_(ioptions), reader_(std::move(reader)) {}
 
   ~ExternalTableReaderAdapter() override {}
 
@@ -193,9 +197,34 @@ class ExternalTableReaderAdapter : public TableReader {
   void SetupForCompaction() override {}
 
   std::shared_ptr<const TableProperties> GetTableProperties() const override {
-    std::shared_ptr<TableProperties> props =
-        std::make_shared<TableProperties>(*reader_->GetTableProperties());
-    props->key_largest_seqno = 0;
+    std::shared_ptr<TableProperties> props;
+    std::unique_ptr<char[]> property_block;
+    uint64_t property_block_size = 0;
+    uint64_t property_block_offset = 0;
+    Status s;
+    // Get the raw properties block from the external table reader. We don't
+    // support writing the global sequence number, but we still get and return
+    // the correct global seqno offset in the file to prevent accidental
+    // corruption.
+    s = reader_->GetPropertiesBlock(&property_block, &property_block_size,
+                                    &property_block_offset);
+    if (s.ok()) {
+      std::unique_ptr<TableProperties> table_properties =
+          std::make_unique<TableProperties>();
+      BlockContents block_contents(std::move(property_block),
+                                   property_block_size);
+      Block block(std::move(block_contents));
+      s = ParsePropertiesBlock(ioptions_, property_block_offset, block,
+                               table_properties);
+      if (s.ok()) {
+        props.reset(table_properties.release());
+      }
+    } else {
+      // Fallback to getting a minimal table properties structure from the
+      // external table reader
+      props = std::make_shared<TableProperties>(*reader_->GetTableProperties());
+      props->key_largest_seqno = 0;
+    }
     return props;
   }
 
@@ -213,14 +242,54 @@ class ExternalTableReaderAdapter : public TableReader {
   }
 
  private:
+  const ImmutableOptions& ioptions_;
   std::unique_ptr<ExternalTableReader> reader_;
 };
 
 class ExternalTableBuilderAdapter : public TableBuilder {
  public:
   explicit ExternalTableBuilderAdapter(
-      std::unique_ptr<ExternalTableBuilder>&& builder)
-      : builder_(std::move(builder)), num_entries_(0) {}
+      const TableBuilderOptions& topts,
+      std::unique_ptr<ExternalTableBuilder>&& builder,
+      std::unique_ptr<FSWritableFile>&& file)
+      : builder_(std::move(builder)),
+        file_(std::move(file)),
+        ioptions_(topts.ioptions) {
+    properties_.num_data_blocks = 1;
+    properties_.index_size = 0;
+    properties_.filter_size = 0;
+    properties_.format_version = 0;
+    properties_.key_largest_seqno = 0;
+    properties_.column_family_id = topts.column_family_id;
+    properties_.column_family_name = topts.column_family_name;
+    properties_.db_id = topts.db_id;
+    properties_.db_session_id = topts.db_session_id;
+    properties_.db_host_id = topts.ioptions.db_host_id;
+    if (!ReifyDbHostIdProperty(topts.ioptions.env, &properties_.db_host_id)
+             .ok()) {
+      ROCKS_LOG_INFO(topts.ioptions.logger,
+                     "db_host_id property will not be set");
+    }
+    properties_.orig_file_number = topts.cur_file_num;
+    properties_.comparator_name = topts.ioptions.user_comparator != nullptr
+                                      ? topts.ioptions.user_comparator->Name()
+                                      : "nullptr";
+    properties_.prefix_extractor_name =
+        topts.moptions.prefix_extractor != nullptr
+            ? topts.moptions.prefix_extractor->AsString()
+            : "nullptr";
+
+    for (auto& factory : *topts.internal_tbl_prop_coll_factories) {
+      assert(factory);
+      std::unique_ptr<InternalTblPropColl> collector{
+          factory->CreateInternalTblPropColl(topts.column_family_id,
+                                             topts.level_at_creation,
+                                             topts.ioptions.num_levels)};
+      if (collector) {
+        table_properties_collectors_.emplace_back(std::move(collector));
+      }
+    }
+  }
 
   void Add(const Slice& key, const Slice& value) override {
     ParsedInternalKey pkey;
@@ -231,7 +300,12 @@ class ExternalTableBuilderAdapter : public TableBuilder {
             "Value type " + std::to_string(pkey.type) + "not supported");
       } else {
         builder_->Add(pkey.user_key, value);
-        num_entries_++;
+        properties_.num_entries++;
+        properties_.raw_key_size += key.size();
+        properties_.raw_value_size += value.size();
+        NotifyCollectTableCollectorsOnAdd(key, value, /*offset=*/0,
+                                          table_properties_collectors_,
+                                          ioptions_.logger);
       }
     }
   }
@@ -246,13 +320,37 @@ class ExternalTableBuilderAdapter : public TableBuilder {
 
   IOStatus io_status() const override { return status_to_io_status(status()); }
 
-  Status Finish() override { return builder_->Finish(); }
+  Status Finish() override {
+    // Approximate the data size
+    properties_.data_size =
+        properties_.raw_key_size + properties_.raw_value_size;
+
+    PropertyBlockBuilder property_block_builder;
+    property_block_builder.AddTableProperty(properties_);
+    UserCollectedProperties more_user_collected_properties;
+    NotifyCollectTableCollectorsOnFinish(
+        table_properties_collectors_, ioptions_.logger, &property_block_builder,
+        more_user_collected_properties, properties_.readable_properties);
+    properties_.user_collected_properties.insert(
+        more_user_collected_properties.begin(),
+        more_user_collected_properties.end());
+
+    Slice prop_block = property_block_builder.Finish();
+    Status s = builder_->PutPropertiesBlock(prop_block);
+    if (s.ok() || s.IsNotSupported()) {
+      // If the builder doesn't support writing the properties block,
+      // we still call Finish() and let the external builder handle it.
+      s = builder_->Finish();
+    }
+
+    return s;
+  }
 
   void Abandon() override { builder_->Abandon(); }
 
   uint64_t FileSize() const override { return builder_->FileSize(); }
 
-  uint64_t NumEntries() const override { return num_entries_; }
+  uint64_t NumEntries() const override { return properties_.num_entries; }
 
   TableProperties GetTableProperties() const override {
     return builder_->GetTableProperties();
@@ -269,7 +367,11 @@ class ExternalTableBuilderAdapter : public TableBuilder {
  private:
   Status status_;
   std::unique_ptr<ExternalTableBuilder> builder_;
-  uint64_t num_entries_;
+  std::unique_ptr<FSWritableFile> file_;
+  const ImmutableOptions& ioptions_;
+  TableProperties properties_;
+  std::vector<std::unique_ptr<InternalTblPropColl>>
+      table_properties_collectors_;
 };
 
 class ExternalTableFactoryAdapter : public TableFactory {
@@ -286,16 +388,24 @@ class ExternalTableFactoryAdapter : public TableFactory {
       std::unique_ptr<RandomAccessFileReader>&& file, uint64_t /* file_size */,
       std::unique_ptr<TableReader>* table_reader,
       bool /* prefetch_index_and_filter_in_cache */) const override {
+    // SstFileReader specifies largest_seqno as kMaxSequenceNumber to denote
+    // that its unknown
+    if (topts.largest_seqno > 0 && topts.largest_seqno != kMaxSequenceNumber) {
+      return Status::NotSupported(
+          "Ingesting file with sequence number larger than 0");
+    }
     std::unique_ptr<ExternalTableReader> reader;
-    ExternalTableOptions ext_topts(
-        topts.prefix_extractor, topts.ioptions.user_comparator,
-        topts.ioptions.fs, FileOptions(topts.env_options));
+    FileOptions fopts(topts.env_options);
+    ExternalTableOptions ext_topts(topts.prefix_extractor,
+                                   topts.ioptions.user_comparator,
+                                   topts.ioptions.fs, fopts);
     auto status =
         inner_->NewTableReader(ro, file->file_name(), ext_topts, &reader);
     if (!status.ok()) {
       return status;
     }
-    table_reader->reset(new ExternalTableReaderAdapter(std::move(reader)));
+    table_reader->reset(
+        new ExternalTableReaderAdapter(topts.ioptions, std::move(reader)));
     file.reset();
     return Status::OK();
   }
@@ -308,10 +418,13 @@ class ExternalTableFactoryAdapter : public TableFactory {
         topts.read_options, topts.write_options,
         topts.moptions.prefix_extractor, topts.ioptions.user_comparator,
         topts.column_family_name, topts.reason);
+    auto file_wrapper =
+        std::make_unique<ExternalTableWritableFileWrapper>(file);
     builder.reset(inner_->NewTableBuilder(ext_topts, file->file_name(),
-                                          file->writable_file()));
+                                          file_wrapper.get()));
     if (builder) {
-      return new ExternalTableBuilderAdapter(std::move(builder));
+      return new ExternalTableBuilderAdapter(topts, std::move(builder),
+                                             std::move(file_wrapper));
     }
     return nullptr;
   }
@@ -319,6 +432,42 @@ class ExternalTableFactoryAdapter : public TableFactory {
   std::unique_ptr<TableFactory> Clone() const override { return nullptr; }
 
  private:
+  // An FSWritableFile subclass for wrapping a WritableFileWriter. The
+  // latter is private to RocksDB, so we wrap it here in order to pass it
+  // to the ExternalTableBuilder. This is necessary for WritableFileWriter
+  // to intercept Append so that it can calculate the file checksum.
+  class ExternalTableWritableFileWrapper : public FSWritableFile {
+   public:
+    explicit ExternalTableWritableFileWrapper(WritableFileWriter* writer)
+        : writer_(writer) {}
+
+    using FSWritableFile::Append;
+    IOStatus Append(const Slice& data, const IOOptions& options,
+                    IODebugContext* /*dbg*/) override {
+      return writer_->Append(options, data);
+    }
+
+    IOStatus Close(const IOOptions& options, IODebugContext* /*dbg*/) override {
+      return writer_->Close(options);
+    }
+
+    IOStatus Flush(const IOOptions& options, IODebugContext* /*dbg*/) override {
+      return writer_->Flush(options);
+    }
+
+    IOStatus Sync(const IOOptions& options, IODebugContext* /*dbg*/) override {
+      return writer_->Sync(options, /*use_fsync=*/false);
+    }
+
+    uint64_t GetFileSize(const IOOptions& options,
+                         IODebugContext* dbg) override {
+      return writer_->writable_file()->GetFileSize(options, dbg);
+    }
+
+   private:
+    WritableFileWriter* writer_;
+  };
+
   std::shared_ptr<ExternalTableFactory> inner_;
 };
 

table/meta_blocks.cc

@@ -253,6 +253,144 @@ bool NotifyCollectTableCollectorsOnFinish(
   return all_succeeded;
 }
 
+Status ParsePropertiesBlock(
+    const ImmutableOptions& ioptions, uint64_t offset, Block& properties_block,
+    std::unique_ptr<TableProperties>& new_table_properties) {
+  std::unique_ptr<MetaBlockIter> iter(properties_block.NewMetaIterator());
+
+  //  All pre-defined properties of type uint64_t
+  std::unordered_map<std::string, uint64_t*> predefined_uint64_properties = {
+      {TablePropertiesNames::kOriginalFileNumber,
+       &new_table_properties->orig_file_number},
+      {TablePropertiesNames::kDataSize, &new_table_properties->data_size},
+      {TablePropertiesNames::kIndexSize, &new_table_properties->index_size},
+      {TablePropertiesNames::kIndexPartitions,
+       &new_table_properties->index_partitions},
+      {TablePropertiesNames::kTopLevelIndexSize,
+       &new_table_properties->top_level_index_size},
+      {TablePropertiesNames::kIndexKeyIsUserKey,
+       &new_table_properties->index_key_is_user_key},
+      {TablePropertiesNames::kIndexValueIsDeltaEncoded,
+       &new_table_properties->index_value_is_delta_encoded},
+      {TablePropertiesNames::kFilterSize, &new_table_properties->filter_size},
+      {TablePropertiesNames::kRawKeySize, &new_table_properties->raw_key_size},
+      {TablePropertiesNames::kRawValueSize,
+       &new_table_properties->raw_value_size},
+      {TablePropertiesNames::kNumDataBlocks,
+       &new_table_properties->num_data_blocks},
+      {TablePropertiesNames::kNumEntries, &new_table_properties->num_entries},
+      {TablePropertiesNames::kNumFilterEntries,
+       &new_table_properties->num_filter_entries},
+      {TablePropertiesNames::kDeletedKeys,
+       &new_table_properties->num_deletions},
+      {TablePropertiesNames::kMergeOperands,
+       &new_table_properties->num_merge_operands},
+      {TablePropertiesNames::kNumRangeDeletions,
+       &new_table_properties->num_range_deletions},
+      {TablePropertiesNames::kFormatVersion,
+       &new_table_properties->format_version},
+      {TablePropertiesNames::kFixedKeyLen,
+       &new_table_properties->fixed_key_len},
+      {TablePropertiesNames::kColumnFamilyId,
+       &new_table_properties->column_family_id},
+      {TablePropertiesNames::kCreationTime,
+       &new_table_properties->creation_time},
+      {TablePropertiesNames::kOldestKeyTime,
+       &new_table_properties->oldest_key_time},
+      {TablePropertiesNames::kNewestKeyTime,
+       &new_table_properties->newest_key_time},
+      {TablePropertiesNames::kFileCreationTime,
+       &new_table_properties->file_creation_time},
+      {TablePropertiesNames::kSlowCompressionEstimatedDataSize,
+       &new_table_properties->slow_compression_estimated_data_size},
+      {TablePropertiesNames::kFastCompressionEstimatedDataSize,
+       &new_table_properties->fast_compression_estimated_data_size},
+      {TablePropertiesNames::kTailStartOffset,
+       &new_table_properties->tail_start_offset},
+      {TablePropertiesNames::kUserDefinedTimestampsPersisted,
+       &new_table_properties->user_defined_timestamps_persisted},
+      {TablePropertiesNames::kKeyLargestSeqno,
+       &new_table_properties->key_largest_seqno},
+  };
+
+  Status s;
+  std::string last_key;
+  for (iter->SeekToFirst(); iter->Valid(); iter->Next()) {
+    s = iter->status();
+    if (!s.ok()) {
+      break;
+    }
+
+    auto key = iter->key().ToString();
+    // properties block should be strictly sorted with no duplicate key.
+    if (!last_key.empty() &&
+        BytewiseComparator()->Compare(key, last_key) <= 0) {
+      s = Status::Corruption("properties unsorted");
+      break;
+    }
+    last_key = key;
+
+    auto raw_val = iter->value();
+    auto pos = predefined_uint64_properties.find(key);
+
+    if (key == ExternalSstFilePropertyNames::kGlobalSeqno) {
+      new_table_properties->external_sst_file_global_seqno_offset =
+          offset + iter->ValueOffset();
+    }
+
+    if (pos != predefined_uint64_properties.end()) {
+      if (key == TablePropertiesNames::kDeletedKeys ||
+          key == TablePropertiesNames::kMergeOperands) {
+        // Insert in user-collected properties for API backwards compatibility
+        new_table_properties->user_collected_properties.insert(
+            {key, raw_val.ToString()});
+      }
+      // handle predefined rocksdb properties
+      uint64_t val;
+      if (!GetVarint64(&raw_val, &val)) {
+        // skip malformed value
+        auto error_msg =
+            "Detect malformed value in properties meta-block:"
+            "\tkey: " +
+            key + "\tval: " + raw_val.ToString();
+        ROCKS_LOG_ERROR(ioptions.logger, "%s", error_msg.c_str());
+        continue;
+      }
+      *(pos->second) = val;
+    } else if (key == TablePropertiesNames::kDbId) {
+      new_table_properties->db_id = raw_val.ToString();
+    } else if (key == TablePropertiesNames::kDbSessionId) {
+      new_table_properties->db_session_id = raw_val.ToString();
+    } else if (key == TablePropertiesNames::kDbHostId) {
+      new_table_properties->db_host_id = raw_val.ToString();
+    } else if (key == TablePropertiesNames::kFilterPolicy) {
+      new_table_properties->filter_policy_name = raw_val.ToString();
+    } else if (key == TablePropertiesNames::kColumnFamilyName) {
+      new_table_properties->column_family_name = raw_val.ToString();
+    } else if (key == TablePropertiesNames::kComparator) {
+      new_table_properties->comparator_name = raw_val.ToString();
+    } else if (key == TablePropertiesNames::kMergeOperator) {
+      new_table_properties->merge_operator_name = raw_val.ToString();
+    } else if (key == TablePropertiesNames::kPrefixExtractorName) {
+      new_table_properties->prefix_extractor_name = raw_val.ToString();
+    } else if (key == TablePropertiesNames::kPropertyCollectors) {
+      new_table_properties->property_collectors_names = raw_val.ToString();
+    } else if (key == TablePropertiesNames::kCompression) {
+      new_table_properties->compression_name = raw_val.ToString();
+    } else if (key == TablePropertiesNames::kCompressionOptions) {
+      new_table_properties->compression_options = raw_val.ToString();
+    } else if (key == TablePropertiesNames::kSequenceNumberTimeMapping) {
+      new_table_properties->seqno_to_time_mapping = raw_val.ToString();
+    } else {
+      // handle user-collected properties
+      new_table_properties->user_collected_properties.insert(
+          {key, raw_val.ToString()});
+    }
+  }
+
+  return s;
+}
+
 // FIXME: should be a parameter for reading table properties to use persistent
 // cache?
 Status ReadTablePropertiesHelper(
@@ -324,140 +462,9 @@ Status ReadTablePropertiesHelper(
 
     uint64_t block_size = block_contents.data.size();
     Block properties_block(std::move(block_contents));
-    // Unfortunately, Block::size() might not equal block_contents.data.size(),
-    // and Block hides block_contents
-    std::unique_ptr<MetaBlockIter> iter(properties_block.NewMetaIterator());
-
     std::unique_ptr<TableProperties> new_table_properties{new TableProperties};
-    // All pre-defined properties of type uint64_t
-    std::unordered_map<std::string, uint64_t*> predefined_uint64_properties = {
-        {TablePropertiesNames::kOriginalFileNumber,
-         &new_table_properties->orig_file_number},
-        {TablePropertiesNames::kDataSize, &new_table_properties->data_size},
-        {TablePropertiesNames::kIndexSize, &new_table_properties->index_size},
-        {TablePropertiesNames::kIndexPartitions,
-         &new_table_properties->index_partitions},
-        {TablePropertiesNames::kTopLevelIndexSize,
-         &new_table_properties->top_level_index_size},
-        {TablePropertiesNames::kIndexKeyIsUserKey,
-         &new_table_properties->index_key_is_user_key},
-        {TablePropertiesNames::kIndexValueIsDeltaEncoded,
-         &new_table_properties->index_value_is_delta_encoded},
-        {TablePropertiesNames::kFilterSize, &new_table_properties->filter_size},
-        {TablePropertiesNames::kRawKeySize,
-         &new_table_properties->raw_key_size},
-        {TablePropertiesNames::kRawValueSize,
-         &new_table_properties->raw_value_size},
-        {TablePropertiesNames::kNumDataBlocks,
-         &new_table_properties->num_data_blocks},
-        {TablePropertiesNames::kNumEntries, &new_table_properties->num_entries},
-        {TablePropertiesNames::kNumFilterEntries,
-         &new_table_properties->num_filter_entries},
-        {TablePropertiesNames::kDeletedKeys,
-         &new_table_properties->num_deletions},
-        {TablePropertiesNames::kMergeOperands,
-         &new_table_properties->num_merge_operands},
-        {TablePropertiesNames::kNumRangeDeletions,
-         &new_table_properties->num_range_deletions},
-        {TablePropertiesNames::kFormatVersion,
-         &new_table_properties->format_version},
-        {TablePropertiesNames::kFixedKeyLen,
-         &new_table_properties->fixed_key_len},
-        {TablePropertiesNames::kColumnFamilyId,
-         &new_table_properties->column_family_id},
-        {TablePropertiesNames::kCreationTime,
-         &new_table_properties->creation_time},
-        {TablePropertiesNames::kOldestKeyTime,
-         &new_table_properties->oldest_key_time},
-        {TablePropertiesNames::kNewestKeyTime,
-         &new_table_properties->newest_key_time},
-        {TablePropertiesNames::kFileCreationTime,
-         &new_table_properties->file_creation_time},
-        {TablePropertiesNames::kSlowCompressionEstimatedDataSize,
-         &new_table_properties->slow_compression_estimated_data_size},
-        {TablePropertiesNames::kFastCompressionEstimatedDataSize,
-         &new_table_properties->fast_compression_estimated_data_size},
-        {TablePropertiesNames::kTailStartOffset,
-         &new_table_properties->tail_start_offset},
-        {TablePropertiesNames::kUserDefinedTimestampsPersisted,
-         &new_table_properties->user_defined_timestamps_persisted},
-        {TablePropertiesNames::kKeyLargestSeqno,
-         &new_table_properties->key_largest_seqno},
-    };
-
-    std::string last_key;
-    for (iter->SeekToFirst(); iter->Valid(); iter->Next()) {
-      s = iter->status();
-      if (!s.ok()) {
-        break;
-      }
-
-      auto key = iter->key().ToString();
-      // properties block should be strictly sorted with no duplicate key.
-      if (!last_key.empty() &&
-          BytewiseComparator()->Compare(key, last_key) <= 0) {
-        s = Status::Corruption("properties unsorted");
-        break;
-      }
-      last_key = key;
-
-      auto raw_val = iter->value();
-      auto pos = predefined_uint64_properties.find(key);
-
-      if (key == ExternalSstFilePropertyNames::kGlobalSeqno) {
-        new_table_properties->external_sst_file_global_seqno_offset =
-            handle.offset() + iter->ValueOffset();
-      }
-
-      if (pos != predefined_uint64_properties.end()) {
-        if (key == TablePropertiesNames::kDeletedKeys ||
-            key == TablePropertiesNames::kMergeOperands) {
-          // Insert in user-collected properties for API backwards compatibility
-          new_table_properties->user_collected_properties.insert(
-              {key, raw_val.ToString()});
-        }
-        // handle predefined rocksdb properties
-        uint64_t val;
-        if (!GetVarint64(&raw_val, &val)) {
-          // skip malformed value
-          auto error_msg =
-              "Detect malformed value in properties meta-block:"
-              "\tkey: " +
-              key + "\tval: " + raw_val.ToString();
-          ROCKS_LOG_ERROR(ioptions.logger, "%s", error_msg.c_str());
-          continue;
-        }
-        *(pos->second) = val;
-      } else if (key == TablePropertiesNames::kDbId) {
-        new_table_properties->db_id = raw_val.ToString();
-      } else if (key == TablePropertiesNames::kDbSessionId) {
-        new_table_properties->db_session_id = raw_val.ToString();
-      } else if (key == TablePropertiesNames::kDbHostId) {
-        new_table_properties->db_host_id = raw_val.ToString();
-      } else if (key == TablePropertiesNames::kFilterPolicy) {
-        new_table_properties->filter_policy_name = raw_val.ToString();
-      } else if (key == TablePropertiesNames::kColumnFamilyName) {
-        new_table_properties->column_family_name = raw_val.ToString();
-      } else if (key == TablePropertiesNames::kComparator) {
-        new_table_properties->comparator_name = raw_val.ToString();
-      } else if (key == TablePropertiesNames::kMergeOperator) {
-        new_table_properties->merge_operator_name = raw_val.ToString();
-      } else if (key == TablePropertiesNames::kPrefixExtractorName) {
-        new_table_properties->prefix_extractor_name = raw_val.ToString();
-      } else if (key == TablePropertiesNames::kPropertyCollectors) {
-        new_table_properties->property_collectors_names = raw_val.ToString();
-      } else if (key == TablePropertiesNames::kCompression) {
-        new_table_properties->compression_name = raw_val.ToString();
-      } else if (key == TablePropertiesNames::kCompressionOptions) {
-        new_table_properties->compression_options = raw_val.ToString();
-      } else if (key == TablePropertiesNames::kSequenceNumberTimeMapping) {
-        new_table_properties->seqno_to_time_mapping = raw_val.ToString();
-      } else {
-        // handle user-collected properties
-        new_table_properties->user_collected_properties.insert(
-            {key, raw_val.ToString()});
-      }
-    }
+    s = ParsePropertiesBlock(ioptions, handle.offset(), properties_block,
+                             new_table_properties);
 
     // Modified version of BlockFetcher checksum verification
     // (See write_global_seqno comment above)

table/meta_blocks.h

@@ -22,6 +22,7 @@
 
 namespace ROCKSDB_NAMESPACE {
 
+class Block;
 class BlockBuilder;
 class BlockHandle;
 class Env;
@@ -110,6 +111,10 @@ bool NotifyCollectTableCollectorsOnFinish(
     UserCollectedProperties& user_collected_properties,
     UserCollectedProperties& readable_properties);
 
+Status ParsePropertiesBlock(
+    const ImmutableOptions& ioptions, uint64_t offset, Block& block,
+    std::unique_ptr<TableProperties>& new_table_properties);
+
 // Read table properties from a file using known BlockHandle.
 // @returns a status to indicate if the operation succeeded. On success,
 //          *table_properties will point to a heap-allocated TableProperties

table/table_test.cc

@@ -6525,10 +6525,10 @@ TEST_F(CacheUsageOptionsOverridesTest, SanitizeAndValidateOptions) {
   Destroy(options);
 }
 
-class ExternalTableReaderTest : public DBTestBase {
+class ExternalTableTest : public DBTestBase {
  public:
-  ExternalTableReaderTest()
-      : DBTestBase("external_table_reader_test", /*env_do_fsync=*/false) {}
+  ExternalTableTest()
+      : DBTestBase("external_table_test", /*env_do_fsync=*/false) {}
 
  protected:
   class DummyExternalTableFile {
@@ -6541,6 +6541,13 @@ class ExternalTableReaderTest : public DBTestBase {
 
     Status Serialize(
         const std::vector<std::pair<std::string, std::string>>& kv_vec) {
+      // First append the property block if one exists
+      uint32_t prop_block_size = static_cast<uint32_t>(prop_block_.length());
+      buf_.append(static_cast<char*>(static_cast<void*>(&prop_block_size)),
+                  sizeof(prop_block_size));
+      if (!prop_block_.empty()) {
+        buf_.append(prop_block_);
+      }
       for (auto& kv : kv_vec) {
         SerializeOne(kv.first, kv.second);
         props_.raw_key_size += kv.first.length();
@@ -6561,6 +6568,12 @@ class ExternalTableReaderTest : public DBTestBase {
         return s;
       }
 
+      uint32_t prop_block_size = 0;
+      buf_.copy(static_cast<char*>(static_cast<void*>(&prop_block_size)),
+                sizeof(prop_block_size));
+      buf_.erase(0, sizeof(prop_block_size));
+      prop_block_.assign(buf_.substr(0, prop_block_size));
+      buf_.erase(0, prop_block_size);
       while (buf_.length() > 0) {
         std::pair<std::string, std::string> kv;
         s = DeserializeOne(kv);
@@ -6577,6 +6590,24 @@ class ExternalTableReaderTest : public DBTestBase {
       return s;
     }
 
+    Status PutPropertiesBlock(const Slice& prop_block) {
+      prop_block_.assign(prop_block.data(), prop_block.size());
+      return Status::OK();
+    }
+
+    Status GetPropertiesBlock(std::unique_ptr<char[]>* block, uint64_t* size,
+                              uint64_t* file_offset) {
+      if (!prop_block_.empty()) {
+        *block = std::make_unique<char[]>(prop_block_.length());
+        memcpy(block->get(), prop_block_.data(), prop_block_.length());
+        *size = prop_block_.length();
+        *file_offset = sizeof(uint32_t);
+      } else {
+        *size = 0;
+      }
+      return Status::OK();
+    }
+
     TableProperties GetTableProperties() const { return props_; }
 
     uint64_t FileSize() const { return file_size_; }
@@ -6619,6 +6650,7 @@ class ExternalTableReaderTest : public DBTestBase {
     std::string buf_;
     TableProperties props_;
     uint64_t file_size_;
+    std::string prop_block_;
   };
 
   class DummyExternalTableIterator : public ExternalTableIterator {
@@ -6764,8 +6796,10 @@ class ExternalTableReaderTest : public DBTestBase {
 
   class DummyExternalTableReader : public ExternalTableReader {
    public:
-    explicit DummyExternalTableReader(const std::string& file_path)
-        : file_(file_path, /*file=*/nullptr) {
+    explicit DummyExternalTableReader(const std::string& file_path,
+                                      bool support_property_block)
+        : file_(file_path, /*file=*/nullptr),
+          support_property_block_(support_property_block) {
       Status s = file_.Deserialize(kv_map_);
       EXPECT_OK(s);
     }
@@ -6800,6 +6834,14 @@ class ExternalTableReaderTest : public DBTestBase {
       }
     }
 
+    Status GetPropertiesBlock(std::unique_ptr<char[]>* block, uint64_t* size,
+                              uint64_t* file_offset) override {
+      if (!support_property_block_) {
+        return Status::NotSupported();
+      }
+      return file_.GetPropertiesBlock(block, size, file_offset);
+    }
+
     std::shared_ptr<const TableProperties> GetTableProperties() const override {
       std::shared_ptr<TableProperties> props =
           std::make_shared<TableProperties>();
@@ -6813,13 +6855,16 @@ class ExternalTableReaderTest : public DBTestBase {
    private:
     std::map<std::string, std::string> kv_map_;
     DummyExternalTableFile file_;
+    bool support_property_block_;
   };
 
   class DummyExternalTableBuilder : public ExternalTableBuilder {
    public:
     explicit DummyExternalTableBuilder(const std::string& file_path,
-                                       FSWritableFile* file)
-        : file_(file_path, file) {}
+                                       FSWritableFile* file,
+                                       bool support_property_block)
+        : file_(file_path, file),
+          support_property_block_(support_property_block) {}
 
     void Add(const Slice& key, const Slice& value) override {
       if (!kv_vec_.empty()) {
@@ -6837,6 +6882,13 @@ class ExternalTableReaderTest : public DBTestBase {
 
     uint64_t FileSize() const override { return file_.FileSize(); }
 
+    Status PutPropertiesBlock(const Slice& block) override {
+      if (!support_property_block_) {
+        return Status::NotSupported();
+      }
+      return file_.PutPropertiesBlock(block);
+    }
+
     TableProperties GetTableProperties() const override {
       return file_.GetTableProperties();
     }
@@ -6847,31 +6899,43 @@ class ExternalTableReaderTest : public DBTestBase {
     std::vector<std::pair<std::string, std::string>> kv_vec_;
     DummyExternalTableFile file_;
     Status status_;
+    bool support_property_block_;
   };
 
   class DummyExternalTableFactory : public ExternalTableFactory {
    public:
+    explicit DummyExternalTableFactory(bool support_property_block)
+        : support_property_block_(support_property_block) {}
     const char* Name() const override { return "DummyExternalTableFactory"; }
 
     Status NewTableReader(
         const ReadOptions& /*read_options*/, const std::string& file_path,
-        const ExternalTableOptions& /*topts*/,
+        const ExternalTableOptions& topts,
         std::unique_ptr<ExternalTableReader>* table_reader) const override {
-      table_reader->reset(new DummyExternalTableReader(file_path));
+      // Sanity check some options
+      EXPECT_EQ(topts.file_options.handoff_checksum_type,
+                ChecksumType::kCRC32c);
+      table_reader->reset(
+          new DummyExternalTableReader(file_path, support_property_block_));
       return Status::OK();
     }
 
     ExternalTableBuilder* NewTableBuilder(
         const ExternalTableBuilderOptions& /*opts*/,
         const std::string& file_path, FSWritableFile* file) const override {
-      return new DummyExternalTableBuilder(file_path, file);
+      return new DummyExternalTableBuilder(file_path, file,
+                                           support_property_block_);
     }
+
+   private:
+    bool support_property_block_;
   };
 };
 
-TEST_F(ExternalTableReaderTest, BasicTest) {
+TEST_F(ExternalTableTest, BasicTest) {
   std::shared_ptr<ExternalTableFactory> factory =
-      std::make_shared<DummyExternalTableFactory>();
+      std::make_shared<DummyExternalTableFactory>(
+          /*support_property_block=*/false);
 
   std::string file_path = test::PerThreadDBPath("external_table");
   {
@@ -6917,16 +6981,19 @@ TEST_F(ExternalTableReaderTest, BasicTest) {
   ASSERT_EQ(statuses[1], Status::NotFound());
 }
 
-TEST_F(ExternalTableReaderTest, SstReaderTest) {
+TEST_F(ExternalTableTest, SstReaderTest) {
+  if (encrypted_env_) {
+    ROCKSDB_GTEST_SKIP("Test requires non-encrypted environment");
+    return;
+  }
   Options options = GetDefaultOptions();
-  std::string dbname = test::PerThreadDBPath("external_table_reader_test");
+  std::string dbname = test::PerThreadDBPath("external_table_test");
   std::string ingest_file = dbname + "test.immutabledb";
   dbname += "_db";
-  // This test doesn't work with some custom Envs, like EncryptedEnv
-  options.env = Env::Default();
 
   std::shared_ptr<ExternalTableFactory> factory =
-      std::make_shared<DummyExternalTableFactory>();
+      std::make_shared<DummyExternalTableFactory>(
+          /*support_property_block=*/false);
   options.table_factory = NewExternalTableFactory(factory);
 
   std::unique_ptr<SstFileWriter> writer;
@@ -6950,17 +7017,56 @@ TEST_F(ExternalTableReaderTest, SstReaderTest) {
   ASSERT_TRUE(iter->status().ok());
 }
 
-TEST_F(ExternalTableReaderTest, DBIterTest) {
+TEST_F(ExternalTableTest, ExternalFileChecksumTest) {
+  if (encrypted_env_) {
+    ROCKSDB_GTEST_SKIP("Test requires non-encrypted environment");
+    return;
+  }
   Options options = GetDefaultOptions();
-  std::string dbname = test::PerThreadDBPath("external_table_reader_test");
+  std::string dbname = test::PerThreadDBPath("external_table_test");
   std::string ingest_file = dbname + "test.immutable";
   dbname += "_db";
-  // This test doesn't work with some custom Envs, like EncryptedEnv
-  options.env = Env::Default();
   ASSERT_OK(DestroyDB(dbname, options));
 
   std::shared_ptr<ExternalTableFactory> factory =
-      std::make_shared<DummyExternalTableFactory>();
+      std::make_shared<DummyExternalTableFactory>(
+          /*support_property_block=*/true);
+  options.table_factory = NewExternalTableFactory(factory);
+
+  // Create a file
+  options.file_checksum_gen_factory = GetFileChecksumGenCrc32cFactory();
+  std::unique_ptr<SstFileWriter> writer;
+  writer.reset(new SstFileWriter(EnvOptions(), options));
+  ASSERT_OK(writer->Open(ingest_file));
+  ASSERT_OK(writer->Put("foo", "bar"));
+  ASSERT_OK(writer->Put("foo2", "bar2"));
+  ExternalSstFileInfo info;
+  ASSERT_OK(writer->Finish(&info));
+  writer.reset();
+
+  FileChecksumGenContext cksum_ctx;
+  FileChecksumGenCrc32c cksum_gen(cksum_ctx);
+  std::string file_data;
+  ASSERT_OK(ReadFileToString(options.env, ingest_file, &file_data));
+  cksum_gen.Update(file_data.data(), file_data.size());
+  cksum_gen.Finalize();
+  ASSERT_EQ(info.file_checksum, cksum_gen.GetChecksum());
+}
+
+TEST_F(ExternalTableTest, DBIterTest) {
+  if (encrypted_env_) {
+    ROCKSDB_GTEST_SKIP("Test requires non-encrypted environment");
+    return;
+  }
+  Options options = GetDefaultOptions();
+  std::string dbname = test::PerThreadDBPath("external_table_test");
+  std::string ingest_file = dbname + "test.immutable";
+  dbname += "_db";
+  ASSERT_OK(DestroyDB(dbname, options));
+
+  std::shared_ptr<ExternalTableFactory> factory =
+      std::make_shared<DummyExternalTableFactory>(
+          /*support_property_block=*/true);
   options.table_factory = NewExternalTableFactory(factory);
 
   // Create a file
@@ -7004,17 +7110,20 @@ TEST_F(ExternalTableReaderTest, DBIterTest) {
   ASSERT_OK(db->Close());
 }
 
-TEST_F(ExternalTableReaderTest, DBMultiScanTest) {
+TEST_F(ExternalTableTest, DBMultiScanTest) {
+  if (encrypted_env_) {
+    ROCKSDB_GTEST_SKIP("Test requires non-encrypted environment");
+    return;
+  }
   Options options = GetDefaultOptions();
-  std::string dbname = test::PerThreadDBPath("external_table_reader_test");
+  std::string dbname = test::PerThreadDBPath("external_table_test");
   std::string ingest_file = dbname + "test.immutable";
   dbname += "_db";
-  // This test doesn't work with some custom Envs, like EncryptedEnv
-  options.env = Env::Default();
   ASSERT_OK(DestroyDB(dbname, options));
 
   std::shared_ptr<ExternalTableFactory> factory =
-      std::make_shared<DummyExternalTableFactory>();
+      std::make_shared<DummyExternalTableFactory>(
+          /*support_property_block=*/true);
   options.table_factory = NewExternalTableFactory(factory);
 
   // Create a file
@@ -7158,6 +7267,111 @@ TEST_F(ExternalTableReaderTest, DBMultiScanTest) {
   ASSERT_OK(db->DestroyColumnFamilyHandle(cfh));
   ASSERT_OK(db->Close());
 }
+
+TEST_F(ExternalTableTest, IngestionTest) {
+  if (encrypted_env_) {
+    ROCKSDB_GTEST_SKIP("Test requires non-encrypted environment");
+    return;
+  }
+  Options options = GetDefaultOptions();
+  std::string dbname = test::PerThreadDBPath("external_table_test");
+  std::string ingest_file = dbname + "test.immutable";
+  dbname += "_db";
+  ASSERT_OK(DestroyDB(dbname, options));
+
+  std::shared_ptr<ExternalTableFactory> factory =
+      std::make_shared<DummyExternalTableFactory>(
+          /*support_property_block=*/true);
+  options.table_factory = NewExternalTableFactory(factory);
+
+  // Create a file
+  std::unique_ptr<SstFileWriter> writer;
+  writer.reset(new SstFileWriter(EnvOptions(), options));
+  ASSERT_OK(writer->Open(ingest_file));
+  ASSERT_OK(writer->Put("foo", "bar"));
+  ASSERT_OK(writer->Put("foo2", "bar2"));
+  ASSERT_OK(writer->Finish());
+  writer.reset();
+
+  std::unique_ptr<DB> db;
+  options.create_if_missing = true;
+  Status s = DB::Open(options, dbname, &db);
+  ASSERT_OK(s);
+  ASSERT_TRUE(db != nullptr);
+  ColumnFamilyHandle* cfh = nullptr;
+  ASSERT_OK(db->CreateColumnFamily(options, "new_cf", &cfh));
+
+  IngestExternalFileOptions ifo;
+  ifo.allow_db_generated_files = false;
+  ifo.fill_cache = false;
+  s = db->IngestExternalFile(cfh, {ingest_file}, ifo);
+  ASSERT_OK(s);
+
+  std::unique_ptr<Iterator> iter(db->NewIterator({}, cfh));
+  ASSERT_NE(iter, nullptr);
+  iter->Seek("foo");
+  ASSERT_TRUE(iter->Valid() && iter->status().ok());
+  ASSERT_EQ(iter->value(), "bar");
+  iter->Next();
+  ASSERT_TRUE(iter->Valid() && iter->status().ok());
+  ASSERT_EQ(iter->key(), "foo2");
+  ASSERT_EQ(iter->value(), "bar2");
+  iter->Next();
+  ASSERT_FALSE(iter->Valid());
+  ASSERT_OK(iter->status());
+  iter.reset();
+
+  // Create an overlapping file to ingest with atomic_replace_range option
+  ingest_file += "2";
+  writer.reset(new SstFileWriter(EnvOptions(), options));
+  ASSERT_OK(writer->Open(ingest_file));
+  ASSERT_OK(writer->Put("foo", "val"));
+  ASSERT_OK(writer->Put("foo2", "val2"));
+  ASSERT_OK(writer->Finish());
+  writer.reset();
+
+  ifo.snapshot_consistency = false;
+  s = db->IngestExternalFiles({{cfh,
+                                {ingest_file},
+                                ifo,
+                                {},
+                                {},
+                                Temperature::kUnknown,
+                                {{nullptr, nullptr}}}});
+  ASSERT_OK(s);
+
+  iter.reset(db->NewIterator({}, cfh));
+  ASSERT_NE(iter, nullptr);
+  iter->Seek("foo");
+  ASSERT_TRUE(iter->Valid() && iter->status().ok());
+  ASSERT_EQ(iter->value(), "val");
+  iter->Next();
+  ASSERT_TRUE(iter->Valid() && iter->status().ok());
+  ASSERT_EQ(iter->key(), "foo2");
+  ASSERT_EQ(iter->value(), "val2");
+  iter->Next();
+  ASSERT_FALSE(iter->Valid());
+  ASSERT_OK(iter->status());
+  iter.reset();
+
+  // Create an overlapping file to ingest without atomic_replace_range option.
+  // This should fail as we don't support ingesting an external file with
+  // non-zero assigned sequence number.
+  ingest_file += "3";
+  writer.reset(new SstFileWriter(EnvOptions(), options));
+  ASSERT_OK(writer->Open(ingest_file));
+  ASSERT_OK(writer->Put("foo", "newval"));
+  ASSERT_OK(writer->Put("foo2", "newval2"));
+  ASSERT_OK(writer->Finish());
+  writer.reset();
+
+  s = db->IngestExternalFiles(
+      {{cfh, {ingest_file}, ifo, {}, {}, Temperature::kUnknown, {}}});
+  ASSERT_EQ(s, Status::NotSupported());
+
+  ASSERT_OK(db->DestroyColumnFamilyHandle(cfh));
+  ASSERT_OK(db->Close());
+}
 }  // namespace ROCKSDB_NAMESPACE
 
 int main(int argc, char** argv) {

unreleased_history/behavior_changes/ra_stats_user_only.md

@@ -1 +0,0 @@
-Make stats `PREFETCH_BYTES_USEFUL`, `PREFETCH_HITS`, `PREFETCH_BYTES` only account for prefetching during user initiated scan

unreleased_history/bug_fixes/fix_reopened_writable_file_size_incorrect.md

@@ -1 +0,0 @@
-Fix a bug in Posix file system that the FSWritableFile created via `FileSystem::ReopenWritableFile` internally does not track the correct file size.

unreleased_history/bug_fixes/remote_compact_populate.md

@@ -1 +0,0 @@
-Fix a bug where tail size of remote compaction output is not persisted in primary db's manifest

unreleased_history/new_features/compact_ra_stats.md

@@ -1,2 +0,0 @@
-Provide histogram stats `COMPACTION_PREFETCH_BYTES` to measure number of bytes for RocksDB's prefetching (as opposed to file
-system's prefetch) on SST file during compaction read

unreleased_history/new_features/get_newest_udt.md

@@ -1 +0,0 @@
-A new API DB::GetNewestUserDefinedTimestamp is added to return the newest user defined timestamp seen in a column family

unreleased_history/new_features/ingest_wbwi.md

@@ -1 +0,0 @@
-* Introduce API `IngestWriteBatchWithIndex()` for ingesting updates into DB while bypassing memtable writes. This improves performance when writing a large write batch to the DB.

unreleased_history/new_features/tombstone_scan_flush_trigger.md

@@ -1 +0,0 @@
-* Add a new CF option `memtable_op_scan_flush_trigger` that triggers a flush of the memtable if an iterator's Seek()/Next() scans over a certain number of invisible entries from the memtable.

unreleased_history/public_api_changes/delete_max_write_buffer_number_to_maintain.md

@@ -1 +0,0 @@
-AdvancedColumnFamilyOptions.max_write_buffer_number_to_maintain is deleted. It's deprecated since introduction of a better option max_write_buffer_size_to_maintain since RocksDB 6.5.0.

unreleased_history/public_api_changes/dep_max_compact_memtable_level.md

@@ -1 +0,0 @@
-* Deprecated API `DB::MaxMemCompactionLevel()`.

unreleased_history/public_api_changes/deprecate-ignore-range-del.md

@@ -1 +0,0 @@
-* Deprecated `ReadOptions::ignore_range_deletions`.

unreleased_history/public_api_changes/deprecate-promote-l0.md

@@ -1 +0,0 @@
-* Deprecated API `experimental::PromoteL0()`.

unreleased_history/public_api_changes/options_map_in_compaction_service_options_override.md

@@ -1 +0,0 @@
-Added arbitrary string map for additional options to be overriden for remote compactions

unreleased_history/public_api_changes/remove_fail_if_options_file_error.md

@@ -1 +0,0 @@
-The fail_if_options_file_error option in DBOptions has been removed. The behavior now is to always return failure in any API that fails to persist the OPTIONS file.