Fix WriteBatchBase::DeleteRange API comment

The `DeleteRange` end key is exclusive, not inclusive. Updated API comment accordingly.

CMakeLists.txt

@@ -501,7 +501,6 @@ set(SOURCES
         db/flush_scheduler.cc
         db/forward_iterator.cc
         db/internal_stats.cc
-        db/in_memory_stats_history.cc
         db/logs_with_prep_tracker.cc
         db/log_reader.cc
         db/log_writer.cc
@@ -531,6 +530,7 @@ set(SOURCES
         env/env_hdfs.cc
         env/mock_env.cc
         memtable/alloc_tracker.cc
+        memtable/hash_cuckoo_rep.cc
         memtable/hash_linklist_rep.cc
         memtable/hash_skiplist_rep.cc
         memtable/skiplistrep.cc
@@ -648,6 +648,7 @@ set(SOURCES
         utilities/env_mirror.cc
         utilities/env_timed.cc
         utilities/leveldb_options/leveldb_options.cc
+        utilities/lua/rocks_lua_compaction_filter.cc
         utilities/memory/memory_util.cc
         utilities/merge_operators/bytesxor.cc
         utilities/merge_operators/max.cc
@@ -964,6 +965,7 @@ if(WITH_TESTS)
         utilities/cassandra/cassandra_row_merge_test.cc
         utilities/cassandra/cassandra_serialize_test.cc
         utilities/checkpoint/checkpoint_test.cc
+        utilities/lua/rocks_lua_test.cc
         utilities/memory/memory_test.cc
         utilities/merge_operators/string_append/stringappend_test.cc
         utilities/object_registry_test.cc

HISTORY.md

@@ -1,24 +1,5 @@
 # Rocksdb Change Log
-
-## 6.0.2 (4/23/2019)
-## Bug Fixes
-* Fix build failures due to missing JEMALLOC_CXX_THROW macro (#5053)
-* Fix compilation on db_bench_tool.cc on Windows (#5227)
-
-## 6.0.1 (3/26/2019)
-### New Features
-### Public API Change
-* Added many new features to the Java API to bring it closer to the C++ API.
-
-### Bug Fixes
-* Make BlobDB wait for all background tasks on shutdown.
-* Fixed a BlobDB issue where some trash files are not tracked causing them to remain forever.
-
-### Bug Fixes
-* Fix JEMALLOC_CXX_THROW macro missing from older Jemalloc versions, causing build failures on some platforms.
-
-
-## 6.0.0 (2/19/2019)
+## Unreleased
 ### New Features
 * Enabled checkpoint on readonly db (DBImplReadOnly).
 * Make DB ignore dropped column families while committing results of atomic flush.
@@ -26,16 +7,8 @@
 * For users of dictionary compression with ZSTD v0.7.0+, we now reuse the same digested dictionary when compressing each of an SST file's data blocks for faster compression speeds.
 * For all users of dictionary compression who set `cache_index_and_filter_blocks == true`, we now store dictionary data used for decompression in the block cache for better control over memory usage. For users of ZSTD v1.1.4+ who compile with -DZSTD_STATIC_LINKING_ONLY, this includes a digested dictionary, which is used to increase decompression speed.
 * Add support for block checksums verification for external SST files before ingestion.
-* Introduce stats history which periodically saves Statistics snapshots and added `GetStatsHistory` API to retrieve these snapshots.
-* Add a place holder in manifest which indicate a record from future that can be safely ignored.
-* Add support for trace sampling.
-* Enable properties block checksum verification for block-based tables.
-* For all users of dictionary compression, we now generate a separate dictionary for compressing each bottom-level SST file. Previously we reused a single dictionary for a whole compaction to bottom level. The new approach achieves better compression ratios; however, it uses more memory and CPU for buffering/sampling data blocks and training dictionaries.
-* Add whole key bloom filter support in memtable.
-* Files written by `SstFileWriter` will now use dictionary compression if it is configured in the file writer's `CompressionOptions`.
 
 ### Public API Change
-* Disallow CompactionFilter::IgnoreSnapshots() = false, because it is not very useful and the behavior is confusing. The filter will filter everything if there is no snapshot declared by the time the compaction starts. However, users can define a snapshot after the compaction starts and before it finishes and this new snapshot won't be repeatable, because after the compaction finishes, some keys may be dropped.
 * CompactionPri = kMinOverlappingRatio also uses compensated file size, which boosts file with lots of tombstones to be compacted first.
 * Transaction::GetForUpdate is extended with a do_validate parameter with default value of true. If false it skips validating the snapshot before doing the read. Similarly ::Merge, ::Put, ::Delete, and ::SingleDelete are extended with assume_tracked with default value of false. If true it indicates that call is assumed to be after a ::GetForUpdate.
 * `TableProperties::num_entries` and `TableProperties::num_deletions` now also account for number of range tombstones.
@@ -43,11 +16,7 @@
 * With "ldb ----try_load_options", when wal_dir specified by the option file doesn't exist, ignore it.
 * Change time resolution in FileOperationInfo.
 * Deleting Blob files also go through SStFileManager.
-* Remove CuckooHash memtable.
-* The counter stat `number.block.not_compressed` now also counts blocks not compressed due to poor compression ratio.
-* Remove ttl option from `CompactionOptionsFIFO`. The option has been deprecated and ttl in `ColumnFamilyOptions` is used instead.
-* Support SST file ingestion across multiple column families via DB::IngestExternalFiles. See the function's comment about atomicity.
-* Remove Lua compaction filter.
+* Remove PlainTable's store_index_in_file feature. When opening an existing DB with index in SST files, the index and bloom filter will still be rebuild while SST files are opened, in the same way as there is no index in the file.
 
 ### Bug Fixes
 * Fix a deadlock caused by compaction and file ingestion waiting for each other in the event of write stalls.

Makefile

@@ -535,6 +535,7 @@ TESTS = \
 	ldb_cmd_test \
 	persistent_cache_test \
 	statistics_test \
+	lua_test \
 	lru_cache_test \
 	object_registry_test \
 	repair_test \
@@ -686,8 +687,7 @@ endif  # PLATFORM_SHARED_EXT
 .PHONY: blackbox_crash_test check clean coverage crash_test ldb_tests package \
 	release tags tags0 valgrind_check whitebox_crash_test format static_lib shared_lib all \
 	dbg rocksdbjavastatic rocksdbjava install install-static install-shared uninstall \
-	analyze tools tools_lib \
-	blackbox_crash_test_with_atomic_flush whitebox_crash_test_with_atomic_flush
+	analyze tools tools_lib
 
 
 all: $(LIBRARY) $(BENCHMARKS) tools tools_lib test_libs $(TESTS)
@@ -895,14 +895,10 @@ ldb_tests: ldb
 
 crash_test: whitebox_crash_test blackbox_crash_test
 
-crash_test_with_atomic_flush: whitebox_crash_test_with_atomic_flush blackbox_crash_test_with_atomic_flush
-
 blackbox_crash_test: db_stress
 	python -u tools/db_crashtest.py --simple blackbox $(CRASH_TEST_EXT_ARGS)
-	python -u tools/db_crashtest.py blackbox $(CRASH_TEST_EXT_ARGS)
-
-blackbox_crash_test_with_atomic_flush: db_stress
 	python -u tools/db_crashtest.py --enable_atomic_flush blackbox $(CRASH_TEST_EXT_ARGS)
+	python -u tools/db_crashtest.py blackbox $(CRASH_TEST_EXT_ARGS)
 
 ifeq ($(CRASH_TEST_KILL_ODD),)
   CRASH_TEST_KILL_ODD=888887
@@ -911,12 +907,10 @@ endif
 whitebox_crash_test: db_stress
 	python -u tools/db_crashtest.py --simple whitebox --random_kill_odd \
       $(CRASH_TEST_KILL_ODD) $(CRASH_TEST_EXT_ARGS)
-	python -u tools/db_crashtest.py whitebox  --random_kill_odd \
-      $(CRASH_TEST_KILL_ODD) $(CRASH_TEST_EXT_ARGS)
-
-whitebox_crash_test_with_atomic_flush: db_stress
 	python -u tools/db_crashtest.py --enable_atomic_flush whitebox  --random_kill_odd \
       $(CRASH_TEST_KILL_ODD) $(CRASH_TEST_EXT_ARGS)
+	python -u tools/db_crashtest.py whitebox  --random_kill_odd \
+      $(CRASH_TEST_KILL_ODD) $(CRASH_TEST_EXT_ARGS)
 
 asan_check:
 	$(MAKE) clean
@@ -928,11 +922,6 @@ asan_crash_test:
 	COMPILE_WITH_ASAN=1 $(MAKE) crash_test
 	$(MAKE) clean
 
-asan_crash_test_with_atomic_flush:
-	$(MAKE) clean
-	COMPILE_WITH_ASAN=1 $(MAKE) crash_test_with_atomic_flush
-	$(MAKE) clean
-
 ubsan_check:
 	$(MAKE) clean
 	COMPILE_WITH_UBSAN=1 $(MAKE) check -j32
@@ -943,11 +932,6 @@ ubsan_crash_test:
 	COMPILE_WITH_UBSAN=1 $(MAKE) crash_test
 	$(MAKE) clean
 
-ubsan_crash_test_with_atomic_flush:
-	$(MAKE) clean
-	COMPILE_WITH_UBSAN=1 $(MAKE) crash_test_with_atomic_flush
-	$(MAKE) clean
-
 valgrind_test:
 	ROCKSDB_VALGRIND_RUN=1 DISABLE_JEMALLOC=1 $(MAKE) valgrind_check
 
@@ -1553,6 +1537,9 @@ statistics_test: monitoring/statistics_test.o $(LIBOBJECTS) $(TESTHARNESS)
 lru_cache_test: cache/lru_cache_test.o $(LIBOBJECTS) $(TESTHARNESS)
 	$(AM_LINK)
 
+lua_test: utilities/lua/rocks_lua_test.o db/db_test_util.o $(LIBOBJECTS) $(TESTHARNESS)
+	$(AM_LINK)
+
 range_del_aggregator_test: db/range_del_aggregator_test.o db/db_test_util.o $(LIBOBJECTS) $(TESTHARNESS)
 	$(AM_LINK)
 
@@ -1726,7 +1713,7 @@ endif
 	fi
 	tar xvzf snappy-$(SNAPPY_VER).tar.gz
 	mkdir snappy-$(SNAPPY_VER)/build
-	cd snappy-$(SNAPPY_VER)/build && CFLAGS='${EXTRA_CFLAGS}' CXXFLAGS='${EXTRA_CXXFLAGS}' LDFLAGS='${EXTRA_LDFLAGS}' cmake -DCMAKE_POSITION_INDEPENDENT_CODE=ON .. && $(MAKE) ${SNAPPY_MAKE_TARGET}
+	cd snappy-$(SNAPPY_VER)/build && CFLAGS='${EXTRA_CFLAGS}' CXXFLAGS='${EXTRA_CXXFLAGS}' LDFLAGS='${EXTRA_LDFLAGS}' cmake .. && $(MAKE) ${SNAPPY_MAKE_TARGET}
 	cp snappy-$(SNAPPY_VER)/build/libsnappy.a .
 
 liblz4.a:

TARGETS

@@ -110,7 +110,6 @@ cpp_library(
         "db/flush_job.cc",
         "db/flush_scheduler.cc",
         "db/forward_iterator.cc",
-        "db/in_memory_stats_history.cc",
         "db/internal_stats.cc",
         "db/log_reader.cc",
         "db/log_writer.cc",
@@ -143,6 +142,7 @@ cpp_library(
         "env/io_posix.cc",
         "env/mock_env.cc",
         "memtable/alloc_tracker.cc",
+        "memtable/hash_cuckoo_rep.cc",
         "memtable/hash_linklist_rep.cc",
         "memtable/hash_skiplist_rep.cc",
         "memtable/skiplistrep.cc",
@@ -261,6 +261,7 @@ cpp_library(
         "utilities/env_mirror.cc",
         "utilities/env_timed.cc",
         "utilities/leveldb_options/leveldb_options.cc",
+        "utilities/lua/rocks_lua_compaction_filter.cc",
         "utilities/memory/memory_util.cc",
         "utilities/merge_operators/bytesxor.cc",
         "utilities/merge_operators/max.cc",

buckifier/targets_cfg.py

@@ -3,7 +3,6 @@ from __future__ import division
 from __future__ import print_function
 from __future__ import unicode_literals
 rocksdb_target_header = """load("@fbcode_macros//build_defs:auto_headers.bzl", "AutoHeaders")
-load("@fbcode_macros//build_defs:cpp_library.bzl", "cpp_library")
 load(":defs.bzl", "test_binary")
 
 REPO_PATH = package_name() + "/"
@@ -24,7 +23,6 @@ ROCKSDB_COMPILER_FLAGS = [
     "-DBZIP2",
     "-DLZ4",
     "-DZSTD",
-    "-DZSTD_STATIC_LINKING_ONLY",
     "-DGFLAGS=gflags",
     "-DNUMA",
     "-DTBB",

build_tools/error_filter.py

@@ -132,14 +132,11 @@ _TEST_NAME_TO_PARSERS = {
     'lite': [CompilerErrorParser],
     'lite_test': [CompilerErrorParser, GTestErrorParser],
     'stress_crash': [CompilerErrorParser, DbCrashErrorParser],
-    'stress_crash_with_atomic_flush': [CompilerErrorParser, DbCrashErrorParser],
     'write_stress': [CompilerErrorParser, WriteStressErrorParser],
     'asan': [CompilerErrorParser, GTestErrorParser, AsanErrorParser],
     'asan_crash': [CompilerErrorParser, AsanErrorParser, DbCrashErrorParser],
-    'asan_crash_with_atomic_flush': [CompilerErrorParser, AsanErrorParser, DbCrashErrorParser],
     'ubsan': [CompilerErrorParser, GTestErrorParser, UbsanErrorParser],
     'ubsan_crash': [CompilerErrorParser, UbsanErrorParser, DbCrashErrorParser],
-    'ubsan_crash_with_atomic_flush': [CompilerErrorParser, UbsanErrorParser, DbCrashErrorParser],
     'valgrind': [CompilerErrorParser, GTestErrorParser, ValgrindErrorParser],
     'tsan': [CompilerErrorParser, GTestErrorParser, TsanErrorParser],
     'format_compatible': [CompilerErrorParser, CompatErrorParser],

build_tools/rocksdb-lego-determinator

@@ -400,35 +400,6 @@ STRESS_CRASH_TEST_COMMANDS="[
     }
 ]"
 
-#
-# RocksDB stress/crash test with atomic flush
-#
-STRESS_CRASH_TEST_WITH_ATOMIC_FLUSH_COMMANDS="[
-    {
-        'name':'Rocksdb Stress/Crash Test (atomic flush)',
-        'oncall':'$ONCALL',
-        'timeout': 86400,
-        'steps': [
-            $CLEANUP_ENV,
-            {
-                'name':'Build and run RocksDB debug stress tests',
-                'shell':'$SHM $DEBUG $NON_TSAN_CRASH make J=1 db_stress || $CONTRUN_NAME=db_stress $TASK_CREATION_TOOL',
-                'user':'root',
-                $PARSER
-            },
-            {
-                'name':'Build and run RocksDB debug crash tests with atomic flush',
-                'timeout': 86400,
-                'shell':'$SHM $DEBUG $NON_TSAN_CRASH make J=1 crash_test_with_atomic_flush || $CONTRUN_NAME=crash_test_with_atomic_flush $TASK_CREATION_TOOL',
-                'user':'root',
-                $PARSER
-            }
-        ],
-        $ARTIFACTS,
-        $REPORT
-    }
-]"
-
 # RocksDB write stress test.
 # We run on disk device on purpose (i.e. no $SHM)
 # because we want to add some randomness to fsync commands
@@ -493,28 +464,6 @@ ASAN_CRASH_TEST_COMMANDS="[
     }
 ]"
 
-#
-# RocksDB crash testing with atomic flush under address sanitizer
-#
-ASAN_CRASH_TEST_WITH_ATOMIC_FLUSH_COMMANDS="[
-    {
-        'name':'Rocksdb crash test (atomic flush) under ASAN',
-        'oncall':'$ONCALL',
-        'timeout': 86400,
-        'steps': [
-            $CLEANUP_ENV,
-            {
-                'name':'Build and run RocksDB debug asan_crash_test_with_atomic_flush',
-                'timeout': 86400,
-                'shell':'$SHM $DEBUG $NON_TSAN_CRASH make J=1 asan_crash_test_with_atomic_flush || $CONTRUN_NAME=asan_crash_test_with_atomic_flush $TASK_CREATION_TOOL',
-                'user':'root',
-                $PARSER
-            },
-        ],
-        $REPORT
-    }
-]"
-
 #
 # RocksDB test under undefined behavior sanitizer
 #
@@ -557,28 +506,6 @@ UBSAN_CRASH_TEST_COMMANDS="[
     }
 ]"
 
-#
-# RocksDB crash testing with atomic flush under undefined behavior sanitizer
-#
-UBSAN_CRASH_TEST_WITH_ATOMIC_FLUSH_COMMANDS="[
-    {
-        'name':'Rocksdb crash test (atomic flush) under UBSAN',
-        'oncall':'$ONCALL',
-        'timeout': 86400,
-        'steps': [
-            $CLEANUP_ENV,
-            {
-                'name':'Build and run RocksDB debug ubsan_crash_test_with_atomic_flush',
-                'timeout': 86400,
-                'shell':'$SHM $DEBUG $NON_TSAN_CRASH make J=1 ubsan_crash_test_with_atomic_flush || $CONTRUN_NAME=ubsan_crash_test_with_atomic_flush $TASK_CREATION_TOOL',
-                'user':'root',
-                $PARSER
-            },
-        ],
-        $REPORT
-    }
-]"
-
 #
 # RocksDB unit test under valgrind
 #
@@ -645,28 +572,6 @@ TSAN_CRASH_TEST_COMMANDS="[
     }
 ]"
 
-#
-# RocksDB crash test with atomic flush under TSAN
-#
-TSAN_CRASH_TEST_WITH_ATOMIC_FLUSH_COMMANDS="[
-    {
-        'name':'Rocksdb Crash Test with atomic flush under TSAN',
-        'oncall':'$ONCALL',
-        'timeout': 86400,
-        'steps': [
-            $CLEANUP_ENV,
-            {
-                'name':'Compile and run',
-                'timeout': 86400,
-                'shell':'set -o pipefail && $SHM $DEBUG $TSAN $TSAN_CRASH CRASH_TEST_KILL_ODD=1887 make J=1 crash_test_with_atomic_flush || $CONTRUN_NAME=tsan_crash_test_with_atomic_flush $TASK_CREATION_TOOL',
-                'user':'root',
-                $PARSER
-            },
-        ],
-        $REPORT
-    }
-]"
-
 #
 # RocksDB format compatible
 #
@@ -706,7 +611,7 @@ run_no_compression()
   rm -rf /dev/shm/rocksdb
   mkdir /dev/shm/rocksdb
   make clean
-  cat build_tools/fbcode_config.sh | grep -iv dzstd | grep -iv dzlib | grep -iv dlz4 | grep -iv dsnappy | grep -iv dbzip2 > .tmp.fbcode_config.sh
+  cat build_tools/fbcode_config.sh | grep -iv dzlib | grep -iv dlz4 | grep -iv dsnappy | grep -iv dbzip2 > .tmp.fbcode_config.sh
   mv .tmp.fbcode_config.sh build_tools/fbcode_config.sh
   cat Makefile | grep -v tools/ldb_test.py > .tmp.Makefile
   mv .tmp.Makefile Makefile
@@ -848,9 +753,6 @@ case $1 in
   stress_crash)
     echo $STRESS_CRASH_TEST_COMMANDS
     ;;
-  stress_crash_with_atomic_flush)
-    echo $STRESS_CRASH_TEST_WITH_ATOMIC_FLUSH_COMMANDS
-    ;;
   write_stress)
     echo $WRITE_STRESS_COMMANDS
     ;;
@@ -860,18 +762,12 @@ case $1 in
   asan_crash)
     echo $ASAN_CRASH_TEST_COMMANDS
     ;;
-  asan_crash_with_atomic_flush)
-    echo $ASAN_CRASH_TEST_WITH_ATOMIC_FLUSH_COMMANDS
-    ;;
   ubsan)
     echo $UBSAN_TEST_COMMANDS
     ;;
   ubsan_crash)
     echo $UBSAN_CRASH_TEST_COMMANDS
     ;;
-  ubsan_crash_with_atomic_flush)
-    echo $UBSAN_CRASH_TEST_WITH_ATOMIC_FLUSH_COMMANDS
-    ;;
   valgrind)
     echo $VALGRIND_TEST_COMMANDS
     ;;
@@ -881,9 +777,6 @@ case $1 in
   tsan_crash)
     echo $TSAN_CRASH_TEST_COMMANDS
     ;;
-  tsan_crash_with_atomic_flush)
-    echo $TSAN_CRASH_TEST_WITH_ATOMIC_FLUSH_COMMANDS
-    ;;
   format_compatible)
     echo $FORMAT_COMPATIBLE_COMMANDS
     ;;

cache/cache_test.cc

@@ -73,7 +73,8 @@ class CacheTest : public testing::TestWithParam<std::string> {
     current_ = this;
   }
 
-  ~CacheTest() override {}
+  ~CacheTest() {
+  }
 
   std::shared_ptr<Cache> NewCache(size_t capacity) {
     auto type = GetParam();

cache/clock_cache.cc

@@ -240,29 +240,32 @@ class ClockCacheShard : public CacheShard {
   typedef tbb::concurrent_hash_map<CacheKey, CacheHandle*, CacheKey> HashTable;
 
   ClockCacheShard();
-  ~ClockCacheShard() override;
+  ~ClockCacheShard();
 
   // Interfaces
-  void SetCapacity(size_t capacity) override;
-  void SetStrictCapacityLimit(bool strict_capacity_limit) override;
-  Status Insert(const Slice& key, uint32_t hash, void* value, size_t charge,
-                void (*deleter)(const Slice& key, void* value),
-                Cache::Handle** handle, Cache::Priority priority) override;
-  Cache::Handle* Lookup(const Slice& key, uint32_t hash) override;
+  virtual void SetCapacity(size_t capacity) override;
+  virtual void SetStrictCapacityLimit(bool strict_capacity_limit) override;
+  virtual Status Insert(const Slice& key, uint32_t hash, void* value,
+                        size_t charge,
+                        void (*deleter)(const Slice& key, void* value),
+                        Cache::Handle** handle,
+                        Cache::Priority priority) override;
+  virtual Cache::Handle* Lookup(const Slice& key, uint32_t hash) override;
   // If the entry in in cache, increase reference count and return true.
   // Return false otherwise.
   //
   // Not necessary to hold mutex_ before being called.
-  bool Ref(Cache::Handle* handle) override;
-  bool Release(Cache::Handle* handle, bool force_erase = false) override;
-  void Erase(const Slice& key, uint32_t hash) override;
+  virtual bool Ref(Cache::Handle* handle) override;
+  virtual bool Release(Cache::Handle* handle,
+                       bool force_erase = false) override;
+  virtual void Erase(const Slice& key, uint32_t hash) override;
   bool EraseAndConfirm(const Slice& key, uint32_t hash,
                        CleanupContext* context);
-  size_t GetUsage() const override;
-  size_t GetPinnedUsage() const override;
-  void EraseUnRefEntries() override;
-  void ApplyToAllCacheEntries(void (*callback)(void*, size_t),
-                              bool thread_safe) override;
+  virtual size_t GetUsage() const override;
+  virtual size_t GetPinnedUsage() const override;
+  virtual void EraseUnRefEntries() override;
+  virtual void ApplyToAllCacheEntries(void (*callback)(void*, size_t),
+                                      bool thread_safe) override;
 
  private:
   static const uint32_t kInCacheBit = 1;
@@ -682,31 +685,31 @@ class ClockCache : public ShardedCache {
     SetStrictCapacityLimit(strict_capacity_limit);
   }
 
-  ~ClockCache() override { delete[] shards_; }
+  virtual ~ClockCache() { delete[] shards_; }
 
-  const char* Name() const override { return "ClockCache"; }
+  virtual const char* Name() const override { return "ClockCache"; }
 
-  CacheShard* GetShard(int shard) override {
+  virtual CacheShard* GetShard(int shard) override {
     return reinterpret_cast<CacheShard*>(&shards_[shard]);
   }
 
-  const CacheShard* GetShard(int shard) const override {
+  virtual const CacheShard* GetShard(int shard) const override {
     return reinterpret_cast<CacheShard*>(&shards_[shard]);
   }
 
-  void* Value(Handle* handle) override {
+  virtual void* Value(Handle* handle) override {
     return reinterpret_cast<const CacheHandle*>(handle)->value;
   }
 
-  size_t GetCharge(Handle* handle) const override {
+  virtual size_t GetCharge(Handle* handle) const override {
     return reinterpret_cast<const CacheHandle*>(handle)->charge;
   }
 
-  uint32_t GetHash(Handle* handle) const override {
+  virtual uint32_t GetHash(Handle* handle) const override {
     return reinterpret_cast<const CacheHandle*>(handle)->hash;
   }
 
-  void DisownData() override { shards_ = nullptr; }
+  virtual void DisownData() override { shards_ = nullptr; }
 
  private:
   ClockCacheShard* shards_;

cache/lru_cache_test.cc

@@ -15,7 +15,7 @@ namespace rocksdb {
 class LRUCacheTest : public testing::Test {
  public:
   LRUCacheTest() {}
-  ~LRUCacheTest() override { DeleteCache(); }
+  ~LRUCacheTest() { DeleteCache(); }
 
   void DeleteCache() {
     if (cache_ != nullptr) {

db/builder.cc

@@ -48,17 +48,17 @@ TableBuilder* NewTableBuilder(
     uint32_t column_family_id, const std::string& column_family_name,
     WritableFileWriter* file, const CompressionType compression_type,
     const CompressionOptions& compression_opts, int level,
-    const bool skip_filters, const uint64_t creation_time,
-    const uint64_t oldest_key_time, const uint64_t target_file_size) {
+    const std::string* compression_dict, const bool skip_filters,
+    const uint64_t creation_time, const uint64_t oldest_key_time) {
   assert((column_family_id ==
           TablePropertiesCollectorFactory::Context::kUnknownColumnFamily) ==
          column_family_name.empty());
   return ioptions.table_factory->NewTableBuilder(
       TableBuilderOptions(ioptions, moptions, internal_comparator,
                           int_tbl_prop_collector_factories, compression_type,
-                          compression_opts, skip_filters, column_family_name,
-                          level, creation_time, oldest_key_time,
-                          target_file_size),
+                          compression_opts, compression_dict, skip_filters,
+                          column_family_name, level, creation_time,
+                          oldest_key_time),
       column_family_id, file);
 }
 
@@ -105,11 +105,6 @@ Status BuildTable(
   if (iter->Valid() || !range_del_agg->IsEmpty()) {
     TableBuilder* builder;
     std::unique_ptr<WritableFileWriter> file_writer;
-    // Currently we only enable dictionary compression during compaction to the
-    // bottommost level.
-    CompressionOptions compression_opts_for_flush(compression_opts);
-    compression_opts_for_flush.max_dict_bytes = 0;
-    compression_opts_for_flush.zstd_max_train_bytes = 0;
     {
       std::unique_ptr<WritableFile> file;
 #ifndef NDEBUG
@@ -132,8 +127,8 @@ Status BuildTable(
       builder = NewTableBuilder(
           ioptions, mutable_cf_options, internal_comparator,
           int_tbl_prop_collector_factories, column_family_id,
-          column_family_name, file_writer.get(), compression,
-          compression_opts_for_flush, level, false /* skip_filters */,
+          column_family_name, file_writer.get(), compression, compression_opts,
+          level, nullptr /* compression_dict */, false /* skip_filters */,
           creation_time, oldest_key_time);
     }
 

db/builder.h

@@ -40,6 +40,8 @@ class InternalStats;
 // @param column_family_name Name of the column family that is also identified
 //    by column_family_id, or empty string if unknown. It must outlive the
 //    TableBuilder returned by this function.
+// @param compression_dict Data for presetting the compression library's
+//    dictionary, or nullptr.
 TableBuilder* NewTableBuilder(
     const ImmutableCFOptions& options, const MutableCFOptions& moptions,
     const InternalKeyComparator& internal_comparator,
@@ -48,8 +50,9 @@ TableBuilder* NewTableBuilder(
     uint32_t column_family_id, const std::string& column_family_name,
     WritableFileWriter* file, const CompressionType compression_type,
     const CompressionOptions& compression_opts, int level,
+    const std::string* compression_dict = nullptr,
     const bool skip_filters = false, const uint64_t creation_time = 0,
-    const uint64_t oldest_key_time = 0, const uint64_t target_file_size = 0);
+    const uint64_t oldest_key_time = 0);
 
 // Build a Table file from the contents of *iter.  The generated file
 // will be named according to number specified in meta. On success, the rest of

db/c.cc

@@ -208,10 +208,13 @@ struct rocksdb_compactionfilter_t : public CompactionFilter {
   const char* (*name_)(void*);
   unsigned char ignore_snapshots_;
 
-  ~rocksdb_compactionfilter_t() override { (*destructor_)(state_); }
+  virtual ~rocksdb_compactionfilter_t() {
+    (*destructor_)(state_);
+  }
 
-  bool Filter(int level, const Slice& key, const Slice& existing_value,
-              std::string* new_value, bool* value_changed) const override {
+  virtual bool Filter(int level, const Slice& key, const Slice& existing_value,
+                      std::string* new_value,
+                      bool* value_changed) const override {
     char* c_new_value = nullptr;
     size_t new_value_length = 0;
     unsigned char c_value_changed = 0;
@@ -228,9 +231,9 @@ struct rocksdb_compactionfilter_t : public CompactionFilter {
     return result;
   }
 
-  const char* Name() const override { return (*name_)(state_); }
+  virtual const char* Name() const override { return (*name_)(state_); }
 
-  bool IgnoreSnapshots() const override { return ignore_snapshots_; }
+  virtual bool IgnoreSnapshots() const override { return ignore_snapshots_; }
 };
 
 struct rocksdb_compactionfilterfactory_t : public CompactionFilterFactory {
@@ -240,9 +243,9 @@ struct rocksdb_compactionfilterfactory_t : public CompactionFilterFactory {
       void*, rocksdb_compactionfiltercontext_t* context);
   const char* (*name_)(void*);
 
-  ~rocksdb_compactionfilterfactory_t() override { (*destructor_)(state_); }
+  virtual ~rocksdb_compactionfilterfactory_t() { (*destructor_)(state_); }
 
-  std::unique_ptr<CompactionFilter> CreateCompactionFilter(
+  virtual std::unique_ptr<CompactionFilter> CreateCompactionFilter(
       const CompactionFilter::Context& context) override {
     rocksdb_compactionfiltercontext_t ccontext;
     ccontext.rep = context;
@@ -250,7 +253,7 @@ struct rocksdb_compactionfilterfactory_t : public CompactionFilterFactory {
     return std::unique_ptr<CompactionFilter>(cf);
   }
 
-  const char* Name() const override { return (*name_)(state_); }
+  virtual const char* Name() const override { return (*name_)(state_); }
 };
 
 struct rocksdb_comparator_t : public Comparator {
@@ -262,17 +265,20 @@ struct rocksdb_comparator_t : public Comparator {
       const char* b, size_t blen);
   const char* (*name_)(void*);
 
-  ~rocksdb_comparator_t() override { (*destructor_)(state_); }
+  virtual ~rocksdb_comparator_t() {
+    (*destructor_)(state_);
+  }
 
-  int Compare(const Slice& a, const Slice& b) const override {
+  virtual int Compare(const Slice& a, const Slice& b) const override {
     return (*compare_)(state_, a.data(), a.size(), b.data(), b.size());
   }
 
-  const char* Name() const override { return (*name_)(state_); }
+  virtual const char* Name() const override { return (*name_)(state_); }
 
   // No-ops since the C binding does not support key shortening methods.
-  void FindShortestSeparator(std::string*, const Slice&) const override {}
-  void FindShortSuccessor(std::string* /*key*/) const override {}
+  virtual void FindShortestSeparator(std::string*,
+                                     const Slice&) const override {}
+  virtual void FindShortSuccessor(std::string* /*key*/) const override {}
 };
 
 struct rocksdb_filterpolicy_t : public FilterPolicy {
@@ -292,11 +298,14 @@ struct rocksdb_filterpolicy_t : public FilterPolicy {
       void*,
       const char* filter, size_t filter_length);
 
-  ~rocksdb_filterpolicy_t() override { (*destructor_)(state_); }
+  virtual ~rocksdb_filterpolicy_t() {
+    (*destructor_)(state_);
+  }
 
-  const char* Name() const override { return (*name_)(state_); }
+  virtual const char* Name() const override { return (*name_)(state_); }
 
-  void CreateFilter(const Slice* keys, int n, std::string* dst) const override {
+  virtual void CreateFilter(const Slice* keys, int n,
+                            std::string* dst) const override {
     std::vector<const char*> key_pointers(n);
     std::vector<size_t> key_sizes(n);
     for (int i = 0; i < n; i++) {
@@ -314,7 +323,8 @@ struct rocksdb_filterpolicy_t : public FilterPolicy {
     }
   }
 
-  bool KeyMayMatch(const Slice& key, const Slice& filter) const override {
+  virtual bool KeyMayMatch(const Slice& key,
+                           const Slice& filter) const override {
     return (*key_match_)(state_, key.data(), key.size(),
                          filter.data(), filter.size());
   }
@@ -339,12 +349,14 @@ struct rocksdb_mergeoperator_t : public MergeOperator {
       void*,
       const char* value, size_t value_length);
 
-  ~rocksdb_mergeoperator_t() override { (*destructor_)(state_); }
+  virtual ~rocksdb_mergeoperator_t() {
+    (*destructor_)(state_);
+  }
 
-  const char* Name() const override { return (*name_)(state_); }
+  virtual const char* Name() const override { return (*name_)(state_); }
 
-  bool FullMergeV2(const MergeOperationInput& merge_in,
-                   MergeOperationOutput* merge_out) const override {
+  virtual bool FullMergeV2(const MergeOperationInput& merge_in,
+                           MergeOperationOutput* merge_out) const override {
     size_t n = merge_in.operand_list.size();
     std::vector<const char*> operand_pointers(n);
     std::vector<size_t> operand_sizes(n);
@@ -378,10 +390,10 @@ struct rocksdb_mergeoperator_t : public MergeOperator {
     return success;
   }
 
-  bool PartialMergeMulti(const Slice& key,
-                         const std::deque<Slice>& operand_list,
-                         std::string* new_value,
-                         Logger* /*logger*/) const override {
+  virtual bool PartialMergeMulti(const Slice& key,
+                                 const std::deque<Slice>& operand_list,
+                                 std::string* new_value,
+                                 Logger* /*logger*/) const override {
     size_t operand_count = operand_list.size();
     std::vector<const char*> operand_pointers(operand_count);
     std::vector<size_t> operand_sizes(operand_count);
@@ -432,21 +444,23 @@ struct rocksdb_slicetransform_t : public SliceTransform {
       void*,
       const char* key, size_t length);
 
-  ~rocksdb_slicetransform_t() override { (*destructor_)(state_); }
+  virtual ~rocksdb_slicetransform_t() {
+    (*destructor_)(state_);
+  }
 
-  const char* Name() const override { return (*name_)(state_); }
+  virtual const char* Name() const override { return (*name_)(state_); }
 
-  Slice Transform(const Slice& src) const override {
+  virtual Slice Transform(const Slice& src) const override {
     size_t len;
     char* dst = (*transform_)(state_, src.data(), src.size(), &len);
     return Slice(dst, len);
   }
 
-  bool InDomain(const Slice& src) const override {
+  virtual bool InDomain(const Slice& src) const override {
     return (*in_domain_)(state_, src.data(), src.size());
   }
 
-  bool InRange(const Slice& src) const override {
+  virtual bool InRange(const Slice& src) const override {
     return (*in_range_)(state_, src.data(), src.size());
   }
 };
@@ -1481,10 +1495,10 @@ class H : public WriteBatch::Handler {
   void* state_;
   void (*put_)(void*, const char* k, size_t klen, const char* v, size_t vlen);
   void (*deleted_)(void*, const char* k, size_t klen);
-  void Put(const Slice& key, const Slice& value) override {
+  virtual void Put(const Slice& key, const Slice& value) override {
     (*put_)(state_, key.data(), key.size(), value.data(), value.size());
   }
-  void Delete(const Slice& key) override {
+  virtual void Delete(const Slice& key) override {
     (*deleted_)(state_, key.data(), key.size());
   }
 };
@@ -2852,7 +2866,7 @@ rocksdb_compactionfilter_t* rocksdb_compactionfilter_create(
   result->state_ = state;
   result->destructor_ = destructor;
   result->filter_ = filter;
-  result->ignore_snapshots_ = true;
+  result->ignore_snapshots_ = false;
   result->name_ = name;
   return result;
 }
@@ -2952,7 +2966,7 @@ rocksdb_filterpolicy_t* rocksdb_filterpolicy_create_bloom_format(int bits_per_ke
   // supplied C functions.
   struct Wrapper : public rocksdb_filterpolicy_t {
     const FilterPolicy* rep_;
-    ~Wrapper() override { delete rep_; }
+    ~Wrapper() { delete rep_; }
     const char* Name() const override { return rep_->Name(); }
     void CreateFilter(const Slice* keys, int n,
                       std::string* dst) const override {
@@ -3407,7 +3421,7 @@ void rocksdb_slicetransform_destroy(rocksdb_slicetransform_t* st) {
 
 struct Wrapper : public rocksdb_slicetransform_t {
   const SliceTransform* rep_;
-  ~Wrapper() override { delete rep_; }
+  ~Wrapper() { delete rep_; }
   const char* Name() const override { return rep_->Name(); }
   Slice Transform(const Slice& src) const override {
     return rep_->Transform(src);

db/column_family_test.cc

@@ -14,7 +14,6 @@
 
 #include "db/db_impl.h"
 #include "db/db_test_util.h"
-#include "memtable/hash_skiplist_rep.h"
 #include "options/options_parser.h"
 #include "port/port.h"
 #include "rocksdb/db.h"
@@ -69,7 +68,7 @@ class ColumnFamilyTestBase : public testing::Test {
     DestroyDB(dbname_, Options(db_options_, column_family_options_));
   }
 
-  ~ColumnFamilyTestBase() override {
+  virtual ~ColumnFamilyTestBase() {
     std::vector<ColumnFamilyDescriptor> column_families;
     for (auto h : handles_) {
       ColumnFamilyDescriptor cfdescriptor;
@@ -1207,32 +1206,29 @@ TEST_P(ColumnFamilyTest, DifferentWriteBufferSizes) {
 }
 #endif  // !ROCKSDB_LITE
 
-// The test is commented out because we want to test that snapshot is
-// not created for memtables not supported it, but There isn't a memtable
-// that doesn't support snapshot right now. If we have one later, we can
-// re-enable the test.
-//
-// #ifndef ROCKSDB_LITE  // Cuckoo is not supported in lite
-//   TEST_P(ColumnFamilyTest, MemtableNotSupportSnapshot) {
-//   db_options_.allow_concurrent_memtable_write = false;
-//   Open();
-//   auto* s1 = dbfull()->GetSnapshot();
-//   ASSERT_TRUE(s1 != nullptr);
-//   dbfull()->ReleaseSnapshot(s1);
+#ifndef ROCKSDB_LITE  // Cuckoo is not supported in lite
+TEST_P(ColumnFamilyTest, MemtableNotSupportSnapshot) {
+  db_options_.allow_concurrent_memtable_write = false;
+  Open();
+  auto* s1 = dbfull()->GetSnapshot();
+  ASSERT_TRUE(s1 != nullptr);
+  dbfull()->ReleaseSnapshot(s1);
 
-//   // Add a column family that doesn't support snapshot
-//   ColumnFamilyOptions first;
-//   first.memtable_factory.reset(new DummyMemtableNotSupportingSnapshot());
-//   CreateColumnFamilies({"first"}, {first});
-//   auto* s2 = dbfull()->GetSnapshot();
-//   ASSERT_TRUE(s2 == nullptr);
+  // Add a column family that doesn't support snapshot
+  ColumnFamilyOptions first;
+  first.memtable_factory.reset(NewHashCuckooRepFactory(1024 * 1024));
+  CreateColumnFamilies({"first"}, {first});
+  auto* s2 = dbfull()->GetSnapshot();
+  ASSERT_TRUE(s2 == nullptr);
 
-//   // Add a column family that supports snapshot. Snapshot stays not
-//   supported. ColumnFamilyOptions second; CreateColumnFamilies({"second"},
-//   {second}); auto* s3 = dbfull()->GetSnapshot(); ASSERT_TRUE(s3 == nullptr);
-//   Close();
-// }
-// #endif  // !ROCKSDB_LITE
+  // Add a column family that supports snapshot. Snapshot stays not supported.
+  ColumnFamilyOptions second;
+  CreateColumnFamilies({"second"}, {second});
+  auto* s3 = dbfull()->GetSnapshot();
+  ASSERT_TRUE(s3 == nullptr);
+  Close();
+}
+#endif  // !ROCKSDB_LITE
 
 class TestComparator : public Comparator {
   int Compare(const rocksdb::Slice& /*a*/,

db/compact_files_test.cc

@@ -35,9 +35,9 @@ class CompactFilesTest : public testing::Test {
 class FlushedFileCollector : public EventListener {
  public:
   FlushedFileCollector() {}
-  ~FlushedFileCollector() override {}
+  ~FlushedFileCollector() {}
 
-  void OnFlushCompleted(DB* /*db*/, const FlushJobInfo& info) override {
+  virtual void OnFlushCompleted(DB* /*db*/, const FlushJobInfo& info) override {
     std::lock_guard<std::mutex> lock(mutex_);
     flushed_files_.push_back(info.file_path);
   }
@@ -256,9 +256,9 @@ TEST_F(CompactFilesTest, CapturingPendingFiles) {
 TEST_F(CompactFilesTest, CompactionFilterWithGetSv) {
   class FilterWithGet : public CompactionFilter {
    public:
-    bool Filter(int /*level*/, const Slice& /*key*/, const Slice& /*value*/,
-                std::string* /*new_value*/,
-                bool* /*value_changed*/) const override {
+    virtual bool Filter(int /*level*/, const Slice& /*key*/,
+                        const Slice& /*value*/, std::string* /*new_value*/,
+                        bool* /*value_changed*/) const override {
       if (db_ == nullptr) {
         return true;
       }
@@ -271,7 +271,7 @@ TEST_F(CompactFilesTest, CompactionFilterWithGetSv) {
       db_ = db;
     }
 
-    const char* Name() const override { return "FilterWithGet"; }
+    virtual const char* Name() const override { return "FilterWithGet"; }
 
    private:
     DB* db_;

db/compaction.cc

@@ -250,12 +250,6 @@ Compaction::Compaction(VersionStorageInfo* vstorage,
   if (max_subcompactions_ == 0) {
     max_subcompactions_ = immutable_cf_options_.max_subcompactions;
   }
-  if (!bottommost_level_) {
-    // Currently we only enable dictionary compression during compaction to the
-    // bottommost level.
-    output_compression_opts_.max_dict_bytes = 0;
-    output_compression_opts_.zstd_max_train_bytes = 0;
-  }
 
 #ifndef NDEBUG
   for (size_t i = 1; i < inputs_.size(); ++i) {

db/compaction_iterator.cc

@@ -72,6 +72,7 @@ CompactionIterator::CompactionIterator(
       compaction_filter_(compaction_filter),
       shutting_down_(shutting_down),
       preserve_deletes_seqnum_(preserve_deletes_seqnum),
+      ignore_snapshots_(false),
       current_user_key_sequence_(0),
       current_user_key_snapshot_(0),
       merge_out_iter_(merge_helper_),
@@ -101,6 +102,13 @@ CompactionIterator::CompactionIterator(
     assert(snapshots_->at(i - 1) < snapshots_->at(i));
   }
 #endif
+  if (compaction_filter_ != nullptr) {
+    if (compaction_filter_->IgnoreSnapshots()) {
+      ignore_snapshots_ = true;
+    }
+  } else {
+    ignore_snapshots_ = false;
+  }
   input_->SetPinnedItersMgr(&pinned_iters_mgr_);
   TEST_SYNC_POINT_CALLBACK("CompactionIterator:AfterInit", compaction_.get());
 }
@@ -172,7 +180,9 @@ void CompactionIterator::Next() {
 void CompactionIterator::InvokeFilterIfNeeded(bool* need_skip,
                                               Slice* skip_until) {
   if (compaction_filter_ != nullptr &&
-      (ikey_.type == kTypeValue || ikey_.type == kTypeBlobIndex)) {
+      (ikey_.type == kTypeValue || ikey_.type == kTypeBlobIndex) &&
+      (visible_at_tip_ || ignore_snapshots_ ||
+       DEFINITELY_NOT_IN_SNAPSHOT(ikey_.sequence, latest_snapshot_))) {
     // If the user has specified a compaction filter and the sequence
     // number is greater than any external snapshot, then invoke the
     // filter. If the return value of the compaction filter is true,
@@ -482,14 +492,11 @@ void CompactionIterator::NextFromInput() {
       // this value, and findEarliestVisibleSnapshot returns the next snapshot
       // as current_user_key_snapshot. In this case last value and current
       // value are both in current_user_key_snapshot currently.
-      // Although last_snapshot is released we might still get a definitive
-      // response when key sequence number changes, e.g., when seq is determined
-      // too old and visible in all snapshots.
       assert(last_snapshot == current_user_key_snapshot_ ||
              (snapshot_checker_ != nullptr &&
               snapshot_checker_->CheckInSnapshot(current_user_key_sequence_,
-                                                 last_snapshot) !=
-                  SnapshotCheckerResult::kNotInSnapshot));
+                                                 last_snapshot) ==
+                  SnapshotCheckerResult::kSnapshotReleased));
 
       ++iter_stats_.num_record_drop_hidden;  // (A)
       input_->Next();
@@ -627,7 +634,8 @@ void CompactionIterator::PrepareOutput() {
   // KeyNotExistsBeyondOutputLevel() return true?
   if ((compaction_ != nullptr && !compaction_->allow_ingest_behind()) &&
       ikeyNotNeededForIncrementalSnapshot() && bottommost_level_ && valid_ &&
-      IN_EARLIEST_SNAPSHOT(ikey_.sequence) && ikey_.type != kTypeMerge) {
+      IN_EARLIEST_SNAPSHOT(ikey_.sequence) && ikey_.type != kTypeMerge &&
+      !cmp_->Equal(compaction_->GetLargestUserKey(), ikey_.user_key)) {
     assert(ikey_.type != kTypeDeletion && ikey_.type != kTypeSingleDeletion);
     ikey_.sequence = 0;
     current_key_.UpdateInternalKey(0, ikey_.type);

db/compaction_iterator.h

@@ -168,6 +168,8 @@ class CompactionIterator {
   SequenceNumber earliest_snapshot_;
   SequenceNumber latest_snapshot_;
 
+  bool ignore_snapshots_;
+
   // State
   //
   // Points to a copy of the current compaction iterator output (current_key_)

db/compaction_iterator_test.cc

@@ -112,39 +112,39 @@ class LoggingForwardVectorIterator : public InternalIterator {
     assert(keys_.size() == values_.size());
   }
 
-  bool Valid() const override { return current_ < keys_.size(); }
+  virtual bool Valid() const override { return current_ < keys_.size(); }
 
-  void SeekToFirst() override {
+  virtual void SeekToFirst() override {
     log.emplace_back(Action::Type::SEEK_TO_FIRST);
     current_ = 0;
   }
-  void SeekToLast() override { assert(false); }
+  virtual void SeekToLast() override { assert(false); }
 
-  void Seek(const Slice& target) override {
+  virtual void Seek(const Slice& target) override {
     log.emplace_back(Action::Type::SEEK, target.ToString());
     current_ = std::lower_bound(keys_.begin(), keys_.end(), target.ToString()) -
                keys_.begin();
   }
 
-  void SeekForPrev(const Slice& /*target*/) override { assert(false); }
+  virtual void SeekForPrev(const Slice& /*target*/) override { assert(false); }
 
-  void Next() override {
+  virtual void Next() override {
     assert(Valid());
     log.emplace_back(Action::Type::NEXT);
     current_++;
   }
-  void Prev() override { assert(false); }
+  virtual void Prev() override { assert(false); }
 
-  Slice key() const override {
+  virtual Slice key() const override {
     assert(Valid());
     return Slice(keys_[current_]);
   }
-  Slice value() const override {
+  virtual Slice value() const override {
     assert(Valid());
     return Slice(values_[current_]);
   }
 
-  Status status() const override { return Status::OK(); }
+  virtual Status status() const override { return Status::OK(); }
 
   std::vector<Action> log;
 
@@ -158,20 +158,22 @@ class FakeCompaction : public CompactionIterator::CompactionProxy {
  public:
   FakeCompaction() = default;
 
-  int level(size_t /*compaction_input_level*/) const override { return 0; }
-  bool KeyNotExistsBeyondOutputLevel(
+  virtual int level(size_t /*compaction_input_level*/) const override {
+    return 0;
+  }
+  virtual bool KeyNotExistsBeyondOutputLevel(
       const Slice& /*user_key*/,
       std::vector<size_t>* /*level_ptrs*/) const override {
     return is_bottommost_level || key_not_exists_beyond_output_level;
   }
-  bool bottommost_level() const override { return is_bottommost_level; }
-  int number_levels() const override { return 1; }
-  Slice GetLargestUserKey() const override {
+  virtual bool bottommost_level() const override { return is_bottommost_level; }
+  virtual int number_levels() const override { return 1; }
+  virtual Slice GetLargestUserKey() const override {
     return "\xff\xff\xff\xff\xff\xff\xff\xff\xff";
   }
-  bool allow_ingest_behind() const override { return false; }
+  virtual bool allow_ingest_behind() const override { return false; }
 
-  bool preserve_deletes() const override { return false; }
+  virtual bool preserve_deletes() const override { return false; }
 
   bool key_not_exists_beyond_output_level = false;
 
@@ -375,9 +377,10 @@ TEST_P(CompactionIteratorTest, RangeDeletionWithSnapshots) {
 
 TEST_P(CompactionIteratorTest, CompactionFilterSkipUntil) {
   class Filter : public CompactionFilter {
-    Decision FilterV2(int /*level*/, const Slice& key, ValueType t,
-                      const Slice& existing_value, std::string* /*new_value*/,
-                      std::string* skip_until) const override {
+    virtual Decision FilterV2(int /*level*/, const Slice& key, ValueType t,
+                              const Slice& existing_value,
+                              std::string* /*new_value*/,
+                              std::string* skip_until) const override {
       std::string k = key.ToString();
       std::string v = existing_value.ToString();
       // See InitIterators() call below for the sequence of keys and their
@@ -557,9 +560,10 @@ TEST_P(CompactionIteratorTest, ShuttingDownInMerge) {
 
 TEST_P(CompactionIteratorTest, SingleMergeOperand) {
   class Filter : public CompactionFilter {
-    Decision FilterV2(int /*level*/, const Slice& key, ValueType t,
-                      const Slice& existing_value, std::string* /*new_value*/,
-                      std::string* /*skip_until*/) const override {
+    virtual Decision FilterV2(int /*level*/, const Slice& key, ValueType t,
+                              const Slice& existing_value,
+                              std::string* /*new_value*/,
+                              std::string* /*skip_until*/) const override {
       std::string k = key.ToString();
       std::string v = existing_value.ToString();
 

db/compaction_job.cc

@@ -157,6 +157,7 @@ struct CompactionJob::SubcompactionState {
   uint64_t overlapped_bytes = 0;
   // A flag determine whether the key has been seen in ShouldStopBefore()
   bool seen_key = false;
+  std::string compression_dict;
 
   SubcompactionState(Compaction* c, Slice* _start, Slice* _end,
                      uint64_t size = 0)
@@ -172,7 +173,8 @@ struct CompactionJob::SubcompactionState {
         approx_size(size),
         grandparent_index(0),
         overlapped_bytes(0),
-        seen_key(false) {
+        seen_key(false),
+        compression_dict() {
     assert(compaction != nullptr);
   }
 
@@ -195,6 +197,7 @@ struct CompactionJob::SubcompactionState {
     grandparent_index = std::move(o.grandparent_index);
     overlapped_bytes = std::move(o.overlapped_bytes);
     seen_key = std::move(o.seen_key);
+    compression_dict = std::move(o.compression_dict);
     return *this;
   }
 
@@ -813,24 +816,6 @@ void CompactionJob::ProcessKeyValueCompaction(SubcompactionState* sub_compact) {
   uint64_t prev_cpu_micros = env_->NowCPUNanos() / 1000;
 
   ColumnFamilyData* cfd = sub_compact->compaction->column_family_data();
-
-  // Create compaction filter and fail the compaction if
-  // IgnoreSnapshots() = false because it is not supported anymore
-  const CompactionFilter* compaction_filter =
-      cfd->ioptions()->compaction_filter;
-  std::unique_ptr<CompactionFilter> compaction_filter_from_factory = nullptr;
-  if (compaction_filter == nullptr) {
-    compaction_filter_from_factory =
-        sub_compact->compaction->CreateCompactionFilter();
-    compaction_filter = compaction_filter_from_factory.get();
-  }
-  if (compaction_filter != nullptr && !compaction_filter->IgnoreSnapshots()) {
-    sub_compact->status = Status::NotSupported(
-        "CompactionFilter::IgnoreSnapshots() = false is not supported "
-        "anymore.");
-    return;
-  }
-
   CompactionRangeDelAggregator range_del_agg(&cfd->internal_comparator(),
                                              existing_snapshots_);
 
@@ -862,6 +847,49 @@ void CompactionJob::ProcessKeyValueCompaction(SubcompactionState* sub_compact) {
     prev_cpu_read_nanos = IOSTATS(cpu_read_nanos);
   }
 
+  const MutableCFOptions* mutable_cf_options =
+      sub_compact->compaction->mutable_cf_options();
+
+  // To build compression dictionary, we sample the first output file, assuming
+  // it'll reach the maximum length. We optionally pass these samples through
+  // zstd's dictionary trainer, or just use them directly. Then, the dictionary
+  // is used for compressing subsequent output files in the same subcompaction.
+  const bool kUseZstdTrainer =
+      sub_compact->compaction->output_compression_opts().zstd_max_train_bytes >
+      0;
+  const size_t kSampleBytes =
+      kUseZstdTrainer
+          ? sub_compact->compaction->output_compression_opts()
+                .zstd_max_train_bytes
+          : sub_compact->compaction->output_compression_opts().max_dict_bytes;
+  const int kSampleLenShift = 6;  // 2^6 = 64-byte samples
+  std::set<size_t> sample_begin_offsets;
+  if (bottommost_level_ && kSampleBytes > 0) {
+    const size_t kMaxSamples = kSampleBytes >> kSampleLenShift;
+    const size_t kOutFileLen =
+        static_cast<size_t>(MaxFileSizeForLevel(*mutable_cf_options,
+            compact_->compaction->output_level(),
+            cfd->ioptions()->compaction_style,
+            compact_->compaction->GetInputBaseLevel(),
+            cfd->ioptions()->level_compaction_dynamic_level_bytes));
+    if (kOutFileLen != port::kMaxSizet) {
+      const size_t kOutFileNumSamples = kOutFileLen >> kSampleLenShift;
+      Random64 generator{versions_->NewFileNumber()};
+      for (size_t i = 0; i < kMaxSamples; ++i) {
+        sample_begin_offsets.insert(
+            static_cast<size_t>(generator.Uniform(kOutFileNumSamples))
+            << kSampleLenShift);
+      }
+    }
+  }
+
+  auto compaction_filter = cfd->ioptions()->compaction_filter;
+  std::unique_ptr<CompactionFilter> compaction_filter_from_factory = nullptr;
+  if (compaction_filter == nullptr) {
+    compaction_filter_from_factory =
+        sub_compact->compaction->CreateCompactionFilter();
+    compaction_filter = compaction_filter_from_factory.get();
+  }
   MergeHelper merge(
       env_, cfd->user_comparator(), cfd->ioptions()->merge_operator,
       compaction_filter, db_options_.info_log.get(),
@@ -899,6 +927,12 @@ void CompactionJob::ProcessKeyValueCompaction(SubcompactionState* sub_compact) {
                                   sub_compact->current_output_file_size);
   }
   const auto& c_iter_stats = c_iter->iter_stats();
+  auto sample_begin_offset_iter = sample_begin_offsets.cbegin();
+  // data_begin_offset and dict_sample_data are only valid while generating
+  // dictionary from the first output file.
+  size_t data_begin_offset = 0;
+  std::string dict_sample_data;
+  dict_sample_data.reserve(kSampleBytes);
 
   while (status.ok() && !cfd->IsDropped() && c_iter->Valid()) {
     // Invariant: c_iter.status() is guaranteed to be OK if c_iter->Valid()
@@ -934,6 +968,55 @@ void CompactionJob::ProcessKeyValueCompaction(SubcompactionState* sub_compact) {
         key, c_iter->ikey().sequence);
     sub_compact->num_output_records++;
 
+    if (sub_compact->outputs.size() == 1) {  // first output file
+      // Check if this key/value overlaps any sample intervals; if so, appends
+      // overlapping portions to the dictionary.
+      for (const auto& data_elmt : {key, value}) {
+        size_t data_end_offset = data_begin_offset + data_elmt.size();
+        while (sample_begin_offset_iter != sample_begin_offsets.cend() &&
+               *sample_begin_offset_iter < data_end_offset) {
+          size_t sample_end_offset =
+              *sample_begin_offset_iter + (1 << kSampleLenShift);
+          // Invariant: Because we advance sample iterator while processing the
+          // data_elmt containing the sample's last byte, the current sample
+          // cannot end before the current data_elmt.
+          assert(data_begin_offset < sample_end_offset);
+
+          size_t data_elmt_copy_offset, data_elmt_copy_len;
+          if (*sample_begin_offset_iter <= data_begin_offset) {
+            // The sample starts before data_elmt starts, so take bytes starting
+            // at the beginning of data_elmt.
+            data_elmt_copy_offset = 0;
+          } else {
+            // data_elmt starts before the sample starts, so take bytes starting
+            // at the below offset into data_elmt.
+            data_elmt_copy_offset =
+                *sample_begin_offset_iter - data_begin_offset;
+          }
+          if (sample_end_offset <= data_end_offset) {
+            // The sample ends before data_elmt ends, so take as many bytes as
+            // needed.
+            data_elmt_copy_len =
+                sample_end_offset - (data_begin_offset + data_elmt_copy_offset);
+          } else {
+            // data_elmt ends before the sample ends, so take all remaining
+            // bytes in data_elmt.
+            data_elmt_copy_len =
+                data_end_offset - (data_begin_offset + data_elmt_copy_offset);
+          }
+          dict_sample_data.append(&data_elmt.data()[data_elmt_copy_offset],
+                                  data_elmt_copy_len);
+          if (sample_end_offset > data_end_offset) {
+            // Didn't finish sample. Try to finish it with the next data_elmt.
+            break;
+          }
+          // Next sample may require bytes from same data_elmt.
+          sample_begin_offset_iter++;
+        }
+        data_begin_offset = data_end_offset;
+      }
+    }
+
     // Close output file if it is big enough. Two possibilities determine it's
     // time to close it: (1) the current key should be this file's last key, (2)
     // the next key should not be in this file.
@@ -975,6 +1058,18 @@ void CompactionJob::ProcessKeyValueCompaction(SubcompactionState* sub_compact) {
                                      &range_del_out_stats, next_key);
       RecordDroppedKeys(range_del_out_stats,
                         &sub_compact->compaction_job_stats);
+      if (sub_compact->outputs.size() == 1) {
+        // Use samples from first output file to create dictionary for
+        // compression of subsequent files.
+        if (kUseZstdTrainer) {
+          sub_compact->compression_dict = ZSTD_TrainDictionary(
+              dict_sample_data, kSampleLenShift,
+              sub_compact->compaction->output_compression_opts()
+                  .max_dict_bytes);
+        } else {
+          sub_compact->compression_dict = std::move(dict_sample_data);
+        }
+      }
     }
   }
 
@@ -1133,19 +1228,10 @@ Status CompactionJob::FinishCompactionOutputFile(
       lower_bound = nullptr;
     }
     if (next_table_min_key != nullptr) {
-      // This may be the last file in the subcompaction in some cases, so we
-      // need to compare the end key of subcompaction with the next file start
-      // key. When the end key is chosen by the subcompaction, we know that
-      // it must be the biggest key in output file. Therefore, it is safe to
-      // use the smaller key as the upper bound of the output file, to ensure
-      // that there is no overlapping between different output files.
+      // This isn't the last file in the subcompaction, so extend until the next
+      // file starts.
       upper_bound_guard = ExtractUserKey(*next_table_min_key);
-      if (sub_compact->end != nullptr &&
-          ucmp->Compare(upper_bound_guard, *sub_compact->end) >= 0) {
-        upper_bound = sub_compact->end;
-      } else {
-        upper_bound = &upper_bound_guard;
-      }
+      upper_bound = &upper_bound_guard;
     } else {
       // This is the last file in the subcompaction, so extend until the
       // subcompaction ends.
@@ -1163,13 +1249,6 @@ Status CompactionJob::FinishCompactionOutputFile(
       has_overlapping_endpoints = false;
     }
 
-    // The end key of the subcompaction must be bigger or equal to the upper
-    // bound. If the end of subcompaction is null or the upper bound is null,
-    // it means that this file is the last file in the compaction. So there
-    // will be no overlapping between this file and others.
-    assert(sub_compact->end == nullptr ||
-           upper_bound == nullptr ||
-           ucmp->Compare(*upper_bound , *sub_compact->end) <= 0);
     auto it = range_del_agg->NewIterator(lower_bound, upper_bound,
                                          has_overlapping_endpoints);
     // Position the range tombstone output iterator. There may be tombstone
@@ -1500,9 +1579,8 @@ Status CompactionJob::OpenCompactionOutputFile(
       cfd->GetID(), cfd->GetName(), sub_compact->outfile.get(),
       sub_compact->compaction->output_compression(),
       sub_compact->compaction->output_compression_opts(),
-      sub_compact->compaction->output_level(), skip_filters,
-      output_file_creation_time, 0 /* oldest_key_time */,
-      sub_compact->compaction->max_output_file_size()));
+      sub_compact->compaction->output_level(), &sub_compact->compression_dict,
+      skip_filters, output_file_creation_time));
   LogFlush(db_options_.info_log);
   return s;
 }

db/compaction_job_stats_test.cc

@@ -113,7 +113,7 @@ class CompactionJobStatsTest : public testing::Test,
     Reopen(options);
   }
 
-  ~CompactionJobStatsTest() override {
+  ~CompactionJobStatsTest() {
     rocksdb::SyncPoint::GetInstance()->DisableProcessing();
     rocksdb::SyncPoint::GetInstance()->LoadDependency({});
     rocksdb::SyncPoint::GetInstance()->ClearAllCallBacks();
@@ -426,7 +426,7 @@ class CompactionJobStatsChecker : public EventListener {
   // Once a compaction completed, this function will verify the returned
   // CompactionJobInfo with the oldest CompactionJobInfo added earlier
   // in "expected_stats_" which has not yet being used for verification.
-  void OnCompactionCompleted(DB* /*db*/, const CompactionJobInfo& ci) override {
+  virtual void OnCompactionCompleted(DB* /*db*/, const CompactionJobInfo& ci) {
     if (verify_next_comp_io_stats_) {
       ASSERT_GT(ci.stats.file_write_nanos, 0);
       ASSERT_GT(ci.stats.file_range_sync_nanos, 0);
@@ -523,7 +523,7 @@ class CompactionJobDeletionStatsChecker : public CompactionJobStatsChecker {
  public:
   // Verifies whether two CompactionJobStats match.
   void Verify(const CompactionJobStats& current_stats,
-              const CompactionJobStats& stats) override {
+              const CompactionJobStats& stats) {
     ASSERT_EQ(
       current_stats.num_input_deletion_records,
       stats.num_input_deletion_records);

db/compaction_job_test.cc

@@ -172,7 +172,7 @@ class CompactionJobTest : public testing::Test {
         // This is how the key will look like once it's written in bottommost
         // file
         InternalKey bottommost_internal_key(
-            key, 0, kTypeValue);
+            key, (key == "9999") ? sequence_number : 0, kTypeValue);
 
         if (corrupt_id(k)) {
           test::CorruptKeyType(&internal_key);
@@ -379,7 +379,7 @@ TEST_F(CompactionJobTest, SimpleOverwrite) {
 
   auto expected_results =
       mock::MakeMockFile({{KeyStr("a", 0U, kTypeValue), "val2"},
-                          {KeyStr("b", 0U, kTypeValue), "val3"}});
+                          {KeyStr("b", 4U, kTypeValue), "val3"}});
 
   SetLastSequence(4U);
   auto files = cfd_->current()->storage_info()->LevelFiles(0);
@@ -432,7 +432,7 @@ TEST_F(CompactionJobTest, SimpleMerge) {
 
   auto expected_results =
       mock::MakeMockFile({{KeyStr("a", 0U, kTypeValue), "3,4,5"},
-                          {KeyStr("b", 0U, kTypeValue), "1,2"}});
+                          {KeyStr("b", 2U, kTypeValue), "1,2"}});
 
   SetLastSequence(5U);
   auto files = cfd_->current()->storage_info()->LevelFiles(0);
@@ -456,7 +456,7 @@ TEST_F(CompactionJobTest, NonAssocMerge) {
 
   auto expected_results =
       mock::MakeMockFile({{KeyStr("a", 0U, kTypeValue), "3,4,5"},
-                          {KeyStr("b", 0U, kTypeValue), "1,2"}});
+                          {KeyStr("b", 2U, kTypeValue), "1,2"}});
 
   SetLastSequence(5U);
   auto files = cfd_->current()->storage_info()->LevelFiles(0);
@@ -483,7 +483,7 @@ TEST_F(CompactionJobTest, MergeOperandFilter) {
 
   auto expected_results =
       mock::MakeMockFile({{KeyStr("a", 0U, kTypeValue), test::EncodeInt(8U)},
-                          {KeyStr("b", 0U, kTypeValue), test::EncodeInt(2U)}});
+                          {KeyStr("b", 2U, kTypeValue), test::EncodeInt(2U)}});
 
   SetLastSequence(5U);
   auto files = cfd_->current()->storage_info()->LevelFiles(0);
@@ -746,7 +746,7 @@ TEST_F(CompactionJobTest, SingleDeleteZeroSeq) {
   AddMockFile(file2);
 
   auto expected_results = mock::MakeMockFile({
-      {KeyStr("dummy", 0U, kTypeValue), "val2"},
+      {KeyStr("dummy", 5U, kTypeValue), "val2"},
   });
 
   SetLastSequence(22U);
@@ -930,7 +930,7 @@ TEST_F(CompactionJobTest, CorruptionAfterDeletion) {
       mock::MakeMockFile({{test::KeyStr("A", 0U, kTypeValue), "val3"},
                           {test::KeyStr("a", 0U, kTypeValue, true), "val"},
                           {test::KeyStr("b", 0U, kTypeValue, true), "val"},
-                          {test::KeyStr("c", 0U, kTypeValue), "val2"}});
+                          {test::KeyStr("c", 1U, kTypeValue), "val2"}});
 
   SetLastSequence(6U);
   auto files = cfd_->current()->storage_info()->LevelFiles(0);

db/compaction_picker_fifo.cc

@@ -41,7 +41,7 @@ bool FIFOCompactionPicker::NeedsCompaction(
 Compaction* FIFOCompactionPicker::PickTTLCompaction(
     const std::string& cf_name, const MutableCFOptions& mutable_cf_options,
     VersionStorageInfo* vstorage, LogBuffer* log_buffer) {
-  assert(mutable_cf_options.ttl > 0);
+  assert(mutable_cf_options.compaction_options_fifo.ttl > 0);
 
   const int kLevel0 = 0;
   const std::vector<FileMetaData*>& level_files = vstorage->LevelFiles(kLevel0);
@@ -63,7 +63,7 @@ Compaction* FIFOCompactionPicker::PickTTLCompaction(
   inputs[0].level = 0;
 
   // avoid underflow
-  if (current_time > mutable_cf_options.ttl) {
+  if (current_time > mutable_cf_options.compaction_options_fifo.ttl) {
     for (auto ritr = level_files.rbegin(); ritr != level_files.rend(); ++ritr) {
       auto f = *ritr;
       if (f->fd.table_reader != nullptr &&
@@ -71,7 +71,8 @@ Compaction* FIFOCompactionPicker::PickTTLCompaction(
         auto creation_time =
             f->fd.table_reader->GetTableProperties()->creation_time;
         if (creation_time == 0 ||
-            creation_time >= (current_time - mutable_cf_options.ttl)) {
+            creation_time >= (current_time -
+                              mutable_cf_options.compaction_options_fifo.ttl)) {
           break;
         }
         total_size -= f->compensated_file_size;
@@ -200,7 +201,7 @@ Compaction* FIFOCompactionPicker::PickCompaction(
   assert(vstorage->num_levels() == 1);
 
   Compaction* c = nullptr;
-  if (mutable_cf_options.ttl > 0) {
+  if (mutable_cf_options.compaction_options_fifo.ttl > 0) {
     c = PickTTLCompaction(cf_name, mutable_cf_options, vstorage, log_buffer);
   }
   if (c == nullptr) {

db/compaction_picker_test.cc

@@ -22,7 +22,9 @@ namespace rocksdb {
 class CountingLogger : public Logger {
  public:
   using Logger::Logv;
-  void Logv(const char* /*format*/, va_list /*ap*/) override { log_count++; }
+  virtual void Logv(const char* /*format*/, va_list /*ap*/) override {
+    log_count++;
+  }
   size_t log_count;
 };
 
@@ -66,7 +68,8 @@ class CompactionPickerTest : public testing::Test {
                                     std::numeric_limits<uint64_t>::max());
   }
 
-  ~CompactionPickerTest() override {}
+  ~CompactionPickerTest() {
+  }
 
   void NewVersionStorage(int num_levels, CompactionStyle style) {
     DeleteVersionStorage();

db/comparator_db_test.cc

@@ -27,24 +27,24 @@ class KVIter : public Iterator {
  public:
   explicit KVIter(const stl_wrappers::KVMap* map)
       : map_(map), iter_(map_->end()) {}
-  bool Valid() const override { return iter_ != map_->end(); }
-  void SeekToFirst() override { iter_ = map_->begin(); }
-  void SeekToLast() override {
+  virtual bool Valid() const override { return iter_ != map_->end(); }
+  virtual void SeekToFirst() override { iter_ = map_->begin(); }
+  virtual void SeekToLast() override {
     if (map_->empty()) {
       iter_ = map_->end();
     } else {
       iter_ = map_->find(map_->rbegin()->first);
     }
   }
-  void Seek(const Slice& k) override {
+  virtual void Seek(const Slice& k) override {
     iter_ = map_->lower_bound(k.ToString());
   }
-  void SeekForPrev(const Slice& k) override {
+  virtual void SeekForPrev(const Slice& k) override {
     iter_ = map_->upper_bound(k.ToString());
     Prev();
   }
-  void Next() override { ++iter_; }
-  void Prev() override {
+  virtual void Next() override { ++iter_; }
+  virtual void Prev() override {
     if (iter_ == map_->begin()) {
       iter_ = map_->end();
       return;
@@ -52,9 +52,9 @@ class KVIter : public Iterator {
     --iter_;
   }
 
-  Slice key() const override { return iter_->first; }
-  Slice value() const override { return iter_->second; }
-  Status status() const override { return Status::OK(); }
+  virtual Slice key() const override { return iter_->first; }
+  virtual Slice value() const override { return iter_->second; }
+  virtual Status status() const override { return Status::OK(); }
 
  private:
   const stl_wrappers::KVMap* const map_;
@@ -171,9 +171,9 @@ class DoubleComparator : public Comparator {
  public:
   DoubleComparator() {}
 
-  const char* Name() const override { return "DoubleComparator"; }
+  virtual const char* Name() const override { return "DoubleComparator"; }
 
-  int Compare(const Slice& a, const Slice& b) const override {
+  virtual int Compare(const Slice& a, const Slice& b) const override {
 #ifndef CYGWIN
     double da = std::stod(a.ToString());
     double db = std::stod(b.ToString());
@@ -189,19 +189,19 @@ class DoubleComparator : public Comparator {
       return -1;
     }
   }
-  void FindShortestSeparator(std::string* /*start*/,
-                             const Slice& /*limit*/) const override {}
+  virtual void FindShortestSeparator(std::string* /*start*/,
+                                     const Slice& /*limit*/) const override {}
 
-  void FindShortSuccessor(std::string* /*key*/) const override {}
+  virtual void FindShortSuccessor(std::string* /*key*/) const override {}
 };
 
 class HashComparator : public Comparator {
  public:
   HashComparator() {}
 
-  const char* Name() const override { return "HashComparator"; }
+  virtual const char* Name() const override { return "HashComparator"; }
 
-  int Compare(const Slice& a, const Slice& b) const override {
+  virtual int Compare(const Slice& a, const Slice& b) const override {
     uint32_t ha = Hash(a.data(), a.size(), 66);
     uint32_t hb = Hash(b.data(), b.size(), 66);
     if (ha == hb) {
@@ -212,19 +212,19 @@ class HashComparator : public Comparator {
       return -1;
     }
   }
-  void FindShortestSeparator(std::string* /*start*/,
-                             const Slice& /*limit*/) const override {}
+  virtual void FindShortestSeparator(std::string* /*start*/,
+                                     const Slice& /*limit*/) const override {}
 
-  void FindShortSuccessor(std::string* /*key*/) const override {}
+  virtual void FindShortSuccessor(std::string* /*key*/) const override {}
 };
 
 class TwoStrComparator : public Comparator {
  public:
   TwoStrComparator() {}
 
-  const char* Name() const override { return "TwoStrComparator"; }
+  virtual const char* Name() const override { return "TwoStrComparator"; }
 
-  int Compare(const Slice& a, const Slice& b) const override {
+  virtual int Compare(const Slice& a, const Slice& b) const override {
     assert(a.size() >= 2);
     assert(b.size() >= 2);
     size_t size_a1 = static_cast<size_t>(a[0]);
@@ -244,10 +244,10 @@ class TwoStrComparator : public Comparator {
     }
     return a2.compare(b2);
   }
-  void FindShortestSeparator(std::string* /*start*/,
-                             const Slice& /*limit*/) const override {}
+  virtual void FindShortestSeparator(std::string* /*start*/,
+                                     const Slice& /*limit*/) const override {}
 
-  void FindShortSuccessor(std::string* /*key*/) const override {}
+  virtual void FindShortSuccessor(std::string* /*key*/) const override {}
 };
 }  // namespace
 
@@ -272,7 +272,7 @@ class ComparatorDBTest
     EXPECT_OK(DestroyDB(dbname_, last_options_));
   }
 
-  ~ComparatorDBTest() override {
+  ~ComparatorDBTest() {
     delete db_;
     EXPECT_OK(DestroyDB(dbname_, last_options_));
     comparator = BytewiseComparator();

db/corruption_test.cc

@@ -62,9 +62,9 @@ class CorruptionTest : public testing::Test {
     options_.create_if_missing = false;
   }
 
-  ~CorruptionTest() override {
-    delete db_;
-    DestroyDB(dbname_, Options());
+  ~CorruptionTest() {
+     delete db_;
+     DestroyDB(dbname_, Options());
   }
 
   void CloseDb() {

db/cuckoo_table_db_test.cc

@@ -31,7 +31,7 @@ class CuckooTableDBTest : public testing::Test {
     Reopen();
   }
 
-  ~CuckooTableDBTest() override {
+  ~CuckooTableDBTest() {
     delete db_;
     EXPECT_OK(DestroyDB(dbname_, Options()));
   }

db/db_basic_test.cc

@@ -215,11 +215,11 @@ TEST_F(DBBasicTest, PutSingleDeleteGet) {
     ASSERT_EQ("v2", Get(1, "foo2"));
     ASSERT_OK(SingleDelete(1, "foo"));
     ASSERT_EQ("NOT_FOUND", Get(1, "foo"));
-    // Ski FIFO and universal compaction because they do not apply to the test
-    // case. Skip MergePut because single delete does not get removed when it
-    // encounters a merge.
-  } while (ChangeOptions(kSkipFIFOCompaction | kSkipUniversalCompaction |
-                         kSkipMergePut));
+    // Skip HashCuckooRep as it does not support single delete. FIFO and
+    // universal compaction do not apply to the test case. Skip MergePut
+    // because single delete does not get removed when it encounters a merge.
+  } while (ChangeOptions(kSkipHashCuckoo | kSkipFIFOCompaction |
+                         kSkipUniversalCompaction | kSkipMergePut));
 }
 
 TEST_F(DBBasicTest, EmptyFlush) {
@@ -237,11 +237,11 @@ TEST_F(DBBasicTest, EmptyFlush) {
     ASSERT_OK(Flush(1));
 
     ASSERT_EQ("[ ]", AllEntriesFor("a", 1));
-    // Skip FIFO and  universal compaction as they do not apply to the test
-    // case. Skip MergePut because merges cannot be combined with single
-    // deletions.
-  } while (ChangeOptions(kSkipFIFOCompaction | kSkipUniversalCompaction |
-                         kSkipMergePut));
+    // Skip HashCuckooRep as it does not support single delete. FIFO and
+    // universal compaction do not apply to the test case. Skip MergePut
+    // because merges cannot be combined with single deletions.
+  } while (ChangeOptions(kSkipHashCuckoo | kSkipFIFOCompaction |
+                         kSkipUniversalCompaction | kSkipMergePut));
 }
 
 TEST_F(DBBasicTest, GetFromVersions) {
@@ -265,6 +265,11 @@ TEST_F(DBBasicTest, GetSnapshot) {
       std::string key = (i == 0) ? std::string("foo") : std::string(200, 'x');
       ASSERT_OK(Put(1, key, "v1"));
       const Snapshot* s1 = db_->GetSnapshot();
+      if (option_config_ == kHashCuckoo) {
+        // Unsupported case.
+        ASSERT_TRUE(s1 == nullptr);
+        break;
+      }
       ASSERT_OK(Put(1, key, "v2"));
       ASSERT_EQ("v2", Get(1, key));
       ASSERT_EQ("v1", Get(1, key, s1));
@@ -505,7 +510,7 @@ TEST_F(DBBasicTest, Snapshot) {
     ASSERT_EQ(0U, GetNumSnapshots());
     ASSERT_EQ("0v4", Get(0, "foo"));
     ASSERT_EQ("1v4", Get(1, "foo"));
-  } while (ChangeOptions());
+  } while (ChangeOptions(kSkipHashCuckoo));
 }
 
 #endif  // ROCKSDB_LITE
@@ -561,7 +566,8 @@ TEST_F(DBBasicTest, CompactBetweenSnapshots) {
                            nullptr);
     ASSERT_EQ("sixth", Get(1, "foo"));
     ASSERT_EQ(AllEntriesFor("foo", 1), "[ sixth ]");
-  } while (ChangeOptions(kSkipFIFOCompaction));
+    // skip HashCuckooRep as it does not support snapshot
+  } while (ChangeOptions(kSkipHashCuckoo | kSkipFIFOCompaction));
 }
 
 TEST_F(DBBasicTest, DBOpen_Options) {
@@ -852,17 +858,18 @@ class TestEnv : public EnvWrapper {
       public:
         using Logger::Logv;
         TestLogger(TestEnv *env_ptr) : Logger() { env = env_ptr; }
-        ~TestLogger() override {
+        ~TestLogger() {
           if (!closed_) {
             CloseHelper();
           }
         }
-        void Logv(const char* /*format*/, va_list /*ap*/) override{};
+        virtual void Logv(const char* /*format*/, va_list /*ap*/) override{};
 
        protected:
-        Status CloseImpl() override { return CloseHelper(); }
-
-       private:
+        virtual Status CloseImpl() override {
+          return CloseHelper();
+        }
+      private:
         Status CloseHelper() {
           env->CloseCountInc();;
           return Status::IOError();
@@ -874,8 +881,8 @@ class TestEnv : public EnvWrapper {
 
     int GetCloseCount() { return close_count; }
 
-    Status NewLogger(const std::string& /*fname*/,
-                     std::shared_ptr<Logger>* result) override {
+    virtual Status NewLogger(const std::string& /*fname*/,
+                             std::shared_ptr<Logger>* result) {
       result->reset(new TestLogger(this));
       return Status::OK();
     }

db/db_block_cache_test.cc

@@ -395,9 +395,9 @@ class MockCache : public LRUCache {
                  false /*strict_capacity_limit*/, 0.0 /*high_pri_pool_ratio*/) {
   }
 
-  Status Insert(const Slice& key, void* value, size_t charge,
-                void (*deleter)(const Slice& key, void* value), Handle** handle,
-                Priority priority) override {
+  virtual Status Insert(const Slice& key, void* value, size_t charge,
+                        void (*deleter)(const Slice& key, void* value),
+                        Handle** handle, Priority priority) override {
     if (priority == Priority::LOW) {
       low_pri_insert_count++;
     } else {
@@ -633,7 +633,7 @@ TEST_F(DBBlockCacheTest, CompressedCache) {
 
 TEST_F(DBBlockCacheTest, CacheCompressionDict) {
   const int kNumFiles = 4;
-  const int kNumEntriesPerFile = 128;
+  const int kNumEntriesPerFile = 32;
   const int kNumBytesPerEntry = 1024;
 
   // Try all the available libraries that support dictionary compression

db/db_bloom_filter_test.cc

@@ -32,7 +32,7 @@ class DBBloomFilterTestWithParam
  public:
   DBBloomFilterTestWithParam() : DBTestBase("/db_bloom_filter_tests") {}
 
-  ~DBBloomFilterTestWithParam() override {}
+  ~DBBloomFilterTestWithParam() {}
 
   void SetUp() override {
     use_block_based_filter_ = std::get<0>(GetParam());
@@ -642,7 +642,7 @@ class WrappedBloom : public FilterPolicy {
   explicit WrappedBloom(int bits_per_key)
       : filter_(NewBloomFilterPolicy(bits_per_key)), counter_(0) {}
 
-  ~WrappedBloom() override { delete filter_; }
+  ~WrappedBloom() { delete filter_; }
 
   const char* Name() const override { return "WrappedRocksDbFilterPolicy"; }
 
@@ -786,56 +786,6 @@ TEST_F(DBBloomFilterTest, PrefixExtractorBlockFilter) {
   delete iter;
 }
 
-TEST_F(DBBloomFilterTest, MemtableWholeKeyBloomFilter) {
-  // regression test for #2743. the range delete tombstones in memtable should
-  // be added even when Get() skips searching due to its prefix bloom filter
-  const int kMemtableSize = 1 << 20;              // 1MB
-  const int kMemtablePrefixFilterSize = 1 << 13;  // 8KB
-  const int kPrefixLen = 4;
-  Options options = CurrentOptions();
-  options.memtable_prefix_bloom_size_ratio =
-      static_cast<double>(kMemtablePrefixFilterSize) / kMemtableSize;
-  options.prefix_extractor.reset(rocksdb::NewFixedPrefixTransform(kPrefixLen));
-  options.write_buffer_size = kMemtableSize;
-  options.memtable_whole_key_filtering = false;
-  Reopen(options);
-  std::string key1("AAAABBBB");
-  std::string key2("AAAACCCC");  // not in DB
-  std::string key3("AAAADDDD");
-  std::string key4("AAAAEEEE");
-  std::string value1("Value1");
-  std::string value3("Value3");
-  std::string value4("Value4");
-
-  ASSERT_OK(Put(key1, value1, WriteOptions()));
-
-  // check memtable bloom stats
-  ASSERT_EQ("NOT_FOUND", Get(key2));
-  ASSERT_EQ(0, get_perf_context()->bloom_memtable_miss_count);
-  // same prefix, bloom filter false positive
-  ASSERT_EQ(1, get_perf_context()->bloom_memtable_hit_count);
-
-  // enable whole key bloom filter
-  options.memtable_whole_key_filtering = true;
-  Reopen(options);
-  // check memtable bloom stats
-  ASSERT_OK(Put(key3, value3, WriteOptions()));
-  ASSERT_EQ("NOT_FOUND", Get(key2));
-  // whole key bloom filter kicks in and determines it's a miss
-  ASSERT_EQ(1, get_perf_context()->bloom_memtable_miss_count);
-  ASSERT_EQ(1, get_perf_context()->bloom_memtable_hit_count);
-
-  // verify whole key filtering does not depend on prefix_extractor
-  options.prefix_extractor.reset();
-  Reopen(options);
-  // check memtable bloom stats
-  ASSERT_OK(Put(key4, value4, WriteOptions()));
-  ASSERT_EQ("NOT_FOUND", Get(key2));
-  // whole key bloom filter kicks in and determines it's a miss
-  ASSERT_EQ(2, get_perf_context()->bloom_memtable_miss_count);
-  ASSERT_EQ(1, get_perf_context()->bloom_memtable_hit_count);
-}
-
 #ifndef ROCKSDB_LITE
 class BloomStatsTestWithParam
     : public DBBloomFilterTest,
@@ -873,7 +823,7 @@ class BloomStatsTestWithParam
     DestroyAndReopen(options_);
   }
 
-  ~BloomStatsTestWithParam() override {
+  ~BloomStatsTestWithParam() {
     get_perf_context()->Reset();
     Destroy(options_);
   }

db/db_compaction_filter_test.cc

@@ -63,33 +63,33 @@ INSTANTIATE_TEST_CASE_P(DBTestCompactionFilterWithCompactOption,
 
 class KeepFilter : public CompactionFilter {
  public:
-  bool Filter(int /*level*/, const Slice& /*key*/, const Slice& /*value*/,
-              std::string* /*new_value*/,
-              bool* /*value_changed*/) const override {
+  virtual bool Filter(int /*level*/, const Slice& /*key*/,
+                      const Slice& /*value*/, std::string* /*new_value*/,
+                      bool* /*value_changed*/) const override {
     cfilter_count++;
     return false;
   }
 
-  const char* Name() const override { return "KeepFilter"; }
+  virtual const char* Name() const override { return "KeepFilter"; }
 };
 
 class DeleteFilter : public CompactionFilter {
  public:
-  bool Filter(int /*level*/, const Slice& /*key*/, const Slice& /*value*/,
-              std::string* /*new_value*/,
-              bool* /*value_changed*/) const override {
+  virtual bool Filter(int /*level*/, const Slice& /*key*/,
+                      const Slice& /*value*/, std::string* /*new_value*/,
+                      bool* /*value_changed*/) const override {
     cfilter_count++;
     return true;
   }
 
-  const char* Name() const override { return "DeleteFilter"; }
+  virtual const char* Name() const override { return "DeleteFilter"; }
 };
 
 class DeleteISFilter : public CompactionFilter {
  public:
-  bool Filter(int /*level*/, const Slice& key, const Slice& /*value*/,
-              std::string* /*new_value*/,
-              bool* /*value_changed*/) const override {
+  virtual bool Filter(int /*level*/, const Slice& key, const Slice& /*value*/,
+                      std::string* /*new_value*/,
+                      bool* /*value_changed*/) const override {
     cfilter_count++;
     int i = std::stoi(key.ToString());
     if (i > 5 && i <= 105) {
@@ -98,18 +98,20 @@ class DeleteISFilter : public CompactionFilter {
     return false;
   }
 
-  bool IgnoreSnapshots() const override { return true; }
+  virtual bool IgnoreSnapshots() const override { return true; }
 
-  const char* Name() const override { return "DeleteFilter"; }
+  virtual const char* Name() const override { return "DeleteFilter"; }
 };
 
 // Skip x if floor(x/10) is even, use range skips. Requires that keys are
 // zero-padded to length 10.
 class SkipEvenFilter : public CompactionFilter {
  public:
-  Decision FilterV2(int /*level*/, const Slice& key, ValueType /*value_type*/,
-                    const Slice& /*existing_value*/, std::string* /*new_value*/,
-                    std::string* skip_until) const override {
+  virtual Decision FilterV2(int /*level*/, const Slice& key,
+                            ValueType /*value_type*/,
+                            const Slice& /*existing_value*/,
+                            std::string* /*new_value*/,
+                            std::string* skip_until) const override {
     cfilter_count++;
     int i = std::stoi(key.ToString());
     if (i / 10 % 2 == 0) {
@@ -122,22 +124,22 @@ class SkipEvenFilter : public CompactionFilter {
     return Decision::kKeep;
   }
 
-  bool IgnoreSnapshots() const override { return true; }
+  virtual bool IgnoreSnapshots() const override { return true; }
 
-  const char* Name() const override { return "DeleteFilter"; }
+  virtual const char* Name() const override { return "DeleteFilter"; }
 };
 
 class DelayFilter : public CompactionFilter {
  public:
   explicit DelayFilter(DBTestBase* d) : db_test(d) {}
-  bool Filter(int /*level*/, const Slice& /*key*/, const Slice& /*value*/,
-              std::string* /*new_value*/,
-              bool* /*value_changed*/) const override {
+  virtual bool Filter(int /*level*/, const Slice& /*key*/,
+                      const Slice& /*value*/, std::string* /*new_value*/,
+                      bool* /*value_changed*/) const override {
     db_test->env_->addon_time_.fetch_add(1000);
     return true;
   }
 
-  const char* Name() const override { return "DelayFilter"; }
+  virtual const char* Name() const override { return "DelayFilter"; }
 
  private:
   DBTestBase* db_test;
@@ -147,13 +149,13 @@ class ConditionalFilter : public CompactionFilter {
  public:
   explicit ConditionalFilter(const std::string* filtered_value)
       : filtered_value_(filtered_value) {}
-  bool Filter(int /*level*/, const Slice& /*key*/, const Slice& value,
-              std::string* /*new_value*/,
-              bool* /*value_changed*/) const override {
+  virtual bool Filter(int /*level*/, const Slice& /*key*/, const Slice& value,
+                      std::string* /*new_value*/,
+                      bool* /*value_changed*/) const override {
     return value.ToString() == *filtered_value_;
   }
 
-  const char* Name() const override { return "ConditionalFilter"; }
+  virtual const char* Name() const override { return "ConditionalFilter"; }
 
  private:
   const std::string* filtered_value_;
@@ -163,15 +165,16 @@ class ChangeFilter : public CompactionFilter {
  public:
   explicit ChangeFilter() {}
 
-  bool Filter(int /*level*/, const Slice& /*key*/, const Slice& /*value*/,
-              std::string* new_value, bool* value_changed) const override {
+  virtual bool Filter(int /*level*/, const Slice& /*key*/,
+                      const Slice& /*value*/, std::string* new_value,
+                      bool* value_changed) const override {
     assert(new_value != nullptr);
     *new_value = NEW_VALUE;
     *value_changed = true;
     return false;
   }
 
-  const char* Name() const override { return "ChangeFilter"; }
+  virtual const char* Name() const override { return "ChangeFilter"; }
 };
 
 class KeepFilterFactory : public CompactionFilterFactory {
@@ -182,7 +185,7 @@ class KeepFilterFactory : public CompactionFilterFactory {
         check_context_cf_id_(check_context_cf_id),
         compaction_filter_created_(false) {}
 
-  std::unique_ptr<CompactionFilter> CreateCompactionFilter(
+  virtual std::unique_ptr<CompactionFilter> CreateCompactionFilter(
       const CompactionFilter::Context& context) override {
     if (check_context_) {
       EXPECT_EQ(expect_full_compaction_.load(), context.is_full_compaction);
@@ -197,7 +200,7 @@ class KeepFilterFactory : public CompactionFilterFactory {
 
   bool compaction_filter_created() const { return compaction_filter_created_; }
 
-  const char* Name() const override { return "KeepFilterFactory"; }
+  virtual const char* Name() const override { return "KeepFilterFactory"; }
   bool check_context_;
   bool check_context_cf_id_;
   std::atomic_bool expect_full_compaction_;
@@ -208,7 +211,7 @@ class KeepFilterFactory : public CompactionFilterFactory {
 
 class DeleteFilterFactory : public CompactionFilterFactory {
  public:
-  std::unique_ptr<CompactionFilter> CreateCompactionFilter(
+  virtual std::unique_ptr<CompactionFilter> CreateCompactionFilter(
       const CompactionFilter::Context& context) override {
     if (context.is_manual_compaction) {
       return std::unique_ptr<CompactionFilter>(new DeleteFilter());
@@ -217,13 +220,13 @@ class DeleteFilterFactory : public CompactionFilterFactory {
     }
   }
 
-  const char* Name() const override { return "DeleteFilterFactory"; }
+  virtual const char* Name() const override { return "DeleteFilterFactory"; }
 };
 
 // Delete Filter Factory which ignores snapshots
 class DeleteISFilterFactory : public CompactionFilterFactory {
  public:
-  std::unique_ptr<CompactionFilter> CreateCompactionFilter(
+  virtual std::unique_ptr<CompactionFilter> CreateCompactionFilter(
       const CompactionFilter::Context& context) override {
     if (context.is_manual_compaction) {
       return std::unique_ptr<CompactionFilter>(new DeleteISFilter());
@@ -232,12 +235,12 @@ class DeleteISFilterFactory : public CompactionFilterFactory {
     }
   }
 
-  const char* Name() const override { return "DeleteFilterFactory"; }
+  virtual const char* Name() const override { return "DeleteFilterFactory"; }
 };
 
 class SkipEvenFilterFactory : public CompactionFilterFactory {
  public:
-  std::unique_ptr<CompactionFilter> CreateCompactionFilter(
+  virtual std::unique_ptr<CompactionFilter> CreateCompactionFilter(
       const CompactionFilter::Context& context) override {
     if (context.is_manual_compaction) {
       return std::unique_ptr<CompactionFilter>(new SkipEvenFilter());
@@ -246,18 +249,18 @@ class SkipEvenFilterFactory : public CompactionFilterFactory {
     }
   }
 
-  const char* Name() const override { return "SkipEvenFilterFactory"; }
+  virtual const char* Name() const override { return "SkipEvenFilterFactory"; }
 };
 
 class DelayFilterFactory : public CompactionFilterFactory {
  public:
   explicit DelayFilterFactory(DBTestBase* d) : db_test(d) {}
-  std::unique_ptr<CompactionFilter> CreateCompactionFilter(
+  virtual std::unique_ptr<CompactionFilter> CreateCompactionFilter(
       const CompactionFilter::Context& /*context*/) override {
     return std::unique_ptr<CompactionFilter>(new DelayFilter(db_test));
   }
 
-  const char* Name() const override { return "DelayFilterFactory"; }
+  virtual const char* Name() const override { return "DelayFilterFactory"; }
 
  private:
   DBTestBase* db_test;
@@ -268,13 +271,15 @@ class ConditionalFilterFactory : public CompactionFilterFactory {
   explicit ConditionalFilterFactory(const Slice& filtered_value)
       : filtered_value_(filtered_value.ToString()) {}
 
-  std::unique_ptr<CompactionFilter> CreateCompactionFilter(
+  virtual std::unique_ptr<CompactionFilter> CreateCompactionFilter(
       const CompactionFilter::Context& /*context*/) override {
     return std::unique_ptr<CompactionFilter>(
         new ConditionalFilter(&filtered_value_));
   }
 
-  const char* Name() const override { return "ConditionalFilterFactory"; }
+  virtual const char* Name() const override {
+    return "ConditionalFilterFactory";
+  }
 
  private:
   std::string filtered_value_;
@@ -284,12 +289,12 @@ class ChangeFilterFactory : public CompactionFilterFactory {
  public:
   explicit ChangeFilterFactory() {}
 
-  std::unique_ptr<CompactionFilter> CreateCompactionFilter(
+  virtual std::unique_ptr<CompactionFilter> CreateCompactionFilter(
       const CompactionFilter::Context& /*context*/) override {
     return std::unique_ptr<CompactionFilter>(new ChangeFilter());
   }
 
-  const char* Name() const override { return "ChangeFilterFactory"; }
+  virtual const char* Name() const override { return "ChangeFilterFactory"; }
 };
 
 #ifndef ROCKSDB_LITE
@@ -352,7 +357,7 @@ TEST_F(DBTestCompactionFilter, CompactionFilter) {
     }
   }
   ASSERT_EQ(total, 100000);
-  ASSERT_EQ(count, 0);
+  ASSERT_EQ(count, 1);
 
   // overwrite all the 100K keys once again.
   for (int i = 0; i < 100000; i++) {
@@ -659,7 +664,7 @@ TEST_F(DBTestCompactionFilter, CompactionFilterContextManual) {
       iter->Next();
     }
     ASSERT_EQ(total, 700);
-    ASSERT_EQ(count, 0);
+    ASSERT_EQ(count, 1);
   }
 }
 #endif  // ROCKSDB_LITE
@@ -694,7 +699,44 @@ TEST_F(DBTestCompactionFilter, CompactionFilterContextCfId) {
 }
 
 #ifndef ROCKSDB_LITE
-// Compaction filters aplies to all records, regardless snapshots.
+// Compaction filters should only be applied to records that are newer than the
+// latest snapshot. This test inserts records and applies a delete filter.
+TEST_F(DBTestCompactionFilter, CompactionFilterSnapshot) {
+  Options options = CurrentOptions();
+  options.compaction_filter_factory = std::make_shared<DeleteFilterFactory>();
+  options.disable_auto_compactions = true;
+  options.create_if_missing = true;
+  DestroyAndReopen(options);
+
+  // Put some data.
+  const Snapshot* snapshot = nullptr;
+  for (int table = 0; table < 4; ++table) {
+    for (int i = 0; i < 10; ++i) {
+      Put(ToString(table * 100 + i), "val");
+    }
+    Flush();
+
+    if (table == 0) {
+      snapshot = db_->GetSnapshot();
+    }
+  }
+  assert(snapshot != nullptr);
+
+  cfilter_count = 0;
+  ASSERT_OK(db_->CompactRange(CompactRangeOptions(), nullptr, nullptr));
+  // The filter should delete 10 records.
+  ASSERT_EQ(30U, cfilter_count);
+
+  // Release the snapshot and compact again -> now all records should be
+  // removed.
+  db_->ReleaseSnapshot(snapshot);
+  ASSERT_OK(db_->CompactRange(CompactRangeOptions(), nullptr, nullptr));
+  ASSERT_EQ(0U, CountLiveFiles());
+}
+
+// Compaction filters should only be applied to records that are newer than the
+// latest snapshot. However, if the compaction filter asks to ignore snapshots
+// records newer than the snapshot will also be processed
 TEST_F(DBTestCompactionFilter, CompactionFilterIgnoreSnapshot) {
   std::string five = ToString(5);
   Options options = CurrentOptions();
@@ -832,38 +874,6 @@ TEST_F(DBTestCompactionFilter, SkipUntilWithBloomFilter) {
   EXPECT_EQ("v50", val);
 }
 
-class TestNotSupportedFilter : public CompactionFilter {
- public:
-  bool Filter(int /*level*/, const Slice& /*key*/, const Slice& /*value*/,
-              std::string* /*new_value*/,
-              bool* /*value_changed*/) const override {
-    return true;
-  }
-
-  const char* Name() const override { return "NotSupported"; }
-  bool IgnoreSnapshots() const override { return false; }
-};
-
-TEST_F(DBTestCompactionFilter, IgnoreSnapshotsFalse) {
-  Options options = CurrentOptions();
-  options.compaction_filter = new TestNotSupportedFilter();
-  DestroyAndReopen(options);
-
-  Put("a", "v10");
-  Put("z", "v20");
-  Flush();
-
-  Put("a", "v10");
-  Put("z", "v20");
-  Flush();
-
-  // Comapction should fail because IgnoreSnapshots() = false
-  EXPECT_TRUE(db_->CompactRange(CompactRangeOptions(), nullptr, nullptr)
-                  .IsNotSupported());
-
-  delete options.compaction_filter;
-}
-
 }  // namespace rocksdb
 
 int main(int argc, char** argv) {

db/db_compaction_test.cc

@@ -55,9 +55,9 @@ namespace {
 class FlushedFileCollector : public EventListener {
  public:
   FlushedFileCollector() {}
-  ~FlushedFileCollector() override {}
+  ~FlushedFileCollector() {}
 
-  void OnFlushCompleted(DB* /*db*/, const FlushJobInfo& info) override {
+  virtual void OnFlushCompleted(DB* /*db*/, const FlushJobInfo& info) override {
     std::lock_guard<std::mutex> lock(mutex_);
     flushed_files_.push_back(info.file_path);
   }
@@ -87,23 +87,24 @@ public:
     }
   }
 
-  ~CompactionStatsCollector() override {}
+  ~CompactionStatsCollector() {}
 
-  void OnCompactionCompleted(DB* /* db */,
-                             const CompactionJobInfo& info) override {
+  virtual void OnCompactionCompleted(DB* /* db */,
+      const CompactionJobInfo& info) override {
     int k = static_cast<int>(info.compaction_reason);
     int num_of_reasons = static_cast<int>(CompactionReason::kNumOfReasons);
     assert(k >= 0 && k < num_of_reasons);
     compaction_completed_[k]++;
   }
 
-  void OnExternalFileIngested(
-      DB* /* db */, const ExternalFileIngestionInfo& /* info */) override {
+  virtual void OnExternalFileIngested(DB* /* db */,
+      const ExternalFileIngestionInfo& /* info */) override {
     int k = static_cast<int>(CompactionReason::kExternalSstIngestion);
     compaction_completed_[k]++;
   }
 
-  void OnFlushCompleted(DB* /* db */, const FlushJobInfo& /* info */) override {
+  virtual void OnFlushCompleted(DB* /* db */,
+      const FlushJobInfo& /* info */) override {
     int k = static_cast<int>(CompactionReason::kFlush);
     compaction_completed_[k]++;
   }
@@ -4186,14 +4187,14 @@ class NoopMergeOperator : public MergeOperator {
  public:
   NoopMergeOperator() {}
 
-  bool FullMergeV2(const MergeOperationInput& /*merge_in*/,
-                   MergeOperationOutput* merge_out) const override {
+  virtual bool FullMergeV2(const MergeOperationInput& /*merge_in*/,
+                           MergeOperationOutput* merge_out) const override {
     std::string val("bar");
     merge_out->new_value = val;
     return true;
   }
 
-  const char* Name() const override { return "Noop"; }
+  virtual const char* Name() const override { return "Noop"; }
 };
 
 TEST_F(DBCompactionTest, PartialManualCompaction) {

db/db_impl.cc

@@ -37,7 +37,6 @@
 #include "db/external_sst_file_ingestion_job.h"
 #include "db/flush_job.h"
 #include "db/forward_iterator.h"
-#include "db/in_memory_stats_history.h"
 #include "db/job_context.h"
 #include "db/log_reader.h"
 #include "db/log_writer.h"
@@ -70,7 +69,6 @@
 #include "rocksdb/env.h"
 #include "rocksdb/merge_operator.h"
 #include "rocksdb/statistics.h"
-#include "rocksdb/stats_history.h"
 #include "rocksdb/status.h"
 #include "rocksdb/table.h"
 #include "rocksdb/write_buffer_manager.h"
@@ -389,15 +387,16 @@ void DBImpl::CancelAllBackgroundWork(bool wait) {
   ROCKS_LOG_INFO(immutable_db_options_.info_log,
                  "Shutdown: canceling all background work");
 
+  InstrumentedMutexLock l(&mutex_);
+  // To avoid deadlock, `thread_dump_stats_->cancel()` needs to be called
+  // before grabbing db mutex because the actual worker function
+  // `DBImpl::DumpStats()` also holds db mutex
   if (thread_dump_stats_ != nullptr) {
+    mutex_.Unlock();
     thread_dump_stats_->cancel();
+    mutex_.Lock();
     thread_dump_stats_.reset();
   }
-  if (thread_persist_stats_ != nullptr) {
-    thread_persist_stats_->cancel();
-    thread_persist_stats_.reset();
-  }
-  InstrumentedMutexLock l(&mutex_);
   if (!shutting_down_.load(std::memory_order_acquire) &&
       has_unpersisted_data_.load(std::memory_order_relaxed) &&
       !mutable_db_options_.avoid_flush_during_shutdown) {
@@ -614,14 +613,13 @@ const Status DBImpl::CreateArchivalDirectory() {
 void DBImpl::PrintStatistics() {
   auto dbstats = immutable_db_options_.statistics.get();
   if (dbstats) {
-    ROCKS_LOG_INFO(immutable_db_options_.info_log, "STATISTICS:\n %s",
+    ROCKS_LOG_WARN(immutable_db_options_.info_log, "STATISTICS:\n %s",
                    dbstats->ToString().c_str());
   }
 }
 
 void DBImpl::StartTimedTasks() {
   unsigned int stats_dump_period_sec = 0;
-  unsigned int stats_persist_period_sec = 0;
   {
     InstrumentedMutexLock l(&mutex_);
     stats_dump_period_sec = mutable_db_options_.stats_dump_period_sec;
@@ -632,114 +630,9 @@ void DBImpl::StartTimedTasks() {
             stats_dump_period_sec * 1000000));
       }
     }
-    stats_persist_period_sec = mutable_db_options_.stats_persist_period_sec;
-    if (stats_persist_period_sec > 0) {
-      if (!thread_persist_stats_) {
-        thread_persist_stats_.reset(new rocksdb::RepeatableThread(
-            [this]() { DBImpl::PersistStats(); }, "pst_st", env_,
-            stats_persist_period_sec * 1000000));
-      }
-    }
   }
 }
 
-// esitmate the total size of stats_history_
-size_t DBImpl::EstiamteStatsHistorySize() const {
-  size_t size_total =
-      sizeof(std::map<uint64_t, std::map<std::string, uint64_t>>);
-  if (stats_history_.size() == 0) return size_total;
-  size_t size_per_slice =
-      sizeof(uint64_t) + sizeof(std::map<std::string, uint64_t>);
-  // non-empty map, stats_history_.begin() guaranteed to exist
-  std::map<std::string, uint64_t> sample_slice(stats_history_.begin()->second);
-  for (const auto& pairs : sample_slice) {
-    size_per_slice +=
-        pairs.first.capacity() + sizeof(pairs.first) + sizeof(pairs.second);
-  }
-  size_total = size_per_slice * stats_history_.size();
-  return size_total;
-}
-
-void DBImpl::PersistStats() {
-  TEST_SYNC_POINT("DBImpl::PersistStats:Entry");
-#ifndef ROCKSDB_LITE
-  if (shutdown_initiated_) {
-    return;
-  }
-  uint64_t now_micros = env_->NowMicros();
-  Statistics* statistics = immutable_db_options_.statistics.get();
-  if (!statistics) {
-    return;
-  }
-  size_t stats_history_size_limit = 0;
-  {
-    InstrumentedMutexLock l(&mutex_);
-    stats_history_size_limit = mutable_db_options_.stats_history_buffer_size;
-  }
-
-  // TODO(Zhongyi): also persist immutable_db_options_.statistics
-  {
-    std::map<std::string, uint64_t> stats_map;
-    if (!statistics->getTickerMap(&stats_map)) {
-      return;
-    }
-    InstrumentedMutexLock l(&stats_history_mutex_);
-    // calculate the delta from last time
-    if (stats_slice_initialized_) {
-      std::map<std::string, uint64_t> stats_delta;
-      for (const auto& stat : stats_map) {
-        if (stats_slice_.find(stat.first) != stats_slice_.end()) {
-          stats_delta[stat.first] = stat.second - stats_slice_[stat.first];
-        }
-      }
-      stats_history_[now_micros] = stats_delta;
-    }
-    stats_slice_initialized_ = true;
-    std::swap(stats_slice_, stats_map);
-    TEST_SYNC_POINT("DBImpl::PersistStats:StatsCopied");
-
-    // delete older stats snapshots to control memory consumption
-    bool purge_needed = EstiamteStatsHistorySize() > stats_history_size_limit;
-    while (purge_needed && !stats_history_.empty()) {
-      stats_history_.erase(stats_history_.begin());
-      purge_needed = EstiamteStatsHistorySize() > stats_history_size_limit;
-    }
-  }
-  // TODO: persist stats to disk
-#endif  // !ROCKSDB_LITE
-}
-
-bool DBImpl::FindStatsByTime(uint64_t start_time, uint64_t end_time,
-                             uint64_t* new_time,
-                             std::map<std::string, uint64_t>* stats_map) {
-  assert(new_time);
-  assert(stats_map);
-  if (!new_time || !stats_map) return false;
-  // lock when search for start_time
-  {
-    InstrumentedMutexLock l(&stats_history_mutex_);
-    auto it = stats_history_.lower_bound(start_time);
-    if (it != stats_history_.end() && it->first < end_time) {
-      // make a copy for timestamp and stats_map
-      *new_time = it->first;
-      *stats_map = it->second;
-      return true;
-    } else {
-      return false;
-    }
-  }
-}
-
-Status DBImpl::GetStatsHistory(uint64_t start_time, uint64_t end_time,
-    std::unique_ptr<StatsHistoryIterator>* stats_iterator) {
-  if (!stats_iterator) {
-    return Status::InvalidArgument("stats_iterator not preallocated.");
-  }
-  stats_iterator->reset(
-      new InMemoryStatsHistoryIterator(start_time, end_time, this));
-  return (*stats_iterator)->status();
-}
-
 void DBImpl::DumpStats() {
   TEST_SYNC_POINT("DBImpl::DumpStats:1");
 #ifndef ROCKSDB_LITE
@@ -772,16 +665,16 @@ void DBImpl::DumpStats() {
     }
   }
   TEST_SYNC_POINT("DBImpl::DumpStats:2");
-  ROCKS_LOG_INFO(immutable_db_options_.info_log,
+  ROCKS_LOG_WARN(immutable_db_options_.info_log,
                  "------- DUMPING STATS -------");
-  ROCKS_LOG_INFO(immutable_db_options_.info_log, "%s", stats.c_str());
+  ROCKS_LOG_WARN(immutable_db_options_.info_log, "%s", stats.c_str());
   if (immutable_db_options_.dump_malloc_stats) {
     stats.clear();
     DumpMallocStats(&stats);
     if (!stats.empty()) {
-      ROCKS_LOG_INFO(immutable_db_options_.info_log,
+      ROCKS_LOG_WARN(immutable_db_options_.info_log,
                      "------- Malloc STATS -------");
-      ROCKS_LOG_INFO(immutable_db_options_.info_log, "%s", stats.c_str());
+      ROCKS_LOG_WARN(immutable_db_options_.info_log, "%s", stats.c_str());
     }
   }
 #endif  // !ROCKSDB_LITE
@@ -926,21 +819,6 @@ Status DBImpl::SetDBOptions(
             thread_dump_stats_.reset();
           }
       }
-      if (new_options.stats_persist_period_sec !=
-          mutable_db_options_.stats_persist_period_sec) {
-        if (thread_persist_stats_) {
-          mutex_.Unlock();
-          thread_persist_stats_->cancel();
-          mutex_.Lock();
-        }
-        if (new_options.stats_persist_period_sec > 0) {
-          thread_persist_stats_.reset(new rocksdb::RepeatableThread(
-              [this]() { DBImpl::PersistStats(); }, "pst_st", env_,
-              new_options.stats_persist_period_sec * 1000000));
-        } else {
-          thread_persist_stats_.reset();
-        }
-      }
       write_controller_.set_max_delayed_write_rate(
           new_options.delayed_write_rate);
       table_cache_.get()->SetCapacity(new_options.max_open_files == -1
@@ -1441,13 +1319,10 @@ Status DBImpl::GetImpl(const ReadOptions& read_options,
     ReturnAndCleanupSuperVersion(cfd, sv);
 
     RecordTick(stats_, NUMBER_KEYS_READ);
-    size_t size = 0;
-    if (s.ok()) {
-      size = pinnable_val->size();
-      RecordTick(stats_, BYTES_READ, size);
-      PERF_COUNTER_ADD(get_read_bytes, size);
-    }
+    size_t size = pinnable_val->size();
+    RecordTick(stats_, BYTES_READ, size);
     MeasureTime(stats_, BYTES_PER_READ, size);
+    PERF_COUNTER_ADD(get_read_bytes, size);
   }
   return s;
 }
@@ -3234,124 +3109,74 @@ Status DBImpl::IngestExternalFile(
     ColumnFamilyHandle* column_family,
     const std::vector<std::string>& external_files,
     const IngestExternalFileOptions& ingestion_options) {
-  IngestExternalFileArg arg;
-  arg.column_family = column_family;
-  arg.external_files = external_files;
-  arg.options = ingestion_options;
-  return IngestExternalFiles({arg});
-}
+  if (external_files.empty()) {
+    return Status::InvalidArgument("external_files is empty");
+  }
 
-Status DBImpl::IngestExternalFiles(
-    const std::vector<IngestExternalFileArg>& args) {
-  if (args.empty()) {
-    return Status::InvalidArgument("ingestion arg list is empty");
-  }
-  {
-    std::unordered_set<ColumnFamilyHandle*> unique_cfhs;
-    for (const auto& arg : args) {
-      if (arg.column_family == nullptr) {
-        return Status::InvalidArgument("column family handle is null");
-      } else if (unique_cfhs.count(arg.column_family) > 0) {
-        return Status::InvalidArgument(
-            "ingestion args have duplicate column families");
-      }
-      unique_cfhs.insert(arg.column_family);
-    }
-  }
-  // Ingest multiple external SST files atomically.
-  size_t num_cfs = args.size();
-  for (size_t i = 0; i != num_cfs; ++i) {
-    if (args[i].external_files.empty()) {
-      char err_msg[128] = {0};
-      snprintf(err_msg, 128, "external_files[%zu] is empty", i);
-      return Status::InvalidArgument(err_msg);
-    }
-  }
-  for (const auto& arg : args) {
-    const IngestExternalFileOptions& ingest_opts = arg.options;
-    if (ingest_opts.ingest_behind &&
-        !immutable_db_options_.allow_ingest_behind) {
+  Status status;
+  auto cfh = reinterpret_cast<ColumnFamilyHandleImpl*>(column_family);
+  auto cfd = cfh->cfd();
+
+  // Ingest should immediately fail if ingest_behind is requested,
+  // but the DB doesn't support it.
+  if (ingestion_options.ingest_behind) {
+    if (!immutable_db_options_.allow_ingest_behind) {
       return Status::InvalidArgument(
-          "can't ingest_behind file in DB with allow_ingest_behind=false");
+          "Can't ingest_behind file in DB with allow_ingest_behind=false");
     }
   }
 
-  // TODO (yanqin) maybe handle the case in which column_families have
-  // duplicates
-  std::list<uint64_t>::iterator pending_output_elem;
-  size_t total = 0;
-  for (const auto& arg : args) {
-    total += arg.external_files.size();
-  }
+  ExternalSstFileIngestionJob ingestion_job(env_, versions_.get(), cfd,
+                                            immutable_db_options_, env_options_,
+                                            &snapshots_, ingestion_options);
+
+  SuperVersionContext dummy_sv_ctx(/* create_superversion */ true);
+  VersionEdit dummy_edit;
   uint64_t next_file_number = 0;
-  Status status = ReserveFileNumbersBeforeIngestion(
-      static_cast<ColumnFamilyHandleImpl*>(args[0].column_family)->cfd(), total,
-      &pending_output_elem, &next_file_number);
-  if (!status.ok()) {
-    InstrumentedMutexLock l(&mutex_);
-    ReleaseFileNumberFromPendingOutputs(pending_output_elem);
-    return status;
-  }
-
-  std::vector<ExternalSstFileIngestionJob> ingestion_jobs;
-  for (const auto& arg : args) {
-    auto* cfd = static_cast<ColumnFamilyHandleImpl*>(arg.column_family)->cfd();
-    ingestion_jobs.emplace_back(env_, versions_.get(), cfd,
-                                immutable_db_options_, env_options_,
-                                &snapshots_, arg.options);
-  }
-  std::vector<std::pair<bool, Status>> exec_results;
-  for (size_t i = 0; i != num_cfs; ++i) {
-    exec_results.emplace_back(false, Status::OK());
-  }
-  // TODO(yanqin) maybe make jobs run in parallel
-  for (size_t i = 1; i != num_cfs; ++i) {
-    uint64_t start_file_number =
-        next_file_number + args[i - 1].external_files.size();
-    auto* cfd =
-        static_cast<ColumnFamilyHandleImpl*>(args[i].column_family)->cfd();
-    SuperVersion* super_version = cfd->GetReferencedSuperVersion(&mutex_);
-    exec_results[i].second = ingestion_jobs[i].Prepare(
-        args[i].external_files, start_file_number, super_version);
-    exec_results[i].first = true;
-    CleanupSuperVersion(super_version);
-  }
-  TEST_SYNC_POINT("DBImpl::IngestExternalFiles:BeforeLastJobPrepare:0");
-  TEST_SYNC_POINT("DBImpl::IngestExternalFiles:BeforeLastJobPrepare:1");
+  std::list<uint64_t>::iterator pending_output_elem;
   {
-    auto* cfd =
-        static_cast<ColumnFamilyHandleImpl*>(args[0].column_family)->cfd();
-    SuperVersion* super_version = cfd->GetReferencedSuperVersion(&mutex_);
-    exec_results[0].second = ingestion_jobs[0].Prepare(
-        args[0].external_files, next_file_number, super_version);
-    exec_results[0].first = true;
-    CleanupSuperVersion(super_version);
-  }
-  for (const auto& exec_result : exec_results) {
-    if (!exec_result.second.ok()) {
-      status = exec_result.second;
-      break;
+    InstrumentedMutexLock l(&mutex_);
+    if (error_handler_.IsDBStopped()) {
+      // Don't ingest files when there is a bg_error
+      return error_handler_.GetBGError();
+    }
+
+    // Make sure that bg cleanup wont delete the files that we are ingesting
+    pending_output_elem = CaptureCurrentFileNumberInPendingOutputs();
+
+    // If crash happen after a hard link established, Recover function may
+    // reuse the file number that has already assigned to the internal file,
+    // and this will overwrite the external file. To protect the external
+    // file, we have to make sure the file number will never being reused.
+    next_file_number = versions_->FetchAddFileNumber(external_files.size());
+    auto cf_options = cfd->GetLatestMutableCFOptions();
+    status = versions_->LogAndApply(cfd, *cf_options, &dummy_edit, &mutex_,
+                                    directories_.GetDbDir());
+    if (status.ok()) {
+      InstallSuperVersionAndScheduleWork(cfd, &dummy_sv_ctx, *cf_options);
     }
   }
+  dummy_sv_ctx.Clean();
   if (!status.ok()) {
-    for (size_t i = 0; i != num_cfs; ++i) {
-      if (exec_results[i].first) {
-        ingestion_jobs[i].Cleanup(status);
-      }
-    }
     InstrumentedMutexLock l(&mutex_);
     ReleaseFileNumberFromPendingOutputs(pending_output_elem);
     return status;
   }
 
-  std::vector<SuperVersionContext> sv_ctxs;
-  for (size_t i = 0; i != num_cfs; ++i) {
-    sv_ctxs.emplace_back(true /* create_superversion */);
+  SuperVersion* super_version = cfd->GetReferencedSuperVersion(&mutex_);
+  status =
+      ingestion_job.Prepare(external_files, next_file_number, super_version);
+  CleanupSuperVersion(super_version);
+  if (!status.ok()) {
+    InstrumentedMutexLock l(&mutex_);
+    ReleaseFileNumberFromPendingOutputs(pending_output_elem);
+    return status;
   }
-  TEST_SYNC_POINT("DBImpl::IngestExternalFiles:BeforeJobsRun:0");
-  TEST_SYNC_POINT("DBImpl::IngestExternalFiles:BeforeJobsRun:1");
+
+  SuperVersionContext sv_context(/* create_superversion */ true);
   TEST_SYNC_POINT("DBImpl::AddFile:Start");
   {
+    // Lock db mutex
     InstrumentedMutexLock l(&mutex_);
     TEST_SYNC_POINT("DBImpl::AddFile:MutexLock");
 
@@ -3363,133 +3188,55 @@ Status DBImpl::IngestExternalFiles(
       nonmem_write_thread_.EnterUnbatched(&nonmem_w, &mutex_);
     }
 
-    num_running_ingest_file_ += static_cast<int>(num_cfs);
+    num_running_ingest_file_++;
     TEST_SYNC_POINT("DBImpl::IngestExternalFile:AfterIncIngestFileCounter");
 
-    bool at_least_one_cf_need_flush = false;
-    std::vector<bool> need_flush(num_cfs, false);
-    for (size_t i = 0; i != num_cfs; ++i) {
-      auto* cfd =
-          static_cast<ColumnFamilyHandleImpl*>(args[i].column_family)->cfd();
-      if (cfd->IsDropped()) {
-        // TODO (yanqin) investigate whether we should abort ingestion or
-        // proceed with other non-dropped column families.
-        status = Status::InvalidArgument(
-            "cannot ingest an external file into a dropped CF");
-        break;
-      }
-      bool tmp = false;
-      status = ingestion_jobs[i].NeedsFlush(&tmp, cfd->GetSuperVersion());
-      need_flush[i] = tmp;
-      at_least_one_cf_need_flush = (at_least_one_cf_need_flush || tmp);
-      if (!status.ok()) {
-        break;
-      }
+    // We cannot ingest a file into a dropped CF
+    if (cfd->IsDropped()) {
+      status = Status::InvalidArgument(
+          "Cannot ingest an external file into a dropped CF");
     }
-    TEST_SYNC_POINT_CALLBACK("DBImpl::IngestExternalFile:NeedFlush",
-                             &at_least_one_cf_need_flush);
 
-    if (status.ok() && at_least_one_cf_need_flush) {
-      FlushOptions flush_opts;
-      flush_opts.allow_write_stall = true;
-      if (immutable_db_options_.atomic_flush) {
-        autovector<ColumnFamilyData*> cfds_to_flush;
-        SelectColumnFamiliesForAtomicFlush(&cfds_to_flush);
-        mutex_.Unlock();
-        status = AtomicFlushMemTables(cfds_to_flush, flush_opts,
-                                      FlushReason::kExternalFileIngestion,
-                                      true /* writes_stopped */);
+    // Figure out if we need to flush the memtable first
+    if (status.ok()) {
+      bool need_flush = false;
+      status = ingestion_job.NeedsFlush(&need_flush, cfd->GetSuperVersion());
+      TEST_SYNC_POINT_CALLBACK("DBImpl::IngestExternalFile:NeedFlush",
+                               &need_flush);
+      if (status.ok() && need_flush) {
+        FlushOptions flush_opts;
+        flush_opts.allow_write_stall = true;
+        if (immutable_db_options_.atomic_flush) {
+          autovector<ColumnFamilyData*> cfds;
+          SelectColumnFamiliesForAtomicFlush(&cfds);
+          mutex_.Unlock();
+          status = AtomicFlushMemTables(cfds, flush_opts,
+                                        FlushReason::kExternalFileIngestion,
+                                        true /* writes_stopped */);
+        } else {
+          mutex_.Unlock();
+          status = FlushMemTable(cfd, flush_opts,
+                                 FlushReason::kExternalFileIngestion,
+                                 true /* writes_stopped */);
+        }
         mutex_.Lock();
-      } else {
-        for (size_t i = 0; i != num_cfs; ++i) {
-          if (need_flush[i]) {
-            mutex_.Unlock();
-            auto* cfd =
-                static_cast<ColumnFamilyHandleImpl*>(args[i].column_family)
-                    ->cfd();
-            status = FlushMemTable(cfd, flush_opts,
-                                   FlushReason::kExternalFileIngestion,
-                                   true /* writes_stopped */);
-            mutex_.Lock();
-            if (!status.ok()) {
-              break;
-            }
-          }
-        }
       }
     }
-    // Run ingestion jobs.
-    if (status.ok()) {
-      for (size_t i = 0; i != num_cfs; ++i) {
-        status = ingestion_jobs[i].Run();
-        if (!status.ok()) {
-          break;
-        }
-      }
-    }
-    if (status.ok()) {
-      bool should_increment_last_seqno =
-          ingestion_jobs[0].ShouldIncrementLastSequence();
-#ifndef NDEBUG
-      for (size_t i = 1; i != num_cfs; ++i) {
-        assert(should_increment_last_seqno ==
-               ingestion_jobs[i].ShouldIncrementLastSequence());
-      }
-#endif
-      if (should_increment_last_seqno) {
-        const SequenceNumber last_seqno = versions_->LastSequence();
-        versions_->SetLastAllocatedSequence(last_seqno + 1);
-        versions_->SetLastPublishedSequence(last_seqno + 1);
-        versions_->SetLastSequence(last_seqno + 1);
-      }
-      autovector<ColumnFamilyData*> cfds_to_commit;
-      autovector<const MutableCFOptions*> mutable_cf_options_list;
-      autovector<autovector<VersionEdit*>> edit_lists;
-      uint32_t num_entries = 0;
-      for (size_t i = 0; i != num_cfs; ++i) {
-        auto* cfd =
-            static_cast<ColumnFamilyHandleImpl*>(args[i].column_family)->cfd();
-        if (cfd->IsDropped()) {
-          continue;
-        }
-        cfds_to_commit.push_back(cfd);
-        mutable_cf_options_list.push_back(cfd->GetLatestMutableCFOptions());
-        autovector<VersionEdit*> edit_list;
-        edit_list.push_back(ingestion_jobs[i].edit());
-        edit_lists.push_back(edit_list);
-        ++num_entries;
-      }
-      // Mark the version edits as an atomic group if the number of version
-      // edits exceeds 1.
-      if (cfds_to_commit.size() > 1) {
-        for (auto& edits : edit_lists) {
-          assert(edits.size() == 1);
-          edits[0]->MarkAtomicGroup(--num_entries);
-        }
-        assert(0 == num_entries);
-      }
-      status =
-          versions_->LogAndApply(cfds_to_commit, mutable_cf_options_list,
-                                 edit_lists, &mutex_, directories_.GetDbDir());
-    }
 
+    // Run the ingestion job
     if (status.ok()) {
-      for (size_t i = 0; i != num_cfs; ++i) {
-        auto* cfd =
-            static_cast<ColumnFamilyHandleImpl*>(args[i].column_family)->cfd();
-        if (!cfd->IsDropped()) {
-          InstallSuperVersionAndScheduleWork(cfd, &sv_ctxs[i],
-                                             *cfd->GetLatestMutableCFOptions());
-#ifndef NDEBUG
-          if (0 == i && num_cfs > 1) {
-            TEST_SYNC_POINT(
-                "DBImpl::IngestExternalFiles:InstallSVForFirstCF:0");
-            TEST_SYNC_POINT(
-                "DBImpl::IngestExternalFiles:InstallSVForFirstCF:1");
-          }
-#endif  // !NDEBUG
-        }
-      }
+      status = ingestion_job.Run();
+    }
+
+    // Install job edit [Mutex will be unlocked here]
+    auto mutable_cf_options = cfd->GetLatestMutableCFOptions();
+    if (status.ok()) {
+      status =
+          versions_->LogAndApply(cfd, *mutable_cf_options, ingestion_job.edit(),
+                                 &mutex_, directories_.GetDbDir());
+    }
+    if (status.ok()) {
+      InstallSuperVersionAndScheduleWork(cfd, &sv_context, *mutable_cf_options);
     }
 
     // Resume writes to the DB
@@ -3498,36 +3245,30 @@ Status DBImpl::IngestExternalFiles(
     }
     write_thread_.ExitUnbatched(&w);
 
+    // Update stats
     if (status.ok()) {
-      for (auto& job : ingestion_jobs) {
-        job.UpdateStats();
-      }
+      ingestion_job.UpdateStats();
     }
+
     ReleaseFileNumberFromPendingOutputs(pending_output_elem);
-    num_running_ingest_file_ -= static_cast<int>(num_cfs);
-    if (0 == num_running_ingest_file_) {
+
+    num_running_ingest_file_--;
+    if (num_running_ingest_file_ == 0) {
       bg_cv_.SignalAll();
     }
+
     TEST_SYNC_POINT("DBImpl::AddFile:MutexUnlock");
   }
   // mutex_ is unlocked here
 
   // Cleanup
-  for (size_t i = 0; i != num_cfs; ++i) {
-    sv_ctxs[i].Clean();
-    // This may rollback jobs that have completed successfully. This is
-    // intended for atomicity.
-    ingestion_jobs[i].Cleanup(status);
-  }
+  sv_context.Clean();
+  ingestion_job.Cleanup(status);
+
   if (status.ok()) {
-    for (size_t i = 0; i != num_cfs; ++i) {
-      auto* cfd =
-          static_cast<ColumnFamilyHandleImpl*>(args[i].column_family)->cfd();
-      if (!cfd->IsDropped()) {
-        NotifyOnExternalFileIngested(cfd, ingestion_jobs[i]);
-      }
-    }
+    NotifyOnExternalFileIngested(cfd, ingestion_job);
   }
+
   return status;
 }
 
@@ -3647,35 +3388,6 @@ Status DBImpl::TraceIteratorSeekForPrev(const uint32_t& cf_id,
   return s;
 }
 
-Status DBImpl::ReserveFileNumbersBeforeIngestion(
-    ColumnFamilyData* cfd, uint64_t num,
-    std::list<uint64_t>::iterator* pending_output_elem,
-    uint64_t* next_file_number) {
-  Status s;
-  SuperVersionContext dummy_sv_ctx(true /* create_superversion */);
-  assert(nullptr != pending_output_elem);
-  assert(nullptr != next_file_number);
-  InstrumentedMutexLock l(&mutex_);
-  if (error_handler_.IsDBStopped()) {
-    // Do not ingest files when there is a bg_error
-    return error_handler_.GetBGError();
-  }
-  *pending_output_elem = CaptureCurrentFileNumberInPendingOutputs();
-  *next_file_number = versions_->FetchAddFileNumber(static_cast<uint64_t>(num));
-  auto cf_options = cfd->GetLatestMutableCFOptions();
-  VersionEdit dummy_edit;
-  // If crash happen after a hard link established, Recover function may
-  // reuse the file number that has already assigned to the internal file,
-  // and this will overwrite the external file. To protect the external
-  // file, we have to make sure the file number will never being reused.
-  s = versions_->LogAndApply(cfd, *cf_options, &dummy_edit, &mutex_,
-                             directories_.GetDbDir());
-  if (s.ok()) {
-    InstallSuperVersionAndScheduleWork(cfd, &dummy_sv_ctx, *cf_options);
-  }
-  dummy_sv_ctx.Clean();
-  return s;
-}
 #endif  // ROCKSDB_LITE
 
 }  // namespace rocksdb

db/db_impl.h

@@ -63,7 +63,6 @@ namespace rocksdb {
 
 class Arena;
 class ArenaWrappedDBIter;
-class InMemoryStatsHistoryIterator;
 class MemTable;
 class TableCache;
 class TaskLimiterToken;
@@ -347,10 +346,6 @@ class DBImpl : public DB {
       const std::vector<std::string>& external_files,
       const IngestExternalFileOptions& ingestion_options) override;
 
-  using DB::IngestExternalFiles;
-  virtual Status IngestExternalFiles(
-      const std::vector<IngestExternalFileArg>& args) override;
-
   virtual Status VerifyChecksum() override;
 
   using DB::StartTrace;
@@ -483,10 +478,7 @@ class DBImpl : public DB {
   int TEST_BGCompactionsAllowed() const;
   int TEST_BGFlushesAllowed() const;
   size_t TEST_GetWalPreallocateBlockSize(uint64_t write_buffer_size) const;
-  void TEST_WaitForDumpStatsRun(std::function<void()> callback) const;
-  void TEST_WaitForPersistStatsRun(std::function<void()> callback) const;
-  bool TEST_IsPersistentStatsEnabled() const;
-  size_t TEST_EstiamteStatsHistorySize() const;
+  void TEST_WaitForTimedTaskRun(std::function<void()> callback) const;
 
 #endif  // NDEBUG
 
@@ -732,17 +724,6 @@ class DBImpl : public DB {
   static Status CreateAndNewDirectory(Env* env, const std::string& dirname,
                                       std::unique_ptr<Directory>* directory);
 
-  // Given a time window, return an iterator for accessing stats history
-  Status GetStatsHistory(
-      uint64_t start_time, uint64_t end_time,
-      std::unique_ptr<StatsHistoryIterator>* stats_iterator) override;
-
-  // find stats map from stats_history_ with smallest timestamp in
-  // the range of [start_time, end_time)
-  bool FindStatsByTime(uint64_t start_time, uint64_t end_time,
-                       uint64_t* new_time,
-                       std::map<std::string, uint64_t>* stats_map);
-
  protected:
   Env* const env_;
   const std::string dbname_;
@@ -1150,11 +1131,6 @@ class DBImpl : public DB {
 
   void PrintStatistics();
 
-  size_t EstiamteStatsHistorySize() const;
-
-  // persist stats to column family "_persistent_stats"
-  void PersistStats();
-
   // dump rocksdb.stats to LOG
   void DumpStats();
 
@@ -1197,8 +1173,6 @@ class DBImpl : public DB {
   // Lock over the persistent DB state.  Non-nullptr iff successfully acquired.
   FileLock* db_lock_;
 
-  // In addition to mutex_, log_write_mutex_ protected writes to stats_history_
-  InstrumentedMutex stats_history_mutex_;
   // In addition to mutex_, log_write_mutex_ protected writes to logs_ and
   // logfile_number_. With two_write_queues it also protects alive_log_files_,
   // and log_empty_. Refer to the definition of each variable below for more
@@ -1321,12 +1295,6 @@ class DBImpl : public DB {
 
   bool is_snapshot_supported_;
 
-  std::map<uint64_t, std::map<std::string, uint64_t>> stats_history_;
-
-  std::map<std::string, uint64_t> stats_slice_;
-
-  bool stats_slice_initialized_ = false;
-
   // Class to maintain directories for all database paths other than main one.
   class Directories {
    public:
@@ -1572,14 +1540,10 @@ class DBImpl : public DB {
   // Only to be set during initialization
   std::unique_ptr<PreReleaseCallback> recoverable_state_pre_release_callback_;
 
-  // handle for scheduling stats dumping at fixed intervals
+  // handle for scheduling jobs at fixed intervals
   // REQUIRES: mutex locked
   std::unique_ptr<rocksdb::RepeatableThread> thread_dump_stats_;
 
-  // handle for scheduling stats snapshoting at fixed intervals
-  // REQUIRES: mutex locked
-  std::unique_ptr<rocksdb::RepeatableThread> thread_persist_stats_;
-
   // No copying allowed
   DBImpl(const DBImpl&);
   void operator=(const DBImpl&);
@@ -1624,14 +1588,7 @@ class DBImpl : public DB {
                               const CompactionJobStats& compaction_job_stats,
                               const int job_id, const Version* current,
                               CompactionJobInfo* compaction_job_info) const;
-  // Reserve the next 'num' file numbers for to-be-ingested external SST files,
-  // and return the current file_number in 'next_file_number'.
-  // Write a version edit to the MANIFEST.
-  Status ReserveFileNumbersBeforeIngestion(
-      ColumnFamilyData* cfd, uint64_t num,
-      std::list<uint64_t>::iterator* pending_output_elem,
-      uint64_t* next_file_number);
-#endif  //! ROCKSDB_LITE
+#endif
 
   bool ShouldPurge(uint64_t file_number) const;
   void MarkAsGrabbedForPurge(uint64_t file_number);

db/db_impl_compaction_flush.cc

@@ -296,7 +296,6 @@ Status DBImpl::AtomicFlushMemTablesToOutputFiles(
                      &earliest_write_conflict_snapshot, &snapshot_checker);
 
   autovector<Directory*> distinct_output_dirs;
-  autovector<std::string> distinct_output_dir_paths;
   std::vector<FlushJob> jobs;
   std::vector<MutableCFOptions> all_mutable_cf_options;
   int num_cfs = static_cast<int>(cfds.size());
@@ -304,20 +303,18 @@ Status DBImpl::AtomicFlushMemTablesToOutputFiles(
   for (int i = 0; i < num_cfs; ++i) {
     auto cfd = cfds[i];
     Directory* data_dir = GetDataDir(cfd, 0U);
-    const std::string& curr_path = cfd->ioptions()->cf_paths[0].path;
 
     // Add to distinct output directories if eligible. Use linear search. Since
     // the number of elements in the vector is not large, performance should be
     // tolerable.
     bool found = false;
-    for (const auto& path : distinct_output_dir_paths) {
-      if (path == curr_path) {
+    for (const auto dir : distinct_output_dirs) {
+      if (dir == data_dir) {
         found = true;
         break;
       }
     }
     if (!found) {
-      distinct_output_dir_paths.emplace_back(curr_path);
       distinct_output_dirs.emplace_back(data_dir);
     }
 
@@ -325,7 +322,7 @@ Status DBImpl::AtomicFlushMemTablesToOutputFiles(
     const MutableCFOptions& mutable_cf_options = all_mutable_cf_options.back();
     const uint64_t* max_memtable_id = &(bg_flush_args[i].max_memtable_id_);
     jobs.emplace_back(
-        dbname_, cfd, immutable_db_options_, mutable_cf_options,
+        dbname_, cfds[i], immutable_db_options_, mutable_cf_options,
         max_memtable_id, env_options_for_compaction_, versions_.get(), &mutex_,
         &shutting_down_, snapshot_seqs, earliest_write_conflict_snapshot,
         snapshot_checker, job_context, log_buffer, directories_.GetDbDir(),
@@ -335,18 +332,21 @@ Status DBImpl::AtomicFlushMemTablesToOutputFiles(
     jobs.back().PickMemTable();
   }
 
-  std::vector<FileMetaData> file_meta(num_cfs);
+  autovector<FileMetaData> file_meta;
   Status s;
   assert(num_cfs == static_cast<int>(jobs.size()));
 
-#ifndef ROCKSDB_LITE
   for (int i = 0; i != num_cfs; ++i) {
-    const MutableCFOptions& mutable_cf_options = all_mutable_cf_options.at(i);
+    file_meta.emplace_back();
+
+#ifndef ROCKSDB_LITE
+    const MutableCFOptions& mutable_cf_options =
+        *cfds[i]->GetLatestMutableCFOptions();
     // may temporarily unlock and lock the mutex.
     NotifyOnFlushBegin(cfds[i], &file_meta[i], mutable_cf_options,
                        job_context->job_id, jobs[i].GetTableProperties());
-  }
 #endif /* !ROCKSDB_LITE */
+  }
 
   if (logfile_number_ > 0) {
     // TODO (yanqin) investigate whether we should sync the closed logs for
@@ -450,21 +450,19 @@ Status DBImpl::AtomicFlushMemTablesToOutputFiles(
     autovector<ColumnFamilyData*> tmp_cfds;
     autovector<const autovector<MemTable*>*> mems_list;
     autovector<const MutableCFOptions*> mutable_cf_options_list;
-    autovector<FileMetaData*> tmp_file_meta;
     for (int i = 0; i != num_cfs; ++i) {
       const auto& mems = jobs[i].GetMemTables();
       if (!cfds[i]->IsDropped() && !mems.empty()) {
         tmp_cfds.emplace_back(cfds[i]);
         mems_list.emplace_back(&mems);
         mutable_cf_options_list.emplace_back(&all_mutable_cf_options[i]);
-        tmp_file_meta.emplace_back(&file_meta[i]);
       }
     }
 
     s = InstallMemtableAtomicFlushResults(
         nullptr /* imm_lists */, tmp_cfds, mutable_cf_options_list, mems_list,
-        versions_.get(), &mutex_, tmp_file_meta,
-        &job_context->memtables_to_free, directories_.GetDbDir(), log_buffer);
+        versions_.get(), &mutex_, file_meta, &job_context->memtables_to_free,
+        directories_.GetDbDir(), log_buffer);
   }
 
   if (s.ok() || s.IsShutdownInProgress()) {
@@ -476,7 +474,7 @@ Status DBImpl::AtomicFlushMemTablesToOutputFiles(
       }
       InstallSuperVersionAndScheduleWork(cfds[i],
                                          &job_context->superversion_contexts[i],
-                                         all_mutable_cf_options[i]);
+                                         *cfds[i]->GetLatestMutableCFOptions());
       VersionStorageInfo::LevelSummaryStorage tmp;
       ROCKS_LOG_BUFFER(log_buffer, "[%s] Level summary: %s\n",
                        cfds[i]->GetName().c_str(),
@@ -492,7 +490,8 @@ Status DBImpl::AtomicFlushMemTablesToOutputFiles(
       if (cfds[i]->IsDropped()) {
         continue;
       }
-      NotifyOnFlushCompleted(cfds[i], &file_meta[i], all_mutable_cf_options[i],
+      NotifyOnFlushCompleted(cfds[i], &file_meta[i],
+                             *cfds[i]->GetLatestMutableCFOptions(),
                              job_context->job_id, jobs[i].GetTableProperties());
       if (sfm) {
         std::string file_path = MakeTableFileName(
@@ -1026,7 +1025,6 @@ Status DBImpl::CompactFilesImpl(
   if (bg_compaction_scheduled_ == 0) {
     bg_cv_.SignalAll();
   }
-  MaybeScheduleFlushOrCompaction();
   TEST_SYNC_POINT("CompactFilesImpl:End");
 
   return status;

db/db_impl_debug.cc

@@ -243,24 +243,10 @@ size_t DBImpl::TEST_GetWalPreallocateBlockSize(
   return GetWalPreallocateBlockSize(write_buffer_size);
 }
 
-void DBImpl::TEST_WaitForDumpStatsRun(std::function<void()> callback) const {
+void DBImpl::TEST_WaitForTimedTaskRun(std::function<void()> callback) const {
   if (thread_dump_stats_ != nullptr) {
     thread_dump_stats_->TEST_WaitForRun(callback);
   }
 }
-
-void DBImpl::TEST_WaitForPersistStatsRun(std::function<void()> callback) const {
-  if (thread_persist_stats_ != nullptr) {
-    thread_persist_stats_->TEST_WaitForRun(callback);
-  }
-}
-
-bool DBImpl::TEST_IsPersistentStatsEnabled() const {
-  return thread_persist_stats_ && thread_persist_stats_->IsRunning();
-}
-
-size_t DBImpl::TEST_EstiamteStatsHistorySize() const {
-  return EstiamteStatsHistorySize();
-}
 }  // namespace rocksdb
 #endif  // NDEBUG

db/db_impl_open.cc

@@ -176,7 +176,7 @@ static Status ValidateOptions(
       return s;
     }
 
-    if (cfd.options.ttl > 0) {
+    if (cfd.options.ttl > 0 || cfd.options.compaction_options_fifo.ttl > 0) {
       if (db_options.max_open_files != -1) {
         return Status::NotSupported(
             "TTL is only supported when files are always "
@@ -512,7 +512,7 @@ Status DBImpl::RecoverLogFiles(const std::vector<uint64_t>& log_numbers,
     Logger* info_log;
     const char* fname;
     Status* status;  // nullptr if immutable_db_options_.paranoid_checks==false
-    void Corruption(size_t bytes, const Status& s) override {
+    virtual void Corruption(size_t bytes, const Status& s) override {
       ROCKS_LOG_WARN(info_log, "%s%s: dropping %d bytes; %s",
                      (this->status == nullptr ? "(ignoring error) " : ""),
                      fname, static_cast<int>(bytes), s.ToString().c_str());
@@ -1316,8 +1316,7 @@ Status DBImpl::Open(const DBOptions& db_options, const std::string& dbname,
 #endif  // !ROCKSDB_LITE
 
   if (s.ok()) {
-    ROCKS_LOG_HEADER(impl->immutable_db_options_.info_log, "DB pointer %p",
-                     impl);
+    ROCKS_LOG_INFO(impl->immutable_db_options_.info_log, "DB pointer %p", impl);
     LogFlush(impl->immutable_db_options_.info_log);
     assert(impl->TEST_WALBufferIsEmpty());
     // If the assert above fails then we need to FlushWAL before returning

db/db_iter.cc

@@ -152,7 +152,7 @@ class DBIter final: public Iterator {
       iter_->SetPinnedItersMgr(&pinned_iters_mgr_);
     }
   }
-  ~DBIter() override {
+  virtual ~DBIter() {
     // Release pinned data if any
     if (pinned_iters_mgr_.PinningEnabled()) {
       pinned_iters_mgr_.ReleasePinnedData();
@@ -175,16 +175,17 @@ class DBIter final: public Iterator {
     return &range_del_agg_;
   }
 
-  bool Valid() const override { return valid_; }
-  Slice key() const override {
+  virtual bool Valid() const override { return valid_; }
+  virtual Slice key() const override {
     assert(valid_);
     if(start_seqnum_ > 0) {
       return saved_key_.GetInternalKey();
     } else {
       return saved_key_.GetUserKey();
     }
+
   }
-  Slice value() const override {
+  virtual Slice value() const override {
     assert(valid_);
     if (current_entry_is_merged_) {
       // If pinned_value_ is set then the result of merge operator is one of
@@ -196,7 +197,7 @@ class DBIter final: public Iterator {
       return iter_->value();
     }
   }
-  Status status() const override {
+  virtual Status status() const override {
     if (status_.ok()) {
       return iter_->status();
     } else {
@@ -209,7 +210,8 @@ class DBIter final: public Iterator {
     return is_blob_;
   }
 
-  Status GetProperty(std::string prop_name, std::string* prop) override {
+  virtual Status GetProperty(std::string prop_name,
+                             std::string* prop) override {
     if (prop == nullptr) {
       return Status::InvalidArgument("prop is nullptr");
     }
@@ -230,12 +232,12 @@ class DBIter final: public Iterator {
     return Status::InvalidArgument("Unidentified property.");
   }
 
-  void Next() override;
-  void Prev() override;
-  void Seek(const Slice& target) override;
-  void SeekForPrev(const Slice& target) override;
-  void SeekToFirst() override;
-  void SeekToLast() override;
+  virtual void Next() override;
+  virtual void Prev() override;
+  virtual void Seek(const Slice& target) override;
+  virtual void SeekForPrev(const Slice& target) override;
+  virtual void SeekToFirst() override;
+  virtual void SeekToLast() override;
   Env* env() { return env_; }
   void set_sequence(uint64_t s) { sequence_ = s; }
   void set_valid(bool v) { valid_ = v; }

db/db_iter_test.cc

@@ -116,12 +116,12 @@ class TestIterator : public InternalIterator {
   // Number of operations done on this iterator since construction.
   size_t steps() const { return steps_; }
 
-  bool Valid() const override {
+  virtual bool Valid() const override {
     assert(initialized_);
     return valid_;
   }
 
-  void SeekToFirst() override {
+  virtual void SeekToFirst() override {
     assert(initialized_);
     ++steps_;
     DeleteCurrentIfNeeded();
@@ -129,7 +129,7 @@ class TestIterator : public InternalIterator {
     iter_ = 0;
   }
 
-  void SeekToLast() override {
+  virtual void SeekToLast() override {
     assert(initialized_);
     ++steps_;
     DeleteCurrentIfNeeded();
@@ -137,7 +137,7 @@ class TestIterator : public InternalIterator {
     iter_ = data_.size() - 1;
   }
 
-  void Seek(const Slice& target) override {
+  virtual void Seek(const Slice& target) override {
     assert(initialized_);
     SeekToFirst();
     ++steps_;
@@ -154,13 +154,13 @@ class TestIterator : public InternalIterator {
     }
   }
 
-  void SeekForPrev(const Slice& target) override {
+  virtual void SeekForPrev(const Slice& target) override {
     assert(initialized_);
     DeleteCurrentIfNeeded();
     SeekForPrevImpl(target, &cmp);
   }
 
-  void Next() override {
+  virtual void Next() override {
     assert(initialized_);
     assert(valid_);
     assert(iter_ < data_.size());
@@ -174,7 +174,7 @@ class TestIterator : public InternalIterator {
     valid_ = iter_ < data_.size();
   }
 
-  void Prev() override {
+  virtual void Prev() override {
     assert(initialized_);
     assert(valid_);
     assert(iter_ < data_.size());
@@ -188,23 +188,23 @@ class TestIterator : public InternalIterator {
     }
   }
 
-  Slice key() const override {
+  virtual Slice key() const override {
     assert(initialized_);
     return data_[iter_].first;
   }
 
-  Slice value() const override {
+  virtual Slice value() const override {
     assert(initialized_);
     return data_[iter_].second;
   }
 
-  Status status() const override {
+  virtual Status status() const override {
     assert(initialized_);
     return Status::OK();
   }
 
-  bool IsKeyPinned() const override { return true; }
-  bool IsValuePinned() const override { return true; }
+  virtual bool IsKeyPinned() const override { return true; }
+  virtual bool IsValuePinned() const override { return true; }
 
  private:
   bool initialized_;

db/db_iterator_test.cc

@@ -50,14 +50,13 @@ class DBIteratorTest : public DBTestBase,
 
 class FlushBlockEveryKeyPolicy : public FlushBlockPolicy {
  public:
-  bool Update(const Slice& /*key*/, const Slice& /*value*/) override {
+  virtual bool Update(const Slice& /*key*/, const Slice& /*value*/) override {
     if (!start_) {
       start_ = true;
       return false;
     }
     return true;
   }
-
  private:
   bool start_ = false;
 };
@@ -180,7 +179,9 @@ TEST_P(DBIteratorTest, NonBlockingIteration) {
 
     // This test verifies block cache behaviors, which is not used by plain
     // table format.
-  } while (ChangeOptions(kSkipPlainTable | kSkipNoSeekToLast | kSkipMmapReads));
+    // Exclude kHashCuckoo as it does not support iteration currently
+  } while (ChangeOptions(kSkipPlainTable | kSkipNoSeekToLast | kSkipHashCuckoo |
+                         kSkipMmapReads));
 }
 
 TEST_P(DBIteratorTest, IterSeekBeforePrev) {
@@ -764,7 +765,8 @@ TEST_P(DBIteratorTest, IterWithSnapshot) {
     }
     db_->ReleaseSnapshot(snapshot);
     delete iter;
-  } while (ChangeOptions());
+    // skip as HashCuckooRep does not support snapshot
+  } while (ChangeOptions(kSkipHashCuckoo));
 }
 
 TEST_P(DBIteratorTest, IteratorPinsRef) {

db/db_log_iter_test.cc

@@ -249,20 +249,22 @@ TEST_F(DBTestXactLogIterator, TransactionLogIteratorBlobs) {
   auto res = OpenTransactionLogIter(0)->GetBatch();
   struct Handler : public WriteBatch::Handler {
     std::string seen;
-    Status PutCF(uint32_t cf, const Slice& key, const Slice& value) override {
+    virtual Status PutCF(uint32_t cf, const Slice& key,
+                         const Slice& value) override {
       seen += "Put(" + ToString(cf) + ", " + key.ToString() + ", " +
               ToString(value.size()) + ")";
       return Status::OK();
     }
-    Status MergeCF(uint32_t cf, const Slice& key, const Slice& value) override {
+    virtual Status MergeCF(uint32_t cf, const Slice& key,
+                           const Slice& value) override {
       seen += "Merge(" + ToString(cf) + ", " + key.ToString() + ", " +
               ToString(value.size()) + ")";
       return Status::OK();
     }
-    void LogData(const Slice& blob) override {
+    virtual void LogData(const Slice& blob) override {
       seen += "LogData(" + blob.ToString() + ")";
     }
-    Status DeleteCF(uint32_t cf, const Slice& key) override {
+    virtual Status DeleteCF(uint32_t cf, const Slice& key) override {
       seen += "Delete(" + ToString(cf) + ", " + key.ToString() + ")";
       return Status::OK();
     }

db/db_memtable_test.cc

@@ -25,13 +25,13 @@ class MockMemTableRep : public MemTableRep {
   explicit MockMemTableRep(Allocator* allocator, MemTableRep* rep)
       : MemTableRep(allocator), rep_(rep), num_insert_with_hint_(0) {}
 
-  KeyHandle Allocate(const size_t len, char** buf) override {
+  virtual KeyHandle Allocate(const size_t len, char** buf) override {
     return rep_->Allocate(len, buf);
   }
 
-  void Insert(KeyHandle handle) override { rep_->Insert(handle); }
+  virtual void Insert(KeyHandle handle) override { rep_->Insert(handle); }
 
-  void InsertWithHint(KeyHandle handle, void** hint) override {
+  virtual void InsertWithHint(KeyHandle handle, void** hint) override {
     num_insert_with_hint_++;
     EXPECT_NE(nullptr, hint);
     last_hint_in_ = *hint;
@@ -39,18 +39,21 @@ class MockMemTableRep : public MemTableRep {
     last_hint_out_ = *hint;
   }
 
-  bool Contains(const char* key) const override { return rep_->Contains(key); }
+  virtual bool Contains(const char* key) const override {
+    return rep_->Contains(key);
+  }
 
-  void Get(const LookupKey& k, void* callback_args,
-           bool (*callback_func)(void* arg, const char* entry)) override {
+  virtual void Get(const LookupKey& k, void* callback_args,
+                   bool (*callback_func)(void* arg,
+                                         const char* entry)) override {
     rep_->Get(k, callback_args, callback_func);
   }
 
-  size_t ApproximateMemoryUsage() override {
+  virtual size_t ApproximateMemoryUsage() override {
     return rep_->ApproximateMemoryUsage();
   }
 
-  Iterator* GetIterator(Arena* arena) override {
+  virtual Iterator* GetIterator(Arena* arena) override {
     return rep_->GetIterator(arena);
   }
 
@@ -67,10 +70,10 @@ class MockMemTableRep : public MemTableRep {
 
 class MockMemTableRepFactory : public MemTableRepFactory {
  public:
-  MemTableRep* CreateMemTableRep(const MemTableRep::KeyComparator& cmp,
-                                 Allocator* allocator,
-                                 const SliceTransform* transform,
-                                 Logger* logger) override {
+  virtual MemTableRep* CreateMemTableRep(const MemTableRep::KeyComparator& cmp,
+                                         Allocator* allocator,
+                                         const SliceTransform* transform,
+                                         Logger* logger) override {
     SkipListFactory factory;
     MemTableRep* skiplist_rep =
         factory.CreateMemTableRep(cmp, allocator, transform, logger);
@@ -78,16 +81,16 @@ class MockMemTableRepFactory : public MemTableRepFactory {
     return mock_rep_;
   }
 
-  MemTableRep* CreateMemTableRep(const MemTableRep::KeyComparator& cmp,
-                                 Allocator* allocator,
-                                 const SliceTransform* transform,
-                                 Logger* logger,
-                                 uint32_t column_family_id) override {
+  virtual MemTableRep* CreateMemTableRep(const MemTableRep::KeyComparator& cmp,
+                                         Allocator* allocator,
+                                         const SliceTransform* transform,
+                                         Logger* logger,
+                                         uint32_t column_family_id) override {
     last_column_family_id_ = column_family_id;
     return CreateMemTableRep(cmp, allocator, transform, logger);
   }
 
-  const char* Name() const override { return "MockMemTableRepFactory"; }
+  virtual const char* Name() const override { return "MockMemTableRepFactory"; }
 
   MockMemTableRep* rep() { return mock_rep_; }
 
@@ -103,9 +106,9 @@ class MockMemTableRepFactory : public MemTableRepFactory {
 
 class TestPrefixExtractor : public SliceTransform {
  public:
-  const char* Name() const override { return "TestPrefixExtractor"; }
+  virtual const char* Name() const override { return "TestPrefixExtractor"; }
 
-  Slice Transform(const Slice& key) const override {
+  virtual Slice Transform(const Slice& key) const override {
     const char* p = separator(key);
     if (p == nullptr) {
       return Slice();
@@ -113,11 +116,11 @@ class TestPrefixExtractor : public SliceTransform {
     return Slice(key.data(), p - key.data() + 1);
   }
 
-  bool InDomain(const Slice& key) const override {
+  virtual bool InDomain(const Slice& key) const override {
     return separator(key) != nullptr;
   }
 
-  bool InRange(const Slice& /*key*/) const override { return false; }
+  virtual bool InRange(const Slice& /*key*/) const override { return false; }
 
  private:
   const char* separator(const Slice& key) const {

db/db_merge_operator_test.cc

@@ -332,7 +332,8 @@ TEST_P(MergeOperatorPinningTest, Randomized) {
 
     VerifyDBFromMap(true_data);
 
-  } while (ChangeOptions(kSkipMergePut));
+    // Skip HashCuckoo since it does not support merge operators
+  } while (ChangeOptions(kSkipMergePut | kSkipHashCuckoo));
 }
 
 class MergeOperatorHook : public MergeOperator {
@@ -340,15 +341,15 @@ class MergeOperatorHook : public MergeOperator {
   explicit MergeOperatorHook(std::shared_ptr<MergeOperator> _merge_op)
       : merge_op_(_merge_op) {}
 
-  bool FullMergeV2(const MergeOperationInput& merge_in,
-                   MergeOperationOutput* merge_out) const override {
+  virtual bool FullMergeV2(const MergeOperationInput& merge_in,
+                           MergeOperationOutput* merge_out) const override {
     before_merge_();
     bool res = merge_op_->FullMergeV2(merge_in, merge_out);
     after_merge_();
     return res;
   }
 
-  const char* Name() const override { return merge_op_->Name(); }
+  virtual const char* Name() const override { return merge_op_->Name(); }
 
   std::shared_ptr<MergeOperator> merge_op_;
   std::function<void()> before_merge_ = []() {};

db/db_options_test.cc

@@ -18,15 +18,12 @@
 #include "rocksdb/cache.h"
 #include "rocksdb/convenience.h"
 #include "rocksdb/rate_limiter.h"
-#include "rocksdb/stats_history.h"
 #include "util/random.h"
 #include "util/sync_point.h"
 #include "util/testutil.h"
 
 namespace rocksdb {
 
-const int kMicrosInSec = 1000000;
-
 class DBOptionsTest : public DBTestBase {
  public:
   DBOptionsTest() : DBTestBase("/db_options_test") {}
@@ -511,11 +508,10 @@ TEST_F(DBOptionsTest, SetStatsDumpPeriodSec) {
 
   for (int i = 0; i < 20; i++) {
     int num = rand() % 5000 + 1;
-    ASSERT_OK(
-        dbfull()->SetDBOptions({{"stats_dump_period_sec", ToString(num)}}));
+    ASSERT_OK(dbfull()->SetDBOptions(
+        {{"stats_dump_period_sec", std::to_string(num)}}));
     ASSERT_EQ(num, dbfull()->GetDBOptions().stats_dump_period_sec);
   }
-  Close();
 }
 
 TEST_F(DBOptionsTest, RunStatsDumpPeriodSec) {
@@ -534,210 +530,17 @@ TEST_F(DBOptionsTest, RunStatsDumpPeriodSec) {
   rocksdb::SyncPoint::GetInstance()->EnableProcessing();
   Reopen(options);
   ASSERT_EQ(5, dbfull()->GetDBOptions().stats_dump_period_sec);
-  dbfull()->TEST_WaitForDumpStatsRun([&] { mock_env->set_current_time(5); });
+  dbfull()->TEST_WaitForTimedTaskRun([&] { mock_env->set_current_time(5); });
   ASSERT_GE(counter, 1);
 
   // Test cacel job through SetOptions
   ASSERT_OK(dbfull()->SetDBOptions({{"stats_dump_period_sec", "0"}}));
   int old_val = counter;
-  for (int i = 6; i < 20; ++i) {
-    dbfull()->TEST_WaitForDumpStatsRun([&] { mock_env->set_current_time(i); });
-  }
+  env_->SleepForMicroseconds(10000000);
   ASSERT_EQ(counter, old_val);
   Close();
 }
 
-// Test persistent stats background thread scheduling and cancelling
-TEST_F(DBOptionsTest, StatsPersistScheduling) {
-  Options options;
-  options.create_if_missing = true;
-  options.stats_persist_period_sec = 5;
-  std::unique_ptr<rocksdb::MockTimeEnv> mock_env;
-  mock_env.reset(new rocksdb::MockTimeEnv(env_));
-  mock_env->set_current_time(0);  // in seconds
-  options.env = mock_env.get();
-  int counter = 0;
-  rocksdb::SyncPoint::GetInstance()->SetCallBack(
-      "DBImpl::PersistStats:Entry", [&](void* /*arg*/) { counter++; });
-  rocksdb::SyncPoint::GetInstance()->EnableProcessing();
-  Reopen(options);
-  ASSERT_EQ(5, dbfull()->GetDBOptions().stats_persist_period_sec);
-  dbfull()->TEST_WaitForPersistStatsRun([&] { mock_env->set_current_time(5); });
-  ASSERT_GE(counter, 1);
-
-  // Test cacel job through SetOptions
-  ASSERT_TRUE(dbfull()->TEST_IsPersistentStatsEnabled());
-  ASSERT_OK(dbfull()->SetDBOptions({{"stats_persist_period_sec", "0"}}));
-  ASSERT_FALSE(dbfull()->TEST_IsPersistentStatsEnabled());
-  Close();
-}
-
-// Test enabling persistent stats for the first time
-TEST_F(DBOptionsTest, PersistentStatsFreshInstall) {
-  Options options;
-  options.create_if_missing = true;
-  options.stats_persist_period_sec = 0;
-  std::unique_ptr<rocksdb::MockTimeEnv> mock_env;
-  mock_env.reset(new rocksdb::MockTimeEnv(env_));
-  mock_env->set_current_time(0);  // in seconds
-  options.env = mock_env.get();
-  int counter = 0;
-  rocksdb::SyncPoint::GetInstance()->SetCallBack(
-      "DBImpl::PersistStats:Entry", [&](void* /*arg*/) { counter++; });
-  rocksdb::SyncPoint::GetInstance()->EnableProcessing();
-  Reopen(options);
-  ASSERT_OK(dbfull()->SetDBOptions({{"stats_persist_period_sec", "5"}}));
-  ASSERT_EQ(5, dbfull()->GetDBOptions().stats_persist_period_sec);
-  dbfull()->TEST_WaitForPersistStatsRun([&] { mock_env->set_current_time(5); });
-  ASSERT_GE(counter, 1);
-  Close();
-}
-
-TEST_F(DBOptionsTest, SetOptionsStatsPersistPeriodSec) {
-  Options options;
-  options.create_if_missing = true;
-  options.stats_persist_period_sec = 5;
-  options.env = env_;
-  Reopen(options);
-  ASSERT_EQ(5, dbfull()->GetDBOptions().stats_persist_period_sec);
-
-  ASSERT_OK(dbfull()->SetDBOptions({{"stats_persist_period_sec", "12345"}}));
-  ASSERT_EQ(12345, dbfull()->GetDBOptions().stats_persist_period_sec);
-  ASSERT_NOK(dbfull()->SetDBOptions({{"stats_persist_period_sec", "abcde"}}));
-  ASSERT_EQ(12345, dbfull()->GetDBOptions().stats_persist_period_sec);
-}
-
-TEST_F(DBOptionsTest, GetStatsHistory) {
-  Options options;
-  options.create_if_missing = true;
-  options.stats_persist_period_sec = 5;
-  options.statistics = rocksdb::CreateDBStatistics();
-  std::unique_ptr<rocksdb::MockTimeEnv> mock_env;
-  mock_env.reset(new rocksdb::MockTimeEnv(env_));
-  mock_env->set_current_time(0);  // in seconds
-  options.env = mock_env.get();
-  CreateColumnFamilies({"pikachu"}, options);
-  ASSERT_OK(Put("foo", "bar"));
-  ReopenWithColumnFamilies({"default", "pikachu"}, options);
-
-  int mock_time = 1;
-  // Wait for stats persist to finish
-  dbfull()->TEST_WaitForPersistStatsRun([&] { mock_env->set_current_time(5); });
-  std::unique_ptr<StatsHistoryIterator> stats_iter;
-  db_->GetStatsHistory(0, 6 * kMicrosInSec, &stats_iter);
-  ASSERT_TRUE(stats_iter != nullptr);
-  // disabled stats snapshots
-  ASSERT_OK(dbfull()->SetDBOptions({{"stats_persist_period_sec", "0"}}));
-  size_t stats_count = 0;
-  for (; stats_iter->Valid(); stats_iter->Next()) {
-    auto stats_map = stats_iter->GetStatsMap();
-    stats_count += stats_map.size();
-  }
-  ASSERT_GT(stats_count, 0);
-  // Wait a bit and verify no more stats are found
-  for (mock_time = 6; mock_time < 20; ++mock_time) {
-    dbfull()->TEST_WaitForPersistStatsRun(
-        [&] { mock_env->set_current_time(mock_time); });
-  }
-  db_->GetStatsHistory(0, 20 * kMicrosInSec, &stats_iter);
-  ASSERT_TRUE(stats_iter != nullptr);
-  size_t stats_count_new = 0;
-  for (; stats_iter->Valid(); stats_iter->Next()) {
-    stats_count_new += stats_iter->GetStatsMap().size();
-  }
-  ASSERT_EQ(stats_count_new, stats_count);
-  Close();
-}
-
-TEST_F(DBOptionsTest, InMemoryStatsHistoryPurging) {
-  Options options;
-  options.create_if_missing = true;
-  options.statistics = rocksdb::CreateDBStatistics();
-  options.stats_persist_period_sec = 1;
-  std::unique_ptr<rocksdb::MockTimeEnv> mock_env;
-  mock_env.reset(new rocksdb::MockTimeEnv(env_));
-  mock_env->set_current_time(0);  // in seconds
-  options.env = mock_env.get();
-  CreateColumnFamilies({"pikachu"}, options);
-  ASSERT_OK(Put("foo", "bar"));
-  ReopenWithColumnFamilies({"default", "pikachu"}, options);
-  // some random operation to populate statistics
-  ASSERT_OK(Delete("foo"));
-  ASSERT_OK(Put("sol", "sol"));
-  ASSERT_OK(Put("epic", "epic"));
-  ASSERT_OK(Put("ltd", "ltd"));
-  ASSERT_EQ("sol", Get("sol"));
-  ASSERT_EQ("epic", Get("epic"));
-  ASSERT_EQ("ltd", Get("ltd"));
-  Iterator* iterator = db_->NewIterator(ReadOptions());
-  for (iterator->SeekToFirst(); iterator->Valid(); iterator->Next()) {
-    ASSERT_TRUE(iterator->key() == iterator->value());
-  }
-  delete iterator;
-  ASSERT_OK(Flush());
-  ASSERT_OK(Delete("sol"));
-  db_->CompactRange(CompactRangeOptions(), nullptr, nullptr);
-  int mock_time = 1;
-  // Wait for stats persist to finish
-  for (; mock_time < 5; ++mock_time) {
-    dbfull()->TEST_WaitForPersistStatsRun(
-        [&] { mock_env->set_current_time(mock_time); });
-  }
-
-  // second round of ops
-  ASSERT_OK(Put("saigon", "saigon"));
-  ASSERT_OK(Put("noodle talk", "noodle talk"));
-  ASSERT_OK(Put("ping bistro", "ping bistro"));
-  iterator = db_->NewIterator(ReadOptions());
-  for (iterator->SeekToFirst(); iterator->Valid(); iterator->Next()) {
-    ASSERT_TRUE(iterator->key() == iterator->value());
-  }
-  delete iterator;
-  ASSERT_OK(Flush());
-  db_->CompactRange(CompactRangeOptions(), nullptr, nullptr);
-  for (; mock_time < 10; ++mock_time) {
-    dbfull()->TEST_WaitForPersistStatsRun(
-        [&] { mock_env->set_current_time(mock_time); });
-  }
-  std::unique_ptr<StatsHistoryIterator> stats_iter;
-  db_->GetStatsHistory(0, 10 * kMicrosInSec, &stats_iter);
-  ASSERT_TRUE(stats_iter != nullptr);
-  size_t stats_count = 0;
-  int slice_count = 0;
-  for (; stats_iter->Valid(); stats_iter->Next()) {
-    slice_count++;
-    auto stats_map = stats_iter->GetStatsMap();
-    stats_count += stats_map.size();
-  }
-  size_t stats_history_size = dbfull()->TEST_EstiamteStatsHistorySize();
-  ASSERT_GE(slice_count, 9);
-  ASSERT_GE(stats_history_size, 12000);
-  // capping memory cost at 12000 bytes since one slice is around 10000~12000
-  ASSERT_OK(dbfull()->SetDBOptions({{"stats_history_buffer_size", "12000"}}));
-  ASSERT_EQ(12000, dbfull()->GetDBOptions().stats_history_buffer_size);
-  // Wait for stats persist to finish
-  for (; mock_time < 20; ++mock_time) {
-    dbfull()->TEST_WaitForPersistStatsRun(
-        [&] { mock_env->set_current_time(mock_time); });
-  }
-  db_->GetStatsHistory(0, 20 * kMicrosInSec, &stats_iter);
-  ASSERT_TRUE(stats_iter != nullptr);
-  size_t stats_count_reopen = 0;
-  slice_count = 0;
-  for (; stats_iter->Valid(); stats_iter->Next()) {
-    slice_count++;
-    auto stats_map = stats_iter->GetStatsMap();
-    stats_count_reopen += stats_map.size();
-  }
-  size_t stats_history_size_reopen = dbfull()->TEST_EstiamteStatsHistorySize();
-  // only one slice can fit under the new stats_history_buffer_size
-  ASSERT_LT(slice_count, 2);
-  ASSERT_TRUE(stats_history_size_reopen < 12000 &&
-              stats_history_size_reopen > 0);
-  ASSERT_TRUE(stats_count_reopen < stats_count && stats_count_reopen > 0);
-  Close();
-}
-
 static void assert_candidate_files_empty(DBImpl* dbfull, const bool empty) {
   dbfull->TEST_LockMutex();
   JobContext job_context(0);
@@ -825,9 +628,9 @@ TEST_F(DBOptionsTest, SetFIFOCompactionOptions) {
   env_->time_elapse_only_sleep_ = false;
   options.env = env_;
 
-  // Test dynamically changing ttl.
+  // Test dynamically changing compaction_options_fifo.ttl
   env_->addon_time_.store(0);
-  options.ttl = 1 * 60 * 60;  // 1 hour
+  options.compaction_options_fifo.ttl = 1 * 60 * 60;  // 1 hour
   ASSERT_OK(TryReopen(options));
 
   Random rnd(301);
@@ -845,13 +648,13 @@ TEST_F(DBOptionsTest, SetFIFOCompactionOptions) {
   env_->addon_time_.fetch_add(61);
 
   // No files should be compacted as ttl is set to 1 hour.
-  ASSERT_EQ(dbfull()->GetOptions().ttl, 3600);
+  ASSERT_EQ(dbfull()->GetOptions().compaction_options_fifo.ttl, 3600);
   dbfull()->CompactRange(CompactRangeOptions(), nullptr, nullptr);
   ASSERT_EQ(NumTableFilesAtLevel(0), 10);
 
   // Set ttl to 1 minute. So all files should get deleted.
-  ASSERT_OK(dbfull()->SetOptions({{"ttl", "60"}}));
-  ASSERT_EQ(dbfull()->GetOptions().ttl, 60);
+  ASSERT_OK(dbfull()->SetOptions({{"compaction_options_fifo", "{ttl=60;}"}}));
+  ASSERT_EQ(dbfull()->GetOptions().compaction_options_fifo.ttl, 60);
   dbfull()->CompactRange(CompactRangeOptions(), nullptr, nullptr);
   ASSERT_OK(dbfull()->TEST_WaitForCompact());
   ASSERT_EQ(NumTableFilesAtLevel(0), 0);
@@ -859,7 +662,7 @@ TEST_F(DBOptionsTest, SetFIFOCompactionOptions) {
   // Test dynamically changing compaction_options_fifo.max_table_files_size
   env_->addon_time_.store(0);
   options.compaction_options_fifo.max_table_files_size = 500 << 10;  // 00KB
-  options.ttl = 0;
+  options.compaction_options_fifo.ttl = 0;
   DestroyAndReopen(options);
 
   for (int i = 0; i < 10; i++) {
@@ -889,7 +692,7 @@ TEST_F(DBOptionsTest, SetFIFOCompactionOptions) {
 
   // Test dynamically changing compaction_options_fifo.allow_compaction
   options.compaction_options_fifo.max_table_files_size = 500 << 10;  // 500KB
-  options.ttl = 0;
+  options.compaction_options_fifo.ttl = 0;
   options.compaction_options_fifo.allow_compaction = false;
   options.level0_file_num_compaction_trigger = 6;
   DestroyAndReopen(options);

db/db_properties_test.cc

@@ -1094,7 +1094,7 @@ class CountingUserTblPropCollector : public TablePropertiesCollector {
     return Status::OK();
   }
 
-  UserCollectedProperties GetReadableProperties() const override {
+  virtual UserCollectedProperties GetReadableProperties() const override {
     return UserCollectedProperties{};
   }
 
@@ -1110,7 +1110,7 @@ class CountingUserTblPropCollectorFactory
       uint32_t expected_column_family_id)
       : expected_column_family_id_(expected_column_family_id),
         num_created_(0) {}
-  TablePropertiesCollector* CreateTablePropertiesCollector(
+  virtual TablePropertiesCollector* CreateTablePropertiesCollector(
       TablePropertiesCollectorFactory::Context context) override {
     EXPECT_EQ(expected_column_family_id_, context.column_family_id);
     num_created_++;
@@ -1158,7 +1158,7 @@ class CountingDeleteTabPropCollector : public TablePropertiesCollector {
 class CountingDeleteTabPropCollectorFactory
     : public TablePropertiesCollectorFactory {
  public:
-  TablePropertiesCollector* CreateTablePropertiesCollector(
+  virtual TablePropertiesCollector* CreateTablePropertiesCollector(
       TablePropertiesCollectorFactory::Context /*context*/) override {
     return new CountingDeleteTabPropCollector();
   }
@@ -1432,7 +1432,7 @@ TEST_F(DBPropertiesTest, EstimateOldestKeyTime) {
   }
 
   options.compaction_style = kCompactionStyleFIFO;
-  options.ttl = 300;
+  options.compaction_options_fifo.ttl = 300;
   options.compaction_options_fifo.allow_compaction = false;
   DestroyAndReopen(options);
 

db/db_range_del_test.cc

@@ -72,8 +72,8 @@ TEST_F(DBRangeDelTest, CompactionOutputHasOnlyRangeTombstone) {
     // Skip cuckoo memtables, which do not support snapshots. Skip non-leveled
     // compactions as the above assertions about the number of files in a level
     // do not hold true.
-  } while (ChangeOptions(kRangeDelSkipConfigs | kSkipUniversalCompaction |
-                         kSkipFIFOCompaction));
+  } while (ChangeOptions(kRangeDelSkipConfigs | kSkipHashCuckoo |
+                         kSkipUniversalCompaction | kSkipFIFOCompaction));
 }
 
 TEST_F(DBRangeDelTest, CompactionOutputFilesExactlyFilled) {
@@ -645,7 +645,8 @@ TEST_F(DBRangeDelTest, GetCoveredKeyFromSst) {
     std::string value;
     ASSERT_TRUE(db_->Get(read_opts, "key", &value).IsNotFound());
     db_->ReleaseSnapshot(snapshot);
-  } while (ChangeOptions(kRangeDelSkipConfigs));
+    // Cuckoo memtables do not support snapshots.
+  } while (ChangeOptions(kRangeDelSkipConfigs | kSkipHashCuckoo));
 }
 
 TEST_F(DBRangeDelTest, GetCoveredMergeOperandFromMemtable) {
@@ -1111,14 +1112,14 @@ class MockMergeOperator : public MergeOperator {
   // Mock non-associative operator. Non-associativity is expressed by lack of
   // implementation for any `PartialMerge*` functions.
  public:
-  bool FullMergeV2(const MergeOperationInput& merge_in,
-                   MergeOperationOutput* merge_out) const override {
+  virtual bool FullMergeV2(const MergeOperationInput& merge_in,
+                           MergeOperationOutput* merge_out) const override {
     assert(merge_out != nullptr);
     merge_out->new_value = merge_in.operand_list.back().ToString();
     return true;
   }
 
-  const char* Name() const override { return "MockMergeOperator"; }
+  virtual const char* Name() const override { return "MockMergeOperator"; }
 };
 
 TEST_F(DBRangeDelTest, KeyAtOverlappingEndpointReappears) {

db/db_sst_test.cc

@@ -25,9 +25,9 @@ class DBSSTTest : public DBTestBase {
 class FlushedFileCollector : public EventListener {
  public:
   FlushedFileCollector() {}
-  ~FlushedFileCollector() override {}
+  ~FlushedFileCollector() {}
 
-  void OnFlushCompleted(DB* /*db*/, const FlushJobInfo& info) override {
+  virtual void OnFlushCompleted(DB* /*db*/, const FlushJobInfo& info) override {
     std::lock_guard<std::mutex> lock(mutex_);
     flushed_files_.push_back(info.file_path);
   }

db/db_test.cc

@@ -487,11 +487,11 @@ TEST_F(DBTest, PutSingleDeleteGet) {
     ASSERT_EQ("v2", Get(1, "foo2"));
     ASSERT_OK(SingleDelete(1, "foo"));
     ASSERT_EQ("NOT_FOUND", Get(1, "foo"));
-    // Skip FIFO and universal compaction beccause they do not apply to the test
-    // case. Skip MergePut because single delete does not get removed when it
-    // encounters a merge.
-  } while (ChangeOptions(kSkipFIFOCompaction | kSkipUniversalCompaction |
-                         kSkipMergePut));
+    // Skip HashCuckooRep as it does not support single delete. FIFO and
+    // universal compaction do not apply to the test case. Skip MergePut
+    // because single delete does not get removed when it encounters a merge.
+  } while (ChangeOptions(kSkipHashCuckoo | kSkipFIFOCompaction |
+                         kSkipUniversalCompaction | kSkipMergePut));
 }
 
 TEST_F(DBTest, ReadFromPersistedTier) {
@@ -604,7 +604,7 @@ TEST_F(DBTest, ReadFromPersistedTier) {
         DestroyAndReopen(options);
       }
     }
-  } while (ChangeOptions());
+  } while (ChangeOptions(kSkipHashCuckoo));
 }
 
 TEST_F(DBTest, SingleDeleteFlush) {
@@ -640,11 +640,11 @@ TEST_F(DBTest, SingleDeleteFlush) {
 
     ASSERT_EQ("NOT_FOUND", Get(1, "bar"));
     ASSERT_EQ("NOT_FOUND", Get(1, "foo"));
-    // Skip FIFO and universal compaction beccause they do not apply to the test
-    // case. Skip MergePut because single delete does not get removed when it
-    // encounters a merge.
-  } while (ChangeOptions(kSkipFIFOCompaction | kSkipUniversalCompaction |
-                         kSkipMergePut));
+    // Skip HashCuckooRep as it does not support single delete. FIFO and
+    // universal compaction do not apply to the test case. Skip MergePut
+    // because merges cannot be combined with single deletions.
+  } while (ChangeOptions(kSkipHashCuckoo | kSkipFIFOCompaction |
+                         kSkipUniversalCompaction | kSkipMergePut));
 }
 
 TEST_F(DBTest, SingleDeletePutFlush) {
@@ -663,11 +663,11 @@ TEST_F(DBTest, SingleDeletePutFlush) {
     ASSERT_OK(Flush(1));
 
     ASSERT_EQ("[ ]", AllEntriesFor("a", 1));
-    // Skip FIFO and universal compaction beccause they do not apply to the test
-    // case. Skip MergePut because single delete does not get removed when it
-    // encounters a merge.
-  } while (ChangeOptions(kSkipFIFOCompaction | kSkipUniversalCompaction |
-                         kSkipMergePut));
+    // Skip HashCuckooRep as it does not support single delete. FIFO and
+    // universal compaction do not apply to the test case. Skip MergePut
+    // because merges cannot be combined with single deletions.
+  } while (ChangeOptions(kSkipHashCuckoo | kSkipFIFOCompaction |
+                         kSkipUniversalCompaction | kSkipMergePut));
 }
 
 // Disable because not all platform can run it.
@@ -940,13 +940,13 @@ TEST_F(DBTest, FlushSchedule) {
 namespace {
 class KeepFilter : public CompactionFilter {
  public:
-  bool Filter(int /*level*/, const Slice& /*key*/, const Slice& /*value*/,
-              std::string* /*new_value*/,
-              bool* /*value_changed*/) const override {
+  virtual bool Filter(int /*level*/, const Slice& /*key*/,
+                      const Slice& /*value*/, std::string* /*new_value*/,
+                      bool* /*value_changed*/) const override {
     return false;
   }
 
-  const char* Name() const override { return "KeepFilter"; }
+  virtual const char* Name() const override { return "KeepFilter"; }
 };
 
 class KeepFilterFactory : public CompactionFilterFactory {
@@ -954,7 +954,7 @@ class KeepFilterFactory : public CompactionFilterFactory {
   explicit KeepFilterFactory(bool check_context = false)
       : check_context_(check_context) {}
 
-  std::unique_ptr<CompactionFilter> CreateCompactionFilter(
+  virtual std::unique_ptr<CompactionFilter> CreateCompactionFilter(
       const CompactionFilter::Context& context) override {
     if (check_context_) {
       EXPECT_EQ(expect_full_compaction_.load(), context.is_full_compaction);
@@ -963,7 +963,7 @@ class KeepFilterFactory : public CompactionFilterFactory {
     return std::unique_ptr<CompactionFilter>(new KeepFilter());
   }
 
-  const char* Name() const override { return "KeepFilterFactory"; }
+  virtual const char* Name() const override { return "KeepFilterFactory"; }
   bool check_context_;
   std::atomic_bool expect_full_compaction_;
   std::atomic_bool expect_manual_compaction_;
@@ -972,14 +972,14 @@ class KeepFilterFactory : public CompactionFilterFactory {
 class DelayFilter : public CompactionFilter {
  public:
   explicit DelayFilter(DBTestBase* d) : db_test(d) {}
-  bool Filter(int /*level*/, const Slice& /*key*/, const Slice& /*value*/,
-              std::string* /*new_value*/,
-              bool* /*value_changed*/) const override {
+  virtual bool Filter(int /*level*/, const Slice& /*key*/,
+                      const Slice& /*value*/, std::string* /*new_value*/,
+                      bool* /*value_changed*/) const override {
     db_test->env_->addon_time_.fetch_add(1000);
     return true;
   }
 
-  const char* Name() const override { return "DelayFilter"; }
+  virtual const char* Name() const override { return "DelayFilter"; }
 
  private:
   DBTestBase* db_test;
@@ -988,12 +988,12 @@ class DelayFilter : public CompactionFilter {
 class DelayFilterFactory : public CompactionFilterFactory {
  public:
   explicit DelayFilterFactory(DBTestBase* d) : db_test(d) {}
-  std::unique_ptr<CompactionFilter> CreateCompactionFilter(
+  virtual std::unique_ptr<CompactionFilter> CreateCompactionFilter(
       const CompactionFilter::Context& /*context*/) override {
     return std::unique_ptr<CompactionFilter>(new DelayFilter(db_test));
   }
 
-  const char* Name() const override { return "DelayFilterFactory"; }
+  virtual const char* Name() const override { return "DelayFilterFactory"; }
 
  private:
   DBTestBase* db_test;
@@ -1569,7 +1569,7 @@ TEST_F(DBTest, Snapshot) {
     ASSERT_EQ(0U, GetNumSnapshots());
     ASSERT_EQ("0v4", Get(0, "foo"));
     ASSERT_EQ("1v4", Get(1, "foo"));
-  } while (ChangeOptions());
+  } while (ChangeOptions(kSkipHashCuckoo));
 }
 
 TEST_F(DBTest, HiddenValuesAreRemoved) {
@@ -1606,8 +1606,9 @@ TEST_F(DBTest, HiddenValuesAreRemoved) {
     ASSERT_TRUE(Between(Size("", "pastfoo", 1), 0, 1000));
     // ApproximateOffsetOf() is not yet implemented in plain table format,
     // which is used by Size().
+    // skip HashCuckooRep as it does not support snapshot
   } while (ChangeOptions(kSkipUniversalCompaction | kSkipFIFOCompaction |
-                         kSkipPlainTable));
+                         kSkipPlainTable | kSkipHashCuckoo));
 }
 #endif  // ROCKSDB_LITE
 
@@ -1653,11 +1654,11 @@ TEST_F(DBTest, UnremovableSingleDelete) {
     ASSERT_EQ("first", Get(1, "foo", snapshot));
     ASSERT_EQ("NOT_FOUND", Get(1, "foo"));
     db_->ReleaseSnapshot(snapshot);
-    // Skip FIFO and universal compaction beccause they do not apply to the test
-    // case. Skip MergePut because single delete does not get removed when it
-    // encounters a merge.
-  } while (ChangeOptions(kSkipFIFOCompaction | kSkipUniversalCompaction |
-                         kSkipMergePut));
+    // Skip HashCuckooRep as it does not support single delete.  FIFO and
+    // universal compaction do not apply to the test case.  Skip MergePut
+    // because single delete does not get removed when it encounters a merge.
+  } while (ChangeOptions(kSkipHashCuckoo | kSkipFIFOCompaction |
+                         kSkipUniversalCompaction | kSkipMergePut));
 }
 
 #ifndef ROCKSDB_LITE
@@ -1775,14 +1776,17 @@ TEST_F(DBTest, OverlapInLevel0) {
 TEST_F(DBTest, ComparatorCheck) {
   class NewComparator : public Comparator {
    public:
-    const char* Name() const override { return "rocksdb.NewComparator"; }
-    int Compare(const Slice& a, const Slice& b) const override {
+    virtual const char* Name() const override {
+      return "rocksdb.NewComparator";
+    }
+    virtual int Compare(const Slice& a, const Slice& b) const override {
       return BytewiseComparator()->Compare(a, b);
     }
-    void FindShortestSeparator(std::string* s, const Slice& l) const override {
+    virtual void FindShortestSeparator(std::string* s,
+                                       const Slice& l) const override {
       BytewiseComparator()->FindShortestSeparator(s, l);
     }
-    void FindShortSuccessor(std::string* key) const override {
+    virtual void FindShortSuccessor(std::string* key) const override {
       BytewiseComparator()->FindShortSuccessor(key);
     }
   };
@@ -1805,15 +1809,18 @@ TEST_F(DBTest, ComparatorCheck) {
 TEST_F(DBTest, CustomComparator) {
   class NumberComparator : public Comparator {
    public:
-    const char* Name() const override { return "test.NumberComparator"; }
-    int Compare(const Slice& a, const Slice& b) const override {
+    virtual const char* Name() const override {
+      return "test.NumberComparator";
+    }
+    virtual int Compare(const Slice& a, const Slice& b) const override {
       return ToNumber(a) - ToNumber(b);
     }
-    void FindShortestSeparator(std::string* s, const Slice& l) const override {
+    virtual void FindShortestSeparator(std::string* s,
+                                       const Slice& l) const override {
       ToNumber(*s);  // Check format
       ToNumber(l);   // Check format
     }
-    void FindShortSuccessor(std::string* key) const override {
+    virtual void FindShortSuccessor(std::string* key) const override {
       ToNumber(*key);  // Check format
     }
 
@@ -2247,12 +2254,15 @@ static void MTThreadBody(void* arg) {
 class MultiThreadedDBTest : public DBTest,
                             public ::testing::WithParamInterface<int> {
  public:
-  void SetUp() override { option_config_ = GetParam(); }
+  virtual void SetUp() override { option_config_ = GetParam(); }
 
   static std::vector<int> GenerateOptionConfigs() {
     std::vector<int> optionConfigs;
     for (int optionConfig = kDefault; optionConfig < kEnd; ++optionConfig) {
-      optionConfigs.push_back(optionConfig);
+      // skip as HashCuckooRep does not support snapshot
+      if (optionConfig != kHashCuckoo) {
+        optionConfigs.push_back(optionConfig);
+      }
     }
     return optionConfigs;
   }
@@ -2392,7 +2402,7 @@ class ModelDB : public DB {
    public:
     KVMap map_;
 
-    SequenceNumber GetSequenceNumber() const override {
+    virtual SequenceNumber GetSequenceNumber() const override {
       // no need to call this
       assert(false);
       return 0;
@@ -2401,43 +2411,43 @@ class ModelDB : public DB {
 
   explicit ModelDB(const Options& options) : options_(options) {}
   using DB::Put;
-  Status Put(const WriteOptions& o, ColumnFamilyHandle* cf, const Slice& k,
-             const Slice& v) override {
+  virtual Status Put(const WriteOptions& o, ColumnFamilyHandle* cf,
+                     const Slice& k, const Slice& v) override {
     WriteBatch batch;
     batch.Put(cf, k, v);
     return Write(o, &batch);
   }
   using DB::Close;
-  Status Close() override { return Status::OK(); }
+  virtual Status Close() override { return Status::OK(); }
   using DB::Delete;
-  Status Delete(const WriteOptions& o, ColumnFamilyHandle* cf,
-                const Slice& key) override {
+  virtual Status Delete(const WriteOptions& o, ColumnFamilyHandle* cf,
+                        const Slice& key) override {
     WriteBatch batch;
     batch.Delete(cf, key);
     return Write(o, &batch);
   }
   using DB::SingleDelete;
-  Status SingleDelete(const WriteOptions& o, ColumnFamilyHandle* cf,
-                      const Slice& key) override {
+  virtual Status SingleDelete(const WriteOptions& o, ColumnFamilyHandle* cf,
+                              const Slice& key) override {
     WriteBatch batch;
     batch.SingleDelete(cf, key);
     return Write(o, &batch);
   }
   using DB::Merge;
-  Status Merge(const WriteOptions& o, ColumnFamilyHandle* cf, const Slice& k,
-               const Slice& v) override {
+  virtual Status Merge(const WriteOptions& o, ColumnFamilyHandle* cf,
+                       const Slice& k, const Slice& v) override {
     WriteBatch batch;
     batch.Merge(cf, k, v);
     return Write(o, &batch);
   }
   using DB::Get;
-  Status Get(const ReadOptions& /*options*/, ColumnFamilyHandle* /*cf*/,
-             const Slice& key, PinnableSlice* /*value*/) override {
+  virtual Status Get(const ReadOptions& /*options*/, ColumnFamilyHandle* /*cf*/,
+                     const Slice& key, PinnableSlice* /*value*/) override {
     return Status::NotSupported(key);
   }
 
   using DB::MultiGet;
-  std::vector<Status> MultiGet(
+  virtual std::vector<Status> MultiGet(
       const ReadOptions& /*options*/,
       const std::vector<ColumnFamilyHandle*>& /*column_family*/,
       const std::vector<Slice>& keys,
@@ -2449,31 +2459,25 @@ class ModelDB : public DB {
 
 #ifndef ROCKSDB_LITE
   using DB::IngestExternalFile;
-  Status IngestExternalFile(
+  virtual Status IngestExternalFile(
       ColumnFamilyHandle* /*column_family*/,
       const std::vector<std::string>& /*external_files*/,
       const IngestExternalFileOptions& /*options*/) override {
     return Status::NotSupported("Not implemented.");
   }
 
-  using DB::IngestExternalFiles;
-  Status IngestExternalFiles(
-      const std::vector<IngestExternalFileArg>& /*args*/) override {
-    return Status::NotSupported("Not implemented");
-  }
-
-  Status VerifyChecksum() override {
+  virtual Status VerifyChecksum() override {
     return Status::NotSupported("Not implemented.");
   }
 
   using DB::GetPropertiesOfAllTables;
-  Status GetPropertiesOfAllTables(
+  virtual Status GetPropertiesOfAllTables(
       ColumnFamilyHandle* /*column_family*/,
       TablePropertiesCollection* /*props*/) override {
     return Status();
   }
 
-  Status GetPropertiesOfTablesInRange(
+  virtual Status GetPropertiesOfTablesInRange(
       ColumnFamilyHandle* /*column_family*/, const Range* /*range*/,
       std::size_t /*n*/, TablePropertiesCollection* /*props*/) override {
     return Status();
@@ -2481,18 +2485,19 @@ class ModelDB : public DB {
 #endif  // ROCKSDB_LITE
 
   using DB::KeyMayExist;
-  bool KeyMayExist(const ReadOptions& /*options*/,
-                   ColumnFamilyHandle* /*column_family*/, const Slice& /*key*/,
-                   std::string* /*value*/,
-                   bool* value_found = nullptr) override {
+  virtual bool KeyMayExist(const ReadOptions& /*options*/,
+                           ColumnFamilyHandle* /*column_family*/,
+                           const Slice& /*key*/, std::string* /*value*/,
+                           bool* value_found = nullptr) override {
     if (value_found != nullptr) {
       *value_found = false;
     }
     return true;  // Not Supported directly
   }
   using DB::NewIterator;
-  Iterator* NewIterator(const ReadOptions& options,
-                        ColumnFamilyHandle* /*column_family*/) override {
+  virtual Iterator* NewIterator(
+      const ReadOptions& options,
+      ColumnFamilyHandle* /*column_family*/) override {
     if (options.snapshot == nullptr) {
       KVMap* saved = new KVMap;
       *saved = map_;
@@ -2503,33 +2508,38 @@ class ModelDB : public DB {
       return new ModelIter(snapshot_state, false);
     }
   }
-  Status NewIterators(const ReadOptions& /*options*/,
-                      const std::vector<ColumnFamilyHandle*>& /*column_family*/,
-                      std::vector<Iterator*>* /*iterators*/) override {
+  virtual Status NewIterators(
+      const ReadOptions& /*options*/,
+      const std::vector<ColumnFamilyHandle*>& /*column_family*/,
+      std::vector<Iterator*>* /*iterators*/) override {
     return Status::NotSupported("Not supported yet");
   }
-  const Snapshot* GetSnapshot() override {
+  virtual const Snapshot* GetSnapshot() override {
     ModelSnapshot* snapshot = new ModelSnapshot;
     snapshot->map_ = map_;
     return snapshot;
   }
 
-  void ReleaseSnapshot(const Snapshot* snapshot) override {
+  virtual void ReleaseSnapshot(const Snapshot* snapshot) override {
     delete reinterpret_cast<const ModelSnapshot*>(snapshot);
   }
 
-  Status Write(const WriteOptions& /*options*/, WriteBatch* batch) override {
+  virtual Status Write(const WriteOptions& /*options*/,
+                       WriteBatch* batch) override {
     class Handler : public WriteBatch::Handler {
      public:
       KVMap* map_;
-      void Put(const Slice& key, const Slice& value) override {
+      virtual void Put(const Slice& key, const Slice& value) override {
         (*map_)[key.ToString()] = value.ToString();
       }
-      void Merge(const Slice& /*key*/, const Slice& /*value*/) override {
+      virtual void Merge(const Slice& /*key*/,
+                         const Slice& /*value*/) override {
         // ignore merge for now
         // (*map_)[key.ToString()] = value.ToString();
       }
-      void Delete(const Slice& key) override { map_->erase(key.ToString()); }
+      virtual void Delete(const Slice& key) override {
+        map_->erase(key.ToString());
+      }
     };
     Handler handler;
     handler.map_ = &map_;
@@ -2537,58 +2547,61 @@ class ModelDB : public DB {
   }
 
   using DB::GetProperty;
-  bool GetProperty(ColumnFamilyHandle* /*column_family*/,
-                   const Slice& /*property*/, std::string* /*value*/) override {
+  virtual bool GetProperty(ColumnFamilyHandle* /*column_family*/,
+                           const Slice& /*property*/,
+                           std::string* /*value*/) override {
     return false;
   }
   using DB::GetIntProperty;
-  bool GetIntProperty(ColumnFamilyHandle* /*column_family*/,
-                      const Slice& /*property*/, uint64_t* /*value*/) override {
+  virtual bool GetIntProperty(ColumnFamilyHandle* /*column_family*/,
+                              const Slice& /*property*/,
+                              uint64_t* /*value*/) override {
     return false;
   }
   using DB::GetMapProperty;
-  bool GetMapProperty(ColumnFamilyHandle* /*column_family*/,
-                      const Slice& /*property*/,
-                      std::map<std::string, std::string>* /*value*/) override {
+  virtual bool GetMapProperty(
+      ColumnFamilyHandle* /*column_family*/, const Slice& /*property*/,
+      std::map<std::string, std::string>* /*value*/) override {
     return false;
   }
   using DB::GetAggregatedIntProperty;
-  bool GetAggregatedIntProperty(const Slice& /*property*/,
-                                uint64_t* /*value*/) override {
+  virtual bool GetAggregatedIntProperty(const Slice& /*property*/,
+                                        uint64_t* /*value*/) override {
     return false;
   }
   using DB::GetApproximateSizes;
-  void GetApproximateSizes(ColumnFamilyHandle* /*column_family*/,
-                           const Range* /*range*/, int n, uint64_t* sizes,
-                           uint8_t /*include_flags*/
-                           = INCLUDE_FILES) override {
+  virtual void GetApproximateSizes(ColumnFamilyHandle* /*column_family*/,
+                                   const Range* /*range*/, int n,
+                                   uint64_t* sizes,
+                                   uint8_t /*include_flags*/
+                                   = INCLUDE_FILES) override {
     for (int i = 0; i < n; i++) {
       sizes[i] = 0;
     }
   }
   using DB::GetApproximateMemTableStats;
-  void GetApproximateMemTableStats(ColumnFamilyHandle* /*column_family*/,
-                                   const Range& /*range*/,
-                                   uint64_t* const count,
-                                   uint64_t* const size) override {
+  virtual void GetApproximateMemTableStats(
+      ColumnFamilyHandle* /*column_family*/, const Range& /*range*/,
+      uint64_t* const count, uint64_t* const size) override {
     *count = 0;
     *size = 0;
   }
   using DB::CompactRange;
-  Status CompactRange(const CompactRangeOptions& /*options*/,
-                      ColumnFamilyHandle* /*column_family*/,
-                      const Slice* /*start*/, const Slice* /*end*/) override {
+  virtual Status CompactRange(const CompactRangeOptions& /*options*/,
+                              ColumnFamilyHandle* /*column_family*/,
+                              const Slice* /*start*/,
+                              const Slice* /*end*/) override {
     return Status::NotSupported("Not supported operation.");
   }
 
-  Status SetDBOptions(
+  virtual Status SetDBOptions(
       const std::unordered_map<std::string, std::string>& /*new_options*/)
       override {
     return Status::NotSupported("Not supported operation.");
   }
 
   using DB::CompactFiles;
-  Status CompactFiles(
+  virtual Status CompactFiles(
       const CompactionOptions& /*compact_options*/,
       ColumnFamilyHandle* /*column_family*/,
       const std::vector<std::string>& /*input_file_names*/,
@@ -2613,60 +2626,69 @@ class ModelDB : public DB {
   }
 
   using DB::NumberLevels;
-  int NumberLevels(ColumnFamilyHandle* /*column_family*/) override { return 1; }
+  virtual int NumberLevels(ColumnFamilyHandle* /*column_family*/) override {
+    return 1;
+  }
 
   using DB::MaxMemCompactionLevel;
-  int MaxMemCompactionLevel(ColumnFamilyHandle* /*column_family*/) override {
+  virtual int MaxMemCompactionLevel(
+      ColumnFamilyHandle* /*column_family*/) override {
     return 1;
   }
 
   using DB::Level0StopWriteTrigger;
-  int Level0StopWriteTrigger(ColumnFamilyHandle* /*column_family*/) override {
+  virtual int Level0StopWriteTrigger(
+      ColumnFamilyHandle* /*column_family*/) override {
     return -1;
   }
 
-  const std::string& GetName() const override { return name_; }
+  virtual const std::string& GetName() const override { return name_; }
 
-  Env* GetEnv() const override { return nullptr; }
+  virtual Env* GetEnv() const override { return nullptr; }
 
   using DB::GetOptions;
-  Options GetOptions(ColumnFamilyHandle* /*column_family*/) const override {
+  virtual Options GetOptions(
+      ColumnFamilyHandle* /*column_family*/) const override {
     return options_;
   }
 
   using DB::GetDBOptions;
-  DBOptions GetDBOptions() const override { return options_; }
+  virtual DBOptions GetDBOptions() const override { return options_; }
 
   using DB::Flush;
-  Status Flush(const rocksdb::FlushOptions& /*options*/,
-               ColumnFamilyHandle* /*column_family*/) override {
+  virtual Status Flush(const rocksdb::FlushOptions& /*options*/,
+                       ColumnFamilyHandle* /*column_family*/) override {
     Status ret;
     return ret;
   }
-  Status Flush(
+  virtual Status Flush(
       const rocksdb::FlushOptions& /*options*/,
       const std::vector<ColumnFamilyHandle*>& /*column_families*/) override {
     return Status::OK();
   }
 
-  Status SyncWAL() override { return Status::OK(); }
+  virtual Status SyncWAL() override { return Status::OK(); }
 
 #ifndef ROCKSDB_LITE
-  Status DisableFileDeletions() override { return Status::OK(); }
+  virtual Status DisableFileDeletions() override { return Status::OK(); }
 
-  Status EnableFileDeletions(bool /*force*/) override { return Status::OK(); }
-  Status GetLiveFiles(std::vector<std::string>&, uint64_t* /*size*/,
-                      bool /*flush_memtable*/ = true) override {
+  virtual Status EnableFileDeletions(bool /*force*/) override {
+    return Status::OK();
+  }
+  virtual Status GetLiveFiles(std::vector<std::string>&, uint64_t* /*size*/,
+                              bool /*flush_memtable*/ = true) override {
     return Status::OK();
   }
 
-  Status GetSortedWalFiles(VectorLogPtr& /*files*/) override {
+  virtual Status GetSortedWalFiles(VectorLogPtr& /*files*/) override {
     return Status::OK();
   }
 
-  Status DeleteFile(std::string /*name*/) override { return Status::OK(); }
+  virtual Status DeleteFile(std::string /*name*/) override {
+    return Status::OK();
+  }
 
-  Status GetUpdatesSince(
+  virtual Status GetUpdatesSince(
       rocksdb::SequenceNumber,
       std::unique_ptr<rocksdb::TransactionLogIterator>*,
       const TransactionLogIterator::ReadOptions& /*read_options*/ =
@@ -2674,48 +2696,52 @@ class ModelDB : public DB {
     return Status::NotSupported("Not supported in Model DB");
   }
 
-  void GetColumnFamilyMetaData(ColumnFamilyHandle* /*column_family*/,
-                               ColumnFamilyMetaData* /*metadata*/) override {}
+  virtual void GetColumnFamilyMetaData(
+      ColumnFamilyHandle* /*column_family*/,
+      ColumnFamilyMetaData* /*metadata*/) override {}
 #endif  // ROCKSDB_LITE
 
-  Status GetDbIdentity(std::string& /*identity*/) const override {
+  virtual Status GetDbIdentity(std::string& /*identity*/) const override {
     return Status::OK();
   }
 
-  SequenceNumber GetLatestSequenceNumber() const override { return 0; }
+  virtual SequenceNumber GetLatestSequenceNumber() const override { return 0; }
 
-  bool SetPreserveDeletesSequenceNumber(SequenceNumber /*seqnum*/) override {
+  virtual bool SetPreserveDeletesSequenceNumber(
+      SequenceNumber /*seqnum*/) override {
     return true;
   }
 
-  ColumnFamilyHandle* DefaultColumnFamily() const override { return nullptr; }
+  virtual ColumnFamilyHandle* DefaultColumnFamily() const override {
+    return nullptr;
+  }
 
  private:
   class ModelIter : public Iterator {
    public:
     ModelIter(const KVMap* map, bool owned)
         : map_(map), owned_(owned), iter_(map_->end()) {}
-    ~ModelIter() override {
+    ~ModelIter() {
       if (owned_) delete map_;
     }
-    bool Valid() const override { return iter_ != map_->end(); }
-    void SeekToFirst() override { iter_ = map_->begin(); }
-    void SeekToLast() override {
+    virtual bool Valid() const override { return iter_ != map_->end(); }
+    virtual void SeekToFirst() override { iter_ = map_->begin(); }
+    virtual void SeekToLast() override {
       if (map_->empty()) {
         iter_ = map_->end();
       } else {
         iter_ = map_->find(map_->rbegin()->first);
       }
     }
-    void Seek(const Slice& k) override {
+    virtual void Seek(const Slice& k) override {
       iter_ = map_->lower_bound(k.ToString());
     }
-    void SeekForPrev(const Slice& k) override {
+    virtual void SeekForPrev(const Slice& k) override {
       iter_ = map_->upper_bound(k.ToString());
       Prev();
     }
-    void Next() override { ++iter_; }
-    void Prev() override {
+    virtual void Next() override { ++iter_; }
+    virtual void Prev() override {
       if (iter_ == map_->begin()) {
         iter_ = map_->end();
         return;
@@ -2723,9 +2749,9 @@ class ModelDB : public DB {
       --iter_;
     }
 
-    Slice key() const override { return iter_->first; }
-    Slice value() const override { return iter_->second; }
-    Status status() const override { return Status::OK(); }
+    virtual Slice key() const override { return iter_->first; }
+    virtual Slice value() const override { return iter_->second; }
+    virtual Status status() const override { return Status::OK(); }
 
    private:
     const KVMap* const map_;
@@ -2793,14 +2819,15 @@ static bool CompareIterators(int step, DB* model, DB* db,
 class DBTestRandomized : public DBTest,
                          public ::testing::WithParamInterface<int> {
  public:
-  void SetUp() override { option_config_ = GetParam(); }
+  virtual void SetUp() override { option_config_ = GetParam(); }
 
   static std::vector<int> GenerateOptionConfigs() {
     std::vector<int> option_configs;
     // skip cuckoo hash as it does not support snapshot.
     for (int option_config = kDefault; option_config < kEnd; ++option_config) {
-      if (!ShouldSkipOptions(option_config,
-                             kSkipDeletesFilterFirst | kSkipNoSeekToLast)) {
+      if (!ShouldSkipOptions(option_config, kSkipDeletesFilterFirst |
+                                                kSkipNoSeekToLast |
+                                                kSkipHashCuckoo)) {
         option_configs.push_back(option_config);
       }
     }
@@ -2830,6 +2857,7 @@ TEST_P(DBTestRandomized, Randomized) {
     int p = rnd.Uniform(100);
     int minimum = 0;
     if (option_config_ == kHashSkipList || option_config_ == kHashLinkList ||
+        option_config_ == kHashCuckoo ||
         option_config_ == kPlainTableFirstBytePrefix ||
         option_config_ == kBlockBasedTableWithWholeKeyHashIndex ||
         option_config_ == kBlockBasedTableWithPrefixHashIndex) {
@@ -3083,7 +3111,7 @@ TEST_F(DBTest, FIFOCompactionWithTTLAndMaxOpenFilesTest) {
   Options options;
   options.compaction_style = kCompactionStyleFIFO;
   options.create_if_missing = true;
-  options.ttl = 600;  // seconds
+  options.compaction_options_fifo.ttl = 600;  // seconds
 
   // Check that it is not supported with max_open_files != -1.
   options.max_open_files = 100;
@@ -3099,7 +3127,7 @@ TEST_F(DBTest, FIFOCompactionWithTTLAndVariousTableFormatsTest) {
   Options options;
   options.compaction_style = kCompactionStyleFIFO;
   options.create_if_missing = true;
-  options.ttl = 600;  // seconds
+  options.compaction_options_fifo.ttl = 600;  // seconds
 
   options = CurrentOptions(options);
   options.table_factory.reset(NewBlockBasedTableFactory());
@@ -3109,6 +3137,10 @@ TEST_F(DBTest, FIFOCompactionWithTTLAndVariousTableFormatsTest) {
   options.table_factory.reset(NewPlainTableFactory());
   ASSERT_TRUE(TryReopen(options).IsNotSupported());
 
+  Destroy(options);
+  options.table_factory.reset(NewCuckooTableFactory());
+  ASSERT_TRUE(TryReopen(options).IsNotSupported());
+
   Destroy(options);
   options.table_factory.reset(NewAdaptiveTableFactory());
   ASSERT_TRUE(TryReopen(options).IsNotSupported());
@@ -3130,7 +3162,7 @@ TEST_F(DBTest, FIFOCompactionWithTTLTest) {
     env_->addon_time_.store(0);
     options.compaction_options_fifo.max_table_files_size = 150 << 10;  // 150KB
     options.compaction_options_fifo.allow_compaction = false;
-    options.ttl = 1 * 60 * 60 ;  // 1 hour
+    options.compaction_options_fifo.ttl = 1 * 60 * 60 ;  // 1 hour
     options = CurrentOptions(options);
     DestroyAndReopen(options);
 
@@ -3165,7 +3197,7 @@ TEST_F(DBTest, FIFOCompactionWithTTLTest) {
   {
     options.compaction_options_fifo.max_table_files_size = 150 << 10;  // 150KB
     options.compaction_options_fifo.allow_compaction = false;
-    options.ttl = 1 * 60 * 60;  // 1 hour
+    options.compaction_options_fifo.ttl = 1 * 60 * 60;  // 1 hour
     options = CurrentOptions(options);
     DestroyAndReopen(options);
 
@@ -3207,7 +3239,7 @@ TEST_F(DBTest, FIFOCompactionWithTTLTest) {
     options.write_buffer_size = 10 << 10;                              // 10KB
     options.compaction_options_fifo.max_table_files_size = 150 << 10;  // 150KB
     options.compaction_options_fifo.allow_compaction = false;
-    options.ttl =  1 * 60 * 60;  // 1 hour
+    options.compaction_options_fifo.ttl =  1 * 60 * 60;  // 1 hour
     options = CurrentOptions(options);
     DestroyAndReopen(options);
 
@@ -3244,7 +3276,7 @@ TEST_F(DBTest, FIFOCompactionWithTTLTest) {
   {
     options.compaction_options_fifo.max_table_files_size = 150 << 10;  // 150KB
     options.compaction_options_fifo.allow_compaction = true;
-    options.ttl = 1 * 60 * 60;  // 1 hour
+    options.compaction_options_fifo.ttl = 1 * 60 * 60;  // 1 hour
     options.level0_file_num_compaction_trigger = 6;
     options = CurrentOptions(options);
     DestroyAndReopen(options);
@@ -3288,7 +3320,7 @@ TEST_F(DBTest, FIFOCompactionWithTTLTest) {
     options.write_buffer_size = 20 << 10;                               // 20K
     options.compaction_options_fifo.max_table_files_size = 1500 << 10;  // 1.5MB
     options.compaction_options_fifo.allow_compaction = true;
-    options.ttl = 1 * 60 * 60;  // 1 hour
+    options.compaction_options_fifo.ttl = 1 * 60 * 60;  // 1 hour
     options.level0_file_num_compaction_trigger = 6;
     options = CurrentOptions(options);
     DestroyAndReopen(options);
@@ -4587,7 +4619,7 @@ TEST_F(DBTest, DynamicCompactionOptions) {
   ASSERT_LT(NumTableFilesAtLevel(0), 4);
 }
 
-// Test dynamic FIFO compaction options.
+// Test dynamic FIFO copmaction options.
 // This test covers just option parsing and makes sure that the options are
 // correctly assigned. Also look at DBOptionsTest.SetFIFOCompactionOptions
 // test which makes sure that the FIFO compaction funcionality is working
@@ -4601,7 +4633,7 @@ TEST_F(DBTest, DynamicFIFOCompactionOptions) {
   // Initial defaults
   ASSERT_EQ(dbfull()->GetOptions().compaction_options_fifo.max_table_files_size,
             1024 * 1024 * 1024);
-  ASSERT_EQ(dbfull()->GetOptions().ttl, 0);
+  ASSERT_EQ(dbfull()->GetOptions().compaction_options_fifo.ttl, 0);
   ASSERT_EQ(dbfull()->GetOptions().compaction_options_fifo.allow_compaction,
             false);
 
@@ -4609,21 +4641,21 @@ TEST_F(DBTest, DynamicFIFOCompactionOptions) {
       {{"compaction_options_fifo", "{max_table_files_size=23;}"}}));
   ASSERT_EQ(dbfull()->GetOptions().compaction_options_fifo.max_table_files_size,
             23);
-  ASSERT_EQ(dbfull()->GetOptions().ttl, 0);
+  ASSERT_EQ(dbfull()->GetOptions().compaction_options_fifo.ttl, 0);
   ASSERT_EQ(dbfull()->GetOptions().compaction_options_fifo.allow_compaction,
             false);
 
-  ASSERT_OK(dbfull()->SetOptions({{"ttl", "97"}}));
+  ASSERT_OK(dbfull()->SetOptions({{"compaction_options_fifo", "{ttl=97}"}}));
   ASSERT_EQ(dbfull()->GetOptions().compaction_options_fifo.max_table_files_size,
             23);
-  ASSERT_EQ(dbfull()->GetOptions().ttl, 97);
+  ASSERT_EQ(dbfull()->GetOptions().compaction_options_fifo.ttl, 97);
   ASSERT_EQ(dbfull()->GetOptions().compaction_options_fifo.allow_compaction,
             false);
 
-  ASSERT_OK(dbfull()->SetOptions({{"ttl", "203"}}));
+  ASSERT_OK(dbfull()->SetOptions({{"compaction_options_fifo", "{ttl=203;}"}}));
   ASSERT_EQ(dbfull()->GetOptions().compaction_options_fifo.max_table_files_size,
             23);
-  ASSERT_EQ(dbfull()->GetOptions().ttl, 203);
+  ASSERT_EQ(dbfull()->GetOptions().compaction_options_fifo.ttl, 203);
   ASSERT_EQ(dbfull()->GetOptions().compaction_options_fifo.allow_compaction,
             false);
 
@@ -4631,25 +4663,24 @@ TEST_F(DBTest, DynamicFIFOCompactionOptions) {
       {{"compaction_options_fifo", "{allow_compaction=true;}"}}));
   ASSERT_EQ(dbfull()->GetOptions().compaction_options_fifo.max_table_files_size,
             23);
-  ASSERT_EQ(dbfull()->GetOptions().ttl, 203);
+  ASSERT_EQ(dbfull()->GetOptions().compaction_options_fifo.ttl, 203);
   ASSERT_EQ(dbfull()->GetOptions().compaction_options_fifo.allow_compaction,
             true);
 
   ASSERT_OK(dbfull()->SetOptions(
-      {{"compaction_options_fifo", "{max_table_files_size=31;}"}}));
+      {{"compaction_options_fifo", "{max_table_files_size=31;ttl=19;}"}}));
   ASSERT_EQ(dbfull()->GetOptions().compaction_options_fifo.max_table_files_size,
             31);
-  ASSERT_EQ(dbfull()->GetOptions().ttl, 203);
+  ASSERT_EQ(dbfull()->GetOptions().compaction_options_fifo.ttl, 19);
   ASSERT_EQ(dbfull()->GetOptions().compaction_options_fifo.allow_compaction,
             true);
 
   ASSERT_OK(dbfull()->SetOptions(
       {{"compaction_options_fifo",
-        "{max_table_files_size=51;allow_compaction=true;}"}}));
-  ASSERT_OK(dbfull()->SetOptions({{"ttl", "49"}}));
+        "{max_table_files_size=51;ttl=49;allow_compaction=true;}"}}));
   ASSERT_EQ(dbfull()->GetOptions().compaction_options_fifo.max_table_files_size,
             51);
-  ASSERT_EQ(dbfull()->GetOptions().ttl, 49);
+  ASSERT_EQ(dbfull()->GetOptions().compaction_options_fifo.ttl, 49);
   ASSERT_EQ(dbfull()->GetOptions().compaction_options_fifo.allow_compaction,
             true);
 }
@@ -5016,14 +5047,14 @@ class DelayedMergeOperator : public MergeOperator {
  public:
   explicit DelayedMergeOperator(DBTest* d) : db_test_(d) {}
 
-  bool FullMergeV2(const MergeOperationInput& /*merge_in*/,
-                   MergeOperationOutput* merge_out) const override {
+  virtual bool FullMergeV2(const MergeOperationInput& /*merge_in*/,
+                           MergeOperationOutput* merge_out) const override {
     db_test_->env_->addon_time_.fetch_add(1000);
     merge_out->new_value = "";
     return true;
   }
 
-  const char* Name() const override { return "DelayedMergeOperator"; }
+  virtual const char* Name() const override { return "DelayedMergeOperator"; }
 };
 
 TEST_F(DBTest, MergeTestTime) {
@@ -5159,7 +5190,7 @@ TEST_F(DBTest, EmptyCompactedDB) {
 TEST_F(DBTest, SuggestCompactRangeTest) {
   class CompactionFilterFactoryGetContext : public CompactionFilterFactory {
    public:
-    std::unique_ptr<CompactionFilter> CreateCompactionFilter(
+    virtual std::unique_ptr<CompactionFilter> CreateCompactionFilter(
         const CompactionFilter::Context& context) override {
       saved_context = context;
       std::unique_ptr<CompactionFilter> empty_filter;
@@ -5409,69 +5440,6 @@ TEST_F(DBTest, AutomaticConflictsWithManualCompaction) {
   dbfull()->TEST_WaitForCompact();
 }
 
-#ifndef ROCKSDB_LITE
-TEST_F(DBTest, CompactFilesShouldTriggerAutoCompaction) {
-  Options options = CurrentOptions();
-  options.max_background_compactions = 1;
-  options.level0_file_num_compaction_trigger = 4;
-  options.level0_slowdown_writes_trigger = 36;
-  options.level0_stop_writes_trigger = 36;
-  DestroyAndReopen(options);
-
-  // generate files for manual compaction
-  Random rnd(301);
-  for (int i = 0; i < 2; ++i) {
-    // put two keys to ensure no trivial move
-    for (int j = 0; j < 2; ++j) {
-      ASSERT_OK(Put(Key(j), RandomString(&rnd, 1024)));
-    }
-    ASSERT_OK(Flush());
-  }
-
-  rocksdb::ColumnFamilyMetaData cf_meta_data;
-  db_->GetColumnFamilyMetaData(db_->DefaultColumnFamily(), &cf_meta_data);
-
-  std::vector<std::string> input_files;
-  input_files.push_back(cf_meta_data.levels[0].files[0].name);
-
-  SyncPoint::GetInstance()->LoadDependency({
-      {"CompactFilesImpl:0",
-       "DBTest::CompactFilesShouldTriggerAutoCompaction:Begin"},
-      {"DBTest::CompactFilesShouldTriggerAutoCompaction:End",
-       "CompactFilesImpl:1"},
-  });
-
-  SyncPoint::GetInstance()->EnableProcessing();
-
-  port::Thread manual_compaction_thread([&]() {
-      auto s = db_->CompactFiles(CompactionOptions(),
-          db_->DefaultColumnFamily(), input_files, 0);
-  });
-
-  TEST_SYNC_POINT(
-          "DBTest::CompactFilesShouldTriggerAutoCompaction:Begin");
-  // generate enough files to trigger compaction
-  for (int i = 0; i < 20; ++i) {
-    for (int j = 0; j < 2; ++j) {
-      ASSERT_OK(Put(Key(j), RandomString(&rnd, 1024)));
-    }
-    ASSERT_OK(Flush());
-  }
-  db_->GetColumnFamilyMetaData(db_->DefaultColumnFamily(), &cf_meta_data);
-  ASSERT_GT(cf_meta_data.levels[0].files.size(),
-      options.level0_file_num_compaction_trigger);
-  TEST_SYNC_POINT(
-          "DBTest::CompactFilesShouldTriggerAutoCompaction:End");
-
-  manual_compaction_thread.join();
-  dbfull()->TEST_WaitForCompact();
-
-  db_->GetColumnFamilyMetaData(db_->DefaultColumnFamily(), &cf_meta_data);
-  ASSERT_LE(cf_meta_data.levels[0].files.size(),
-      options.level0_file_num_compaction_trigger);
-}
-#endif  // ROCKSDB_LITE
-
 // Github issue #595
 // Large write batch with column families
 TEST_F(DBTest, LargeBatchWithColumnFamilies) {

db/db_test2.cc

@@ -30,7 +30,7 @@ class PrefixFullBloomWithReverseComparator
  public:
   PrefixFullBloomWithReverseComparator()
       : DBTestBase("/prefix_bloom_reverse") {}
-  void SetUp() override { if_cache_filter_ = GetParam(); }
+  virtual void SetUp() override { if_cache_filter_ = GetParam(); }
   bool if_cache_filter_;
 };
 
@@ -515,9 +515,9 @@ TEST_F(DBTest2, WalFilterTest) {
       apply_option_at_record_index_(apply_option_for_record_index),
       current_record_index_(0) {}
 
-    WalProcessingOption LogRecord(const WriteBatch& /*batch*/,
-                                  WriteBatch* /*new_batch*/,
-                                  bool* /*batch_changed*/) const override {
+    virtual WalProcessingOption LogRecord(
+        const WriteBatch& /*batch*/, WriteBatch* /*new_batch*/,
+        bool* /*batch_changed*/) const override {
       WalFilter::WalProcessingOption option_to_return;
 
       if (current_record_index_ == apply_option_at_record_index_) {
@@ -535,7 +535,7 @@ TEST_F(DBTest2, WalFilterTest) {
       return option_to_return;
     }
 
-    const char* Name() const override { return "TestWalFilter"; }
+    virtual const char* Name() const override { return "TestWalFilter"; }
   };
 
   // Create 3 batches with two keys each
@@ -687,7 +687,7 @@ TEST_F(DBTest2, WalFilterTestWithChangeBatch) {
       : new_write_batch_(new_write_batch),
       num_keys_to_add_in_new_batch_(num_keys_to_add_in_new_batch),
       num_keys_added_(0) {}
-    void Put(const Slice& key, const Slice& value) override {
+    virtual void Put(const Slice& key, const Slice& value) override {
       if (num_keys_added_ < num_keys_to_add_in_new_batch_) {
         new_write_batch_->Put(key, value);
         ++num_keys_added_;
@@ -711,9 +711,9 @@ TEST_F(DBTest2, WalFilterTestWithChangeBatch) {
       num_keys_to_add_in_new_batch_(num_keys_to_add_in_new_batch),
       current_record_index_(0) {}
 
-    WalProcessingOption LogRecord(const WriteBatch& batch,
-                                  WriteBatch* new_batch,
-                                  bool* batch_changed) const override {
+    virtual WalProcessingOption LogRecord(const WriteBatch& batch,
+      WriteBatch* new_batch,
+      bool* batch_changed) const override {
       if (current_record_index_ >= change_records_from_index_) {
         ChangeBatchHandler handler(new_batch, num_keys_to_add_in_new_batch_);
         batch.Iterate(&handler);
@@ -729,7 +729,9 @@ TEST_F(DBTest2, WalFilterTestWithChangeBatch) {
       return WalProcessingOption::kContinueProcessing;
     }
 
-    const char* Name() const override { return "TestWalFilterWithChangeBatch"; }
+    virtual const char* Name() const override {
+      return "TestWalFilterWithChangeBatch";
+    }
   };
 
   std::vector<std::vector<std::string>> batch_keys(3);
@@ -807,17 +809,18 @@ TEST_F(DBTest2, WalFilterTestWithChangeBatch) {
 TEST_F(DBTest2, WalFilterTestWithChangeBatchExtraKeys) {
   class TestWalFilterWithChangeBatchAddExtraKeys : public WalFilter {
   public:
-   WalProcessingOption LogRecord(const WriteBatch& batch, WriteBatch* new_batch,
-                                 bool* batch_changed) const override {
-     *new_batch = batch;
-     new_batch->Put("key_extra", "value_extra");
-     *batch_changed = true;
-     return WalProcessingOption::kContinueProcessing;
-   }
+    virtual WalProcessingOption LogRecord(const WriteBatch& batch,
+      WriteBatch* new_batch,
+      bool* batch_changed) const override {
+      *new_batch = batch;
+      new_batch->Put("key_extra", "value_extra");
+      *batch_changed = true;
+      return WalProcessingOption::kContinueProcessing;
+    }
 
-   const char* Name() const override {
-     return "WalFilterTestWithChangeBatchExtraKeys";
-   }
+    virtual const char* Name() const override {
+      return "WalFilterTestWithChangeBatchExtraKeys";
+    }
   };
 
   std::vector<std::vector<std::string>> batch_keys(3);
@@ -881,19 +884,18 @@ TEST_F(DBTest2, WalFilterTestWithColumnFamilies) {
     // for verification against the keys we expect.
     std::map<uint32_t, std::vector<std::string>> cf_wal_keys_;
   public:
-   void ColumnFamilyLogNumberMap(
-       const std::map<uint32_t, uint64_t>& cf_lognumber_map,
-       const std::map<std::string, uint32_t>& cf_name_id_map) override {
-     cf_log_number_map_ = cf_lognumber_map;
-     cf_name_id_map_ = cf_name_id_map;
-   }
+    virtual void ColumnFamilyLogNumberMap(
+      const std::map<uint32_t, uint64_t>& cf_lognumber_map,
+      const std::map<std::string, uint32_t>& cf_name_id_map) override {
+      cf_log_number_map_ = cf_lognumber_map;
+      cf_name_id_map_ = cf_name_id_map;
+    }
 
-   WalProcessingOption LogRecordFound(unsigned long long log_number,
-                                      const std::string& /*log_file_name*/,
-                                      const WriteBatch& batch,
-                                      WriteBatch* /*new_batch*/,
-                                      bool* /*batch_changed*/) override {
-     class LogRecordBatchHandler : public WriteBatch::Handler {
+    virtual WalProcessingOption LogRecordFound(
+        unsigned long long log_number, const std::string& /*log_file_name*/,
+        const WriteBatch& batch, WriteBatch* /*new_batch*/,
+        bool* /*batch_changed*/) override {
+      class LogRecordBatchHandler : public WriteBatch::Handler {
       private:
         const std::map<uint32_t, uint64_t> & cf_log_number_map_;
         std::map<uint32_t, std::vector<std::string>> & cf_wal_keys_;
@@ -906,8 +908,8 @@ TEST_F(DBTest2, WalFilterTestWithColumnFamilies) {
           cf_wal_keys_(cf_wal_keys),
           log_number_(current_log_number){}
 
-        Status PutCF(uint32_t column_family_id, const Slice& key,
-                     const Slice& /*value*/) override {
+        virtual Status PutCF(uint32_t column_family_id, const Slice& key,
+          const Slice& /*value*/) override {
           auto it = cf_log_number_map_.find(column_family_id);
           assert(it != cf_log_number_map_.end());
           unsigned long long log_number_for_cf = it->second;
@@ -925,11 +927,11 @@ TEST_F(DBTest2, WalFilterTestWithColumnFamilies) {
       batch.Iterate(&handler);
 
       return WalProcessingOption::kContinueProcessing;
-   }
+    }
 
-   const char* Name() const override {
-     return "WalFilterTestWithColumnFamilies";
-   }
+    virtual const char* Name() const override {
+      return "WalFilterTestWithColumnFamilies";
+    }
 
     const std::map<uint32_t, std::vector<std::string>>& GetColumnFamilyKeys() {
       return cf_wal_keys_;
@@ -1036,19 +1038,18 @@ TEST_F(DBTest2, WalFilterTestWithColumnFamilies) {
   ASSERT_TRUE(index == keys_cf.size());
 }
 
-// Temporarily disable it because the test is flaky.
-TEST_F(DBTest2, DISABLED_PresetCompressionDict) {
-  // Verifies that compression ratio improves when dictionary is enabled, and
-  // improves even further when the dictionary is trained by ZSTD.
+TEST_F(DBTest2, PresetCompressionDict) {
   const size_t kBlockSizeBytes = 4 << 10;
   const size_t kL0FileBytes = 128 << 10;
   const size_t kApproxPerBlockOverheadBytes = 50;
   const int kNumL0Files = 5;
+  const int kZstdTrainFactor = 16;
 
   Options options;
   options.env = CurrentOptions().env; // Make sure to use any custom env that the test is configured with.
   options.allow_concurrent_memtable_write = false;
   options.arena_block_size = kBlockSizeBytes;
+  options.compaction_style = kCompactionStyleUniversal;
   options.create_if_missing = true;
   options.disable_auto_compactions = true;
   options.level0_file_num_compaction_trigger = kNumL0Files;
@@ -1090,15 +1091,16 @@ TEST_F(DBTest2, DISABLED_PresetCompressionDict) {
           options.compression_opts.zstd_max_train_bytes = 0;
           break;
         case 1:
-          options.compression_opts.max_dict_bytes = 4 * kBlockSizeBytes;
+          options.compression_opts.max_dict_bytes = kBlockSizeBytes;
           options.compression_opts.zstd_max_train_bytes = 0;
           break;
         case 2:
           if (compression_type != kZSTD) {
             continue;
           }
-          options.compression_opts.max_dict_bytes = 4 * kBlockSizeBytes;
-          options.compression_opts.zstd_max_train_bytes = kL0FileBytes;
+          options.compression_opts.max_dict_bytes = kBlockSizeBytes;
+          options.compression_opts.zstd_max_train_bytes =
+              kZstdTrainFactor * kBlockSizeBytes;
           break;
         default:
           assert(false);
@@ -1108,24 +1110,20 @@ TEST_F(DBTest2, DISABLED_PresetCompressionDict) {
       options.table_factory.reset(NewBlockBasedTableFactory(table_options));
       CreateAndReopenWithCF({"pikachu"}, options);
       Random rnd(301);
-      std::string seq_datas[10];
-      for (int j = 0; j < 10; ++j) {
-        seq_datas[j] =
-            RandomString(&rnd, kBlockSizeBytes - kApproxPerBlockOverheadBytes);
-      }
+      std::string seq_data =
+          RandomString(&rnd, kBlockSizeBytes - kApproxPerBlockOverheadBytes);
 
       ASSERT_EQ(0, NumTableFilesAtLevel(0, 1));
       for (int j = 0; j < kNumL0Files; ++j) {
         for (size_t k = 0; k < kL0FileBytes / kBlockSizeBytes + 1; ++k) {
-          auto key_num = j * (kL0FileBytes / kBlockSizeBytes) + k;
-          ASSERT_OK(Put(1, Key(static_cast<int>(key_num)),
-                        seq_datas[(key_num / 10) % 10]));
+          ASSERT_OK(Put(1, Key(static_cast<int>(
+                               j * (kL0FileBytes / kBlockSizeBytes) + k)),
+                        seq_data));
         }
         dbfull()->TEST_WaitForFlushMemTable(handles_[1]);
         ASSERT_EQ(j + 1, NumTableFilesAtLevel(0, 1));
       }
-      dbfull()->TEST_CompactRange(0, nullptr, nullptr, handles_[1],
-                                  true /* disallow_trivial_move */);
+      db_->CompactRange(CompactRangeOptions(), handles_[1], nullptr, nullptr);
       ASSERT_EQ(0, NumTableFilesAtLevel(0, 1));
       ASSERT_GT(NumTableFilesAtLevel(1, 1), 0);
 
@@ -1140,7 +1138,7 @@ TEST_F(DBTest2, DISABLED_PresetCompressionDict) {
 
       for (size_t j = 0; j < kNumL0Files * (kL0FileBytes / kBlockSizeBytes);
            j++) {
-        ASSERT_EQ(seq_datas[(j / 10) % 10], Get(1, Key(static_cast<int>(j))));
+        ASSERT_EQ(seq_data, Get(1, Key(static_cast<int>(j))));
       }
       if (i) {
         ASSERT_GT(prev_out_bytes, out_bytes);
@@ -1151,70 +1149,6 @@ TEST_F(DBTest2, DISABLED_PresetCompressionDict) {
   }
 }
 
-TEST_F(DBTest2, PresetCompressionDictLocality) {
-  if (!ZSTD_Supported()) {
-    return;
-  }
-  // Verifies that compression dictionary is generated from local data. The
-  // verification simply checks all output SSTs have different compression
-  // dictionaries. We do not verify effectiveness as that'd likely be flaky in
-  // the future.
-  const int kNumEntriesPerFile = 1 << 10;  // 1KB
-  const int kNumBytesPerEntry = 1 << 10;   // 1KB
-  const int kNumFiles = 4;
-  Options options = CurrentOptions();
-  options.compression = kZSTD;
-  options.compression_opts.max_dict_bytes = 1 << 14;        // 16KB
-  options.compression_opts.zstd_max_train_bytes = 1 << 18;  // 256KB
-  options.statistics = rocksdb::CreateDBStatistics();
-  options.target_file_size_base = kNumEntriesPerFile * kNumBytesPerEntry;
-  BlockBasedTableOptions table_options;
-  table_options.cache_index_and_filter_blocks = true;
-  options.table_factory.reset(new BlockBasedTableFactory(table_options));
-  Reopen(options);
-
-  Random rnd(301);
-  for (int i = 0; i < kNumFiles; ++i) {
-    for (int j = 0; j < kNumEntriesPerFile; ++j) {
-      ASSERT_OK(Put(Key(i * kNumEntriesPerFile + j),
-                    RandomString(&rnd, kNumBytesPerEntry)));
-    }
-    ASSERT_OK(Flush());
-    MoveFilesToLevel(1);
-    ASSERT_EQ(NumTableFilesAtLevel(1), i + 1);
-  }
-
-  // Store all the dictionaries generated during a full compaction.
-  std::vector<std::string> compression_dicts;
-  rocksdb::SyncPoint::GetInstance()->SetCallBack(
-      "BlockBasedTableBuilder::WriteCompressionDictBlock:RawDict",
-      [&](void* arg) {
-        compression_dicts.emplace_back(static_cast<Slice*>(arg)->ToString());
-      });
-  rocksdb::SyncPoint::GetInstance()->EnableProcessing();
-  CompactRangeOptions compact_range_opts;
-  compact_range_opts.bottommost_level_compaction =
-      BottommostLevelCompaction::kForce;
-  ASSERT_OK(db_->CompactRange(compact_range_opts, nullptr, nullptr));
-
-  // Dictionary compression should not be so good as to compress four totally
-  // random files into one. If it does then there's probably something wrong
-  // with the test.
-  ASSERT_GT(NumTableFilesAtLevel(1), 1);
-
-  // Furthermore, there should be one compression dictionary generated per file.
-  // And they should all be different from each other.
-  ASSERT_EQ(NumTableFilesAtLevel(1),
-            static_cast<int>(compression_dicts.size()));
-  for (size_t i = 1; i < compression_dicts.size(); ++i) {
-    std::string& a = compression_dicts[i - 1];
-    std::string& b = compression_dicts[i];
-    size_t alen = a.size();
-    size_t blen = b.size();
-    ASSERT_TRUE(alen != blen || memcmp(a.data(), b.data(), alen) != 0);
-  }
-}
-
 class CompactionCompressionListener : public EventListener {
  public:
   explicit CompactionCompressionListener(Options* db_options)
@@ -1451,7 +1385,7 @@ class PinL0IndexAndFilterBlocksTest
       public testing::WithParamInterface<std::tuple<bool, bool>> {
  public:
   PinL0IndexAndFilterBlocksTest() : DBTestBase("/db_pin_l0_index_bloom_test") {}
-  void SetUp() override {
+  virtual void SetUp() override {
     infinite_max_files_ = std::get<0>(GetParam());
     disallow_preload_ = std::get<1>(GetParam());
   }
@@ -1754,7 +1688,7 @@ class MockPersistentCache : public PersistentCache {
         "GetUniqueIdFromFile:FS_IOC_GETVERSION", UniqueIdCallback);
   }
 
-  ~MockPersistentCache() override {}
+  virtual ~MockPersistentCache() {}
 
   PersistentCache::StatsType Stats() override {
     return PersistentCache::StatsType();
@@ -2645,7 +2579,9 @@ TEST_F(DBTest2, ReadCallbackTest) {
   class TestReadCallback : public ReadCallback {
    public:
     explicit TestReadCallback(SequenceNumber snapshot) : snapshot_(snapshot) {}
-    bool IsVisible(SequenceNumber seq) override { return seq <= snapshot_; }
+    virtual bool IsVisible(SequenceNumber seq) override {
+      return seq <= snapshot_;
+    }
 
    private:
     SequenceNumber snapshot_;
@@ -2946,77 +2882,6 @@ TEST_F(DBTest2, TraceWithLimit) {
   ASSERT_OK(DestroyDB(dbname2, options));
 }
 
-TEST_F(DBTest2, TraceWithSampling) {
-  Options options = CurrentOptions();
-  ReadOptions ro;
-  WriteOptions wo;
-  TraceOptions trace_opts;
-  EnvOptions env_opts;
-  CreateAndReopenWithCF({"pikachu"}, options);
-  Random rnd(301);
-
-  // test the trace file sampling options
-  trace_opts.sampling_frequency = 2;
-  std::string trace_filename = dbname_ + "/rocksdb.trace_sampling";
-  std::unique_ptr<TraceWriter> trace_writer;
-  ASSERT_OK(NewFileTraceWriter(env_, env_opts, trace_filename, &trace_writer));
-  ASSERT_OK(db_->StartTrace(trace_opts, std::move(trace_writer)));
-  ASSERT_OK(Put(0, "a", "1"));
-  ASSERT_OK(Put(0, "b", "2"));
-  ASSERT_OK(Put(0, "c", "3"));
-  ASSERT_OK(Put(0, "d", "4"));
-  ASSERT_OK(Put(0, "e", "5"));
-  ASSERT_OK(db_->EndTrace());
-
-  std::string dbname2 = test::TmpDir(env_) + "/db_replay_sampling";
-  std::string value;
-  ASSERT_OK(DestroyDB(dbname2, options));
-
-  // Using a different name than db2, to pacify infer's use-after-lifetime
-  // warnings (http://fbinfer.com).
-  DB* db2_init = nullptr;
-  options.create_if_missing = true;
-  ASSERT_OK(DB::Open(options, dbname2, &db2_init));
-  ColumnFamilyHandle* cf;
-  ASSERT_OK(
-      db2_init->CreateColumnFamily(ColumnFamilyOptions(), "pikachu", &cf));
-  delete cf;
-  delete db2_init;
-
-  DB* db2 = nullptr;
-  std::vector<ColumnFamilyDescriptor> column_families;
-  ColumnFamilyOptions cf_options;
-  column_families.push_back(ColumnFamilyDescriptor("default", cf_options));
-  column_families.push_back(
-      ColumnFamilyDescriptor("pikachu", ColumnFamilyOptions()));
-  std::vector<ColumnFamilyHandle*> handles;
-  ASSERT_OK(DB::Open(DBOptions(), dbname2, column_families, &handles, &db2));
-
-  env_->SleepForMicroseconds(100);
-  ASSERT_TRUE(db2->Get(ro, handles[0], "a", &value).IsNotFound());
-  ASSERT_TRUE(db2->Get(ro, handles[0], "b", &value).IsNotFound());
-  ASSERT_TRUE(db2->Get(ro, handles[0], "c", &value).IsNotFound());
-  ASSERT_TRUE(db2->Get(ro, handles[0], "d", &value).IsNotFound());
-  ASSERT_TRUE(db2->Get(ro, handles[0], "e", &value).IsNotFound());
-
-  std::unique_ptr<TraceReader> trace_reader;
-  ASSERT_OK(NewFileTraceReader(env_, env_opts, trace_filename, &trace_reader));
-  Replayer replayer(db2, handles_, std::move(trace_reader));
-  ASSERT_OK(replayer.Replay());
-
-  ASSERT_TRUE(db2->Get(ro, handles[0], "a", &value).IsNotFound());
-  ASSERT_FALSE(db2->Get(ro, handles[0], "b", &value).IsNotFound());
-  ASSERT_TRUE(db2->Get(ro, handles[0], "c", &value).IsNotFound());
-  ASSERT_FALSE(db2->Get(ro, handles[0], "d", &value).IsNotFound());
-  ASSERT_TRUE(db2->Get(ro, handles[0], "e", &value).IsNotFound());
-
-  for (auto handle : handles) {
-    delete handle;
-  }
-  delete db2;
-  ASSERT_OK(DestroyDB(dbname2, options));
-}
-
 #endif  // ROCKSDB_LITE
 
 TEST_F(DBTest2, PinnableSliceAndMmapReads) {

db/db_test_util.cc

@@ -101,18 +101,18 @@ DBTestBase::~DBTestBase() {
 bool DBTestBase::ShouldSkipOptions(int option_config, int skip_mask) {
 #ifdef ROCKSDB_LITE
     // These options are not supported in ROCKSDB_LITE
-    if (option_config == kHashSkipList ||
-        option_config == kPlainTableFirstBytePrefix ||
-        option_config == kPlainTableCappedPrefix ||
-        option_config == kPlainTableCappedPrefixNonMmap ||
-        option_config == kPlainTableAllBytesPrefix ||
-        option_config == kVectorRep || option_config == kHashLinkList ||
-        option_config == kUniversalCompaction ||
-        option_config == kUniversalCompactionMultiLevel ||
-        option_config == kUniversalSubcompactions ||
-        option_config == kFIFOCompaction ||
-        option_config == kConcurrentSkipList) {
-      return true;
+  if (option_config == kHashSkipList ||
+      option_config == kPlainTableFirstBytePrefix ||
+      option_config == kPlainTableCappedPrefix ||
+      option_config == kPlainTableCappedPrefixNonMmap ||
+      option_config == kPlainTableAllBytesPrefix ||
+      option_config == kVectorRep || option_config == kHashLinkList ||
+      option_config == kHashCuckoo || option_config == kUniversalCompaction ||
+      option_config == kUniversalCompactionMultiLevel ||
+      option_config == kUniversalSubcompactions ||
+      option_config == kFIFOCompaction ||
+      option_config == kConcurrentSkipList) {
+    return true;
     }
 #endif
 
@@ -141,6 +141,9 @@ bool DBTestBase::ShouldSkipOptions(int option_config, int skip_mask) {
          option_config == kBlockBasedTableWithWholeKeyHashIndex)) {
       return true;
     }
+    if ((skip_mask & kSkipHashCuckoo) && (option_config == kHashCuckoo)) {
+      return true;
+    }
     if ((skip_mask & kSkipFIFOCompaction) && option_config == kFIFOCompaction) {
       return true;
     }
@@ -380,6 +383,11 @@ Options DBTestBase::GetOptions(
           NewHashLinkListRepFactory(4, 0, 3, true, 4));
       options.allow_concurrent_memtable_write = false;
       break;
+    case kHashCuckoo:
+      options.memtable_factory.reset(
+          NewHashCuckooRepFactory(options.write_buffer_size));
+      options.allow_concurrent_memtable_write = false;
+      break;
       case kDirectIO: {
         options.use_direct_reads = true;
         options.use_direct_io_for_flush_and_compaction = true;

db/db_test_util.h

@@ -652,28 +652,29 @@ class DBTestBase : public testing::Test {
     kPlainTableAllBytesPrefix = 6,
     kVectorRep = 7,
     kHashLinkList = 8,
-    kMergePut = 9,
-    kFilter = 10,
-    kFullFilterWithNewTableReaderForCompactions = 11,
-    kUncompressed = 12,
-    kNumLevel_3 = 13,
-    kDBLogDir = 14,
-    kWalDirAndMmapReads = 15,
-    kManifestFileSize = 16,
-    kPerfOptions = 17,
-    kHashSkipList = 18,
-    kUniversalCompaction = 19,
-    kUniversalCompactionMultiLevel = 20,
-    kCompressedBlockCache = 21,
-    kInfiniteMaxOpenFiles = 22,
-    kxxHashChecksum = 23,
-    kFIFOCompaction = 24,
-    kOptimizeFiltersForHits = 25,
-    kRowCache = 26,
-    kRecycleLogFiles = 27,
-    kConcurrentSkipList = 28,
-    kPipelinedWrite = 29,
-    kConcurrentWALWrites = 30,
+    kHashCuckoo = 9,
+    kMergePut = 10,
+    kFilter = 11,
+    kFullFilterWithNewTableReaderForCompactions = 12,
+    kUncompressed = 13,
+    kNumLevel_3 = 14,
+    kDBLogDir = 15,
+    kWalDirAndMmapReads = 16,
+    kManifestFileSize = 17,
+    kPerfOptions = 18,
+    kHashSkipList = 19,
+    kUniversalCompaction = 20,
+    kUniversalCompactionMultiLevel = 21,
+    kCompressedBlockCache = 22,
+    kInfiniteMaxOpenFiles = 23,
+    kxxHashChecksum = 24,
+    kFIFOCompaction = 25,
+    kOptimizeFiltersForHits = 26,
+    kRowCache = 27,
+    kRecycleLogFiles = 28,
+    kConcurrentSkipList = 29,
+    kPipelinedWrite = 30,
+    kConcurrentWALWrites = 31,
     kDirectIO,
     kLevelSubcompactions,
     kBlockBasedTableWithIndexRestartInterval,
@@ -709,6 +710,7 @@ class DBTestBase : public testing::Test {
     kSkipPlainTable = 8,
     kSkipHashIndex = 16,
     kSkipNoSeekToLast = 32,
+    kSkipHashCuckoo = 64,
     kSkipFIFOCompaction = 128,
     kSkipMmapReads = 256,
   };

db/db_universal_compaction_test.cc

@@ -27,7 +27,7 @@ class DBTestUniversalCompactionBase
  public:
   explicit DBTestUniversalCompactionBase(
       const std::string& path) : DBTestBase(path) {}
-  void SetUp() override {
+  virtual void SetUp() override {
     num_levels_ = std::get<0>(GetParam());
     exclusive_manual_compaction_ = std::get<1>(GetParam());
   }
@@ -63,13 +63,13 @@ void VerifyCompactionResult(
 
 class KeepFilter : public CompactionFilter {
  public:
-  bool Filter(int /*level*/, const Slice& /*key*/, const Slice& /*value*/,
-              std::string* /*new_value*/,
-              bool* /*value_changed*/) const override {
+  virtual bool Filter(int /*level*/, const Slice& /*key*/,
+                      const Slice& /*value*/, std::string* /*new_value*/,
+                      bool* /*value_changed*/) const override {
     return false;
   }
 
-  const char* Name() const override { return "KeepFilter"; }
+  virtual const char* Name() const override { return "KeepFilter"; }
 };
 
 class KeepFilterFactory : public CompactionFilterFactory {
@@ -77,7 +77,7 @@ class KeepFilterFactory : public CompactionFilterFactory {
   explicit KeepFilterFactory(bool check_context = false)
       : check_context_(check_context) {}
 
-  std::unique_ptr<CompactionFilter> CreateCompactionFilter(
+  virtual std::unique_ptr<CompactionFilter> CreateCompactionFilter(
       const CompactionFilter::Context& context) override {
     if (check_context_) {
       EXPECT_EQ(expect_full_compaction_.load(), context.is_full_compaction);
@@ -86,7 +86,7 @@ class KeepFilterFactory : public CompactionFilterFactory {
     return std::unique_ptr<CompactionFilter>(new KeepFilter());
   }
 
-  const char* Name() const override { return "KeepFilterFactory"; }
+  virtual const char* Name() const override { return "KeepFilterFactory"; }
   bool check_context_;
   std::atomic_bool expect_full_compaction_;
   std::atomic_bool expect_manual_compaction_;
@@ -95,14 +95,14 @@ class KeepFilterFactory : public CompactionFilterFactory {
 class DelayFilter : public CompactionFilter {
  public:
   explicit DelayFilter(DBTestBase* d) : db_test(d) {}
-  bool Filter(int /*level*/, const Slice& /*key*/, const Slice& /*value*/,
-              std::string* /*new_value*/,
-              bool* /*value_changed*/) const override {
+  virtual bool Filter(int /*level*/, const Slice& /*key*/,
+                      const Slice& /*value*/, std::string* /*new_value*/,
+                      bool* /*value_changed*/) const override {
     db_test->env_->addon_time_.fetch_add(1000);
     return true;
   }
 
-  const char* Name() const override { return "DelayFilter"; }
+  virtual const char* Name() const override { return "DelayFilter"; }
 
  private:
   DBTestBase* db_test;
@@ -111,12 +111,12 @@ class DelayFilter : public CompactionFilter {
 class DelayFilterFactory : public CompactionFilterFactory {
  public:
   explicit DelayFilterFactory(DBTestBase* d) : db_test(d) {}
-  std::unique_ptr<CompactionFilter> CreateCompactionFilter(
+  virtual std::unique_ptr<CompactionFilter> CreateCompactionFilter(
       const CompactionFilter::Context& /*context*/) override {
     return std::unique_ptr<CompactionFilter>(new DelayFilter(db_test));
   }
 
-  const char* Name() const override { return "DelayFilterFactory"; }
+  virtual const char* Name() const override { return "DelayFilterFactory"; }
 
  private:
   DBTestBase* db_test;

db/external_sst_file_basic_test.cc

@@ -134,9 +134,7 @@ class ExternalSSTFileBasicTest
         write_global_seqno, verify_checksums_before_ingest, true_data);
   }
 
-  ~ExternalSSTFileBasicTest() override {
-    test::DestroyDir(env_, sst_files_dir_);
-  }
+  ~ExternalSSTFileBasicTest() { test::DestroyDir(env_, sst_files_dir_); }
 
  protected:
   std::string sst_files_dir_;
@@ -861,66 +859,6 @@ TEST_P(ExternalSSTFileBasicTest, IngestFileWithFirstByteTampered) {
   } while (ChangeOptionsForFileIngestionTest());
 }
 
-TEST_P(ExternalSSTFileBasicTest, IngestExternalFileWithCorruptedPropsBlock) {
-  bool verify_checksums_before_ingest = std::get<1>(GetParam());
-  if (!verify_checksums_before_ingest) {
-    return;
-  }
-  uint64_t props_block_offset = 0;
-  size_t props_block_size = 0;
-  const auto& get_props_block_offset = [&](void* arg) {
-    props_block_offset = *reinterpret_cast<uint64_t*>(arg);
-  };
-  const auto& get_props_block_size = [&](void* arg) {
-    props_block_size = *reinterpret_cast<uint64_t*>(arg);
-  };
-  SyncPoint::GetInstance()->DisableProcessing();
-  SyncPoint::GetInstance()->ClearAllCallBacks();
-  SyncPoint::GetInstance()->SetCallBack(
-      "BlockBasedTableBuilder::WritePropertiesBlock:GetPropsBlockOffset",
-      get_props_block_offset);
-  SyncPoint::GetInstance()->SetCallBack(
-      "BlockBasedTableBuilder::WritePropertiesBlock:GetPropsBlockSize",
-      get_props_block_size);
-  SyncPoint::GetInstance()->EnableProcessing();
-  int file_id = 0;
-  Random64 rand(time(nullptr));
-  do {
-    std::string file_path = sst_files_dir_ + ToString(file_id++);
-    Options options = CurrentOptions();
-    SstFileWriter sst_file_writer(EnvOptions(), options);
-    Status s = sst_file_writer.Open(file_path);
-    ASSERT_OK(s);
-    for (int i = 0; i != 100; ++i) {
-      std::string key = Key(i);
-      std::string value = Key(i) + ToString(0);
-      ASSERT_OK(sst_file_writer.Put(key, value));
-    }
-    ASSERT_OK(sst_file_writer.Finish());
-
-    {
-      std::unique_ptr<RandomRWFile> rwfile;
-      ASSERT_OK(env_->NewRandomRWFile(file_path, &rwfile, EnvOptions()));
-      // Manually corrupt the file
-      ASSERT_GT(props_block_size, 8);
-      uint64_t offset =
-          props_block_offset + rand.Next() % (props_block_size - 8);
-      char scratch[8] = {0};
-      Slice buf;
-      ASSERT_OK(rwfile->Read(offset, sizeof(scratch), &buf, scratch));
-      scratch[0] ^= 0xff;  // flip one bit
-      ASSERT_OK(rwfile->Write(offset, buf));
-    }
-
-    // Ingest file.
-    IngestExternalFileOptions ifo;
-    ifo.write_global_seqno = std::get<0>(GetParam());
-    ifo.verify_checksums_before_ingest = true;
-    s = db_->IngestExternalFile({file_path}, ifo);
-    ASSERT_NOK(s);
-  } while (ChangeOptionsForFileIngestionTest());
-}
-
 INSTANTIATE_TEST_CASE_P(ExternalSSTFileBasicTest, ExternalSSTFileBasicTest,
                         testing::Values(std::make_tuple(true, true),
                                         std::make_tuple(true, false),

db/external_sst_file_ingestion_job.cc

@@ -167,6 +167,7 @@ Status ExternalSstFileIngestionJob::Run() {
   assert(status.ok() && need_flush == false);
 #endif
 
+  bool consumed_seqno = false;
   bool force_global_seqno = false;
 
   if (ingestion_options_.snapshot_consistency && !db_snapshots_->empty()) {
@@ -196,7 +197,7 @@ Status ExternalSstFileIngestionJob::Run() {
     TEST_SYNC_POINT_CALLBACK("ExternalSstFileIngestionJob::Run",
                              &assigned_seqno);
     if (assigned_seqno == last_seqno + 1) {
-      consumed_seqno_ = true;
+      consumed_seqno = true;
     }
     if (!status.ok()) {
       return status;
@@ -206,6 +207,13 @@ Status ExternalSstFileIngestionJob::Run() {
                   f.largest_internal_key(), f.assigned_seqno, f.assigned_seqno,
                   false);
   }
+
+  if (consumed_seqno) {
+    versions_->SetLastAllocatedSequence(last_seqno + 1);
+    versions_->SetLastPublishedSequence(last_seqno + 1);
+    versions_->SetLastSequence(last_seqno + 1);
+  }
+
   return status;
 }
 
@@ -261,7 +269,6 @@ void ExternalSstFileIngestionJob::Cleanup(const Status& status) {
                        f.internal_file_path.c_str(), s.ToString().c_str());
       }
     }
-    consumed_seqno_ = false;
   } else if (status.ok() && ingestion_options_.move_files) {
     // The files were moved and added successfully, remove original file links
     for (IngestedFileInfo& f : files_to_ingest_) {

db/external_sst_file_ingestion_job.h

@@ -85,8 +85,7 @@ class ExternalSstFileIngestionJob {
         env_options_(env_options),
         db_snapshots_(db_snapshots),
         ingestion_options_(ingestion_options),
-        job_start_time_(env_->NowMicros()),
-        consumed_seqno_(false) {}
+        job_start_time_(env_->NowMicros()) {}
 
   // Prepare the job by copying external files into the DB.
   Status Prepare(const std::vector<std::string>& external_files_paths,
@@ -119,9 +118,6 @@ class ExternalSstFileIngestionJob {
     return files_to_ingest_;
   }
 
-  // Whether to increment VersionSet's seqno after this job runs
-  bool ShouldIncrementLastSequence() const { return consumed_seqno_; }
-
  private:
   // Open the external file and populate `file_to_ingest` with all the
   // external information we need to ingest this file.
@@ -163,7 +159,6 @@ class ExternalSstFileIngestionJob {
   const IngestExternalFileOptions& ingestion_options_;
   VersionEdit edit_;
   uint64_t job_start_time_;
-  bool consumed_seqno_;
 };
 
 }  // namespace rocksdb

db/external_sst_file_test.cc

@@ -10,7 +10,6 @@
 #include "port/port.h"
 #include "port/stack_trace.h"
 #include "rocksdb/sst_file_writer.h"
-#include "util/fault_injection_test_env.h"
 #include "util/filename.h"
 #include "util/testutil.h"
 
@@ -30,55 +29,6 @@ class ExternalSSTFileTest
     env_->CreateDir(sst_files_dir_);
   }
 
-  Status GenerateOneExternalFile(
-      const Options& options, ColumnFamilyHandle* cfh,
-      std::vector<std::pair<std::string, std::string>>& data, int file_id,
-      bool sort_data, std::string* external_file_path,
-      std::map<std::string, std::string>* true_data) {
-    // Generate a file id if not provided
-    if (-1 == file_id) {
-      file_id = (++last_file_id_);
-    }
-    // Sort data if asked to do so
-    if (sort_data) {
-      std::sort(data.begin(), data.end(),
-                [&](const std::pair<std::string, std::string>& e1,
-                    const std::pair<std::string, std::string>& e2) {
-                  return options.comparator->Compare(e1.first, e2.first) < 0;
-                });
-      auto uniq_iter = std::unique(
-          data.begin(), data.end(),
-          [&](const std::pair<std::string, std::string>& e1,
-              const std::pair<std::string, std::string>& e2) {
-            return options.comparator->Compare(e1.first, e2.first) == 0;
-          });
-      data.resize(uniq_iter - data.begin());
-    }
-    std::string file_path = sst_files_dir_ + ToString(file_id);
-    SstFileWriter sst_file_writer(EnvOptions(), options, cfh);
-    Status s = sst_file_writer.Open(file_path);
-    if (!s.ok()) {
-      return s;
-    }
-    for (const auto& entry : data) {
-      s = sst_file_writer.Put(entry.first, entry.second);
-      if (!s.ok()) {
-        sst_file_writer.Finish();
-        return s;
-      }
-    }
-    s = sst_file_writer.Finish();
-    if (s.ok() && external_file_path != nullptr) {
-      *external_file_path = file_path;
-    }
-    if (s.ok() && nullptr != true_data) {
-      for (const auto& entry : data) {
-        true_data->insert({entry.first, entry.second});
-      }
-    }
-    return s;
-  }
-
   Status GenerateAndAddExternalFile(
       const Options options,
       std::vector<std::pair<std::string, std::string>> data, int file_id = -1,
@@ -204,41 +154,6 @@ class ExternalSSTFileTest
     return s;
   }
 
-  Status GenerateAndAddExternalFiles(
-      const Options& options,
-      const std::vector<ColumnFamilyHandle*>& column_families,
-      const std::vector<IngestExternalFileOptions>& ifos,
-      std::vector<std::vector<std::pair<std::string, std::string>>>& data,
-      int file_id, bool sort_data,
-      std::vector<std::map<std::string, std::string>>& true_data) {
-    if (-1 == file_id) {
-      file_id = (++last_file_id_);
-    }
-    // Generate external SST files, one for each column family
-    size_t num_cfs = column_families.size();
-    assert(ifos.size() == num_cfs);
-    assert(data.size() == num_cfs);
-    Status s;
-    std::vector<IngestExternalFileArg> args(num_cfs);
-    for (size_t i = 0; i != num_cfs; ++i) {
-      std::string external_file_path;
-      s = GenerateOneExternalFile(
-          options, column_families[i], data[i], file_id, sort_data,
-          &external_file_path,
-          true_data.size() == num_cfs ? &true_data[i] : nullptr);
-      if (!s.ok()) {
-        return s;
-      }
-      ++file_id;
-
-      args[i].column_family = column_families[i];
-      args[i].external_files.push_back(external_file_path);
-      args[i].options = ifos[i];
-    }
-    s = db_->IngestExternalFiles(args);
-    return s;
-  }
-
   Status GenerateAndAddExternalFile(
       const Options options, std::vector<std::pair<int, std::string>> data,
       int file_id = -1, bool allow_global_seqno = false,
@@ -283,7 +198,7 @@ class ExternalSSTFileTest
     return db_->IngestExternalFile(files, opts);
   }
 
-  ~ExternalSSTFileTest() override { test::DestroyDir(env_, sst_files_dir_); }
+  ~ExternalSSTFileTest() { test::DestroyDir(env_, sst_files_dir_); }
 
  protected:
   int last_file_id_ = 0;
@@ -571,7 +486,7 @@ class SstFileWriterCollector : public TablePropertiesCollector {
     return Status::OK();
   }
 
-  UserCollectedProperties GetReadableProperties() const override {
+  virtual UserCollectedProperties GetReadableProperties() const override {
     return UserCollectedProperties{};
   }
 
@@ -585,7 +500,7 @@ class SstFileWriterCollectorFactory : public TablePropertiesCollectorFactory {
  public:
   explicit SstFileWriterCollectorFactory(std::string prefix)
       : prefix_(prefix), num_created_(0) {}
-  TablePropertiesCollector* CreateTablePropertiesCollector(
+  virtual TablePropertiesCollector* CreateTablePropertiesCollector(
       TablePropertiesCollectorFactory::Context /*context*/) override {
     num_created_++;
     return new SstFileWriterCollector(prefix_);
@@ -2318,400 +2233,6 @@ TEST_F(ExternalSSTFileTest, SkipBloomFilter) {
   }
 }
 
-TEST_F(ExternalSSTFileTest, IngestFileWrittenWithCompressionDictionary) {
-  if (!ZSTD_Supported()) {
-    return;
-  }
-  const int kNumEntries = 1 << 10;
-  const int kNumBytesPerEntry = 1 << 10;
-  Options options = CurrentOptions();
-  options.compression = kZSTD;
-  options.compression_opts.max_dict_bytes = 1 << 14;        // 16KB
-  options.compression_opts.zstd_max_train_bytes = 1 << 18;  // 256KB
-  DestroyAndReopen(options);
-
-  std::atomic<int> num_compression_dicts(0);
-  rocksdb::SyncPoint::GetInstance()->SetCallBack(
-      "BlockBasedTableBuilder::WriteCompressionDictBlock:RawDict",
-      [&](void* /* arg */) { ++num_compression_dicts; });
-  rocksdb::SyncPoint::GetInstance()->EnableProcessing();
-
-  Random rnd(301);
-  std::vector<std::pair<std::string, std::string>> random_data;
-  for (int i = 0; i < kNumEntries; i++) {
-    std::string val;
-    test::RandomString(&rnd, kNumBytesPerEntry, &val);
-    random_data.emplace_back(Key(i), std::move(val));
-  }
-  ASSERT_OK(GenerateAndAddExternalFile(options, std::move(random_data)));
-  ASSERT_EQ(1, num_compression_dicts);
-}
-
-TEST_P(ExternalSSTFileTest, IngestFilesIntoMultipleColumnFamilies_Success) {
-  std::unique_ptr<FaultInjectionTestEnv> fault_injection_env(
-      new FaultInjectionTestEnv(env_));
-  Options options = CurrentOptions();
-  options.env = fault_injection_env.get();
-  CreateAndReopenWithCF({"pikachu"}, options);
-  std::vector<ColumnFamilyHandle*> column_families;
-  column_families.push_back(handles_[0]);
-  column_families.push_back(handles_[1]);
-  std::vector<IngestExternalFileOptions> ifos(column_families.size());
-  for (auto& ifo : ifos) {
-    ifo.allow_global_seqno = true;  // Always allow global_seqno
-    // May or may not write global_seqno
-    ifo.write_global_seqno = std::get<0>(GetParam());
-    // Whether to verify checksums before ingestion
-    ifo.verify_checksums_before_ingest = std::get<1>(GetParam());
-  }
-  std::vector<std::vector<std::pair<std::string, std::string>>> data;
-  data.push_back(
-      {std::make_pair("foo1", "fv1"), std::make_pair("foo2", "fv2")});
-  data.push_back(
-      {std::make_pair("bar1", "bv1"), std::make_pair("bar2", "bv2")});
-  // Resize the true_data vector upon construction to avoid re-alloc
-  std::vector<std::map<std::string, std::string>> true_data(
-      column_families.size());
-  Status s = GenerateAndAddExternalFiles(options, column_families, ifos, data,
-                                         -1, true, true_data);
-  ASSERT_OK(s);
-  Close();
-  ReopenWithColumnFamilies({kDefaultColumnFamilyName, "pikachu"}, options);
-  ASSERT_EQ(2, handles_.size());
-  int cf = 0;
-  for (const auto& verify_map : true_data) {
-    for (const auto& elem : verify_map) {
-      const std::string& key = elem.first;
-      const std::string& value = elem.second;
-      ASSERT_EQ(value, Get(cf, key));
-    }
-    ++cf;
-  }
-  Close();
-  Destroy(options, true /* delete_cf_paths */);
-}
-
-TEST_P(ExternalSSTFileTest,
-       IngestFilesIntoMultipleColumnFamilies_NoMixedStateWithSnapshot) {
-  std::unique_ptr<FaultInjectionTestEnv> fault_injection_env(
-      new FaultInjectionTestEnv(env_));
-  SyncPoint::GetInstance()->DisableProcessing();
-  SyncPoint::GetInstance()->ClearAllCallBacks();
-  SyncPoint::GetInstance()->LoadDependency({
-      {"DBImpl::IngestExternalFiles:InstallSVForFirstCF:0",
-       "ExternalSSTFileTest::IngestFilesIntoMultipleColumnFamilies_MixedState:"
-       "BeforeRead"},
-      {"ExternalSSTFileTest::IngestFilesIntoMultipleColumnFamilies_MixedState:"
-       "AfterRead",
-       "DBImpl::IngestExternalFiles:InstallSVForFirstCF:1"},
-  });
-  SyncPoint::GetInstance()->EnableProcessing();
-
-  Options options = CurrentOptions();
-  options.env = fault_injection_env.get();
-  CreateAndReopenWithCF({"pikachu"}, options);
-  const std::vector<std::map<std::string, std::string>> data_before_ingestion =
-      {{{"foo1", "fv1_0"}, {"foo2", "fv2_0"}, {"foo3", "fv3_0"}},
-       {{"bar1", "bv1_0"}, {"bar2", "bv2_0"}, {"bar3", "bv3_0"}}};
-  for (size_t i = 0; i != handles_.size(); ++i) {
-    int cf = static_cast<int>(i);
-    const auto& orig_data = data_before_ingestion[i];
-    for (const auto& kv : orig_data) {
-      ASSERT_OK(Put(cf, kv.first, kv.second));
-    }
-    ASSERT_OK(Flush(cf));
-  }
-
-  std::vector<ColumnFamilyHandle*> column_families;
-  column_families.push_back(handles_[0]);
-  column_families.push_back(handles_[1]);
-  std::vector<IngestExternalFileOptions> ifos(column_families.size());
-  for (auto& ifo : ifos) {
-    ifo.allow_global_seqno = true;  // Always allow global_seqno
-    // May or may not write global_seqno
-    ifo.write_global_seqno = std::get<0>(GetParam());
-    // Whether to verify checksums before ingestion
-    ifo.verify_checksums_before_ingest = std::get<1>(GetParam());
-  }
-  std::vector<std::vector<std::pair<std::string, std::string>>> data;
-  data.push_back(
-      {std::make_pair("foo1", "fv1"), std::make_pair("foo2", "fv2")});
-  data.push_back(
-      {std::make_pair("bar1", "bv1"), std::make_pair("bar2", "bv2")});
-  // Resize the true_data vector upon construction to avoid re-alloc
-  std::vector<std::map<std::string, std::string>> true_data(
-      column_families.size());
-  // Take snapshot before ingestion starts
-  ReadOptions read_opts;
-  read_opts.total_order_seek = true;
-  read_opts.snapshot = dbfull()->GetSnapshot();
-  std::vector<Iterator*> iters(handles_.size());
-
-  // Range scan checks first kv of each CF before ingestion starts.
-  for (size_t i = 0; i != handles_.size(); ++i) {
-    iters[i] = dbfull()->NewIterator(read_opts, handles_[i]);
-    iters[i]->SeekToFirst();
-    ASSERT_TRUE(iters[i]->Valid());
-    const std::string& key = iters[i]->key().ToString();
-    const std::string& value = iters[i]->value().ToString();
-    const std::map<std::string, std::string>& orig_data =
-        data_before_ingestion[i];
-    std::map<std::string, std::string>::const_iterator it = orig_data.find(key);
-    ASSERT_NE(orig_data.end(), it);
-    ASSERT_EQ(it->second, value);
-    iters[i]->Next();
-  }
-  port::Thread ingest_thread([&]() {
-    ASSERT_OK(GenerateAndAddExternalFiles(options, column_families, ifos, data,
-                                          -1, true, true_data));
-  });
-  TEST_SYNC_POINT(
-      "ExternalSSTFileTest::IngestFilesIntoMultipleColumnFamilies_MixedState:"
-      "BeforeRead");
-  // Should see only data before ingestion
-  for (size_t i = 0; i != handles_.size(); ++i) {
-    const auto& orig_data = data_before_ingestion[i];
-    for (; iters[i]->Valid(); iters[i]->Next()) {
-      const std::string& key = iters[i]->key().ToString();
-      const std::string& value = iters[i]->value().ToString();
-      std::map<std::string, std::string>::const_iterator it =
-          orig_data.find(key);
-      ASSERT_NE(orig_data.end(), it);
-      ASSERT_EQ(it->second, value);
-    }
-  }
-  TEST_SYNC_POINT(
-      "ExternalSSTFileTest::IngestFilesIntoMultipleColumnFamilies_MixedState:"
-      "AfterRead");
-  ingest_thread.join();
-  for (auto* iter : iters) {
-    delete iter;
-  }
-  iters.clear();
-  dbfull()->ReleaseSnapshot(read_opts.snapshot);
-
-  Close();
-  ReopenWithColumnFamilies({kDefaultColumnFamilyName, "pikachu"}, options);
-  // Should see consistent state after ingestion for all column families even
-  // without snapshot.
-  ASSERT_EQ(2, handles_.size());
-  int cf = 0;
-  for (const auto& verify_map : true_data) {
-    for (const auto& elem : verify_map) {
-      const std::string& key = elem.first;
-      const std::string& value = elem.second;
-      ASSERT_EQ(value, Get(cf, key));
-    }
-    ++cf;
-  }
-  Close();
-  Destroy(options, true /* delete_cf_paths */);
-}
-
-TEST_P(ExternalSSTFileTest, IngestFilesIntoMultipleColumnFamilies_PrepareFail) {
-  std::unique_ptr<FaultInjectionTestEnv> fault_injection_env(
-      new FaultInjectionTestEnv(env_));
-  Options options = CurrentOptions();
-  options.env = fault_injection_env.get();
-  SyncPoint::GetInstance()->DisableProcessing();
-  SyncPoint::GetInstance()->ClearAllCallBacks();
-  SyncPoint::GetInstance()->LoadDependency({
-      {"DBImpl::IngestExternalFiles:BeforeLastJobPrepare:0",
-       "ExternalSSTFileTest::IngestFilesIntoMultipleColumnFamilies_PrepareFail:"
-       "0"},
-      {"ExternalSSTFileTest::IngestFilesIntoMultipleColumnFamilies:PrepareFail:"
-       "1",
-       "DBImpl::IngestExternalFiles:BeforeLastJobPrepare:1"},
-  });
-  SyncPoint::GetInstance()->EnableProcessing();
-  CreateAndReopenWithCF({"pikachu"}, options);
-  std::vector<ColumnFamilyHandle*> column_families;
-  column_families.push_back(handles_[0]);
-  column_families.push_back(handles_[1]);
-  std::vector<IngestExternalFileOptions> ifos(column_families.size());
-  for (auto& ifo : ifos) {
-    ifo.allow_global_seqno = true;  // Always allow global_seqno
-    // May or may not write global_seqno
-    ifo.write_global_seqno = std::get<0>(GetParam());
-    // Whether to verify block checksums before ingest
-    ifo.verify_checksums_before_ingest = std::get<1>(GetParam());
-  }
-  std::vector<std::vector<std::pair<std::string, std::string>>> data;
-  data.push_back(
-      {std::make_pair("foo1", "fv1"), std::make_pair("foo2", "fv2")});
-  data.push_back(
-      {std::make_pair("bar1", "bv1"), std::make_pair("bar2", "bv2")});
-  // Resize the true_data vector upon construction to avoid re-alloc
-  std::vector<std::map<std::string, std::string>> true_data(
-      column_families.size());
-  port::Thread ingest_thread([&]() {
-    Status s = GenerateAndAddExternalFiles(options, column_families, ifos, data,
-                                           -1, true, true_data);
-    ASSERT_NOK(s);
-  });
-  TEST_SYNC_POINT(
-      "ExternalSSTFileTest::IngestFilesIntoMultipleColumnFamilies_PrepareFail:"
-      "0");
-  fault_injection_env->SetFilesystemActive(false);
-  TEST_SYNC_POINT(
-      "ExternalSSTFileTest::IngestFilesIntoMultipleColumnFamilies:PrepareFail:"
-      "1");
-  ingest_thread.join();
-
-  fault_injection_env->SetFilesystemActive(true);
-  Close();
-  ReopenWithColumnFamilies({kDefaultColumnFamilyName, "pikachu"}, options);
-  ASSERT_EQ(2, handles_.size());
-  int cf = 0;
-  for (const auto& verify_map : true_data) {
-    for (const auto& elem : verify_map) {
-      const std::string& key = elem.first;
-      ASSERT_EQ("NOT_FOUND", Get(cf, key));
-    }
-    ++cf;
-  }
-  Close();
-  Destroy(options, true /* delete_cf_paths */);
-}
-
-TEST_P(ExternalSSTFileTest, IngestFilesIntoMultipleColumnFamilies_CommitFail) {
-  std::unique_ptr<FaultInjectionTestEnv> fault_injection_env(
-      new FaultInjectionTestEnv(env_));
-  Options options = CurrentOptions();
-  options.env = fault_injection_env.get();
-  SyncPoint::GetInstance()->DisableProcessing();
-  SyncPoint::GetInstance()->ClearAllCallBacks();
-  SyncPoint::GetInstance()->LoadDependency({
-      {"DBImpl::IngestExternalFiles:BeforeJobsRun:0",
-       "ExternalSSTFileTest::IngestFilesIntoMultipleColumnFamilies_CommitFail:"
-       "0"},
-      {"ExternalSSTFileTest::IngestFilesIntoMultipleColumnFamilies_CommitFail:"
-       "1",
-       "DBImpl::IngestExternalFiles:BeforeJobsRun:1"},
-  });
-  SyncPoint::GetInstance()->EnableProcessing();
-  CreateAndReopenWithCF({"pikachu"}, options);
-  std::vector<ColumnFamilyHandle*> column_families;
-  column_families.push_back(handles_[0]);
-  column_families.push_back(handles_[1]);
-  std::vector<IngestExternalFileOptions> ifos(column_families.size());
-  for (auto& ifo : ifos) {
-    ifo.allow_global_seqno = true;  // Always allow global_seqno
-    // May or may not write global_seqno
-    ifo.write_global_seqno = std::get<0>(GetParam());
-    // Whether to verify block checksums before ingestion
-    ifo.verify_checksums_before_ingest = std::get<1>(GetParam());
-  }
-  std::vector<std::vector<std::pair<std::string, std::string>>> data;
-  data.push_back(
-      {std::make_pair("foo1", "fv1"), std::make_pair("foo2", "fv2")});
-  data.push_back(
-      {std::make_pair("bar1", "bv1"), std::make_pair("bar2", "bv2")});
-  // Resize the true_data vector upon construction to avoid re-alloc
-  std::vector<std::map<std::string, std::string>> true_data(
-      column_families.size());
-  port::Thread ingest_thread([&]() {
-    Status s = GenerateAndAddExternalFiles(options, column_families, ifos, data,
-                                           -1, true, true_data);
-    ASSERT_NOK(s);
-  });
-  TEST_SYNC_POINT(
-      "ExternalSSTFileTest::IngestFilesIntoMultipleColumnFamilies_CommitFail:"
-      "0");
-  fault_injection_env->SetFilesystemActive(false);
-  TEST_SYNC_POINT(
-      "ExternalSSTFileTest::IngestFilesIntoMultipleColumnFamilies_CommitFail:"
-      "1");
-  ingest_thread.join();
-
-  fault_injection_env->SetFilesystemActive(true);
-  Close();
-  ReopenWithColumnFamilies({kDefaultColumnFamilyName, "pikachu"}, options);
-  ASSERT_EQ(2, handles_.size());
-  int cf = 0;
-  for (const auto& verify_map : true_data) {
-    for (const auto& elem : verify_map) {
-      const std::string& key = elem.first;
-      ASSERT_EQ("NOT_FOUND", Get(cf, key));
-    }
-    ++cf;
-  }
-  Close();
-  Destroy(options, true /* delete_cf_paths */);
-}
-
-TEST_P(ExternalSSTFileTest,
-       IngestFilesIntoMultipleColumnFamilies_PartialManifestWriteFail) {
-  std::unique_ptr<FaultInjectionTestEnv> fault_injection_env(
-      new FaultInjectionTestEnv(env_));
-  Options options = CurrentOptions();
-  options.env = fault_injection_env.get();
-
-  CreateAndReopenWithCF({"pikachu"}, options);
-
-  SyncPoint::GetInstance()->ClearTrace();
-  SyncPoint::GetInstance()->DisableProcessing();
-  SyncPoint::GetInstance()->ClearAllCallBacks();
-  SyncPoint::GetInstance()->LoadDependency({
-      {"VersionSet::ProcessManifestWrites:BeforeWriteLastVersionEdit:0",
-       "ExternalSSTFileTest::IngestFilesIntoMultipleColumnFamilies_"
-       "PartialManifestWriteFail:0"},
-      {"ExternalSSTFileTest::IngestFilesIntoMultipleColumnFamilies_"
-       "PartialManifestWriteFail:1",
-       "VersionSet::ProcessManifestWrites:BeforeWriteLastVersionEdit:1"},
-  });
-  SyncPoint::GetInstance()->EnableProcessing();
-
-  std::vector<ColumnFamilyHandle*> column_families;
-  column_families.push_back(handles_[0]);
-  column_families.push_back(handles_[1]);
-  std::vector<IngestExternalFileOptions> ifos(column_families.size());
-  for (auto& ifo : ifos) {
-    ifo.allow_global_seqno = true;  // Always allow global_seqno
-    // May or may not write global_seqno
-    ifo.write_global_seqno = std::get<0>(GetParam());
-    // Whether to verify block checksums before ingestion
-    ifo.verify_checksums_before_ingest = std::get<1>(GetParam());
-  }
-  std::vector<std::vector<std::pair<std::string, std::string>>> data;
-  data.push_back(
-      {std::make_pair("foo1", "fv1"), std::make_pair("foo2", "fv2")});
-  data.push_back(
-      {std::make_pair("bar1", "bv1"), std::make_pair("bar2", "bv2")});
-  // Resize the true_data vector upon construction to avoid re-alloc
-  std::vector<std::map<std::string, std::string>> true_data(
-      column_families.size());
-  port::Thread ingest_thread([&]() {
-    Status s = GenerateAndAddExternalFiles(options, column_families, ifos, data,
-                                           -1, true, true_data);
-    ASSERT_NOK(s);
-  });
-  TEST_SYNC_POINT(
-      "ExternalSSTFileTest::IngestFilesIntoMultipleColumnFamilies_"
-      "PartialManifestWriteFail:0");
-  fault_injection_env->SetFilesystemActive(false);
-  TEST_SYNC_POINT(
-      "ExternalSSTFileTest::IngestFilesIntoMultipleColumnFamilies_"
-      "PartialManifestWriteFail:1");
-  ingest_thread.join();
-
-  fault_injection_env->DropUnsyncedFileData();
-  fault_injection_env->SetFilesystemActive(true);
-  Close();
-  ReopenWithColumnFamilies({kDefaultColumnFamilyName, "pikachu"}, options);
-  ASSERT_EQ(2, handles_.size());
-  int cf = 0;
-  for (const auto& verify_map : true_data) {
-    for (const auto& elem : verify_map) {
-      const std::string& key = elem.first;
-      ASSERT_EQ("NOT_FOUND", Get(cf, key));
-    }
-    ++cf;
-  }
-  Close();
-  Destroy(options, true /* delete_cf_paths */);
-}
-
 INSTANTIATE_TEST_CASE_P(ExternalSSTFileTest, ExternalSSTFileTest,
                         testing::Values(std::make_tuple(false, false),
                                         std::make_tuple(false, true),

db/fault_injection_test.cc

@@ -83,7 +83,7 @@ class FaultInjectionTest
         env_(nullptr),
         db_(nullptr) {}
 
-  ~FaultInjectionTest() override {
+  ~FaultInjectionTest() {
     rocksdb::SyncPoint::GetInstance()->DisableProcessing();
     rocksdb::SyncPoint::GetInstance()->ClearAllCallBacks();
   }

db/file_indexer_test.cc

@@ -47,7 +47,7 @@ class FileIndexerTest : public testing::Test {
   FileIndexerTest()
       : kNumLevels(4), files(new std::vector<FileMetaData*>[kNumLevels]) {}
 
-  ~FileIndexerTest() override {
+  ~FileIndexerTest() {
     ClearFiles();
     delete[] files;
   }

db/flush_job.cc

@@ -311,7 +311,6 @@ Status FlushJob::WriteLevel0Table() {
     ro.total_order_seek = true;
     Arena arena;
     uint64_t total_num_entries = 0, total_num_deletes = 0;
-    uint64_t total_data_size = 0;
     size_t total_memory_usage = 0;
     for (MemTable* m : mems_) {
       ROCKS_LOG_INFO(
@@ -326,18 +325,16 @@ Status FlushJob::WriteLevel0Table() {
       }
       total_num_entries += m->num_entries();
       total_num_deletes += m->num_deletes();
-      total_data_size += m->get_data_size();
       total_memory_usage += m->ApproximateMemoryUsage();
     }
 
-    event_logger_->Log() << "job" << job_context_->job_id << "event"
-                         << "flush_started"
-                         << "num_memtables" << mems_.size() << "num_entries"
-                         << total_num_entries << "num_deletes"
-                         << total_num_deletes << "total_data_size"
-                         << total_data_size << "memory_usage"
-                         << total_memory_usage << "flush_reason"
-                         << GetFlushReasonString(cfd_->GetFlushReason());
+    event_logger_->Log()
+        << "job" << job_context_->job_id << "event"
+        << "flush_started"
+        << "num_memtables" << mems_.size() << "num_entries" << total_num_entries
+        << "num_deletes" << total_num_deletes << "memory_usage"
+        << total_memory_usage << "flush_reason"
+        << GetFlushReasonString(cfd_->GetFlushReason());
 
     {
       ScopedArenaIterator iter(

db/flush_job_test.cc

@@ -308,9 +308,7 @@ TEST_F(FlushJobTest, FlushMemtablesMultipleColumnFamilies) {
     k++;
   }
   HistogramData hist;
-  std::vector<FileMetaData> file_metas;
-  // Call reserve to avoid auto-resizing
-  file_metas.reserve(flush_jobs.size());
+  autovector<FileMetaData> file_metas;
   mutex_.Lock();
   for (auto& job : flush_jobs) {
     job.PickMemTable();
@@ -321,10 +319,6 @@ TEST_F(FlushJobTest, FlushMemtablesMultipleColumnFamilies) {
     ASSERT_OK(job.Run(nullptr /**/, &meta));
     file_metas.emplace_back(meta);
   }
-  autovector<FileMetaData*> file_meta_ptrs;
-  for (auto& meta : file_metas) {
-    file_meta_ptrs.push_back(&meta);
-  }
   autovector<const autovector<MemTable*>*> mems_list;
   for (size_t i = 0; i != all_cfds.size(); ++i) {
     const auto& mems = flush_jobs[i].GetMemTables();
@@ -337,7 +331,7 @@ TEST_F(FlushJobTest, FlushMemtablesMultipleColumnFamilies) {
 
   Status s = InstallMemtableAtomicFlushResults(
       nullptr /* imm_lists */, all_cfds, mutable_cf_options_list, mems_list,
-      versions_.get(), &mutex_, file_meta_ptrs, &job_context.memtables_to_free,
+      versions_.get(), &mutex_, file_metas, &job_context.memtables_to_free,
       nullptr /* db_directory */, nullptr /* log_buffer */);
   ASSERT_OK(s);
 

db/forward_iterator.cc

@@ -46,7 +46,7 @@ class ForwardLevelIterator : public InternalIterator {
         pinned_iters_mgr_(nullptr),
         prefix_extractor_(prefix_extractor) {}
 
-  ~ForwardLevelIterator() override {
+  ~ForwardLevelIterator() {
     // Reset current pointer
     if (pinned_iters_mgr_ && pinned_iters_mgr_->PinningEnabled()) {
       pinned_iters_mgr_->PinIterator(file_iter_);

db/in_memory_stats_history.cc

@@ -1,45 +0,0 @@
-// Copyright (c) 2011-present, Facebook, Inc.  All rights reserved.
-//  This source code is licensed under both the GPLv2 (found in the
-//  COPYING file in the root directory) and Apache 2.0 License
-//  (found in the LICENSE.Apache file in the root directory).
-// Copyright (c) 2011 The LevelDB Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style license that can be
-// found in the LICENSE file. See the AUTHORS file for names of contributors.
-
-#include "db/db_impl.h"
-#include "db/in_memory_stats_history.h"
-
-namespace rocksdb {
-
-InMemoryStatsHistoryIterator::~InMemoryStatsHistoryIterator() {}
-
-bool InMemoryStatsHistoryIterator::Valid() const { return valid_; }
-
-Status InMemoryStatsHistoryIterator::status() const { return status_; }
-
-void InMemoryStatsHistoryIterator::Next() {
-  // increment start_time by 1 to avoid infinite loop
-  AdvanceIteratorByTime(GetStatsTime() + 1, end_time_);
-}
-
-uint64_t InMemoryStatsHistoryIterator::GetStatsTime() const { return time_; }
-
-const std::map<std::string, uint64_t>&
-InMemoryStatsHistoryIterator::GetStatsMap() const {
-  return stats_map_;
-}
-
-// advance the iterator to the next time between [start_time, end_time)
-// if success, update time_ and stats_map_ with new_time and stats_map
-void InMemoryStatsHistoryIterator::AdvanceIteratorByTime(uint64_t start_time,
-                                                         uint64_t end_time) {
-  // try to find next entry in stats_history_ map
-  if (db_impl_ != nullptr) {
-    valid_ =
-        db_impl_->FindStatsByTime(start_time, end_time, &time_, &stats_map_);
-  } else {
-    valid_ = false;
-  }
-}
-
-}  // namespace rocksdb

db/in_memory_stats_history.h

@@ -1,55 +0,0 @@
-// Copyright (c) 2011-present, Facebook, Inc.  All rights reserved.
-//  This source code is licensed under both the GPLv2 (found in the
-//  COPYING file in the root directory) and Apache 2.0 License
-//  (found in the LICENSE.Apache file in the root directory).
-// Copyright (c) 2011 The LevelDB Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style license that can be
-// found in the LICENSE file. See the AUTHORS file for names of contributors.
-
-#pragma once
-
-#include "rocksdb/stats_history.h"
-
-namespace rocksdb {
-
-class InMemoryStatsHistoryIterator final : public StatsHistoryIterator {
- public:
-  InMemoryStatsHistoryIterator(uint64_t start_time, uint64_t end_time,
-                               DBImpl* db_impl)
-      : start_time_(start_time),
-        end_time_(end_time),
-        valid_(true),
-        db_impl_(db_impl) {
-    AdvanceIteratorByTime(start_time_, end_time_);
-  }
-  ~InMemoryStatsHistoryIterator() override;
-  bool Valid() const override;
-  Status status() const override;
-
-  void Next() override;
-  uint64_t GetStatsTime() const override;
-
-  const std::map<std::string, uint64_t>& GetStatsMap() const override;
-
- private:
-  // advance the iterator to the next stats history record with timestamp
-  // between [start_time, end_time)
-  void AdvanceIteratorByTime(uint64_t start_time, uint64_t end_time);
-
-  // No copying allowed
-  InMemoryStatsHistoryIterator(const InMemoryStatsHistoryIterator&) = delete;
-  void operator=(const InMemoryStatsHistoryIterator&) = delete;
-  InMemoryStatsHistoryIterator(InMemoryStatsHistoryIterator&&) = delete;
-  InMemoryStatsHistoryIterator& operator=(InMemoryStatsHistoryIterator&&) =
-      delete;
-
-  uint64_t time_;
-  uint64_t start_time_;
-  uint64_t end_time_;
-  std::map<std::string, uint64_t> stats_map_;
-  Status status_;
-  bool valid_;
-  DBImpl* db_impl_;
-};
-
-}  // namespace rocksdb

db/listener_test.cc

@@ -46,20 +46,22 @@ class EventListenerTest : public DBTestBase {
 };
 
 struct TestPropertiesCollector : public rocksdb::TablePropertiesCollector {
-  rocksdb::Status AddUserKey(const rocksdb::Slice& /*key*/,
-                             const rocksdb::Slice& /*value*/,
-                             rocksdb::EntryType /*type*/,
-                             rocksdb::SequenceNumber /*seq*/,
-                             uint64_t /*file_size*/) override {
+  virtual rocksdb::Status AddUserKey(const rocksdb::Slice& /*key*/,
+                                     const rocksdb::Slice& /*value*/,
+                                     rocksdb::EntryType /*type*/,
+                                     rocksdb::SequenceNumber /*seq*/,
+                                     uint64_t /*file_size*/) override {
     return Status::OK();
   }
-  rocksdb::Status Finish(
+  virtual rocksdb::Status Finish(
       rocksdb::UserCollectedProperties* properties) override {
     properties->insert({"0", "1"});
     return Status::OK();
   }
 
-  const char* Name() const override { return "TestTablePropertiesCollector"; }
+  virtual const char* Name() const override {
+    return "TestTablePropertiesCollector";
+  }
 
   rocksdb::UserCollectedProperties GetReadableProperties() const override {
     rocksdb::UserCollectedProperties ret;
@@ -70,7 +72,7 @@ struct TestPropertiesCollector : public rocksdb::TablePropertiesCollector {
 
 class TestPropertiesCollectorFactory : public TablePropertiesCollectorFactory {
  public:
-  TablePropertiesCollector* CreateTablePropertiesCollector(
+  virtual TablePropertiesCollector* CreateTablePropertiesCollector(
       TablePropertiesCollectorFactory::Context /*context*/) override {
     return new TestPropertiesCollector;
   }
@@ -601,7 +603,7 @@ class TableFileCreationListener : public EventListener {
 
     Status NewWritableFile(const std::string& fname,
                            std::unique_ptr<WritableFile>* result,
-                           const EnvOptions& options) override {
+                           const EnvOptions& options) {
       if (fname.size() > 4 && fname.substr(fname.size() - 4) == ".sst") {
         if (!status_.ok()) {
           return status_;

db/log_test.cc

@@ -61,7 +61,7 @@ class LogTest : public ::testing::TestWithParam<int> {
       force_eof_position_(0),
       returned_partial_(false) { }
 
-    Status Read(size_t n, Slice* result, char* scratch) override {
+    virtual Status Read(size_t n, Slice* result, char* scratch) override {
       EXPECT_TRUE(!returned_partial_) << "must not Read() after eof/error";
 
       if (force_error_) {
@@ -100,7 +100,7 @@ class LogTest : public ::testing::TestWithParam<int> {
       return Status::OK();
     }
 
-    Status Skip(uint64_t n) override {
+    virtual Status Skip(uint64_t n) override {
       if (n > contents_.size()) {
         contents_.clear();
         return Status::NotFound("in-memory file skipepd past end");
@@ -118,7 +118,7 @@ class LogTest : public ::testing::TestWithParam<int> {
     std::string message_;
 
     ReportCollector() : dropped_bytes_(0) { }
-    void Corruption(size_t bytes, const Status& status) override {
+    virtual void Corruption(size_t bytes, const Status& status) override {
       dropped_bytes_ += bytes;
       message_.append(status.ToString());
     }
@@ -661,7 +661,7 @@ class RetriableLogTest : public ::testing::TestWithParam<int> {
     std::string message_;
 
     ReportCollector() : dropped_bytes_(0) {}
-    void Corruption(size_t bytes, const Status& status) override {
+    virtual void Corruption(size_t bytes, const Status& status) override {
       dropped_bytes_ += bytes;
       message_.append(status.ToString());
     }

db/manual_compaction_test.cc

@@ -51,13 +51,15 @@ class DestroyAllCompactionFilter : public CompactionFilter {
  public:
   DestroyAllCompactionFilter() {}
 
-  bool Filter(int /*level*/, const Slice& /*key*/, const Slice& existing_value,
-              std::string* /*new_value*/,
-              bool* /*value_changed*/) const override {
+  virtual bool Filter(int /*level*/, const Slice& /*key*/,
+                      const Slice& existing_value, std::string* /*new_value*/,
+                      bool* /*value_changed*/) const override {
     return existing_value.ToString() == "destroy";
   }
 
-  const char* Name() const override { return "DestroyAllCompactionFilter"; }
+  virtual const char* Name() const override {
+    return "DestroyAllCompactionFilter";
+  }
 };
 
 TEST_F(ManualCompactionTest, CompactTouchesAllKeys) {

db/memtable.cc

@@ -51,8 +51,6 @@ ImmutableMemTableOptions::ImmutableMemTableOptions(
               mutable_cf_options.memtable_prefix_bloom_size_ratio) *
           8u),
       memtable_huge_page_size(mutable_cf_options.memtable_huge_page_size),
-      memtable_whole_key_filtering(
-          mutable_cf_options.memtable_whole_key_filtering),
       inplace_update_support(ioptions.inplace_update_support),
       inplace_update_num_locks(mutable_cf_options.inplace_update_num_locks),
       inplace_callback(ioptions.inplace_callback),
@@ -111,10 +109,8 @@ MemTable::MemTable(const InternalKeyComparator& cmp,
   // something went wrong if we need to flush before inserting anything
   assert(!ShouldScheduleFlush());
 
-  // use bloom_filter_ for both whole key and prefix bloom filter
-  if ((prefix_extractor_ || moptions_.memtable_whole_key_filtering) &&
-      moptions_.memtable_prefix_bloom_bits > 0) {
-    bloom_filter_.reset(
+  if (prefix_extractor_ && moptions_.memtable_prefix_bloom_bits > 0) {
+    prefix_bloom_.reset(
         new DynamicBloom(&arena_, moptions_.memtable_prefix_bloom_bits,
                          ioptions.bloom_locality, 6 /* hard coded 6 probes */,
                          moptions_.memtable_huge_page_size, ioptions.info_log));
@@ -286,14 +282,14 @@ class MemTableIterator : public InternalIterator {
     if (use_range_del_table) {
       iter_ = mem.range_del_table_->GetIterator(arena);
     } else if (prefix_extractor_ != nullptr && !read_options.total_order_seek) {
-      bloom_ = mem.bloom_filter_.get();
+      bloom_ = mem.prefix_bloom_.get();
       iter_ = mem.table_->GetDynamicPrefixIterator(arena);
     } else {
       iter_ = mem.table_->GetIterator(arena);
     }
   }
 
-  ~MemTableIterator() override {
+  ~MemTableIterator() {
 #ifndef NDEBUG
     // Assert that the MemTableIterator is never deleted while
     // Pinning is Enabled.
@@ -307,18 +303,18 @@ class MemTableIterator : public InternalIterator {
   }
 
 #ifndef NDEBUG
-  void SetPinnedItersMgr(PinnedIteratorsManager* pinned_iters_mgr) override {
+  virtual void SetPinnedItersMgr(
+      PinnedIteratorsManager* pinned_iters_mgr) override {
     pinned_iters_mgr_ = pinned_iters_mgr;
   }
   PinnedIteratorsManager* pinned_iters_mgr_ = nullptr;
 #endif
 
-  bool Valid() const override { return valid_; }
-  void Seek(const Slice& k) override {
+  virtual bool Valid() const override { return valid_; }
+  virtual void Seek(const Slice& k) override {
     PERF_TIMER_GUARD(seek_on_memtable_time);
     PERF_COUNTER_ADD(seek_on_memtable_count, 1);
-    if (bloom_) {
-      // iterator should only use prefix bloom filter
+    if (bloom_ != nullptr) {
       if (!bloom_->MayContain(
               prefix_extractor_->Transform(ExtractUserKey(k)))) {
         PERF_COUNTER_ADD(bloom_memtable_miss_count, 1);
@@ -331,10 +327,10 @@ class MemTableIterator : public InternalIterator {
     iter_->Seek(k, nullptr);
     valid_ = iter_->Valid();
   }
-  void SeekForPrev(const Slice& k) override {
+  virtual void SeekForPrev(const Slice& k) override {
     PERF_TIMER_GUARD(seek_on_memtable_time);
     PERF_COUNTER_ADD(seek_on_memtable_count, 1);
-    if (bloom_) {
+    if (bloom_ != nullptr) {
       if (!bloom_->MayContain(
               prefix_extractor_->Transform(ExtractUserKey(k)))) {
         PERF_COUNTER_ADD(bloom_memtable_miss_count, 1);
@@ -353,44 +349,44 @@ class MemTableIterator : public InternalIterator {
       Prev();
     }
   }
-  void SeekToFirst() override {
+  virtual void SeekToFirst() override {
     iter_->SeekToFirst();
     valid_ = iter_->Valid();
   }
-  void SeekToLast() override {
+  virtual void SeekToLast() override {
     iter_->SeekToLast();
     valid_ = iter_->Valid();
   }
-  void Next() override {
+  virtual void Next() override {
     PERF_COUNTER_ADD(next_on_memtable_count, 1);
     assert(Valid());
     iter_->Next();
     valid_ = iter_->Valid();
   }
-  void Prev() override {
+  virtual void Prev() override {
     PERF_COUNTER_ADD(prev_on_memtable_count, 1);
     assert(Valid());
     iter_->Prev();
     valid_ = iter_->Valid();
   }
-  Slice key() const override {
+  virtual Slice key() const override {
     assert(Valid());
     return GetLengthPrefixedSlice(iter_->key());
   }
-  Slice value() const override {
+  virtual Slice value() const override {
     assert(Valid());
     Slice key_slice = GetLengthPrefixedSlice(iter_->key());
     return GetLengthPrefixedSlice(key_slice.data() + key_slice.size());
   }
 
-  Status status() const override { return Status::OK(); }
+  virtual Status status() const override { return Status::OK(); }
 
-  bool IsKeyPinned() const override {
+  virtual bool IsKeyPinned() const override {
     // memtable data is always pinned
     return true;
   }
 
-  bool IsValuePinned() const override {
+  virtual bool IsValuePinned() const override {
     // memtable value is always pinned, except if we allow inplace update.
     return value_pinned_;
   }
@@ -520,11 +516,9 @@ bool MemTable::Add(SequenceNumber s, ValueType type,
                          std::memory_order_relaxed);
     }
 
-    if (bloom_filter_ && prefix_extractor_) {
-      bloom_filter_->Add(prefix_extractor_->Transform(key));
-    }
-    if (bloom_filter_ && moptions_.memtable_whole_key_filtering) {
-      bloom_filter_->Add(key);
+    if (prefix_bloom_) {
+      assert(prefix_extractor_);
+      prefix_bloom_->Add(prefix_extractor_->Transform(key));
     }
 
     // The first sequence number inserted into the memtable
@@ -553,11 +547,9 @@ bool MemTable::Add(SequenceNumber s, ValueType type,
       post_process_info->num_deletes++;
     }
 
-    if (bloom_filter_ && prefix_extractor_) {
-      bloom_filter_->AddConcurrently(prefix_extractor_->Transform(key));
-    }
-    if (bloom_filter_ && moptions_.memtable_whole_key_filtering) {
-      bloom_filter_->AddConcurrently(key);
+    if (prefix_bloom_) {
+      assert(prefix_extractor_);
+      prefix_bloom_->AddConcurrently(prefix_extractor_->Transform(key));
     }
 
     // atomically update first_seqno_ and earliest_seqno_.
@@ -764,24 +756,16 @@ bool MemTable::Get(const LookupKey& key, std::string* value, Status* s,
   Slice user_key = key.user_key();
   bool found_final_value = false;
   bool merge_in_progress = s->IsMergeInProgress();
-  bool may_contain = true;
-  if (bloom_filter_) {
-    // when both memtable_whole_key_filtering and prefix_extractor_ are set,
-    // only do whole key filtering for Get() to save CPU
-    if (moptions_.memtable_whole_key_filtering) {
-      may_contain = bloom_filter_->MayContain(user_key);
-    } else {
-      assert(prefix_extractor_);
-      may_contain =
-          bloom_filter_->MayContain(prefix_extractor_->Transform(user_key));
-    }
-  }
-  if (bloom_filter_ && !may_contain) {
+  bool const may_contain =
+      nullptr == prefix_bloom_
+          ? false
+          : prefix_bloom_->MayContain(prefix_extractor_->Transform(user_key));
+  if (prefix_bloom_ && !may_contain) {
     // iter is null if prefix bloom says the key does not exist
     PERF_COUNTER_ADD(bloom_memtable_miss_count, 1);
     *seq = kMaxSequenceNumber;
   } else {
-    if (bloom_filter_) {
+    if (prefix_bloom_) {
       PERF_COUNTER_ADD(bloom_memtable_hit_count, 1);
     }
     Saver saver;

db/memtable.h

@@ -41,7 +41,6 @@ struct ImmutableMemTableOptions {
   size_t arena_block_size;
   uint32_t memtable_prefix_bloom_bits;
   size_t memtable_huge_page_size;
-  bool memtable_whole_key_filtering;
   bool inplace_update_support;
   size_t inplace_update_num_locks;
   UpdateStatus (*inplace_callback)(char* existing_value,
@@ -266,16 +265,12 @@ class MemTable {
     return num_deletes_.load(std::memory_order_relaxed);
   }
 
-  uint64_t get_data_size() const {
-    return data_size_.load(std::memory_order_relaxed);
-  }
-
   // Dynamically change the memtable's capacity. If set below the current usage,
   // the next key added will trigger a flush. Can only increase size when
   // memtable prefix bloom is disabled, since we can't easily allocate more
   // space.
   void UpdateWriteBufferSize(size_t new_write_buffer_size) {
-    if (bloom_filter_ == nullptr ||
+    if (prefix_bloom_ == nullptr ||
         new_write_buffer_size < write_buffer_size_) {
       write_buffer_size_.store(new_write_buffer_size,
                                std::memory_order_relaxed);
@@ -455,7 +450,7 @@ class MemTable {
   std::vector<port::RWMutex> locks_;
 
   const SliceTransform* const prefix_extractor_;
-  std::unique_ptr<DynamicBloom> bloom_filter_;
+  std::unique_ptr<DynamicBloom> prefix_bloom_;
 
   std::atomic<FlushStateEnum> flush_state_;
 

db/memtable_list.cc

@@ -533,7 +533,7 @@ Status InstallMemtableAtomicFlushResults(
     const autovector<ColumnFamilyData*>& cfds,
     const autovector<const MutableCFOptions*>& mutable_cf_options_list,
     const autovector<const autovector<MemTable*>*>& mems_list, VersionSet* vset,
-    InstrumentedMutex* mu, const autovector<FileMetaData*>& file_metas,
+    InstrumentedMutex* mu, const autovector<FileMetaData>& file_metas,
     autovector<MemTable*>* to_delete, Directory* db_directory,
     LogBuffer* log_buffer) {
   AutoThreadOperationStageUpdater stage_updater(
@@ -553,11 +553,10 @@ Status InstallMemtableAtomicFlushResults(
       assert((*mems_list[k])[0]->GetID() == imm->GetEarliestMemTableID());
     }
 #endif
-    assert(nullptr != file_metas[k]);
     for (size_t i = 0; i != mems_list[k]->size(); ++i) {
       assert(i == 0 || (*mems_list[k])[i]->GetEdits()->NumEntries() == 0);
       (*mems_list[k])[i]->SetFlushCompleted(true);
-      (*mems_list[k])[i]->SetFileNumber(file_metas[k]->fd.GetNumber());
+      (*mems_list[k])[i]->SetFileNumber(file_metas[k].fd.GetNumber());
     }
   }
 
@@ -573,15 +572,12 @@ Status InstallMemtableAtomicFlushResults(
     ++num_entries;
     edit_lists.emplace_back(edits);
   }
-  // Mark the version edits as an atomic group if the number of version edits
-  // exceeds 1.
-  if (cfds.size() > 1) {
-    for (auto& edits : edit_lists) {
-      assert(edits.size() == 1);
-      edits[0]->MarkAtomicGroup(--num_entries);
-    }
-    assert(0 == num_entries);
+  // Mark the version edits as an atomic group
+  for (auto& edits : edit_lists) {
+    assert(edits.size() == 1);
+    edits[0]->MarkAtomicGroup(--num_entries);
   }
+  assert(0 == num_entries);
 
   // this can release and reacquire the mutex.
   s = vset->LogAndApply(cfds, mutable_cf_options_list, edit_lists, mu,

db/memtable_list.h

@@ -123,7 +123,7 @@ class MemTableListVersion {
       const autovector<const MutableCFOptions*>& mutable_cf_options_list,
       const autovector<const autovector<MemTable*>*>& mems_list,
       VersionSet* vset, InstrumentedMutex* mu,
-      const autovector<FileMetaData*>& file_meta,
+      const autovector<FileMetaData>& file_meta,
       autovector<MemTable*>* to_delete, Directory* db_directory,
       LogBuffer* log_buffer);
 
@@ -301,7 +301,7 @@ class MemTableList {
       const autovector<const MutableCFOptions*>& mutable_cf_options_list,
       const autovector<const autovector<MemTable*>*>& mems_list,
       VersionSet* vset, InstrumentedMutex* mu,
-      const autovector<FileMetaData*>& file_meta,
+      const autovector<FileMetaData>& file_meta,
       autovector<MemTable*>* to_delete, Directory* db_directory,
       LogBuffer* log_buffer);
 
@@ -337,7 +337,7 @@ extern Status InstallMemtableAtomicFlushResults(
     const autovector<ColumnFamilyData*>& cfds,
     const autovector<const MutableCFOptions*>& mutable_cf_options_list,
     const autovector<const autovector<MemTable*>*>& mems_list, VersionSet* vset,
-    InstrumentedMutex* mu, const autovector<FileMetaData*>& file_meta,
+    InstrumentedMutex* mu, const autovector<FileMetaData>& file_meta,
     autovector<MemTable*>* to_delete, Directory* db_directory,
     LogBuffer* log_buffer);
 }  // namespace rocksdb

db/memtable_list_test.cc

@@ -62,7 +62,7 @@ class MemTableListTest : public testing::Test {
     }
   }
 
-  ~MemTableListTest() override {
+  ~MemTableListTest() {
     if (db) {
       std::vector<ColumnFamilyDescriptor> cf_descs(handles.size());
       for (int i = 0; i != static_cast<int>(handles.size()); ++i) {
@@ -161,23 +161,18 @@ class MemTableListTest : public testing::Test {
       cfds.emplace_back(column_family_set->GetColumnFamily(cf_ids[i]));
       EXPECT_NE(nullptr, cfds[i]);
     }
-    std::vector<FileMetaData> file_metas;
-    file_metas.reserve(cf_ids.size());
+    autovector<FileMetaData> file_metas;
     for (size_t i = 0; i != cf_ids.size(); ++i) {
       FileMetaData meta;
       uint64_t file_num = file_number.fetch_add(1);
       meta.fd = FileDescriptor(file_num, 0, 0);
       file_metas.emplace_back(meta);
     }
-    autovector<FileMetaData*> file_meta_ptrs;
-    for (auto& meta : file_metas) {
-      file_meta_ptrs.push_back(&meta);
-    }
     InstrumentedMutex mutex;
     InstrumentedMutexLock l(&mutex);
     return InstallMemtableAtomicFlushResults(
         &lists, cfds, mutable_cf_options_list, mems_list, &versions, &mutex,
-        file_meta_ptrs, to_delete, nullptr, &log_buffer);
+        file_metas, to_delete, nullptr, &log_buffer);
   }
 };
 

db/merge_helper.cc

@@ -138,11 +138,7 @@ Status MergeHelper::MergeUntil(InternalIterator* iter,
   // orig_ikey is backed by original_key if keys_.empty()
   // orig_ikey is backed by keys_.back() if !keys_.empty()
   ParsedInternalKey orig_ikey;
-  bool succ = ParseInternalKey(original_key, &orig_ikey);
-  assert(succ);
-  if (!succ) {
-    return Status::Corruption("Cannot parse key in MergeUntil");
-  }
+  ParseInternalKey(original_key, &orig_ikey);
 
   Status s;
   bool hit_the_next_user_key = false;

db/merge_helper_test.cc

@@ -20,7 +20,7 @@ class MergeHelperTest : public testing::Test {
  public:
   MergeHelperTest() { env_ = Env::Default(); }
 
-  ~MergeHelperTest() override = default;
+  ~MergeHelperTest() = default;
 
   Status Run(SequenceNumber stop_before, bool at_bottom,
              SequenceNumber latest_snapshot = 0) {

db/merge_test.cc

@@ -38,8 +38,11 @@ class CountMergeOperator : public AssociativeMergeOperator {
     mergeOperator_ = MergeOperators::CreateUInt64AddOperator();
   }
 
-  bool Merge(const Slice& key, const Slice* existing_value, const Slice& value,
-             std::string* new_value, Logger* logger) const override {
+  virtual bool Merge(const Slice& key,
+                     const Slice* existing_value,
+                     const Slice& value,
+                     std::string* new_value,
+                     Logger* logger) const override {
     assert(new_value->empty());
     ++num_merge_operator_calls;
     if (existing_value == nullptr) {
@@ -55,17 +58,19 @@ class CountMergeOperator : public AssociativeMergeOperator {
         logger);
   }
 
-  bool PartialMergeMulti(const Slice& key,
-                         const std::deque<Slice>& operand_list,
-                         std::string* new_value,
-                         Logger* logger) const override {
+  virtual bool PartialMergeMulti(const Slice& key,
+                                 const std::deque<Slice>& operand_list,
+                                 std::string* new_value,
+                                 Logger* logger) const override {
     assert(new_value->empty());
     ++num_partial_merge_calls;
     return mergeOperator_->PartialMergeMulti(key, operand_list, new_value,
                                              logger);
   }
 
-  const char* Name() const override { return "UInt64AddOperator"; }
+  virtual const char* Name() const override {
+    return "UInt64AddOperator";
+  }
 
  private:
   std::shared_ptr<MergeOperator> mergeOperator_;
@@ -225,7 +230,7 @@ class MergeBasedCounters : public Counters {
   }
 
   // mapped to a rocksdb Merge operation
-  bool add(const std::string& key, uint64_t value) override {
+  virtual bool add(const std::string& key, uint64_t value) override {
     char encoded[sizeof(uint64_t)];
     EncodeFixed64(encoded, value);
     Slice slice(encoded, sizeof(uint64_t));

db/perf_context_test.cc

@@ -688,89 +688,6 @@ TEST_F(PerfContextTest, MergeOperatorTime) {
   delete db;
 }
 
-TEST_F(PerfContextTest, CopyAndMove) {
-  // Assignment operator
-  {
-    get_perf_context()->Reset();
-    get_perf_context()->EnablePerLevelPerfContext();
-    PERF_COUNTER_BY_LEVEL_ADD(bloom_filter_useful, 1, 5);
-    ASSERT_EQ(
-        1,
-        (*(get_perf_context()->level_to_perf_context))[5].bloom_filter_useful);
-    PerfContext perf_context_assign;
-    perf_context_assign = *get_perf_context();
-    ASSERT_EQ(
-        1,
-        (*(perf_context_assign.level_to_perf_context))[5].bloom_filter_useful);
-    get_perf_context()->ClearPerLevelPerfContext();
-    get_perf_context()->Reset();
-    ASSERT_EQ(
-        1,
-        (*(perf_context_assign.level_to_perf_context))[5].bloom_filter_useful);
-    perf_context_assign.ClearPerLevelPerfContext();
-    perf_context_assign.Reset();
-  }
-  // Copy constructor
-  {
-    get_perf_context()->Reset();
-    get_perf_context()->EnablePerLevelPerfContext();
-    PERF_COUNTER_BY_LEVEL_ADD(bloom_filter_useful, 1, 5);
-    ASSERT_EQ(
-        1,
-        (*(get_perf_context()->level_to_perf_context))[5].bloom_filter_useful);
-    PerfContext perf_context_copy(*get_perf_context());
-    ASSERT_EQ(
-        1, (*(perf_context_copy.level_to_perf_context))[5].bloom_filter_useful);
-    get_perf_context()->ClearPerLevelPerfContext();
-    get_perf_context()->Reset();
-    ASSERT_EQ(
-        1, (*(perf_context_copy.level_to_perf_context))[5].bloom_filter_useful);
-    perf_context_copy.ClearPerLevelPerfContext();
-    perf_context_copy.Reset();
-  }
-  // Move constructor
-  {
-    get_perf_context()->Reset();
-    get_perf_context()->EnablePerLevelPerfContext();
-    PERF_COUNTER_BY_LEVEL_ADD(bloom_filter_useful, 1, 5);
-    ASSERT_EQ(
-        1,
-        (*(get_perf_context()->level_to_perf_context))[5].bloom_filter_useful);
-    PerfContext perf_context_move = std::move(*get_perf_context());
-    ASSERT_EQ(
-        1, (*(perf_context_move.level_to_perf_context))[5].bloom_filter_useful);
-    get_perf_context()->ClearPerLevelPerfContext();
-    get_perf_context()->Reset();
-    ASSERT_EQ(
-        1, (*(perf_context_move.level_to_perf_context))[5].bloom_filter_useful);
-    perf_context_move.ClearPerLevelPerfContext();
-    perf_context_move.Reset();
-  }
-}
-
-TEST_F(PerfContextTest, PerfContextDisableEnable) {
-  get_perf_context()->Reset();
-  get_perf_context()->EnablePerLevelPerfContext();
-  PERF_COUNTER_BY_LEVEL_ADD(bloom_filter_full_positive, 1, 0);
-  get_perf_context()->DisablePerLevelPerfContext();
-  PERF_COUNTER_BY_LEVEL_ADD(bloom_filter_useful, 1, 5);
-  get_perf_context()->EnablePerLevelPerfContext();
-  PERF_COUNTER_BY_LEVEL_ADD(block_cache_hit_count, 1, 0);
-  get_perf_context()->DisablePerLevelPerfContext();
-  PerfContext perf_context_copy(*get_perf_context());
-  ASSERT_EQ(1, (*(perf_context_copy.level_to_perf_context))[0]
-                   .bloom_filter_full_positive);
-  // this was set when per level perf context is disabled, should not be copied
-  ASSERT_NE(
-      1, (*(perf_context_copy.level_to_perf_context))[5].bloom_filter_useful);
-  ASSERT_EQ(
-      1, (*(perf_context_copy.level_to_perf_context))[0].block_cache_hit_count);
-  perf_context_copy.ClearPerLevelPerfContext();
-  perf_context_copy.Reset();
-  get_perf_context()->ClearPerLevelPerfContext();
-  get_perf_context()->Reset();
-}
-
 TEST_F(PerfContextTest, PerfContextByLevelGetSet) {
   get_perf_context()->Reset();
   get_perf_context()->EnablePerLevelPerfContext();

db/plain_table_db_test.cc

@@ -109,7 +109,7 @@ class PlainTableDBTest : public testing::Test,
  public:
   PlainTableDBTest() : env_(Env::Default()) {}
 
-  ~PlainTableDBTest() override {
+  ~PlainTableDBTest() {
     delete db_;
     EXPECT_OK(DestroyDB(dbname_, Options()));
   }
@@ -134,7 +134,6 @@ class PlainTableDBTest : public testing::Test,
     plain_table_options.huge_page_tlb_size = 0;
     plain_table_options.encoding_type = kPrefix;
     plain_table_options.full_scan_mode = false;
-    plain_table_options.store_index_in_file = false;
 
     options.table_factory.reset(NewPlainTableFactory(plain_table_options));
     options.memtable_factory.reset(NewHashLinkListRepFactory(4, 0, 3, true));
@@ -272,8 +271,7 @@ class TestPlainTableReader : public PlainTableReader {
                        std::unique_ptr<RandomAccessFileReader>&& file,
                        const ImmutableCFOptions& ioptions,
                        const SliceTransform* prefix_extractor,
-                       bool* expect_bloom_not_match, bool store_index_in_file,
-                       uint32_t column_family_id,
+                       bool* expect_bloom_not_match, uint32_t column_family_id,
                        const std::string& column_family_name)
       : PlainTableReader(ioptions, std::move(file), env_options, icomparator,
                          encoding_type, file_size, table_properties,
@@ -290,23 +288,12 @@ class TestPlainTableReader : public PlainTableReader {
     TableProperties* props = const_cast<TableProperties*>(table_properties);
     EXPECT_EQ(column_family_id, static_cast<uint32_t>(props->column_family_id));
     EXPECT_EQ(column_family_name, props->column_family_name);
-    if (store_index_in_file) {
-      auto bloom_version_ptr = props->user_collected_properties.find(
-          PlainTablePropertyNames::kBloomVersion);
-      EXPECT_TRUE(bloom_version_ptr != props->user_collected_properties.end());
-      EXPECT_EQ(bloom_version_ptr->second, std::string("1"));
-      if (ioptions.bloom_locality > 0) {
-        auto num_blocks_ptr = props->user_collected_properties.find(
-            PlainTablePropertyNames::kNumBloomBlocks);
-        EXPECT_TRUE(num_blocks_ptr != props->user_collected_properties.end());
-      }
-    }
   }
 
-  ~TestPlainTableReader() override {}
+  virtual ~TestPlainTableReader() {}
 
  private:
-  bool MatchBloom(uint32_t hash) const override {
+  virtual bool MatchBloom(uint32_t hash) const override {
     bool ret = PlainTableReader::MatchBloom(hash);
     if (*expect_bloom_not_match_) {
       EXPECT_TRUE(!ret);
@@ -329,7 +316,6 @@ class TestPlainTableFactory : public PlainTableFactory {
         bloom_bits_per_key_(options.bloom_bits_per_key),
         hash_table_ratio_(options.hash_table_ratio),
         index_sparseness_(options.index_sparseness),
-        store_index_in_file_(options.store_index_in_file),
         expect_bloom_not_match_(expect_bloom_not_match),
         column_family_id_(column_family_id),
         column_family_name_(std::move(column_family_name)) {}
@@ -346,22 +332,6 @@ class TestPlainTableFactory : public PlainTableFactory {
                             true /* compression_type_missing */);
     EXPECT_TRUE(s.ok());
 
-    if (store_index_in_file_) {
-      BlockHandle bloom_block_handle;
-      s = FindMetaBlock(file.get(), file_size, kPlainTableMagicNumber,
-                        table_reader_options.ioptions,
-                        BloomBlockBuilder::kBloomBlock, &bloom_block_handle,
-                        /* compression_type_missing */ true);
-      EXPECT_TRUE(s.ok());
-
-      BlockHandle index_block_handle;
-      s = FindMetaBlock(file.get(), file_size, kPlainTableMagicNumber,
-                        table_reader_options.ioptions,
-                        PlainTableIndexBuilder::kPlainTableIndexBlock,
-                        &index_block_handle, /* compression_type_missing */ true);
-      EXPECT_TRUE(s.ok());
-    }
-
     auto& user_props = props->user_collected_properties;
     auto encoding_type_prop =
         user_props.find(PlainTablePropertyNames::kEncodingType);
@@ -375,7 +345,7 @@ class TestPlainTableFactory : public PlainTableFactory {
         bloom_bits_per_key_, hash_table_ratio_, index_sparseness_, props,
         std::move(file), table_reader_options.ioptions,
         table_reader_options.prefix_extractor, expect_bloom_not_match_,
-        store_index_in_file_, column_family_id_, column_family_name_));
+        column_family_id_, column_family_name_));
 
     *table = std::move(new_reader);
     return s;
@@ -385,7 +355,6 @@ class TestPlainTableFactory : public PlainTableFactory {
   int bloom_bits_per_key_;
   double hash_table_ratio_;
   size_t index_sparseness_;
-  bool store_index_in_file_;
   bool* expect_bloom_not_match_;
   const uint32_t column_family_id_;
   const std::string column_family_name_;
@@ -395,10 +364,8 @@ TEST_P(PlainTableDBTest, Flush) {
   for (size_t huge_page_tlb_size = 0; huge_page_tlb_size <= 2 * 1024 * 1024;
        huge_page_tlb_size += 2 * 1024 * 1024) {
     for (EncodingType encoding_type : {kPlain, kPrefix}) {
-    for (int bloom_bits = 0; bloom_bits <= 117; bloom_bits += 117) {
-      for (int total_order = 0; total_order <= 1; total_order++) {
-        for (int store_index_in_file = 0; store_index_in_file <= 1;
-             ++store_index_in_file) {
+      for (int bloom_bits = 0; bloom_bits <= 117; bloom_bits += 117) {
+        for (int total_order = 0; total_order <= 1; total_order++) {
           Options options = CurrentOptions();
           options.create_if_missing = true;
           // Set only one bucket to force bucket conflict.
@@ -414,7 +381,6 @@ TEST_P(PlainTableDBTest, Flush) {
             plain_table_options.huge_page_tlb_size = huge_page_tlb_size;
             plain_table_options.encoding_type = encoding_type;
             plain_table_options.full_scan_mode = false;
-            plain_table_options.store_index_in_file = store_index_in_file;
 
             options.table_factory.reset(
                 NewPlainTableFactory(plain_table_options));
@@ -427,7 +393,6 @@ TEST_P(PlainTableDBTest, Flush) {
             plain_table_options.huge_page_tlb_size = huge_page_tlb_size;
             plain_table_options.encoding_type = encoding_type;
             plain_table_options.full_scan_mode = false;
-            plain_table_options.store_index_in_file = store_index_in_file;
 
             options.table_factory.reset(
                 NewPlainTableFactory(plain_table_options));
@@ -453,22 +418,15 @@ TEST_P(PlainTableDBTest, Flush) {
           auto row = ptc.begin();
           auto tp = row->second;
 
-          if (!store_index_in_file) {
-            ASSERT_EQ(total_order ? "4" : "12",
-                      (tp->user_collected_properties)
-                          .at("plain_table_hash_table_size"));
-            ASSERT_EQ("0", (tp->user_collected_properties)
-                               .at("plain_table_sub_index_size"));
-          } else {
-            ASSERT_EQ("0", (tp->user_collected_properties)
-                               .at("plain_table_hash_table_size"));
-            ASSERT_EQ("0", (tp->user_collected_properties)
-                               .at("plain_table_sub_index_size"));
-          }
+          ASSERT_EQ(total_order ? "4" : "12",
+                    (tp->user_collected_properties)
+                        .at("plain_table_hash_table_size"));
+          ASSERT_EQ(
+              "0",
+              (tp->user_collected_properties).at("plain_table_sub_index_size"));
           ASSERT_EQ("v3", Get("1000000000000foo"));
           ASSERT_EQ("v2", Get("0000000000000bar"));
         }
-        }
       }
     }
   }
@@ -478,19 +436,11 @@ TEST_P(PlainTableDBTest, Flush2) {
   for (size_t huge_page_tlb_size = 0; huge_page_tlb_size <= 2 * 1024 * 1024;
        huge_page_tlb_size += 2 * 1024 * 1024) {
     for (EncodingType encoding_type : {kPlain, kPrefix}) {
-    for (int bloom_bits = 0; bloom_bits <= 117; bloom_bits += 117) {
-      for (int total_order = 0; total_order <= 1; total_order++) {
-        for (int store_index_in_file = 0; store_index_in_file <= 1;
-             ++store_index_in_file) {
+      for (int bloom_bits = 0; bloom_bits <= 117; bloom_bits += 117) {
+        for (int total_order = 0; total_order <= 1; total_order++) {
           if (encoding_type == kPrefix && total_order) {
             continue;
           }
-          if (!bloom_bits && store_index_in_file) {
-            continue;
-          }
-          if (total_order && store_index_in_file) {
-          continue;
-        }
         bool expect_bloom_not_match = false;
         Options options = CurrentOptions();
         options.create_if_missing = true;
@@ -509,7 +459,6 @@ TEST_P(PlainTableDBTest, Flush2) {
         plain_table_options.bloom_bits_per_key = bloom_bits;
         plain_table_options.huge_page_tlb_size = huge_page_tlb_size;
         plain_table_options.encoding_type = encoding_type;
-        plain_table_options.store_index_in_file = store_index_in_file;
         options.table_factory.reset(new TestPlainTableFactory(
             &expect_bloom_not_match, plain_table_options,
             0 /* column_family_id */, kDefaultColumnFamilyName));
@@ -548,9 +497,8 @@ TEST_P(PlainTableDBTest, Flush2) {
           }
           expect_bloom_not_match = false;
         }
+        }
       }
-      }
-    }
     }
   }
 }

db/prefix_test.cc

@@ -83,7 +83,7 @@ class TestKeyComparator : public Comparator {
 
   // Compare needs to be aware of the possibility of a and/or b is
   // prefix only
-  int Compare(const Slice& a, const Slice& b) const override {
+  virtual int Compare(const Slice& a, const Slice& b) const override {
     const TestKey kkey_a = SliceToTestKey(a);
     const TestKey kkey_b = SliceToTestKey(b);
     const TestKey *key_a = &kkey_a;
@@ -122,12 +122,14 @@ class TestKeyComparator : public Comparator {
     return Compare(TestKeyToSlice(sa, a), TestKeyToSlice(sb, b)) < 0;
   }
 
-  const char* Name() const override { return "TestKeyComparator"; }
+  virtual const char* Name() const override {
+    return "TestKeyComparator";
+  }
 
-  void FindShortestSeparator(std::string* /*start*/,
-                             const Slice& /*limit*/) const override {}
+  virtual void FindShortestSeparator(std::string* /*start*/,
+                                     const Slice& /*limit*/) const override {}
 
-  void FindShortSuccessor(std::string* /*key*/) const override {}
+  virtual void FindShortSuccessor(std::string* /*key*/) const override {}
 };
 
 namespace {
@@ -193,23 +195,27 @@ class SamePrefixTransform : public SliceTransform {
   explicit SamePrefixTransform(const Slice& prefix)
       : prefix_(prefix), name_("rocksdb.SamePrefix." + prefix.ToString()) {}
 
-  const char* Name() const override { return name_.c_str(); }
+  virtual const char* Name() const override { return name_.c_str(); }
 
-  Slice Transform(const Slice& src) const override {
+  virtual Slice Transform(const Slice& src) const override {
     assert(InDomain(src));
     return prefix_;
   }
 
-  bool InDomain(const Slice& src) const override {
+  virtual bool InDomain(const Slice& src) const override {
     if (src.size() >= prefix_.size()) {
       return Slice(src.data(), prefix_.size()) == prefix_;
     }
     return false;
   }
 
-  bool InRange(const Slice& dst) const override { return dst == prefix_; }
+  virtual bool InRange(const Slice& dst) const override {
+    return dst == prefix_;
+  }
 
-  bool FullLengthEnabled(size_t* /*len*/) const override { return false; }
+  virtual bool FullLengthEnabled(size_t* /*len*/) const override {
+    return false;
+  }
 };
 
 }  // namespace
@@ -277,8 +283,9 @@ class PrefixTest : public testing::Test {
   PrefixTest() : option_config_(kBegin) {
     options.comparator = new TestKeyComparator();
   }
-  ~PrefixTest() override { delete options.comparator; }
-
+  ~PrefixTest() {
+    delete options.comparator;
+  }
  protected:
   enum OptionConfig {
     kBegin,

db/repair.cc

@@ -336,7 +336,7 @@ class Repairer {
       Env* env;
       std::shared_ptr<Logger> info_log;
       uint64_t lognum;
-      void Corruption(size_t bytes, const Status& s) override {
+      virtual void Corruption(size_t bytes, const Status& s) override {
         // We print error messages for corruption, but continue repairing.
         ROCKS_LOG_ERROR(info_log, "Log #%" PRIu64 ": dropping %d bytes; %s",
                         lognum, static_cast<int>(bytes), s.ToString().c_str());

db/table_properties_collector_test.cc

@@ -28,7 +28,7 @@ namespace rocksdb {
 class TablePropertiesTest : public testing::Test,
                             public testing::WithParamInterface<bool> {
  public:
-  void SetUp() override { backward_mode_ = GetParam(); }
+  virtual void SetUp() override { backward_mode_ = GetParam(); }
 
   bool backward_mode_;
 };
@@ -106,7 +106,7 @@ class RegularKeysStartWithA: public TablePropertiesCollector {
     return Status::OK();
   }
 
-  UserCollectedProperties GetReadableProperties() const override {
+  virtual UserCollectedProperties GetReadableProperties() const override {
     return UserCollectedProperties{};
   }
 
@@ -143,7 +143,7 @@ class RegularKeysStartWithABackwardCompatible
     return Status::OK();
   }
 
-  UserCollectedProperties GetReadableProperties() const override {
+  virtual UserCollectedProperties GetReadableProperties() const override {
     return UserCollectedProperties{};
   }
 
@@ -172,7 +172,7 @@ class RegularKeysStartWithAInternal : public IntTblPropCollector {
     return Status::OK();
   }
 
-  UserCollectedProperties GetReadableProperties() const override {
+  virtual UserCollectedProperties GetReadableProperties() const override {
     return UserCollectedProperties{};
   }
 
@@ -185,7 +185,7 @@ class RegularKeysStartWithAFactory : public IntTblPropCollectorFactory,
  public:
   explicit RegularKeysStartWithAFactory(bool backward_mode)
       : backward_mode_(backward_mode) {}
-  TablePropertiesCollector* CreateTablePropertiesCollector(
+  virtual TablePropertiesCollector* CreateTablePropertiesCollector(
       TablePropertiesCollectorFactory::Context context) override {
     EXPECT_EQ(kTestColumnFamilyId, context.column_family_id);
     if (!backward_mode_) {
@@ -194,7 +194,7 @@ class RegularKeysStartWithAFactory : public IntTblPropCollectorFactory,
       return new RegularKeysStartWithABackwardCompatible();
     }
   }
-  IntTblPropCollector* CreateIntTblPropCollector(
+  virtual IntTblPropCollector* CreateIntTblPropCollector(
       uint32_t /*column_family_id*/) override {
     return new RegularKeysStartWithAInternal();
   }
@@ -205,7 +205,7 @@ class RegularKeysStartWithAFactory : public IntTblPropCollectorFactory,
 
 class FlushBlockEveryThreePolicy : public FlushBlockPolicy {
  public:
-  bool Update(const Slice& /*key*/, const Slice& /*value*/) override {
+  virtual bool Update(const Slice& /*key*/, const Slice& /*value*/) override {
     return (++count_ % 3U == 0);
   }
 

db/version_builder_test.cc

@@ -37,7 +37,7 @@ class VersionBuilderTest : public testing::Test {
     size_being_compacted_.resize(options_.num_levels);
   }
 
-  ~VersionBuilderTest() override {
+  ~VersionBuilderTest() {
     for (int i = 0; i < vstorage_.num_levels(); i++) {
       for (auto* f : vstorage_.LevelFiles(i)) {
         if (--f->refs == 0) {

db/version_edit.cc

@@ -44,9 +44,6 @@ enum Tag : uint32_t {
   kInAtomicGroup = 300,
 };
 
-// Mask for an identified tag from the future which can be safely ignored.
-uint32_t kTagSafeIgnoreMask = 1 << 13;
-
 enum CustomTag : uint32_t {
   kTerminate = 1,  // The end of customized fields
   kNeedCompaction = 2,
@@ -504,21 +501,7 @@ Status VersionEdit::DecodeFrom(const Slice& src) {
         break;
 
       default:
-        if (tag & kTagSafeIgnoreMask) {
-          // Tag from future which can be safely ignored.
-          // The next field must be the length of the entry.
-          uint32_t field_len;
-          if (!GetVarint32(&input, &field_len) ||
-              static_cast<size_t>(field_len) > input.size()) {
-            if (!msg) {
-              msg = "safely ignoreable tag length error";
-            }
-          } else {
-            input.remove_prefix(static_cast<size_t>(field_len));
-          }
-        } else {
-          msg = "unknown tag";
-        }
+        msg = "unknown tag";
         break;
     }
   }

db/version_edit.h

@@ -229,10 +229,6 @@ class VersionEdit {
 
   uint64_t log_number() { return log_number_; }
 
-  bool has_next_file_number() const { return has_next_file_number_; }
-
-  uint64_t next_file_number() const { return next_file_number_; }
-
   // Add the specified file at the specified number.
   // REQUIRES: This version has not been saved (see VersionSet::SaveTo)
   // REQUIRES: "smallest" and "largest" are smallest and largest keys in file

db/version_edit_test.cc

@@ -8,7 +8,6 @@
 // found in the LICENSE file. See the AUTHORS file for names of contributors.
 
 #include "db/version_edit.h"
-#include "util/coding.h"
 #include "util/sync_point.h"
 #include "util/testharness.h"
 
@@ -198,47 +197,6 @@ TEST_F(VersionEditTest, AtomicGroupTest) {
   TestEncodeDecode(edit);
 }
 
-TEST_F(VersionEditTest, IgnorableField) {
-  VersionEdit ve;
-  std::string encoded;
-
-  // Size of ignorable field is too large
-  PutVarint32Varint64(&encoded, 2 /* kLogNumber */, 66);
-  // This is a customized ignorable tag
-  PutVarint32Varint64(&encoded,
-                      0x2710 /* A field with kTagSafeIgnoreMask set */,
-                      5 /* fieldlength 5 */);
-  encoded += "abc";  // Only fills 3 bytes,
-  ASSERT_NOK(ve.DecodeFrom(encoded));
-
-  encoded.clear();
-  // Error when seeing unidentified tag that is not ignorable
-  PutVarint32Varint64(&encoded, 2 /* kLogNumber */, 66);
-  // This is a customized ignorable tag
-  PutVarint32Varint64(&encoded, 666 /* A field with kTagSafeIgnoreMask unset */,
-                      3 /* fieldlength 3 */);
-  encoded += "abc";  //  Fill 3 bytes
-  PutVarint32Varint64(&encoded, 3 /* next file number */, 88);
-  ASSERT_NOK(ve.DecodeFrom(encoded));
-
-  // Safely ignore an identified but safely ignorable entry
-  encoded.clear();
-  PutVarint32Varint64(&encoded, 2 /* kLogNumber */, 66);
-  // This is a customized ignorable tag
-  PutVarint32Varint64(&encoded,
-                      0x2710 /* A field with kTagSafeIgnoreMask set */,
-                      3 /* fieldlength 3 */);
-  encoded += "abc";  //  Fill 3 bytes
-  PutVarint32Varint64(&encoded, 3 /* kNextFileNumber */, 88);
-
-  ASSERT_OK(ve.DecodeFrom(encoded));
-
-  ASSERT_TRUE(ve.has_log_number());
-  ASSERT_TRUE(ve.has_next_file_number());
-  ASSERT_EQ(66, ve.log_number());
-  ASSERT_EQ(88, ve.next_file_number());
-}
-
 }  // namespace rocksdb
 
 int main(int argc, char** argv) {

db/version_set.cc

@@ -492,38 +492,39 @@ class LevelIterator final : public InternalIterator {
     assert(flevel_ != nullptr && flevel_->num_files > 0);
   }
 
-  ~LevelIterator() override { delete file_iter_.Set(nullptr); }
+  virtual ~LevelIterator() { delete file_iter_.Set(nullptr); }
 
-  void Seek(const Slice& target) override;
-  void SeekForPrev(const Slice& target) override;
-  void SeekToFirst() override;
-  void SeekToLast() override;
-  void Next() override;
-  void Prev() override;
+  virtual void Seek(const Slice& target) override;
+  virtual void SeekForPrev(const Slice& target) override;
+  virtual void SeekToFirst() override;
+  virtual void SeekToLast() override;
+  virtual void Next() override;
+  virtual void Prev() override;
 
-  bool Valid() const override { return file_iter_.Valid(); }
-  Slice key() const override {
+  virtual bool Valid() const override { return file_iter_.Valid(); }
+  virtual Slice key() const override {
     assert(Valid());
     return file_iter_.key();
   }
-  Slice value() const override {
+  virtual Slice value() const override {
     assert(Valid());
     return file_iter_.value();
   }
-  Status status() const override {
+  virtual Status status() const override {
     return file_iter_.iter() ? file_iter_.status() : Status::OK();
   }
-  void SetPinnedItersMgr(PinnedIteratorsManager* pinned_iters_mgr) override {
+  virtual void SetPinnedItersMgr(
+      PinnedIteratorsManager* pinned_iters_mgr) override {
     pinned_iters_mgr_ = pinned_iters_mgr;
     if (file_iter_.iter()) {
       file_iter_.SetPinnedItersMgr(pinned_iters_mgr);
     }
   }
-  bool IsKeyPinned() const override {
+  virtual bool IsKeyPinned() const override {
     return pinned_iters_mgr_ && pinned_iters_mgr_->PinningEnabled() &&
            file_iter_.iter() && file_iter_.IsKeyPinned();
   }
-  bool IsValuePinned() const override {
+  virtual bool IsValuePinned() const override {
     return pinned_iters_mgr_ && pinned_iters_mgr_->PinningEnabled() &&
            file_iter_.iter() && file_iter_.IsValuePinned();
   }
@@ -1596,7 +1597,8 @@ uint32_t GetExpiredTtlFilesCount(const ImmutableCFOptions& ioptions,
         auto creation_time =
             f->fd.table_reader->GetTableProperties()->creation_time;
         if (creation_time > 0 &&
-            creation_time < (current_time - mutable_cf_options.ttl)) {
+            creation_time < (current_time -
+                             mutable_cf_options.compaction_options_fifo.ttl)) {
           ttl_expired_files_count++;
         }
       }
@@ -1651,7 +1653,7 @@ void VersionStorageInfo::ComputeCompactionScore(
                   mutable_cf_options.level0_file_num_compaction_trigger,
               score);
         }
-        if (mutable_cf_options.ttl > 0) {
+        if (mutable_cf_options.compaction_options_fifo.ttl > 0) {
           score = std::max(
               static_cast<double>(GetExpiredTtlFilesCount(
                   immutable_cf_options, mutable_cf_options, files_[level])),
@@ -3058,9 +3060,6 @@ Status VersionSet::ProcessManifestWrites(
 
     // Write new records to MANIFEST log
     if (s.ok()) {
-#ifndef NDEBUG
-      size_t idx = 0;
-#endif
       for (auto& e : batch_edits) {
         std::string record;
         if (!e->EncodeTo(&record)) {
@@ -3070,15 +3069,6 @@ Status VersionSet::ProcessManifestWrites(
         }
         TEST_KILL_RANDOM("VersionSet::LogAndApply:BeforeAddRecord",
                          rocksdb_kill_odds * REDUCE_ODDS2);
-#ifndef NDEBUG
-        if (batch_edits.size() > 1 && batch_edits.size() - 1 == idx) {
-          TEST_SYNC_POINT(
-              "VersionSet::ProcessManifestWrites:BeforeWriteLastVersionEdit:0");
-          TEST_SYNC_POINT(
-              "VersionSet::ProcessManifestWrites:BeforeWriteLastVersionEdit:1");
-        }
-        ++idx;
-#endif /* !NDEBUG */
         s = descriptor_log_->AddRecord(record);
         if (!s.ok()) {
           break;

db/version_set_test.cc

@@ -25,7 +25,7 @@ class GenerateLevelFilesBriefTest : public testing::Test {
 
   GenerateLevelFilesBriefTest() { }
 
-  ~GenerateLevelFilesBriefTest() override {
+  ~GenerateLevelFilesBriefTest() {
     for (size_t i = 0; i < files_.size(); i++) {
       delete files_[i];
     }
@@ -79,7 +79,9 @@ class CountingLogger : public Logger {
  public:
   CountingLogger() : log_count(0) {}
   using Logger::Logv;
-  void Logv(const char* /*format*/, va_list /*ap*/) override { log_count++; }
+  virtual void Logv(const char* /*format*/, va_list /*ap*/) override {
+    log_count++;
+  }
   int log_count;
 };
 
@@ -115,7 +117,7 @@ class VersionStorageInfoTest : public testing::Test {
         mutable_cf_options_(options_),
         vstorage_(&icmp_, ucmp_, 6, kCompactionStyleLevel, nullptr, false) {}
 
-  ~VersionStorageInfoTest() override {
+  ~VersionStorageInfoTest() {
     for (int i = 0; i < vstorage_.num_levels(); i++) {
       for (auto* f : vstorage_.LevelFiles(i)) {
         if (--f->refs == 0) {
@@ -418,7 +420,8 @@ class FindLevelFileTest : public testing::Test {
 
   FindLevelFileTest() : disjoint_sorted_files_(true) { }
 
-  ~FindLevelFileTest() override {}
+  ~FindLevelFileTest() {
+  }
 
   void LevelFileInit(size_t num = 0) {
     char* mem = arena_.AllocateAligned(num * sizeof(FdWithKeyRange));

db/wal_manager.cc

@@ -429,7 +429,7 @@ Status WalManager::ReadFirstLine(const std::string& fname,
 
     Status* status;
     bool ignore_error;  // true if db_options_.paranoid_checks==false
-    void Corruption(size_t bytes, const Status& s) override {
+    virtual void Corruption(size_t bytes, const Status& s) override {
       ROCKS_LOG_WARN(info_log, "[WalManager] %s%s: dropping %d bytes; %s",
                      (this->ignore_error ? "(ignoring error) " : ""), fname,
                      static_cast<int>(bytes), s.ToString().c_str());

db/write_batch.cc

@@ -1094,8 +1094,10 @@ class MemTableInserter : public WriteBatch::Handler {
   }
 
  protected:
-  bool WriteBeforePrepare() const override { return write_before_prepare_; }
-  bool WriteAfterCommit() const override { return write_after_commit_; }
+  virtual bool WriteBeforePrepare() const override {
+    return write_before_prepare_;
+  }
+  virtual bool WriteAfterCommit() const override { return write_after_commit_; }
 
  public:
   // cf_mems should not be shared with concurrent inserters
@@ -1132,7 +1134,7 @@ class MemTableInserter : public WriteBatch::Handler {
     assert(cf_mems_);
   }
 
-  ~MemTableInserter() override {
+  ~MemTableInserter() {
     if (dup_dectector_on_) {
       reinterpret_cast<DuplicateDetector*>
         (&duplicate_detector_)->~DuplicateDetector();
@@ -1322,8 +1324,8 @@ class MemTableInserter : public WriteBatch::Handler {
     return ret_status;
   }
 
-  Status PutCF(uint32_t column_family_id, const Slice& key,
-               const Slice& value) override {
+  virtual Status PutCF(uint32_t column_family_id, const Slice& key,
+                       const Slice& value) override {
     return PutCFImpl(column_family_id, key, value, kTypeValue);
   }
 
@@ -1345,7 +1347,8 @@ class MemTableInserter : public WriteBatch::Handler {
     return ret_status;
   }
 
-  Status DeleteCF(uint32_t column_family_id, const Slice& key) override {
+  virtual Status DeleteCF(uint32_t column_family_id,
+                          const Slice& key) override {
     // optimize for non-recovery mode
     if (UNLIKELY(write_after_commit_ && rebuilding_trx_ != nullptr)) {
       WriteBatchInternal::Delete(rebuilding_trx_, column_family_id, key);
@@ -1378,7 +1381,8 @@ class MemTableInserter : public WriteBatch::Handler {
     return ret_status;
   }
 
-  Status SingleDeleteCF(uint32_t column_family_id, const Slice& key) override {
+  virtual Status SingleDeleteCF(uint32_t column_family_id,
+                                const Slice& key) override {
     // optimize for non-recovery mode
     if (UNLIKELY(write_after_commit_ && rebuilding_trx_ != nullptr)) {
       WriteBatchInternal::SingleDelete(rebuilding_trx_, column_family_id, key);
@@ -1413,8 +1417,9 @@ class MemTableInserter : public WriteBatch::Handler {
     return ret_status;
   }
 
-  Status DeleteRangeCF(uint32_t column_family_id, const Slice& begin_key,
-                       const Slice& end_key) override {
+  virtual Status DeleteRangeCF(uint32_t column_family_id,
+                               const Slice& begin_key,
+                               const Slice& end_key) override {
     // optimize for non-recovery mode
     if (UNLIKELY(write_after_commit_ && rebuilding_trx_ != nullptr)) {
       WriteBatchInternal::DeleteRange(rebuilding_trx_, column_family_id,
@@ -1466,8 +1471,8 @@ class MemTableInserter : public WriteBatch::Handler {
     return ret_status;
   }
 
-  Status MergeCF(uint32_t column_family_id, const Slice& key,
-                 const Slice& value) override {
+  virtual Status MergeCF(uint32_t column_family_id, const Slice& key,
+                         const Slice& value) override {
     assert(!concurrent_memtable_writes_);
     // optimize for non-recovery mode
     if (UNLIKELY(write_after_commit_ && rebuilding_trx_ != nullptr)) {
@@ -1580,8 +1585,8 @@ class MemTableInserter : public WriteBatch::Handler {
     return ret_status;
   }
 
-  Status PutBlobIndexCF(uint32_t column_family_id, const Slice& key,
-                        const Slice& value) override {
+  virtual Status PutBlobIndexCF(uint32_t column_family_id, const Slice& key,
+                                const Slice& value) override {
     // Same as PutCF except for value type.
     return PutCFImpl(column_family_id, key, value, kTypeBlobIndex);
   }

db/write_batch_test.cc

@@ -236,8 +236,8 @@ TEST_F(WriteBatchTest, SingleDeletion) {
 namespace {
   struct TestHandler : public WriteBatch::Handler {
     std::string seen;
-    Status PutCF(uint32_t column_family_id, const Slice& key,
-                 const Slice& value) override {
+    virtual Status PutCF(uint32_t column_family_id, const Slice& key,
+                         const Slice& value) override {
       if (column_family_id == 0) {
         seen += "Put(" + key.ToString() + ", " + value.ToString() + ")";
       } else {
@@ -246,7 +246,8 @@ namespace {
       }
       return Status::OK();
     }
-    Status DeleteCF(uint32_t column_family_id, const Slice& key) override {
+    virtual Status DeleteCF(uint32_t column_family_id,
+                            const Slice& key) override {
       if (column_family_id == 0) {
         seen += "Delete(" + key.ToString() + ")";
       } else {
@@ -255,8 +256,8 @@ namespace {
       }
       return Status::OK();
     }
-    Status SingleDeleteCF(uint32_t column_family_id,
-                          const Slice& key) override {
+    virtual Status SingleDeleteCF(uint32_t column_family_id,
+                                  const Slice& key) override {
       if (column_family_id == 0) {
         seen += "SingleDelete(" + key.ToString() + ")";
       } else {
@@ -265,8 +266,9 @@ namespace {
       }
       return Status::OK();
     }
-    Status DeleteRangeCF(uint32_t column_family_id, const Slice& begin_key,
-                         const Slice& end_key) override {
+    virtual Status DeleteRangeCF(uint32_t column_family_id,
+                                 const Slice& begin_key,
+                                 const Slice& end_key) override {
       if (column_family_id == 0) {
         seen += "DeleteRange(" + begin_key.ToString() + ", " +
                 end_key.ToString() + ")";
@@ -276,8 +278,8 @@ namespace {
       }
       return Status::OK();
     }
-    Status MergeCF(uint32_t column_family_id, const Slice& key,
-                   const Slice& value) override {
+    virtual Status MergeCF(uint32_t column_family_id, const Slice& key,
+                           const Slice& value) override {
       if (column_family_id == 0) {
         seen += "Merge(" + key.ToString() + ", " + value.ToString() + ")";
       } else {
@@ -286,27 +288,27 @@ namespace {
       }
       return Status::OK();
     }
-    void LogData(const Slice& blob) override {
+    virtual void LogData(const Slice& blob) override {
       seen += "LogData(" + blob.ToString() + ")";
     }
-    Status MarkBeginPrepare(bool unprepare) override {
+    virtual Status MarkBeginPrepare(bool unprepare) override {
       seen +=
           "MarkBeginPrepare(" + std::string(unprepare ? "true" : "false") + ")";
       return Status::OK();
     }
-    Status MarkEndPrepare(const Slice& xid) override {
+    virtual Status MarkEndPrepare(const Slice& xid) override {
       seen += "MarkEndPrepare(" + xid.ToString() + ")";
       return Status::OK();
     }
-    Status MarkNoop(bool empty_batch) override {
+    virtual Status MarkNoop(bool empty_batch) override {
       seen += "MarkNoop(" + std::string(empty_batch ? "true" : "false") + ")";
       return Status::OK();
     }
-    Status MarkCommit(const Slice& xid) override {
+    virtual Status MarkCommit(const Slice& xid) override {
       seen += "MarkCommit(" + xid.ToString() + ")";
       return Status::OK();
     }
-    Status MarkRollback(const Slice& xid) override {
+    virtual Status MarkRollback(const Slice& xid) override {
       seen += "MarkRollback(" + xid.ToString() + ")";
       return Status::OK();
     }
@@ -437,8 +439,8 @@ TEST_F(WriteBatchTest, DISABLED_ManyUpdates) {
   struct NoopHandler : public WriteBatch::Handler {
     uint32_t num_seen = 0;
     char expected_char = 'A';
-    Status PutCF(uint32_t /*column_family_id*/, const Slice& key,
-                 const Slice& value) override {
+    virtual Status PutCF(uint32_t /*column_family_id*/, const Slice& key,
+                         const Slice& value) override {
       EXPECT_EQ(kKeyValueSize, key.size());
       EXPECT_EQ(kKeyValueSize, value.size());
       EXPECT_EQ(expected_char, key[0]);
@@ -452,23 +454,23 @@ TEST_F(WriteBatchTest, DISABLED_ManyUpdates) {
       ++num_seen;
       return Status::OK();
     }
-    Status DeleteCF(uint32_t /*column_family_id*/,
-                    const Slice& /*key*/) override {
+    virtual Status DeleteCF(uint32_t /*column_family_id*/,
+                            const Slice& /*key*/) override {
       ADD_FAILURE();
       return Status::OK();
     }
-    Status SingleDeleteCF(uint32_t /*column_family_id*/,
-                          const Slice& /*key*/) override {
+    virtual Status SingleDeleteCF(uint32_t /*column_family_id*/,
+                                  const Slice& /*key*/) override {
       ADD_FAILURE();
       return Status::OK();
     }
-    Status MergeCF(uint32_t /*column_family_id*/, const Slice& /*key*/,
-                   const Slice& /*value*/) override {
+    virtual Status MergeCF(uint32_t /*column_family_id*/, const Slice& /*key*/,
+                           const Slice& /*value*/) override {
       ADD_FAILURE();
       return Status::OK();
     }
-    void LogData(const Slice& /*blob*/) override { ADD_FAILURE(); }
-    bool Continue() override { return num_seen < kNumUpdates; }
+    virtual void LogData(const Slice& /*blob*/) override { ADD_FAILURE(); }
+    virtual bool Continue() override { return num_seen < kNumUpdates; }
   } handler;
 
   batch.Iterate(&handler);
@@ -492,8 +494,8 @@ TEST_F(WriteBatchTest, DISABLED_LargeKeyValue) {
 
   struct NoopHandler : public WriteBatch::Handler {
     int num_seen = 0;
-    Status PutCF(uint32_t /*column_family_id*/, const Slice& key,
-                 const Slice& value) override {
+    virtual Status PutCF(uint32_t /*column_family_id*/, const Slice& key,
+                         const Slice& value) override {
       EXPECT_EQ(kKeyValueSize, key.size());
       EXPECT_EQ(kKeyValueSize, value.size());
       EXPECT_EQ('A' + num_seen, key[0]);
@@ -503,23 +505,23 @@ TEST_F(WriteBatchTest, DISABLED_LargeKeyValue) {
       ++num_seen;
       return Status::OK();
     }
-    Status DeleteCF(uint32_t /*column_family_id*/,
-                    const Slice& /*key*/) override {
+    virtual Status DeleteCF(uint32_t /*column_family_id*/,
+                            const Slice& /*key*/) override {
       ADD_FAILURE();
       return Status::OK();
     }
-    Status SingleDeleteCF(uint32_t /*column_family_id*/,
-                          const Slice& /*key*/) override {
+    virtual Status SingleDeleteCF(uint32_t /*column_family_id*/,
+                                  const Slice& /*key*/) override {
       ADD_FAILURE();
       return Status::OK();
     }
-    Status MergeCF(uint32_t /*column_family_id*/, const Slice& /*key*/,
-                   const Slice& /*value*/) override {
+    virtual Status MergeCF(uint32_t /*column_family_id*/, const Slice& /*key*/,
+                           const Slice& /*value*/) override {
       ADD_FAILURE();
       return Status::OK();
     }
-    void LogData(const Slice& /*blob*/) override { ADD_FAILURE(); }
-    bool Continue() override { return num_seen < 2; }
+    virtual void LogData(const Slice& /*blob*/) override { ADD_FAILURE(); }
+    virtual bool Continue() override { return num_seen < 2; }
   } handler;
 
   batch.Iterate(&handler);
@@ -531,30 +533,31 @@ TEST_F(WriteBatchTest, Continue) {
 
   struct Handler : public TestHandler {
     int num_seen = 0;
-    Status PutCF(uint32_t column_family_id, const Slice& key,
-                 const Slice& value) override {
+    virtual Status PutCF(uint32_t column_family_id, const Slice& key,
+                         const Slice& value) override {
       ++num_seen;
       return TestHandler::PutCF(column_family_id, key, value);
     }
-    Status DeleteCF(uint32_t column_family_id, const Slice& key) override {
+    virtual Status DeleteCF(uint32_t column_family_id,
+                            const Slice& key) override {
       ++num_seen;
       return TestHandler::DeleteCF(column_family_id, key);
     }
-    Status SingleDeleteCF(uint32_t column_family_id,
-                          const Slice& key) override {
+    virtual Status SingleDeleteCF(uint32_t column_family_id,
+                                  const Slice& key) override {
       ++num_seen;
       return TestHandler::SingleDeleteCF(column_family_id, key);
     }
-    Status MergeCF(uint32_t column_family_id, const Slice& key,
-                   const Slice& value) override {
+    virtual Status MergeCF(uint32_t column_family_id, const Slice& key,
+                           const Slice& value) override {
       ++num_seen;
       return TestHandler::MergeCF(column_family_id, key, value);
     }
-    void LogData(const Slice& blob) override {
+    virtual void LogData(const Slice& blob) override {
       ++num_seen;
       TestHandler::LogData(blob);
     }
-    bool Continue() override { return num_seen < 5; }
+    virtual bool Continue() override { return num_seen < 5; }
   } handler;
 
   batch.Put(Slice("k1"), Slice("v1"));

db/write_callback_test.cc

@@ -295,8 +295,8 @@ TEST_F(WriteCallbackTest, WriteWithCallbackTest) {
                    public:
                     PublishSeqCallback(DBImpl* db_impl_in)
                         : db_impl_(db_impl_in) {}
-                    Status Callback(SequenceNumber last_seq,
-                                    bool /*not used*/) override {
+                    virtual Status Callback(SequenceNumber last_seq,
+                                            bool /*not used*/) override {
                       db_impl_->SetLastPublishedSequence(last_seq);
                       return Status::OK();
                     }

db/write_controller_test.cc

@@ -18,7 +18,7 @@ class TimeSetEnv : public EnvWrapper {
  public:
   explicit TimeSetEnv() : EnvWrapper(nullptr) {}
   uint64_t now_micros_ = 6666;
-  uint64_t NowNanos() override { return now_micros_ * std::milli::den; }
+  virtual uint64_t NowNanos() override { return now_micros_ * std::milli::den; }
 };
 
 TEST_F(WriteControllerTest, ChangeDelayRateTest) {

defs.bzl

@@ -1,31 +0,0 @@
-load("@fbcode_macros//build_defs:cpp_binary.bzl", "cpp_binary")
-load("@fbcode_macros//build_defs:custom_unittest.bzl", "custom_unittest")
-
-def test_binary(
-        test_name,
-        test_cc,
-        parallelism,
-        rocksdb_arch_preprocessor_flags,
-        rocksdb_compiler_flags,
-        rocksdb_preprocessor_flags,
-        rocksdb_external_deps):
-    TEST_RUNNER = native.package_name() + "/buckifier/rocks_test_runner.sh"
-
-    ttype = "gtest" if parallelism == "parallel" else "simple"
-    test_bin = test_name + "_bin"
-
-    cpp_binary(
-        name = test_bin,
-        srcs = [test_cc],
-        arch_preprocessor_flags = rocksdb_arch_preprocessor_flags,
-        compiler_flags = rocksdb_compiler_flags,
-        preprocessor_flags = rocksdb_preprocessor_flags,
-        deps = [":rocksdb_test_lib"],
-        external_deps = rocksdb_external_deps,
-    )
-
-    custom_unittest(
-        name = test_name,
-        command = [TEST_RUNNER, "$(location :{})".format(test_bin)],
-        type = ttype,
-    )

env/env.cc

@@ -138,8 +138,6 @@ void Logger::Logv(const InfoLogLevel log_level, const char* format, va_list ap)
     // are INFO level. We don't want to add extra costs to those existing
     // logging.
     Logv(format, ap);
-  } else if (log_level == InfoLogLevel::HEADER_LEVEL) {
-    LogHeader(format, ap);
   } else {
     char new_format[500];
     snprintf(new_format, sizeof(new_format) - 1, "[%s] %s",

env/env_basic_test.cc

@@ -21,8 +21,8 @@ class NormalizingEnvWrapper : public EnvWrapper {
   explicit NormalizingEnvWrapper(Env* base) : EnvWrapper(base) {}
 
   // Removes . and .. from directory listing
-  Status GetChildren(const std::string& dir,
-                     std::vector<std::string>* result) override {
+  virtual Status GetChildren(const std::string& dir,
+                             std::vector<std::string>* result) override {
     Status status = EnvWrapper::GetChildren(dir, result);
     if (status.ok()) {
       result->erase(std::remove_if(result->begin(), result->end(),
@@ -35,7 +35,7 @@ class NormalizingEnvWrapper : public EnvWrapper {
   }
 
   // Removes . and .. from directory listing
-  Status GetChildrenFileAttributes(
+  virtual Status GetChildrenFileAttributes(
       const std::string& dir, std::vector<FileAttributes>* result) override {
     Status status = EnvWrapper::GetChildrenFileAttributes(dir, result);
     if (status.ok()) {
@@ -60,9 +60,11 @@ class EnvBasicTestWithParam : public testing::Test,
     test_dir_ = test::PerThreadDBPath(env_, "env_basic_test");
   }
 
-  void SetUp() override { env_->CreateDirIfMissing(test_dir_); }
+  void SetUp() {
+    env_->CreateDirIfMissing(test_dir_);
+  }
 
-  void TearDown() override {
+  void TearDown() {
     std::vector<std::string> files;
     env_->GetChildren(test_dir_, &files);
     for (const auto& file : files) {

env/env_chroot.cc

@@ -38,9 +38,9 @@ class ChrootEnv : public EnvWrapper {
 #endif
   }
 
-  Status NewSequentialFile(const std::string& fname,
-                           std::unique_ptr<SequentialFile>* result,
-                           const EnvOptions& options) override {
+  virtual Status NewSequentialFile(const std::string& fname,
+                                   std::unique_ptr<SequentialFile>* result,
+                                   const EnvOptions& options) override {
     auto status_and_enc_path = EncodePathWithNewBasename(fname);
     if (!status_and_enc_path.first.ok()) {
       return status_and_enc_path.first;
@@ -49,9 +49,9 @@ class ChrootEnv : public EnvWrapper {
                                          options);
   }
 
-  Status NewRandomAccessFile(const std::string& fname,
-                             std::unique_ptr<RandomAccessFile>* result,
-                             const EnvOptions& options) override {
+  virtual Status NewRandomAccessFile(const std::string& fname,
+                                     std::unique_ptr<RandomAccessFile>* result,
+                                     const EnvOptions& options) override {
     auto status_and_enc_path = EncodePathWithNewBasename(fname);
     if (!status_and_enc_path.first.ok()) {
       return status_and_enc_path.first;
@@ -60,9 +60,9 @@ class ChrootEnv : public EnvWrapper {
                                            options);
   }
 
-  Status NewWritableFile(const std::string& fname,
-                         std::unique_ptr<WritableFile>* result,
-                         const EnvOptions& options) override {
+  virtual Status NewWritableFile(const std::string& fname,
+                                 std::unique_ptr<WritableFile>* result,
+                                 const EnvOptions& options) override {
     auto status_and_enc_path = EncodePathWithNewBasename(fname);
     if (!status_and_enc_path.first.ok()) {
       return status_and_enc_path.first;
@@ -71,10 +71,10 @@ class ChrootEnv : public EnvWrapper {
                                        options);
   }
 
-  Status ReuseWritableFile(const std::string& fname,
-                           const std::string& old_fname,
-                           std::unique_ptr<WritableFile>* result,
-                           const EnvOptions& options) override {
+  virtual Status ReuseWritableFile(const std::string& fname,
+                                   const std::string& old_fname,
+                                   std::unique_ptr<WritableFile>* result,
+                                   const EnvOptions& options) override {
     auto status_and_enc_path = EncodePathWithNewBasename(fname);
     if (!status_and_enc_path.first.ok()) {
       return status_and_enc_path.first;
@@ -88,9 +88,9 @@ class ChrootEnv : public EnvWrapper {
                                          options);
   }
 
-  Status NewRandomRWFile(const std::string& fname,
-                         std::unique_ptr<RandomRWFile>* result,
-                         const EnvOptions& options) override {
+  virtual Status NewRandomRWFile(const std::string& fname,
+                                 std::unique_ptr<RandomRWFile>* result,
+                                 const EnvOptions& options) override {
     auto status_and_enc_path = EncodePathWithNewBasename(fname);
     if (!status_and_enc_path.first.ok()) {
       return status_and_enc_path.first;
@@ -99,8 +99,8 @@ class ChrootEnv : public EnvWrapper {
                                        options);
   }
 
-  Status NewDirectory(const std::string& dir,
-                      std::unique_ptr<Directory>* result) override {
+  virtual Status NewDirectory(const std::string& dir,
+                              std::unique_ptr<Directory>* result) override {
     auto status_and_enc_path = EncodePathWithNewBasename(dir);
     if (!status_and_enc_path.first.ok()) {
       return status_and_enc_path.first;
@@ -108,7 +108,7 @@ class ChrootEnv : public EnvWrapper {
     return EnvWrapper::NewDirectory(status_and_enc_path.second, result);
   }
 
-  Status FileExists(const std::string& fname) override {
+  virtual Status FileExists(const std::string& fname) override {
     auto status_and_enc_path = EncodePathWithNewBasename(fname);
     if (!status_and_enc_path.first.ok()) {
       return status_and_enc_path.first;
@@ -116,8 +116,8 @@ class ChrootEnv : public EnvWrapper {
     return EnvWrapper::FileExists(status_and_enc_path.second);
   }
 
-  Status GetChildren(const std::string& dir,
-                     std::vector<std::string>* result) override {
+  virtual Status GetChildren(const std::string& dir,
+                             std::vector<std::string>* result) override {
     auto status_and_enc_path = EncodePath(dir);
     if (!status_and_enc_path.first.ok()) {
       return status_and_enc_path.first;
@@ -125,7 +125,7 @@ class ChrootEnv : public EnvWrapper {
     return EnvWrapper::GetChildren(status_and_enc_path.second, result);
   }
 
-  Status GetChildrenFileAttributes(
+  virtual Status GetChildrenFileAttributes(
       const std::string& dir, std::vector<FileAttributes>* result) override {
     auto status_and_enc_path = EncodePath(dir);
     if (!status_and_enc_path.first.ok()) {
@@ -135,7 +135,7 @@ class ChrootEnv : public EnvWrapper {
                                                  result);
   }
 
-  Status DeleteFile(const std::string& fname) override {
+  virtual Status DeleteFile(const std::string& fname) override {
     auto status_and_enc_path = EncodePath(fname);
     if (!status_and_enc_path.first.ok()) {
       return status_and_enc_path.first;
@@ -143,7 +143,7 @@ class ChrootEnv : public EnvWrapper {
     return EnvWrapper::DeleteFile(status_and_enc_path.second);
   }
 
-  Status CreateDir(const std::string& dirname) override {
+  virtual Status CreateDir(const std::string& dirname) override {
     auto status_and_enc_path = EncodePathWithNewBasename(dirname);
     if (!status_and_enc_path.first.ok()) {
       return status_and_enc_path.first;
@@ -151,7 +151,7 @@ class ChrootEnv : public EnvWrapper {
     return EnvWrapper::CreateDir(status_and_enc_path.second);
   }
 
-  Status CreateDirIfMissing(const std::string& dirname) override {
+  virtual Status CreateDirIfMissing(const std::string& dirname) override {
     auto status_and_enc_path = EncodePathWithNewBasename(dirname);
     if (!status_and_enc_path.first.ok()) {
       return status_and_enc_path.first;
@@ -159,7 +159,7 @@ class ChrootEnv : public EnvWrapper {
     return EnvWrapper::CreateDirIfMissing(status_and_enc_path.second);
   }
 
-  Status DeleteDir(const std::string& dirname) override {
+  virtual Status DeleteDir(const std::string& dirname) override {
     auto status_and_enc_path = EncodePath(dirname);
     if (!status_and_enc_path.first.ok()) {
       return status_and_enc_path.first;
@@ -167,7 +167,8 @@ class ChrootEnv : public EnvWrapper {
     return EnvWrapper::DeleteDir(status_and_enc_path.second);
   }
 
-  Status GetFileSize(const std::string& fname, uint64_t* file_size) override {
+  virtual Status GetFileSize(const std::string& fname,
+                             uint64_t* file_size) override {
     auto status_and_enc_path = EncodePath(fname);
     if (!status_and_enc_path.first.ok()) {
       return status_and_enc_path.first;
@@ -175,8 +176,8 @@ class ChrootEnv : public EnvWrapper {
     return EnvWrapper::GetFileSize(status_and_enc_path.second, file_size);
   }
 
-  Status GetFileModificationTime(const std::string& fname,
-                                 uint64_t* file_mtime) override {
+  virtual Status GetFileModificationTime(const std::string& fname,
+                                         uint64_t* file_mtime) override {
     auto status_and_enc_path = EncodePath(fname);
     if (!status_and_enc_path.first.ok()) {
       return status_and_enc_path.first;
@@ -185,7 +186,8 @@ class ChrootEnv : public EnvWrapper {
                                                file_mtime);
   }
 
-  Status RenameFile(const std::string& src, const std::string& dest) override {
+  virtual Status RenameFile(const std::string& src,
+                            const std::string& dest) override {
     auto status_and_src_enc_path = EncodePath(src);
     if (!status_and_src_enc_path.first.ok()) {
       return status_and_src_enc_path.first;
@@ -198,7 +200,8 @@ class ChrootEnv : public EnvWrapper {
                                   status_and_dest_enc_path.second);
   }
 
-  Status LinkFile(const std::string& src, const std::string& dest) override {
+  virtual Status LinkFile(const std::string& src,
+                          const std::string& dest) override {
     auto status_and_src_enc_path = EncodePath(src);
     if (!status_and_src_enc_path.first.ok()) {
       return status_and_src_enc_path.first;
@@ -211,7 +214,7 @@ class ChrootEnv : public EnvWrapper {
                                 status_and_dest_enc_path.second);
   }
 
-  Status LockFile(const std::string& fname, FileLock** lock) override {
+  virtual Status LockFile(const std::string& fname, FileLock** lock) override {
     auto status_and_enc_path = EncodePathWithNewBasename(fname);
     if (!status_and_enc_path.first.ok()) {
       return status_and_enc_path.first;
@@ -222,7 +225,7 @@ class ChrootEnv : public EnvWrapper {
     return EnvWrapper::LockFile(status_and_enc_path.second, lock);
   }
 
-  Status GetTestDirectory(std::string* path) override {
+  virtual Status GetTestDirectory(std::string* path) override {
     // Adapted from PosixEnv's implementation since it doesn't provide a way to
     // create directory in the chroot.
     char buf[256];
@@ -234,8 +237,8 @@ class ChrootEnv : public EnvWrapper {
     return Status::OK();
   }
 
-  Status NewLogger(const std::string& fname,
-                   std::shared_ptr<Logger>* result) override {
+  virtual Status NewLogger(const std::string& fname,
+                           std::shared_ptr<Logger>* result) override {
     auto status_and_enc_path = EncodePathWithNewBasename(fname);
     if (!status_and_enc_path.first.ok()) {
       return status_and_enc_path.first;
@@ -243,8 +246,8 @@ class ChrootEnv : public EnvWrapper {
     return EnvWrapper::NewLogger(status_and_enc_path.second, result);
   }
 
-  Status GetAbsolutePath(const std::string& db_path,
-                         std::string* output_path) override {
+  virtual Status GetAbsolutePath(const std::string& db_path,
+                                 std::string* output_path) override {
     auto status_and_enc_path = EncodePath(db_path);
     if (!status_and_enc_path.first.ok()) {
       return status_and_enc_path.first;

env/env_encryption.cc

@@ -42,7 +42,7 @@ class EncryptedSequentialFile : public SequentialFile {
   // If an error was encountered, returns a non-OK status.
   //
   // REQUIRES: External synchronization
-  Status Read(size_t n, Slice* result, char* scratch) override {
+  virtual Status Read(size_t n, Slice* result, char* scratch) override {
     assert(scratch);
     Status status = file_->Read(n, result, scratch);
     if (!status.ok()) {
@@ -60,7 +60,7 @@ class EncryptedSequentialFile : public SequentialFile {
   // file, and Skip will return OK.
   //
   // REQUIRES: External synchronization
-  Status Skip(uint64_t n) override {
+  virtual Status Skip(uint64_t n) override {
     auto status = file_->Skip(n);
     if (!status.ok()) {
       return status;
@@ -71,25 +71,26 @@ class EncryptedSequentialFile : public SequentialFile {
 
   // Indicates the upper layers if the current SequentialFile implementation
   // uses direct IO.
-  bool use_direct_io() const override { return file_->use_direct_io(); }
+  virtual bool use_direct_io() const override { 
+    return file_->use_direct_io(); 
+  }
 
   // Use the returned alignment value to allocate
   // aligned buffer for Direct I/O
-  size_t GetRequiredBufferAlignment() const override {
-    return file_->GetRequiredBufferAlignment();
+  virtual size_t GetRequiredBufferAlignment() const override { 
+    return file_->GetRequiredBufferAlignment(); 
   }
 
   // Remove any kind of caching of data from the offset to offset+length
   // of this file. If the length is 0, then it refers to the end of file.
   // If the system is not caching the file contents, then this is a noop.
-  Status InvalidateCache(size_t offset, size_t length) override {
+  virtual Status InvalidateCache(size_t offset, size_t length) override {
     return file_->InvalidateCache(offset + prefixLength_, length);
   }
 
   // Positioned Read for direct I/O
   // If Direct I/O enabled, offset, n, and scratch should be properly aligned
-  Status PositionedRead(uint64_t offset, size_t n, Slice* result,
-                        char* scratch) override {
+  virtual Status PositionedRead(uint64_t offset, size_t n, Slice* result, char* scratch) override {
     assert(scratch);
     offset += prefixLength_; // Skip prefix
     auto status = file_->PositionedRead(offset, n, result, scratch);
@@ -100,6 +101,7 @@ class EncryptedSequentialFile : public SequentialFile {
     status = stream_->Decrypt(offset, (char*)result->data(), result->size());
     return status;
   }
+
 };
 
 // A file abstraction for randomly reading the contents of a file.
@@ -123,8 +125,7 @@ class EncryptedRandomAccessFile : public RandomAccessFile {
   //
   // Safe for concurrent use by multiple threads.
   // If Direct I/O enabled, offset, n, and scratch should be aligned properly.
-  Status Read(uint64_t offset, size_t n, Slice* result,
-              char* scratch) const override {
+  virtual Status Read(uint64_t offset, size_t n, Slice* result, char* scratch) const override {
     assert(scratch);
     offset += prefixLength_;
     auto status = file_->Read(offset, n, result, scratch);
@@ -136,7 +137,7 @@ class EncryptedRandomAccessFile : public RandomAccessFile {
   }
 
   // Readahead the file starting from offset by n bytes for caching.
-  Status Prefetch(uint64_t offset, size_t n) override {
+  virtual Status Prefetch(uint64_t offset, size_t n) override {
     //return Status::OK();
     return file_->Prefetch(offset + prefixLength_, n);
   }
@@ -156,26 +157,30 @@ class EncryptedRandomAccessFile : public RandomAccessFile {
   // a single varint.
   //
   // Note: these IDs are only valid for the duration of the process.
-  size_t GetUniqueId(char* id, size_t max_size) const override {
+  virtual size_t GetUniqueId(char* id, size_t max_size) const override {
     return file_->GetUniqueId(id, max_size);
   };
 
-  void Hint(AccessPattern pattern) override { file_->Hint(pattern); }
+  virtual void Hint(AccessPattern pattern) override {
+    file_->Hint(pattern);
+  }
 
   // Indicates the upper layers if the current RandomAccessFile implementation
   // uses direct IO.
-  bool use_direct_io() const override { return file_->use_direct_io(); }
+  virtual bool use_direct_io() const override {
+     return file_->use_direct_io(); 
+  }
 
   // Use the returned alignment value to allocate
   // aligned buffer for Direct I/O
-  size_t GetRequiredBufferAlignment() const override {
-    return file_->GetRequiredBufferAlignment();
+  virtual size_t GetRequiredBufferAlignment() const override { 
+    return file_->GetRequiredBufferAlignment(); 
   }
 
   // Remove any kind of caching of data from the offset to offset+length
   // of this file. If the length is 0, then it refers to the end of file.
   // If the system is not caching the file contents, then this is a noop.
-  Status InvalidateCache(size_t offset, size_t length) override {
+  virtual Status InvalidateCache(size_t offset, size_t length) override {
     return file_->InvalidateCache(offset + prefixLength_, length);
   }
 };
@@ -242,18 +247,16 @@ class EncryptedWritableFile : public WritableFileWrapper {
 
   // Indicates the upper layers if the current WritableFile implementation
   // uses direct IO.
-  bool use_direct_io() const override { return file_->use_direct_io(); }
+  virtual bool use_direct_io() const override { return file_->use_direct_io(); }
 
   // Use the returned alignment value to allocate
   // aligned buffer for Direct I/O
-  size_t GetRequiredBufferAlignment() const override {
-    return file_->GetRequiredBufferAlignment();
-  }
+  virtual size_t GetRequiredBufferAlignment() const override { return file_->GetRequiredBufferAlignment(); } 
 
     /*
    * Get the size of valid data in the file.
    */
-  uint64_t GetFileSize() override {
+  virtual uint64_t GetFileSize() override {
     return file_->GetFileSize() - prefixLength_;
   }
 
@@ -261,7 +264,7 @@ class EncryptedWritableFile : public WritableFileWrapper {
   // before closing. It is not always possible to keep track of the file
   // size due to whole pages writes. The behavior is undefined if called
   // with other writes to follow.
-  Status Truncate(uint64_t size) override {
+  virtual Status Truncate(uint64_t size) override {
     return file_->Truncate(size + prefixLength_);
   }
 
@@ -269,7 +272,7 @@ class EncryptedWritableFile : public WritableFileWrapper {
   // of this file. If the length is 0, then it refers to the end of file.
   // If the system is not caching the file contents, then this is a noop.
   // This call has no effect on dirty pages in the cache.
-  Status InvalidateCache(size_t offset, size_t length) override {
+  virtual Status InvalidateCache(size_t offset, size_t length) override {
     return file_->InvalidateCache(offset + prefixLength_, length);
   }
 
@@ -279,7 +282,7 @@ class EncryptedWritableFile : public WritableFileWrapper {
   // This asks the OS to initiate flushing the cached data to disk,
   // without waiting for completion.
   // Default implementation does nothing.
-  Status RangeSync(uint64_t offset, uint64_t nbytes) override {
+  virtual Status RangeSync(uint64_t offset, uint64_t nbytes) override { 
     return file_->RangeSync(offset + prefixLength_, nbytes);
   }
 
@@ -288,12 +291,12 @@ class EncryptedWritableFile : public WritableFileWrapper {
   // of space on devices where it can result in less file
   // fragmentation and/or less waste from over-zealous filesystem
   // pre-allocation.
-  void PrepareWrite(size_t offset, size_t len) override {
+  virtual void PrepareWrite(size_t offset, size_t len) override {
     file_->PrepareWrite(offset + prefixLength_, len);
   }
 
   // Pre-allocates space for a file.
-  Status Allocate(uint64_t offset, uint64_t len) override {
+  virtual Status Allocate(uint64_t offset, uint64_t len) override {
     return file_->Allocate(offset + prefixLength_, len);
   }
 };
@@ -311,17 +314,17 @@ class EncryptedRandomRWFile : public RandomRWFile {
 
   // Indicates if the class makes use of direct I/O
   // If false you must pass aligned buffer to Write()
-  bool use_direct_io() const override { return file_->use_direct_io(); }
+  virtual bool use_direct_io() const override { return file_->use_direct_io(); }
 
   // Use the returned alignment value to allocate
   // aligned buffer for Direct I/O
-  size_t GetRequiredBufferAlignment() const override {
-    return file_->GetRequiredBufferAlignment();
+  virtual size_t GetRequiredBufferAlignment() const override { 
+    return file_->GetRequiredBufferAlignment(); 
   }
 
   // Write bytes in `data` at  offset `offset`, Returns Status::OK() on success.
   // Pass aligned buffer when use_direct_io() returns true.
-  Status Write(uint64_t offset, const Slice& data) override {
+  virtual Status Write(uint64_t offset, const Slice& data) override {
     AlignedBuffer buf;
     Status status;
     Slice dataToWrite(data); 
@@ -344,8 +347,7 @@ class EncryptedRandomRWFile : public RandomRWFile {
   // Read up to `n` bytes starting from offset `offset` and store them in
   // result, provided `scratch` size should be at least `n`.
   // Returns Status::OK() on success.
-  Status Read(uint64_t offset, size_t n, Slice* result,
-              char* scratch) const override {
+  virtual Status Read(uint64_t offset, size_t n, Slice* result, char* scratch) const override { 
     assert(scratch);
     offset += prefixLength_;
     auto status = file_->Read(offset, n, result, scratch);
@@ -356,13 +358,21 @@ class EncryptedRandomRWFile : public RandomRWFile {
     return status;
   }
 
-  Status Flush() override { return file_->Flush(); }
+  virtual Status Flush() override {
+    return file_->Flush();
+  }
 
-  Status Sync() override { return file_->Sync(); }
+  virtual Status Sync() override {
+    return file_->Sync();
+  }
 
-  Status Fsync() override { return file_->Fsync(); }
+  virtual Status Fsync() override { 
+    return file_->Fsync();
+  }
 
-  Status Close() override { return file_->Close(); }
+  virtual Status Close() override {
+    return file_->Close();
+  }
 };
 
 // EncryptedEnv implements an Env wrapper that adds encryption to files stored on disk.
@@ -374,9 +384,9 @@ class EncryptedEnv : public EnvWrapper {
   }
 
   // NewSequentialFile opens a file for sequential reading.
-  Status NewSequentialFile(const std::string& fname,
-                           std::unique_ptr<SequentialFile>* result,
-                           const EnvOptions& options) override {
+  virtual Status NewSequentialFile(const std::string& fname,
+                                   std::unique_ptr<SequentialFile>* result,
+                                   const EnvOptions& options) override {
     result->reset();
     if (options.use_mmap_reads) {
       return Status::InvalidArgument();
@@ -411,9 +421,9 @@ class EncryptedEnv : public EnvWrapper {
   }
 
   // NewRandomAccessFile opens a file for random read access.
-  Status NewRandomAccessFile(const std::string& fname,
-                             std::unique_ptr<RandomAccessFile>* result,
-                             const EnvOptions& options) override {
+  virtual Status NewRandomAccessFile(const std::string& fname,
+                                     std::unique_ptr<RandomAccessFile>* result,
+                                     const EnvOptions& options) override {
     result->reset();
     if (options.use_mmap_reads) {
       return Status::InvalidArgument();
@@ -448,9 +458,9 @@ class EncryptedEnv : public EnvWrapper {
   }
 
   // NewWritableFile opens a file for sequential writing.
-  Status NewWritableFile(const std::string& fname,
-                         std::unique_ptr<WritableFile>* result,
-                         const EnvOptions& options) override {
+  virtual Status NewWritableFile(const std::string& fname,
+                                 std::unique_ptr<WritableFile>* result,
+                                 const EnvOptions& options) override {
     result->reset();
     if (options.use_mmap_writes) {
       return Status::InvalidArgument();
@@ -494,9 +504,9 @@ class EncryptedEnv : public EnvWrapper {
   // returns non-OK.
   //
   // The returned file will only be accessed by one thread at a time.
-  Status ReopenWritableFile(const std::string& fname,
-                            std::unique_ptr<WritableFile>* result,
-                            const EnvOptions& options) override {
+  virtual Status ReopenWritableFile(const std::string& fname,
+                                    std::unique_ptr<WritableFile>* result,
+                                    const EnvOptions& options) override {
     result->reset();
     if (options.use_mmap_writes) {
       return Status::InvalidArgument();
@@ -534,10 +544,10 @@ class EncryptedEnv : public EnvWrapper {
   }
 
   // Reuse an existing file by renaming it and opening it as writable.
-  Status ReuseWritableFile(const std::string& fname,
-                           const std::string& old_fname,
-                           std::unique_ptr<WritableFile>* result,
-                           const EnvOptions& options) override {
+  virtual Status ReuseWritableFile(const std::string& fname,
+                                   const std::string& old_fname,
+                                   std::unique_ptr<WritableFile>* result,
+                                   const EnvOptions& options) override {
     result->reset();
     if (options.use_mmap_writes) {
       return Status::InvalidArgument();
@@ -579,9 +589,9 @@ class EncryptedEnv : public EnvWrapper {
   // *result and returns OK.  On failure returns non-OK.
   //
   // The returned file will only be accessed by one thread at a time.
-  Status NewRandomRWFile(const std::string& fname,
-                         std::unique_ptr<RandomRWFile>* result,
-                         const EnvOptions& options) override {
+  virtual Status NewRandomRWFile(const std::string& fname,
+                                 std::unique_ptr<RandomRWFile>* result,
+                                 const EnvOptions& options) override {
     result->reset();
     if (options.use_mmap_reads || options.use_mmap_writes) {
       return Status::InvalidArgument();
@@ -639,8 +649,7 @@ class EncryptedEnv : public EnvWrapper {
   //         NotFound if "dir" does not exist, the calling process does not have
   //                  permission to access "dir", or if "dir" is invalid.
   //         IOError if an IO Error was encountered
-  Status GetChildrenFileAttributes(
-      const std::string& dir, std::vector<FileAttributes>* result) override {
+  virtual Status GetChildrenFileAttributes(const std::string& dir, std::vector<FileAttributes>* result) override {
     auto status = EnvWrapper::GetChildrenFileAttributes(dir, result);
     if (!status.ok()) {
       return status;
@@ -651,10 +660,10 @@ class EncryptedEnv : public EnvWrapper {
       it->size_bytes -= prefixLength;
     }
     return Status::OK();
-  }
+ }
 
   // Store the size of fname in *file_size.
-  Status GetFileSize(const std::string& fname, uint64_t* file_size) override {
+  virtual Status GetFileSize(const std::string& fname, uint64_t* file_size) override {
     auto status = EnvWrapper::GetFileSize(fname, file_size);
     if (!status.ok()) {
       return status;
@@ -662,7 +671,7 @@ class EncryptedEnv : public EnvWrapper {
     size_t prefixLength = provider_->GetPrefixLength();
     assert(*file_size >= prefixLength);
     *file_size -= prefixLength;
-    return Status::OK();
+    return Status::OK();    
   }
 
  private:

env/env_posix.cc

@@ -119,7 +119,7 @@ class PosixEnv : public Env {
  public:
   PosixEnv();
 
-  ~PosixEnv() override {
+  virtual ~PosixEnv() {
     for (const auto tid : threads_to_join_) {
       pthread_join(tid, nullptr);
     }
@@ -141,9 +141,9 @@ class PosixEnv : public Env {
     }
   }
 
-  Status NewSequentialFile(const std::string& fname,
-                           std::unique_ptr<SequentialFile>* result,
-                           const EnvOptions& options) override {
+  virtual Status NewSequentialFile(const std::string& fname,
+                                   std::unique_ptr<SequentialFile>* result,
+                                   const EnvOptions& options) override {
     result->reset();
     int fd = -1;
     int flags = cloexec_flags(O_RDONLY, &options);
@@ -191,9 +191,9 @@ class PosixEnv : public Env {
     return Status::OK();
   }
 
-  Status NewRandomAccessFile(const std::string& fname,
-                             std::unique_ptr<RandomAccessFile>* result,
-                             const EnvOptions& options) override {
+  virtual Status NewRandomAccessFile(const std::string& fname,
+                                     std::unique_ptr<RandomAccessFile>* result,
+                                     const EnvOptions& options) override {
     result->reset();
     Status s;
     int fd;
@@ -332,22 +332,22 @@ class PosixEnv : public Env {
     return s;
   }
 
-  Status NewWritableFile(const std::string& fname,
-                         std::unique_ptr<WritableFile>* result,
-                         const EnvOptions& options) override {
+  virtual Status NewWritableFile(const std::string& fname,
+                                 std::unique_ptr<WritableFile>* result,
+                                 const EnvOptions& options) override {
     return OpenWritableFile(fname, result, options, false);
   }
 
-  Status ReopenWritableFile(const std::string& fname,
-                            std::unique_ptr<WritableFile>* result,
-                            const EnvOptions& options) override {
+  virtual Status ReopenWritableFile(const std::string& fname,
+                                    std::unique_ptr<WritableFile>* result,
+                                    const EnvOptions& options) override {
     return OpenWritableFile(fname, result, options, true);
   }
 
-  Status ReuseWritableFile(const std::string& fname,
-                           const std::string& old_fname,
-                           std::unique_ptr<WritableFile>* result,
-                           const EnvOptions& options) override {
+  virtual Status ReuseWritableFile(const std::string& fname,
+                                   const std::string& old_fname,
+                                   std::unique_ptr<WritableFile>* result,
+                                   const EnvOptions& options) override {
     result->reset();
     Status s;
     int fd = -1;
@@ -429,9 +429,9 @@ class PosixEnv : public Env {
     return s;
   }
 
-  Status NewRandomRWFile(const std::string& fname,
-                         std::unique_ptr<RandomRWFile>* result,
-                         const EnvOptions& options) override {
+  virtual Status NewRandomRWFile(const std::string& fname,
+                                 std::unique_ptr<RandomRWFile>* result,
+                                 const EnvOptions& options) override {
     int fd = -1;
     int flags = cloexec_flags(O_RDWR, &options);
 
@@ -453,7 +453,7 @@ class PosixEnv : public Env {
     return Status::OK();
   }
 
-  Status NewMemoryMappedFileBuffer(
+  virtual Status NewMemoryMappedFileBuffer(
       const std::string& fname,
       std::unique_ptr<MemoryMappedFileBuffer>* result) override {
     int fd = -1;
@@ -496,8 +496,8 @@ class PosixEnv : public Env {
     return status;
   }
 
-  Status NewDirectory(const std::string& name,
-                      std::unique_ptr<Directory>* result) override {
+  virtual Status NewDirectory(const std::string& name,
+                              std::unique_ptr<Directory>* result) override {
     result->reset();
     int fd;
     int flags = cloexec_flags(0, nullptr);
@@ -513,7 +513,7 @@ class PosixEnv : public Env {
     return Status::OK();
   }
 
-  Status FileExists(const std::string& fname) override {
+  virtual Status FileExists(const std::string& fname) override {
     int result = access(fname.c_str(), F_OK);
 
     if (result == 0) {
@@ -535,8 +535,8 @@ class PosixEnv : public Env {
     }
   }
 
-  Status GetChildren(const std::string& dir,
-                     std::vector<std::string>* result) override {
+  virtual Status GetChildren(const std::string& dir,
+                             std::vector<std::string>* result) override {
     result->clear();
     DIR* d = opendir(dir.c_str());
     if (d == nullptr) {
@@ -557,7 +557,7 @@ class PosixEnv : public Env {
     return Status::OK();
   }
 
-  Status DeleteFile(const std::string& fname) override {
+  virtual Status DeleteFile(const std::string& fname) override {
     Status result;
     if (unlink(fname.c_str()) != 0) {
       result = IOError("while unlink() file", fname, errno);
@@ -565,7 +565,7 @@ class PosixEnv : public Env {
     return result;
   };
 
-  Status CreateDir(const std::string& name) override {
+  virtual Status CreateDir(const std::string& name) override {
     Status result;
     if (mkdir(name.c_str(), 0755) != 0) {
       result = IOError("While mkdir", name, errno);
@@ -573,7 +573,7 @@ class PosixEnv : public Env {
     return result;
   };
 
-  Status CreateDirIfMissing(const std::string& name) override {
+  virtual Status CreateDirIfMissing(const std::string& name) override {
     Status result;
     if (mkdir(name.c_str(), 0755) != 0) {
       if (errno != EEXIST) {
@@ -587,7 +587,7 @@ class PosixEnv : public Env {
     return result;
   };
 
-  Status DeleteDir(const std::string& name) override {
+  virtual Status DeleteDir(const std::string& name) override {
     Status result;
     if (rmdir(name.c_str()) != 0) {
       result = IOError("file rmdir", name, errno);
@@ -595,7 +595,8 @@ class PosixEnv : public Env {
     return result;
   };
 
-  Status GetFileSize(const std::string& fname, uint64_t* size) override {
+  virtual Status GetFileSize(const std::string& fname,
+                             uint64_t* size) override {
     Status s;
     struct stat sbuf;
     if (stat(fname.c_str(), &sbuf) != 0) {
@@ -607,8 +608,8 @@ class PosixEnv : public Env {
     return s;
   }
 
-  Status GetFileModificationTime(const std::string& fname,
-                                 uint64_t* file_mtime) override {
+  virtual Status GetFileModificationTime(const std::string& fname,
+                                         uint64_t* file_mtime) override {
     struct stat s;
     if (stat(fname.c_str(), &s) !=0) {
       return IOError("while stat a file for modification time", fname, errno);
@@ -616,8 +617,8 @@ class PosixEnv : public Env {
     *file_mtime = static_cast<uint64_t>(s.st_mtime);
     return Status::OK();
   }
-  Status RenameFile(const std::string& src,
-                    const std::string& target) override {
+  virtual Status RenameFile(const std::string& src,
+                            const std::string& target) override {
     Status result;
     if (rename(src.c_str(), target.c_str()) != 0) {
       result = IOError("While renaming a file to " + target, src, errno);
@@ -625,7 +626,8 @@ class PosixEnv : public Env {
     return result;
   }
 
-  Status LinkFile(const std::string& src, const std::string& target) override {
+  virtual Status LinkFile(const std::string& src,
+                          const std::string& target) override {
     Status result;
     if (link(src.c_str(), target.c_str()) != 0) {
       if (errno == EXDEV) {
@@ -645,8 +647,8 @@ class PosixEnv : public Env {
     return Status::OK();
   }
 
-  Status AreFilesSame(const std::string& first, const std::string& second,
-                      bool* res) override {
+  virtual Status AreFilesSame(const std::string& first,
+                              const std::string& second, bool* res) override {
     struct stat statbuf[2];
     if (stat(first.c_str(), &statbuf[0]) != 0) {
       return IOError("stat file", first, errno);
@@ -665,7 +667,7 @@ class PosixEnv : public Env {
     return Status::OK();
   }
 
-  Status LockFile(const std::string& fname, FileLock** lock) override {
+  virtual Status LockFile(const std::string& fname, FileLock** lock) override {
     *lock = nullptr;
     Status result;
 
@@ -711,7 +713,7 @@ class PosixEnv : public Env {
     return result;
   }
 
-  Status UnlockFile(FileLock* lock) override {
+  virtual Status UnlockFile(FileLock* lock) override {
     PosixFileLock* my_lock = reinterpret_cast<PosixFileLock*>(lock);
     Status result;
     mutex_lockedFiles.Lock();
@@ -729,19 +731,19 @@ class PosixEnv : public Env {
     return result;
   }
 
-  void Schedule(void (*function)(void* arg1), void* arg, Priority pri = LOW,
-                void* tag = nullptr,
-                void (*unschedFunction)(void* arg) = nullptr) override;
+  virtual void Schedule(void (*function)(void* arg1), void* arg,
+                        Priority pri = LOW, void* tag = nullptr,
+                        void (*unschedFunction)(void* arg) = nullptr) override;
 
-  int UnSchedule(void* arg, Priority pri) override;
+  virtual int UnSchedule(void* arg, Priority pri) override;
 
-  void StartThread(void (*function)(void* arg), void* arg) override;
+  virtual void StartThread(void (*function)(void* arg), void* arg) override;
 
-  void WaitForJoin() override;
+  virtual void WaitForJoin() override;
 
-  unsigned int GetThreadPoolQueueLen(Priority pri = LOW) const override;
+  virtual unsigned int GetThreadPoolQueueLen(Priority pri = LOW) const override;
 
-  Status GetTestDirectory(std::string* result) override {
+  virtual Status GetTestDirectory(std::string* result) override {
     const char* env = getenv("TEST_TMPDIR");
     if (env && env[0] != '\0') {
       *result = env;
@@ -755,7 +757,8 @@ class PosixEnv : public Env {
     return Status::OK();
   }
 
-  Status GetThreadList(std::vector<ThreadStatus>* thread_list) override {
+  virtual Status GetThreadList(
+      std::vector<ThreadStatus>* thread_list) override {
     assert(thread_status_updater_);
     return thread_status_updater_->GetThreadList(thread_list);
   }
@@ -771,9 +774,12 @@ class PosixEnv : public Env {
     return gettid(tid);
   }
 
-  uint64_t GetThreadID() const override { return gettid(pthread_self()); }
+  virtual uint64_t GetThreadID() const override {
+    return gettid(pthread_self());
+  }
 
-  Status GetFreeSpace(const std::string& fname, uint64_t* free_space) override {
+  virtual Status GetFreeSpace(const std::string& fname,
+                              uint64_t* free_space) override {
     struct statvfs sbuf;
 
     if (statvfs(fname.c_str(), &sbuf) < 0) {
@@ -784,8 +790,8 @@ class PosixEnv : public Env {
     return Status::OK();
   }
 
-  Status NewLogger(const std::string& fname,
-                   std::shared_ptr<Logger>* result) override {
+  virtual Status NewLogger(const std::string& fname,
+                           std::shared_ptr<Logger>* result) override {
     FILE* f;
     {
       IOSTATS_TIMER_GUARD(open_nanos);
@@ -811,13 +817,13 @@ class PosixEnv : public Env {
     }
   }
 
-  uint64_t NowMicros() override {
+  virtual uint64_t NowMicros() override {
     struct timeval tv;
     gettimeofday(&tv, nullptr);
     return static_cast<uint64_t>(tv.tv_sec) * 1000000 + tv.tv_usec;
   }
 
-  uint64_t NowNanos() override {
+  virtual uint64_t NowNanos() override {
 #if defined(OS_LINUX) || defined(OS_FREEBSD) || defined(OS_AIX)
     struct timespec ts;
     clock_gettime(CLOCK_MONOTONIC, &ts);
@@ -837,7 +843,7 @@ class PosixEnv : public Env {
 #endif
   }
 
-  uint64_t NowCPUNanos() override {
+  virtual uint64_t NowCPUNanos() override {
 #if defined(OS_LINUX) || defined(OS_FREEBSD) || defined(OS_AIX) || \
     defined(__MACH__)
     struct timespec ts;
@@ -847,9 +853,9 @@ class PosixEnv : public Env {
     return 0;
   }
 
-  void SleepForMicroseconds(int micros) override { usleep(micros); }
+  virtual void SleepForMicroseconds(int micros) override { usleep(micros); }
 
-  Status GetHostName(char* name, uint64_t len) override {
+  virtual Status GetHostName(char* name, uint64_t len) override {
     int ret = gethostname(name, static_cast<size_t>(len));
     if (ret < 0) {
       if (errno == EFAULT || errno == EINVAL)
@@ -860,7 +866,7 @@ class PosixEnv : public Env {
     return Status::OK();
   }
 
-  Status GetCurrentTime(int64_t* unix_time) override {
+  virtual Status GetCurrentTime(int64_t* unix_time) override {
     time_t ret = time(nullptr);
     if (ret == (time_t) -1) {
       return IOError("GetCurrentTime", "", errno);
@@ -869,8 +875,8 @@ class PosixEnv : public Env {
     return Status::OK();
   }
 
-  Status GetAbsolutePath(const std::string& db_path,
-                         std::string* output_path) override {
+  virtual Status GetAbsolutePath(const std::string& db_path,
+                                 std::string* output_path) override {
     if (!db_path.empty() && db_path[0] == '/') {
       *output_path = db_path;
       return Status::OK();
@@ -887,28 +893,28 @@ class PosixEnv : public Env {
   }
 
   // Allow increasing the number of worker threads.
-  void SetBackgroundThreads(int num, Priority pri) override {
+  virtual void SetBackgroundThreads(int num, Priority pri) override {
     assert(pri >= Priority::BOTTOM && pri <= Priority::HIGH);
     thread_pools_[pri].SetBackgroundThreads(num);
   }
 
-  int GetBackgroundThreads(Priority pri) override {
+  virtual int GetBackgroundThreads(Priority pri) override {
     assert(pri >= Priority::BOTTOM && pri <= Priority::HIGH);
     return thread_pools_[pri].GetBackgroundThreads();
   }
 
-  Status SetAllowNonOwnerAccess(bool allow_non_owner_access) override {
+  virtual Status SetAllowNonOwnerAccess(bool allow_non_owner_access) override {
     allow_non_owner_access_ = allow_non_owner_access;
     return Status::OK();
   }
 
   // Allow increasing the number of worker threads.
-  void IncBackgroundThreadsIfNeeded(int num, Priority pri) override {
+  virtual void IncBackgroundThreadsIfNeeded(int num, Priority pri) override {
     assert(pri >= Priority::BOTTOM && pri <= Priority::HIGH);
     thread_pools_[pri].IncBackgroundThreadsIfNeeded(num);
   }
 
-  void LowerThreadPoolIOPriority(Priority pool = LOW) override {
+  virtual void LowerThreadPoolIOPriority(Priority pool = LOW) override {
     assert(pool >= Priority::BOTTOM && pool <= Priority::HIGH);
 #ifdef OS_LINUX
     thread_pools_[pool].LowerIOPriority();
@@ -917,7 +923,7 @@ class PosixEnv : public Env {
 #endif
   }
 
-  void LowerThreadPoolCPUPriority(Priority pool = LOW) override {
+  virtual void LowerThreadPoolCPUPriority(Priority pool = LOW) override {
     assert(pool >= Priority::BOTTOM && pool <= Priority::HIGH);
 #ifdef OS_LINUX
     thread_pools_[pool].LowerCPUPriority();
@@ -926,7 +932,7 @@ class PosixEnv : public Env {
 #endif
   }
 
-  std::string TimeToString(uint64_t secondsSince1970) override {
+  virtual std::string TimeToString(uint64_t secondsSince1970) override {
     const time_t seconds = (time_t)secondsSince1970;
     struct tm t;
     int maxsize = 64;

env/env_test.cc

@@ -118,7 +118,7 @@ class EnvPosixTestWithParam
     }
   }
 
-  ~EnvPosixTestWithParam() override { WaitThreadPoolsEmpty(); }
+  ~EnvPosixTestWithParam() { WaitThreadPoolsEmpty(); }
 };
 
 static void SetBool(void* ptr) {
@@ -1135,7 +1135,7 @@ TEST_P(EnvPosixTestWithParam, InvalidateCache) {
 class TestLogger : public Logger {
  public:
   using Logger::Logv;
-  void Logv(const char* format, va_list ap) override {
+  virtual void Logv(const char* format, va_list ap) override {
     log_count++;
 
     char new_format[550];
@@ -1217,7 +1217,7 @@ class TestLogger2 : public Logger {
  public:
   explicit TestLogger2(size_t max_log_size) : max_log_size_(max_log_size) {}
   using Logger::Logv;
-  void Logv(const char* format, va_list ap) override {
+  virtual void Logv(const char* format, va_list ap) override {
     char new_format[2000];
     std::fill_n(new_format, sizeof(new_format), '2');
     {
@@ -1466,7 +1466,7 @@ TEST_P(EnvPosixTestWithParam, WritableFileWrapper) {
     }
 
    public:
-    ~Base() override { inc(23); }
+    ~Base() { inc(23); }
   };
 
   class Wrapper : public WritableFileWrapper {
@@ -1689,15 +1689,15 @@ class TestEnv : public EnvWrapper {
    public:
     using Logger::Logv;
     TestLogger(TestEnv* env_ptr) : Logger() { env = env_ptr; }
-    ~TestLogger() override {
+    ~TestLogger() {
       if (!closed_) {
         CloseHelper();
       }
     }
-    void Logv(const char* /*format*/, va_list /*ap*/) override{};
+    virtual void Logv(const char* /*format*/, va_list /*ap*/) override{};
 
    protected:
-    Status CloseImpl() override { return CloseHelper(); }
+    virtual Status CloseImpl() override { return CloseHelper(); }
 
    private:
     Status CloseHelper() {
@@ -1711,8 +1711,8 @@ class TestEnv : public EnvWrapper {
 
   int GetCloseCount() { return close_count; }
 
-  Status NewLogger(const std::string& /*fname*/,
-                   std::shared_ptr<Logger>* result) override {
+  virtual Status NewLogger(const std::string& /*fname*/,
+                           std::shared_ptr<Logger>* result) {
     result->reset(new TestLogger(this));
     return Status::OK();
   }

env/io_posix.cc

@@ -83,14 +83,10 @@ size_t GetLogicalBufferSize(int __attribute__((__unused__)) fd) {
   if (!device_dir.empty() && device_dir.back() == '/') {
     device_dir.pop_back();
   }
-  // NOTE: sda3 and nvme0n1p1 do not have a `queue/` subdir, only the parent sda
-  // and nvme0n1 have it.
+  // NOTE: sda3 does not have a `queue/` subdir, only the parent sda has it.
   // $ ls -al '/sys/dev/block/8:3'
   // lrwxrwxrwx. 1 root root 0 Jun 26 01:38 /sys/dev/block/8:3 ->
   // ../../block/sda/sda3
-  // $ ls -al '/sys/dev/block/259:4'
-  // lrwxrwxrwx 1 root root 0 Jan 31 16:04 /sys/dev/block/259:4 ->
-  // ../../devices/pci0000:17/0000:17:00.0/0000:18:00.0/nvme/nvme0/nvme0n1/nvme0n1p1
   size_t parent_end = device_dir.rfind('/', device_dir.length() - 1);
   if (parent_end == std::string::npos) {
     return kDefaultPageSize;
@@ -99,11 +95,8 @@ size_t GetLogicalBufferSize(int __attribute__((__unused__)) fd) {
   if (parent_begin == std::string::npos) {
     return kDefaultPageSize;
   }
-  std::string parent =
-      device_dir.substr(parent_begin + 1, parent_end - parent_begin - 1);
-  std::string child = device_dir.substr(parent_end + 1, std::string::npos);
-  if (parent != "block" &&
-      (child.compare(0, 4, "nvme") || child.find('p') != std::string::npos)) {
+  if (device_dir.substr(parent_begin + 1, parent_end - parent_begin - 1) !=
+      "block") {
     device_dir = device_dir.substr(0, parent_end);
   }
   std::string fname = device_dir + "/queue/logical_block_size";

env/mock_env.cc

@@ -183,9 +183,9 @@ class MockSequentialFile : public SequentialFile {
     file_->Ref();
   }
 
-  ~MockSequentialFile() override { file_->Unref(); }
+  ~MockSequentialFile() { file_->Unref(); }
 
-  Status Read(size_t n, Slice* result, char* scratch) override {
+  virtual Status Read(size_t n, Slice* result, char* scratch) override {
     Status s = file_->Read(pos_, n, result, scratch);
     if (s.ok()) {
       pos_ += result->size();
@@ -193,7 +193,7 @@ class MockSequentialFile : public SequentialFile {
     return s;
   }
 
-  Status Skip(uint64_t n) override {
+  virtual Status Skip(uint64_t n) override {
     if (pos_ > file_->Size()) {
       return Status::IOError("pos_ > file_->Size()");
     }
@@ -214,10 +214,10 @@ class MockRandomAccessFile : public RandomAccessFile {
  public:
   explicit MockRandomAccessFile(MemFile* file) : file_(file) { file_->Ref(); }
 
-  ~MockRandomAccessFile() override { file_->Unref(); }
+  ~MockRandomAccessFile() { file_->Unref(); }
 
-  Status Read(uint64_t offset, size_t n, Slice* result,
-              char* scratch) const override {
+  virtual Status Read(uint64_t offset, size_t n, Slice* result,
+                      char* scratch) const override {
     return file_->Read(offset, n, result, scratch);
   }
 
@@ -229,22 +229,22 @@ class MockRandomRWFile : public RandomRWFile {
  public:
   explicit MockRandomRWFile(MemFile* file) : file_(file) { file_->Ref(); }
 
-  ~MockRandomRWFile() override { file_->Unref(); }
+  ~MockRandomRWFile() { file_->Unref(); }
 
-  Status Write(uint64_t offset, const Slice& data) override {
+  virtual Status Write(uint64_t offset, const Slice& data) override {
     return file_->Write(offset, data);
   }
 
-  Status Read(uint64_t offset, size_t n, Slice* result,
-              char* scratch) const override {
+  virtual Status Read(uint64_t offset, size_t n, Slice* result,
+                      char* scratch) const override {
     return file_->Read(offset, n, result, scratch);
   }
 
-  Status Close() override { return file_->Fsync(); }
+  virtual Status Close() override { return file_->Fsync(); }
 
-  Status Flush() override { return Status::OK(); }
+  virtual Status Flush() override { return Status::OK(); }
 
-  Status Sync() override { return file_->Fsync(); }
+  virtual Status Sync() override { return file_->Fsync(); }
 
  private:
   MemFile* file_;
@@ -257,9 +257,9 @@ class MockWritableFile : public WritableFile {
     file_->Ref();
   }
 
-  ~MockWritableFile() override { file_->Unref(); }
+  ~MockWritableFile() { file_->Unref(); }
 
-  Status Append(const Slice& data) override {
+  virtual Status Append(const Slice& data) override {
     size_t bytes_written = 0;
     while (bytes_written < data.size()) {
       auto bytes = RequestToken(data.size() - bytes_written);
@@ -271,17 +271,17 @@ class MockWritableFile : public WritableFile {
     }
     return Status::OK();
   }
-  Status Truncate(uint64_t size) override {
+  virtual Status Truncate(uint64_t size) override {
     file_->Truncate(static_cast<size_t>(size));
     return Status::OK();
   }
-  Status Close() override { return file_->Fsync(); }
+  virtual Status Close() override { return file_->Fsync(); }
 
-  Status Flush() override { return Status::OK(); }
+  virtual Status Flush() override { return Status::OK(); }
 
-  Status Sync() override { return file_->Fsync(); }
+  virtual Status Sync() override { return file_->Fsync(); }
 
-  uint64_t GetFileSize() override { return file_->Size(); }
+  virtual uint64_t GetFileSize() override { return file_->Size(); }
 
  private:
   inline size_t RequestToken(size_t bytes) {
@@ -299,7 +299,7 @@ class MockWritableFile : public WritableFile {
 
 class MockEnvDirectory : public Directory {
  public:
-  Status Fsync() override { return Status::OK(); }
+  virtual Status Fsync() override { return Status::OK(); }
 };
 
 class MockEnvFileLock : public FileLock {
@@ -330,9 +330,9 @@ class TestMemLogger : public Logger {
         last_flush_micros_(0),
         env_(env),
         flush_pending_(false) {}
-  ~TestMemLogger() override {}
+  virtual ~TestMemLogger() {}
 
-  void Flush() override {
+  virtual void Flush() override {
     if (flush_pending_) {
       flush_pending_ = false;
     }
@@ -340,7 +340,7 @@ class TestMemLogger : public Logger {
   }
 
   using Logger::Logv;
-  void Logv(const char* format, va_list ap) override {
+  virtual void Logv(const char* format, va_list ap) override {
     // We try twice: the first time with a fixed-size stack allocated buffer,
     // and the second time with a much larger dynamically allocated buffer.
     char buffer[500];

env/mock_env_test.cc

@@ -20,7 +20,9 @@ class MockEnvTest : public testing::Test {
   MockEnvTest()
       : env_(new MockEnv(Env::Default())) {
   }
-  ~MockEnvTest() override { delete env_; }
+  ~MockEnvTest() {
+    delete env_;
+  }
 };
 
 TEST_F(MockEnvTest, Corrupt) {