Minor changes to CuckooTableBuilder

Summary: - Copy the key and value to in-memory hash table during Add operation. Also modified cuckoo_table_reader_test to use this. - Store only the user_key in in-memory hash table if it is last level file. - Handle Carryover while chosing unused key in Finish() method in case unused key was never found before Finish() call. Test Plan: cuckoo_table_reader_test --enable_perf cuckoo_table_builder_test valgrind_check asan_check Reviewers: sdong, yhchiang, igor, ljin Reviewed By: ljin Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20715
Add DB::GetIntProperty() to return integer properties to be returned as integers
2026-07-07 14:47:40 +08:00 · 2014-07-28 17:14:25 -07:00 · 2014-07-28 16:55:57 -07:00 · 2014-07-28 15:27:58 -07:00 · 2014-07-28 12:27:08 -07:00 · 2014-07-28 12:22:37 -07:00
194 changed files with 15836 additions and 3789 deletions
@@ -10,9 +10,8 @@ the CLA and we can cross-check with your GitHub username.

 Complete your CLA here: <https://code.facebook.com/cla>

-If you don't have a Facebook account, we can send you a PDF that you can
-sign offline. Send us an e-mail or create a new github issue to
-request the CLA in PDF format.
+If you prefer to sign a paper copy, we can send you a PDF.  Send us an 
+e-mail or create a new github issue to request the CLA in PDF format.

 ## License

@@ -1,15 +1,55 @@
 # Rocksdb Change Log

+### Unreleased
+### New Features
+* Support Multiple DB paths in universal style compactions
+* Add feature of storing plain table index and bloom filter in SST file.
+
+### Public API changes
+* DBOptions.db_paths now is a vector of a DBPath structure which indicates both of path and target size
+* NewPlainTableFactory instead of bunch of parameters now accepts PlainTableOptions, which is defined in include/rocksdb/table.h
+* Moved include/utilities/*.h to include/rocksdb/utilities/*.h
+* Statistics APIs now take uint32_t as type instead of Tickers. Also make two access functions getTickerCount and histogramData const
+* Add DB property rocksdb.estimate-num-keys, estimated number of live keys in DB.
+* Add DB::GetIntProperty(), which returns DB properties that are integer as uint64_t.
+
+## 3.3.0 (7/10/2014)
+### New Features
+* Added JSON API prototype.
+* HashLinklist reduces performance outlier caused by skewed bucket by switching data in the bucket from linked list to skip list. Add parameter threshold_use_skiplist in NewHashLinkListRepFactory().
+* RocksDB is now able to reclaim storage space more effectively during the compaction process.  This is done by compensating the size of each deletion entry by the 2X average value size, which makes compaction to be triggerred by deletion entries more easily.
+* Add TimeOut API to write.  Now WriteOptions have a variable called timeout_hint_us.  With timeout_hint_us set to non-zero, any write associated with this timeout_hint_us may be aborted when it runs longer than the specified timeout_hint_us, and it is guaranteed that any write completes earlier than the specified time-out will not be aborted due to the time-out condition.
+* Add a rate_limiter option, which controls total throughput of flush and compaction. The throughput is specified in bytes/sec. Flush always has precedence over compaction when available bandwidth is constrained.
+
+### Public API changes
+* Removed NewTotalOrderPlainTableFactory because it is not used and implemented semantically incorrect.
+
+## 3.2.0 (06/20/2014)
+
+### Public API changes
+* We removed seek compaction as a concept from RocksDB because:
+1) It makes more sense for spinning disk workloads, while RocksDB is primarily designed for flash and memory,
+2) It added some complexity to the important code-paths,
+3) None of our internal customers were really using it.
+Because of that, Options::disable_seek_compaction is now obsolete. It is still a parameter in Options, so it does not break the build, but it does not have any effect. We plan to completely remove it at some point, so we ask users to please remove this option from your code base.
+* Add two paramters to NewHashLinkListRepFactory() for logging on too many entries in a hash bucket when flushing.
+* Added new option BlockBasedTableOptions::hash_index_allow_collision. When enabled, prefix hash index for block-based table will not store prefix and allow hash collision, reducing memory consumption.
+
+### New Features
+* PlainTable now supports a new key encoding: for keys of the same prefix, the prefix is only written once. It can be enabled through encoding_type paramter of NewPlainTableFactory()
+* Add AdaptiveTableFactory, which is used to convert from a DB of PlainTable to BlockBasedTabe, or vise versa. It can be created using NewAdaptiveTableFactory()
+
+### Performance Improvements
+* Tailing Iterator re-implemeted with ForwardIterator + Cascading Search Hint , see ~20% throughput improvement.
+
 ## 3.1.0 (05/21/2014)

 ### Public API changes
 * Replaced ColumnFamilyOptions::table_properties_collectors with ColumnFamilyOptions::table_properties_collector_factories
-* Add two paramters to NewHashLinkListRepFactory() for logging on too many entries in a hash bucket when flushing.

 ### New Features
 * Hash index for block-based table will be materialized and reconstructed more efficiently. Previously hash index is constructed by scanning the whole table during every table open.
 * FIFO compaction style
-* Add AdaptiveTableFactory, which is used to convert from a DB of PlainTable to BlockBasedTabe, or vise versa. It can be created using NewAdaptiveTableFactory()

 ## 3.0.0 (05/05/2014)

@@ -49,7 +49,7 @@ else
 	PLATFORM_CCFLAGS += $(JEMALLOC_INCLUDE) -DHAVE_JEMALLOC
 endif

-WARNING_FLAGS = -Wall -Werror
+WARNING_FLAGS = -Wall -Werror -Wsign-compare
 CFLAGS += $(WARNING_FLAGS) -I. -I./include $(PLATFORM_CCFLAGS) $(OPT)
 CXXFLAGS += $(WARNING_FLAGS) -I. -I./include $(PLATFORM_CXXFLAGS) $(OPT) -Woverloaded-virtual

@@ -69,6 +69,7 @@ VALGRIND_OPTS = --error-exitcode=$(VALGRIND_ERROR) --leak-check=full

 TESTS = \
 	db_test \
+	db_iter_test \
 	block_hash_index_test \
 	autovector_test \
 	column_family_test \
@@ -104,6 +105,9 @@ TESTS = \
 	stringappend_test \
 	ttl_test \
 	backupable_db_test \
+	document_db_test \
+	json_document_test \
+	spatial_db_test \
 	version_edit_test \
 	version_set_test \
 	file_indexer_test \
@@ -111,7 +115,11 @@ TESTS = \
 	deletefile_test \
 	table_test \
 	thread_local_test \
-        geodb_test
+	geodb_test \
+ 	rate_limiter_test \
+	cuckoo_table_builder_test \
+	options_test \
+	cuckoo_table_reader_test

 TOOLS = \
        sst_dump \
@@ -119,6 +127,7 @@ TOOLS = \
        db_stress \
        ldb \
 	db_repl_stress \
+  options_test \
 	blob_store_bench

 PROGRAMS = db_bench signal_test table_reader_bench log_and_apply_bench $(TOOLS)
@@ -146,7 +155,7 @@ SHARED = $(SHARED1)
 else
 # Update db.h if you change these.
 SHARED_MAJOR = 3
-SHARED_MINOR = 2
+SHARED_MINOR = 4
 SHARED1 = ${LIBNAME}.$(PLATFORM_SHARED_EXT)
 SHARED2 = $(SHARED1).$(SHARED_MAJOR)
 SHARED3 = $(SHARED1).$(SHARED_MAJOR).$(SHARED_MINOR)
@@ -319,6 +328,9 @@ crc32c_test: util/crc32c_test.o $(LIBOBJECTS) $(TESTHARNESS)
 db_test: db/db_test.o $(LIBOBJECTS) $(TESTHARNESS)
 	$(CXX) db/db_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS)

+db_iter_test: db/db_iter_test.o $(LIBOBJECTS) $(TESTHARNESS)
+	$(CXX) db/db_iter_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS)
+
 log_write_bench: util/log_write_bench.o $(LIBOBJECTS) $(TESTHARNESS)
 	$(CXX) util/log_write_bench.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) -pg

@@ -343,6 +355,15 @@ prefix_test: db/prefix_test.o $(LIBOBJECTS) $(TESTHARNESS)
 backupable_db_test: utilities/backupable/backupable_db_test.o $(LIBOBJECTS) $(TESTHARNESS)
 	$(CXX) utilities/backupable/backupable_db_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@  $(LDFLAGS) $(COVERAGEFLAGS)

+document_db_test: utilities/document/document_db_test.o $(LIBOBJECTS) $(TESTHARNESS)
+	$(CXX) utilities/document/document_db_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@  $(LDFLAGS) $(COVERAGEFLAGS)
+
+json_document_test: utilities/document/json_document_test.o $(LIBOBJECTS) $(TESTHARNESS)
+	$(CXX) utilities/document/json_document_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@  $(LDFLAGS) $(COVERAGEFLAGS)
+
+spatial_db_test: utilities/spatialdb/spatial_db_test.o $(LIBOBJECTS) $(TESTHARNESS)
+	$(CXX) utilities/spatialdb/spatial_db_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@  $(LDFLAGS) $(COVERAGEFLAGS)
+
 ttl_test: utilities/ttl/ttl_test.o $(LIBOBJECTS) $(TESTHARNESS)
 	$(CXX) utilities/ttl/ttl_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@  $(LDFLAGS) $(COVERAGEFLAGS)

@@ -352,6 +373,9 @@ dbformat_test: db/dbformat_test.o $(LIBOBJECTS) $(TESTHARNESS)
 env_test: util/env_test.o $(LIBOBJECTS) $(TESTHARNESS)
 	$(CXX) util/env_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS)

+rate_limiter_test: util/rate_limiter_test.o $(LIBOBJECTS) $(TESTHARNESS)
+	$(CXX) util/rate_limiter_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS)
+
 filename_test: db/filename_test.o $(LIBOBJECTS) $(TESTHARNESS)
 	$(CXX) db/filename_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS)

@@ -394,6 +418,15 @@ deletefile_test: db/deletefile_test.o $(LIBOBJECTS) $(TESTHARNESS)
 geodb_test: utilities/geodb/geodb_test.o $(LIBOBJECTS) $(TESTHARNESS)
 	$(CXX) utilities/geodb/geodb_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS)

+cuckoo_table_builder_test: table/cuckoo_table_builder_test.o $(LIBOBJECTS) $(TESTHARNESS)
+	$(CXX) table/cuckoo_table_builder_test.o  $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS)
+
+cuckoo_table_reader_test: table/cuckoo_table_reader_test.o $(LIBOBJECTS) $(TESTHARNESS) $(BENCHHARNESS)
+	$(CXX) table/cuckoo_table_reader_test.o  $(LIBOBJECTS) $(TESTHARNESS) $(BENCHHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS)
+
+options_test: util/options_test.o $(LIBOBJECTS) $(TESTHARNESS)
+	$(CXX) util/options_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS)
+
 $(MEMENVLIBRARY) : $(MEMENVOBJECTS)
 	rm -f $@
 	$(AR) -rs $@ $(MEMENVOBJECTS)
@@ -428,18 +461,20 @@ ldb: tools/ldb.o $(LIBOBJECTS)

 JNI_NATIVE_SOURCES = ./java/rocksjni/*.cc
 JAVA_INCLUDE = -I$(JAVA_HOME)/include/ -I$(JAVA_HOME)/include/linux
-ROCKSDBJNILIB = ./java/librocksdbjni.so
+ROCKSDBJNILIB = librocksdbjni.so
+ROCKSDB_JAR = rocksdbjni.jar

 ifeq ($(PLATFORM), OS_MACOSX)
-ROCKSDBJNILIB = ./java/librocksdbjni.jnilib
+ROCKSDBJNILIB = librocksdbjni.jnilib
 JAVA_INCLUDE = -I/System/Library/Frameworks/JavaVM.framework/Headers/
 endif

-rocksdbjava: clean
+rocksdbjava:
 	OPT="-fPIC -DNDEBUG -O2" $(MAKE) $(LIBRARY) -j32
 	cd java;$(MAKE) java;
-	rm -f $(ROCKSDBJNILIB)
-	$(CXX) $(CXXFLAGS) -I./java/. $(JAVA_INCLUDE) -shared -fPIC -o $(ROCKSDBJNILIB) $(JNI_NATIVE_SOURCES) $(LIBOBJECTS) $(LDFLAGS) $(COVERAGEFLAGS)
+	rm -f ./java/$(ROCKSDBJNILIB)
+	$(CXX) $(CXXFLAGS) -I./java/. $(JAVA_INCLUDE) -shared -fPIC -o ./java/$(ROCKSDBJNILIB) $(JNI_NATIVE_SOURCES) $(LIBOBJECTS) $(JAVA_LDFLAGS) $(COVERAGEFLAGS)
+	cd java;jar -cf $(ROCKSDB_JAR) org/rocksdb/*.class org/rocksdb/util/*.class HISTORY*.md $(ROCKSDBJNILIB)

 jclean:
 	cd java;$(MAKE) clean;
@@ -8,7 +8,7 @@ and Jeff Dean (jeff@google.com)

 This code is a library that forms the core building block for a fast
 key value server, especially suited for storing data on flash drives.
-It has an Log-Structured-Merge-Database (LSM) design with flexible tradeoffs
+It has a Log-Structured-Merge-Database (LSM) design with flexible tradeoffs
 between Write-Amplification-Factor (WAF), Read-Amplification-Factor (RAF)
 and Space-Amplification-Factor (SAF). It has multi-threaded compactions,
 making it specially suitable for storing multiple terabytes of data in a
@@ -7,6 +7,7 @@
 #   CC                          C Compiler path
 #   CXX                         C++ Compiler path
 #   PLATFORM_LDFLAGS            Linker flags
+#   JAVA_LDFLAGS                Linker flags for RocksDBJava
 #   PLATFORM_SHARED_EXT         Extension for shared libraries
 #   PLATFORM_SHARED_LDFLAGS     Flags for building shared library
 #   PLATFORM_SHARED_CFLAGS      Flags for compiling objects for shared library
@@ -21,6 +22,7 @@
 #       -DLEVELDB_PLATFORM_NOATOMIC if it is not
 #       -DSNAPPY                    if the Snappy library is present
 #       -DLZ4                       if the LZ4 library is present
+#       -DNUMA                      if the NUMA library is present
 #
 # Using gflags in rocksdb:
 # Our project depends on gflags, which requires users to take some extra steps
@@ -156,6 +158,8 @@ case "$TARGET_OS" in
        exit 1
 esac

+JAVA_LDFLAGS="$PLATFORM_LDFLAGS"
+
 if test -z "$DO_NOT_RUN_BUILD_DETECT_VERSION"; then
  "$PWD/build_tools/build_detect_version"
 fi
@@ -214,6 +218,7 @@ EOF
    if [ "$?" = 0 ]; then
        COMMON_FLAGS="$COMMON_FLAGS -DSNAPPY"
        PLATFORM_LDFLAGS="$PLATFORM_LDFLAGS -lsnappy"
+        JAVA_LDFLAGS="$JAVA_LDFLAGS -lsnappy"
    fi


@@ -249,6 +254,7 @@ EOF
    if [ "$?" = 0 ]; then
        COMMON_FLAGS="$COMMON_FLAGS -DZLIB"
        PLATFORM_LDFLAGS="$PLATFORM_LDFLAGS -lz"
+        JAVA_LDFLAGS="$JAVA_LDFLAGS -lz"
    fi

    # Test whether bzip library is installed
@@ -259,6 +265,7 @@ EOF
    if [ "$?" = 0 ]; then
        COMMON_FLAGS="$COMMON_FLAGS -DBZIP2"
        PLATFORM_LDFLAGS="$PLATFORM_LDFLAGS -lbz2"
+        JAVA_LDFLAGS="$JAVA_LDFLAGS -lbz2"
    fi

    # Test whether lz4 library is installed
@@ -270,6 +277,19 @@ EOF
    if [ "$?" = 0 ]; then
        COMMON_FLAGS="$COMMON_FLAGS -DLZ4"
        PLATFORM_LDFLAGS="$PLATFORM_LDFLAGS -llz4"
+        JAVA_LDFLAGS="$JAVA_LDFLAGS -llz4"
+    fi
+
+    # Test whether numa is available
+    $CXX $CFLAGS -x c++ - -o /dev/null -lnuma 2>/dev/null  <<EOF
+      #include <numa.h>
+      #inlcude <numaif.h>
+      int main() {}
+EOF
+    if [ "$?" = 0 ]; then
+        COMMON_FLAGS="$COMMON_FLAGS -DNUMA"
+        PLATFORM_LDFLAGS="$PLATFORM_LDFLAGS -lnuma"
+        JAVA_LDFLAGS="$JAVA_LDFLAGS -lnuma"
    fi

    # Test whether tcmalloc is available
@@ -278,6 +298,7 @@ EOF
 EOF
    if [ "$?" = 0 ]; then
        PLATFORM_LDFLAGS="$PLATFORM_LDFLAGS -ltcmalloc"
+        JAVA_LDFLAGS="$JAVA_LDFLAGS -ltcmalloc"
    fi
 fi

@@ -294,6 +315,7 @@ if test "$USE_HDFS"; then
  HDFS_LDFLAGS="$HDFS_LDFLAGS -ldl -lverify -ljava -ljvm"
  COMMON_FLAGS="$COMMON_FLAGS $HDFS_CCFLAGS"
  PLATFORM_LDFLAGS="$PLATFORM_LDFLAGS $HDFS_LDFLAGS"
+  JAVA_LDFLAGS="$JAVA_LDFLAGS $HDFS_LDFLAGS"
 fi

 # if Intel SSE instruction set is supported, set USE_SSE=" -msse -msse4.2 "
@@ -308,6 +330,7 @@ echo "CC=$CC" >> "$OUTPUT"
 echo "CXX=$CXX" >> "$OUTPUT"
 echo "PLATFORM=$PLATFORM" >> "$OUTPUT"
 echo "PLATFORM_LDFLAGS=$PLATFORM_LDFLAGS" >> "$OUTPUT"
+echo "JAVA_LDFLAGS=$JAVA_LDFLAGS" >> "$OUTPUT"
 echo "VALGRIND_VER=$VALGRIND_VER" >> "$OUTPUT"
 echo "PLATFORM_CCFLAGS=$PLATFORM_CCFLAGS" >> "$OUTPUT"
 echo "PLATFORM_CXXFLAGS=$PLATFORM_CXXFLAGS" >> "$OUTPUT"
@@ -55,22 +55,27 @@ GFLAGS_LIBS=" $TOOLCHAIN_LIB_BASE/gflags/gflags-1.6/c3f970a/lib/libgflags.a"
 JEMALLOC_INCLUDE=" -I $TOOLCHAIN_LIB_BASE/jemalloc/jemalloc-3.4.1/4d53c6f/include/"
 JEMALLOC_LIB=" -Wl,--whole-archive $TOOLCHAIN_LIB_BASE/jemalloc/jemalloc-3.4.1/4d53c6f/lib/libjemalloc.a"

+# location of numa
+NUMA_REV=829d10dac0230f99cd7e1778869d2adf3da24b65
+NUMA_INCLUDE=" -I /mnt/gvfs/third-party2/numa/$NUMA_REV/2.0.8/gcc-4.8.1-glibc-2.17/c3f970a/include/"
+NUMA_LIB=" /mnt/gvfs/third-party2/numa/$NUMA_REV/2.0.8/gcc-4.8.1-glibc-2.17/c3f970a/lib/libnuma.a"
+
 # use Intel SSE support for checksum calculations
 export USE_SSE=" -msse -msse4.2 "

 CC="$TOOLCHAIN_EXECUTABLES/gcc/gcc-4.8.1/cc6c9dc/bin/gcc"
-CXX="$TOOLCHAIN_EXECUTABLES/gcc/gcc-4.8.1/cc6c9dc/bin/g++ $JINCLUDE $SNAPPY_INCLUDE $ZLIB_INCLUDE $BZIP_INCLUDE $LZ4_INCLUDE $GFLAGS_INCLUDE"
+CXX="$TOOLCHAIN_EXECUTABLES/gcc/gcc-4.8.1/cc6c9dc/bin/g++ $JINCLUDE $SNAPPY_INCLUDE $ZLIB_INCLUDE $BZIP_INCLUDE $LZ4_INCLUDE $GFLAGS_INCLUDE $NUMA_INCLUDE"
 AR=$TOOLCHAIN_EXECUTABLES/binutils/binutils-2.21.1/da39a3e/bin/ar
 RANLIB=$TOOLCHAIN_EXECUTABLES/binutils/binutils-2.21.1/da39a3e/bin/ranlib

 CFLAGS="-B$TOOLCHAIN_EXECUTABLES/binutils/binutils-2.21.1/da39a3e/bin/gold -m64 -mtune=generic"
 CFLAGS+=" $LIBGCC_INCLUDE $GLIBC_INCLUDE"
 CFLAGS+=" -DROCKSDB_PLATFORM_POSIX -DROCKSDB_ATOMIC_PRESENT -DROCKSDB_FALLOCATE_PRESENT"
-CFLAGS+=" -DSNAPPY -DGFLAGS=google -DZLIB -DBZIP2 -DLZ4"
+CFLAGS+=" -DSNAPPY -DGFLAGS=google -DZLIB -DBZIP2 -DLZ4 -DNUMA"

 EXEC_LDFLAGS="-Wl,--dynamic-linker,/usr/local/fbcode/gcc-4.8.1-glibc-2.17/lib/ld.so"
 EXEC_LDFLAGS+=" -Wl,--no-whole-archive $TOOLCHAIN_LIB_BASE/libunwind/libunwind-1.0.1/675d945/lib/libunwind.a"
-EXEC_LDFLAGS+=" $HDFSLIB $SNAPPY_LIBS $ZLIB_LIBS $BZIP_LIBS $LZ4_LIBS $GFLAGS_LIBS"
+EXEC_LDFLAGS+=" $HDFSLIB $SNAPPY_LIBS $ZLIB_LIBS $BZIP_LIBS $LZ4_LIBS $GFLAGS_LIBS $NUMA_LIB"

 PLATFORM_LDFLAGS="$LIBGCC_LIBS $GLIBC_LIBS "

@@ -22,4 +22,4 @@ echo ""
 echo "-----------------------------------------------------------------------------"
 echo "|                         Installation Completed                            |"
 echo "-----------------------------------------------------------------------------"
-echo "Please run `. ~/bash_profile` to be able to compile with gflags"
+echo "Please run \`. ~/.bash_profile\` to be able to compile with gflags"
@@ -109,7 +109,6 @@ make release
    --cache_numshardbits=6 \
    --table_cache_numshardbits=4 \
    --open_files=55000 \
-    --disable_seek_compaction=1 \
    --statistics=1 \
    --histogram=1 \
    --disable_data_sync=1 \
@@ -129,7 +128,6 @@ make release
    --cache_numshardbits=6 \
    --table_cache_numshardbits=4 \
    --open_files=55000 \
-    --disable_seek_compaction=1 \
    --use_tailing_iterator=1 \
    --statistics=1 \
    --histogram=1 \
@@ -150,7 +148,6 @@ make release
    --cache_numshardbits=6 \
    --table_cache_numshardbits=4 \
    --open_files=55000 \
-    --disable_seek_compaction=1 \
    --statistics=1 \
    --histogram=1 \
    --disable_data_sync=1 \
@@ -172,7 +169,6 @@ make release
    --table_cache_numshardbits=4 \
    --write_buffer_size=1000000000 \
    --open_files=55000 \
-    --disable_seek_compaction=1 \
    --statistics=1 \
    --histogram=1 \
    --disable_data_sync=1 \
@@ -231,7 +227,6 @@ make release
    --cache_numshardbits=6 \
    --table_cache_numshardbits=4 \
    --open_files=55000 \
-    --disable_seek_compaction=1 \
    --disable_auto_compactions=1 \
    --statistics=1 \
    --histogram=1 \
@@ -254,7 +249,6 @@ make release
    --cache_numshardbits=6 \
    --table_cache_numshardbits=4 \
    --open_files=55000 \
-    --disable_seek_compaction=1 \
    --statistics=1 \
    --histogram=1 \
    --disable_data_sync=1 \
@@ -274,7 +268,6 @@ make release
    --table_cache_numshardbits=4 \
    --write_buffer_size=1000000000 \
    --open_files=55000 \
-    --disable_seek_compaction=1 \
    --statistics=1 \
    --histogram=1 \
    --disable_data_sync=1 \
@@ -291,7 +284,6 @@ common_in_mem_args="--db=/dev/shm/rocksdb \
    --value_size=100 \
    --compression_type=none \
    --compression_ratio=1 \
-    --disable_seek_compaction=1 \
    --hard_rate_limit=2 \
    --write_buffer_size=134217728 \
    --max_write_buffer_number=4 \
@@ -40,7 +40,8 @@ Status BuildTable(const std::string& dbname, Env* env, const Options& options,
                  const InternalKeyComparator& internal_comparator,
                  const SequenceNumber newest_snapshot,
                  const SequenceNumber earliest_seqno_in_memtable,
-                  const CompressionType compression) {
+                  const CompressionType compression,
+                  const Env::IOPriority io_priority) {
  Status s;
  meta->fd.file_size = 0;
  meta->smallest_seqno = meta->largest_seqno = 0;
@@ -54,13 +55,15 @@ Status BuildTable(const std::string& dbname, Env* env, const Options& options,
    purge = false;
  }

-  std::string fname = TableFileName(dbname, meta->fd.GetNumber());
+  std::string fname = TableFileName(options.db_paths, meta->fd.GetNumber(),
+                                    meta->fd.GetPathId());
  if (iter->Valid()) {
    unique_ptr<WritableFile> file;
    s = env->NewWritableFile(fname, &file, soptions);
    if (!s.ok()) {
      return s;
    }
+    file->SetIOPriority(io_priority);

    TableBuilder* builder =
        NewTableBuilder(options, internal_comparator, file.get(), compression);
@@ -7,6 +7,7 @@
 // found in the LICENSE file. See the AUTHORS file for names of contributors.
 #pragma once
 #include "rocksdb/comparator.h"
+#include "rocksdb/env.h"
 #include "rocksdb/status.h"
 #include "rocksdb/types.h"
 #include "rocksdb/options.h"
@@ -40,6 +41,7 @@ extern Status BuildTable(const std::string& dbname, Env* env,
                         const InternalKeyComparator& internal_comparator,
                         const SequenceNumber newest_snapshot,
                         const SequenceNumber earliest_seqno_in_memtable,
-                         const CompressionType compression);
+                         const CompressionType compression,
+                         const Env::IOPriority io_priority = Env::IO_HIGH);

 }  // namespace rocksdb
@@ -31,10 +31,16 @@
 #include "rocksdb/table.h"

 using rocksdb::Cache;
+using rocksdb::ColumnFamilyDescriptor;
+using rocksdb::ColumnFamilyHandle;
+using rocksdb::ColumnFamilyOptions;
 using rocksdb::CompactionFilter;
+using rocksdb::CompactionFilterFactory;
+using rocksdb::CompactionOptionsFIFO;
 using rocksdb::Comparator;
 using rocksdb::CompressionType;
 using rocksdb::DB;
+using rocksdb::DBOptions;
 using rocksdb::Env;
 using rocksdb::InfoLogLevel;
 using rocksdb::FileLock;
@@ -68,6 +74,7 @@ struct rocksdb_iterator_t        { Iterator*         rep; };
 struct rocksdb_writebatch_t      { WriteBatch        rep; };
 struct rocksdb_snapshot_t        { const Snapshot*   rep; };
 struct rocksdb_flushoptions_t    { FlushOptions      rep; };
+struct rocksdb_fifo_compaction_options_t { CompactionOptionsFIFO rep; };
 struct rocksdb_readoptions_t     { ReadOptions       rep; };
 struct rocksdb_writeoptions_t    { WriteOptions      rep; };
 struct rocksdb_options_t         { Options           rep; };
@@ -78,6 +85,11 @@ struct rocksdb_filelock_t        { FileLock*         rep; };
 struct rocksdb_logger_t          { shared_ptr<Logger>  rep; };
 struct rocksdb_cache_t           { shared_ptr<Cache>   rep; };
 struct rocksdb_livefiles_t       { std::vector<LiveFileMetaData> rep; };
+struct rocksdb_column_family_handle_t  { ColumnFamilyHandle* rep; };
+
+struct rocksdb_compactionfiltercontext_t {
+  CompactionFilter::Context rep;
+};

 struct rocksdb_compactionfilter_t : public CompactionFilter {
  void* state_;
@@ -122,6 +134,26 @@ struct rocksdb_compactionfilter_t : public CompactionFilter {
  }
 };

+struct rocksdb_compactionfilterfactory_t : public CompactionFilterFactory {
+  void* state_;
+  void (*destructor_)(void*);
+  rocksdb_compactionfilter_t* (*create_compaction_filter_)(
+      void*, rocksdb_compactionfiltercontext_t* context);
+  const char* (*name_)(void*);
+
+  virtual ~rocksdb_compactionfilterfactory_t() { (*destructor_)(state_); }
+
+  virtual std::unique_ptr<CompactionFilter> CreateCompactionFilter(
+      const CompactionFilter::Context& context) {
+    rocksdb_compactionfiltercontext_t ccontext;
+    ccontext.rep = context;
+    CompactionFilter* cf = (*create_compaction_filter_)(state_, &ccontext);
+    return std::unique_ptr<CompactionFilter>(cf);
+  }
+
+  virtual const char* Name() const { return (*name_)(state_); }
+};
+
 struct rocksdb_comparator_t : public Comparator {
  void* state_;
  void (*destructor_)(void*);
@@ -394,6 +426,120 @@ void rocksdb_close(rocksdb_t* db) {
  delete db;
 }

+rocksdb_t* rocksdb_open_column_families(
+    const rocksdb_options_t* db_options,
+    const char* name,
+    int num_column_families,
+    const char** column_family_names,
+    const rocksdb_options_t** column_family_options,
+    rocksdb_column_family_handle_t** column_family_handles,
+    char** errptr) {
+  std::vector<ColumnFamilyDescriptor> column_families;
+  for (int i = 0; i < num_column_families; i++) {
+    column_families.push_back(ColumnFamilyDescriptor(
+        std::string(column_family_names[i]),
+        ColumnFamilyOptions(column_family_options[i]->rep)));
+  }
+
+  DB* db;
+  std::vector<ColumnFamilyHandle*> handles;
+  if (SaveError(errptr, DB::Open(DBOptions(db_options->rep),
+          std::string(name), column_families, &handles, &db))) {
+    return nullptr;
+  }
+
+  for (size_t i = 0; i < handles.size(); i++) {
+    rocksdb_column_family_handle_t* c_handle = new rocksdb_column_family_handle_t;
+    c_handle->rep = handles[i];
+    column_family_handles[i] = c_handle;
+  }
+  rocksdb_t* result = new rocksdb_t;
+  result->rep = db;
+  return result;
+}
+
+rocksdb_t* rocksdb_open_for_read_only_column_families(
+    const rocksdb_options_t* db_options,
+    const char* name,
+    int num_column_families,
+    const char** column_family_names,
+    const rocksdb_options_t** column_family_options,
+    rocksdb_column_family_handle_t** column_family_handles,
+    unsigned char error_if_log_file_exist,
+    char** errptr) {
+  std::vector<ColumnFamilyDescriptor> column_families;
+  for (int i = 0; i < num_column_families; i++) {
+    column_families.push_back(ColumnFamilyDescriptor(
+        std::string(column_family_names[i]),
+        ColumnFamilyOptions(column_family_options[i]->rep)));
+  }
+
+  DB* db;
+  std::vector<ColumnFamilyHandle*> handles;
+  if (SaveError(errptr, DB::OpenForReadOnly(DBOptions(db_options->rep),
+          std::string(name), column_families, &handles, &db, error_if_log_file_exist))) {
+    return nullptr;
+  }
+
+  for (size_t i = 0; i < handles.size(); i++) {
+    rocksdb_column_family_handle_t* c_handle = new rocksdb_column_family_handle_t;
+    c_handle->rep = handles[i];
+    column_family_handles[i] = c_handle;
+  }
+  rocksdb_t* result = new rocksdb_t;
+  result->rep = db;
+  return result;
+}
+
+char** rocksdb_list_column_families(
+    const rocksdb_options_t* options,
+    const char* name,
+    size_t* lencfs,
+    char** errptr) {
+  std::vector<std::string> fams;
+  SaveError(errptr,
+      DB::ListColumnFamilies(DBOptions(options->rep),
+        std::string(name), &fams));
+
+  *lencfs = fams.size();
+  char** column_families = static_cast<char**>(malloc(sizeof(char*) * fams.size()));
+  for (size_t i = 0; i < fams.size(); i++) {
+    column_families[i] = strdup(fams[i].c_str());
+  }
+  return column_families;
+}
+
+void rocksdb_list_column_families_destroy(char** list, size_t len) {
+  for (size_t i = 0; i < len; ++i) {
+    free(list[i]);
+  }
+  free(list);
+}
+
+rocksdb_column_family_handle_t* rocksdb_create_column_family(
+    rocksdb_t* db,
+    const rocksdb_options_t* column_family_options,
+    const char* column_family_name,
+    char** errptr) {
+  rocksdb_column_family_handle_t* handle = new rocksdb_column_family_handle_t;
+  SaveError(errptr,
+      db->rep->CreateColumnFamily(ColumnFamilyOptions(column_family_options->rep),
+        std::string(column_family_name), &(handle->rep)));
+  return handle;
+}
+
+void rocksdb_drop_column_family(
+    rocksdb_t* db,
+    rocksdb_column_family_handle_t* handle,
+    char** errptr) {
+  SaveError(errptr, db->rep->DropColumnFamily(handle->rep));
+}
+
+void rocksdb_column_family_handle_destroy(rocksdb_column_family_handle_t* handle) {
+  delete handle->rep;
+  delete handle;
+}
+
 void rocksdb_put(
    rocksdb_t* db,
    const rocksdb_writeoptions_t* options,
@@ -404,6 +550,18 @@ void rocksdb_put(
            db->rep->Put(options->rep, Slice(key, keylen), Slice(val, vallen)));
 }

+void rocksdb_put_cf(
+    rocksdb_t* db,
+    const rocksdb_writeoptions_t* options,
+    rocksdb_column_family_handle_t* column_family,
+    const char* key, size_t keylen,
+    const char* val, size_t vallen,
+    char** errptr) {
+  SaveError(errptr,
+            db->rep->Put(options->rep, column_family->rep,
+              Slice(key, keylen), Slice(val, vallen)));
+}
+
 void rocksdb_delete(
    rocksdb_t* db,
    const rocksdb_writeoptions_t* options,
@@ -412,6 +570,16 @@ void rocksdb_delete(
  SaveError(errptr, db->rep->Delete(options->rep, Slice(key, keylen)));
 }

+void rocksdb_delete_cf(
+    rocksdb_t* db,
+    const rocksdb_writeoptions_t* options,
+    rocksdb_column_family_handle_t* column_family,
+    const char* key, size_t keylen,
+    char** errptr) {
+  SaveError(errptr, db->rep->Delete(options->rep, column_family->rep,
+        Slice(key, keylen)));
+}
+
 void rocksdb_merge(
    rocksdb_t* db,
    const rocksdb_writeoptions_t* options,
@@ -422,6 +590,18 @@ void rocksdb_merge(
            db->rep->Merge(options->rep, Slice(key, keylen), Slice(val, vallen)));
 }

+void rocksdb_merge_cf(
+    rocksdb_t* db,
+    const rocksdb_writeoptions_t* options,
+    rocksdb_column_family_handle_t* column_family,
+    const char* key, size_t keylen,
+    const char* val, size_t vallen,
+    char** errptr) {
+  SaveError(errptr,
+            db->rep->Merge(options->rep, column_family->rep,
+              Slice(key, keylen), Slice(val, vallen)));
+}
+
 void rocksdb_write(
    rocksdb_t* db,
    const rocksdb_writeoptions_t* options,
@@ -451,6 +631,29 @@ char* rocksdb_get(
  return result;
 }

+char* rocksdb_get_cf(
+    rocksdb_t* db,
+    const rocksdb_readoptions_t* options,
+    rocksdb_column_family_handle_t* column_family,
+    const char* key, size_t keylen,
+    size_t* vallen,
+    char** errptr) {
+  char* result = nullptr;
+  std::string tmp;
+  Status s = db->rep->Get(options->rep, column_family->rep,
+      Slice(key, keylen), &tmp);
+  if (s.ok()) {
+    *vallen = tmp.size();
+    result = CopyString(tmp);
+  } else {
+    *vallen = 0;
+    if (!s.IsNotFound()) {
+      SaveError(errptr, s);
+    }
+  }
+  return result;
+}
+
 rocksdb_iterator_t* rocksdb_create_iterator(
    rocksdb_t* db,
    const rocksdb_readoptions_t* options) {
@@ -459,6 +662,15 @@ rocksdb_iterator_t* rocksdb_create_iterator(
  return result;
 }

+rocksdb_iterator_t* rocksdb_create_iterator_cf(
+    rocksdb_t* db,
+    const rocksdb_readoptions_t* options,
+    rocksdb_column_family_handle_t* column_family) {
+  rocksdb_iterator_t* result = new rocksdb_iterator_t;
+  result->rep = db->rep->NewIterator(options->rep, column_family->rep);
+  return result;
+}
+
 const rocksdb_snapshot_t* rocksdb_create_snapshot(
    rocksdb_t* db) {
  rocksdb_snapshot_t* result = new rocksdb_snapshot_t;
@@ -485,6 +697,19 @@ char* rocksdb_property_value(
  }
 }

+char* rocksdb_property_value_cf(
+    rocksdb_t* db,
+    rocksdb_column_family_handle_t* column_family,
+    const char* propname) {
+  std::string tmp;
+  if (db->rep->GetProperty(column_family->rep, Slice(propname), &tmp)) {
+    // We use strdup() since we expect human readable output.
+    return strdup(tmp.c_str());
+  } else {
+    return nullptr;
+  }
+}
+
 void rocksdb_approximate_sizes(
    rocksdb_t* db,
    int num_ranges,
@@ -500,6 +725,22 @@ void rocksdb_approximate_sizes(
  delete[] ranges;
 }

+void rocksdb_approximate_sizes_cf(
+    rocksdb_t* db,
+    rocksdb_column_family_handle_t* column_family,
+    int num_ranges,
+    const char* const* range_start_key, const size_t* range_start_key_len,
+    const char* const* range_limit_key, const size_t* range_limit_key_len,
+    uint64_t* sizes) {
+  Range* ranges = new Range[num_ranges];
+  for (int i = 0; i < num_ranges; i++) {
+    ranges[i].start = Slice(range_start_key[i], range_start_key_len[i]);
+    ranges[i].limit = Slice(range_limit_key[i], range_limit_key_len[i]);
+  }
+  db->rep->GetApproximateSizes(column_family->rep, ranges, num_ranges, sizes);
+  delete[] ranges;
+}
+
 void rocksdb_delete_file(
    rocksdb_t* db,
    const char* name) {
@@ -524,6 +765,19 @@ void rocksdb_compact_range(
      (limit_key ? (b = Slice(limit_key, limit_key_len), &b) : nullptr));
 }

+void rocksdb_compact_range_cf(
+    rocksdb_t* db,
+    rocksdb_column_family_handle_t* column_family,
+    const char* start_key, size_t start_key_len,
+    const char* limit_key, size_t limit_key_len) {
+  Slice a, b;
+  db->rep->CompactRange(
+      column_family->rep,
+      // Pass nullptr Slice if corresponding "const char*" is nullptr
+      (start_key ? (a = Slice(start_key, start_key_len), &a) : nullptr),
+      (limit_key ? (b = Slice(limit_key, limit_key_len), &b) : nullptr));
+}
+
 void rocksdb_flush(
    rocksdb_t* db,
    const rocksdb_flushoptions_t* options,
@@ -607,6 +861,13 @@ rocksdb_writebatch_t* rocksdb_writebatch_create() {
  return new rocksdb_writebatch_t;
 }

+rocksdb_writebatch_t* rocksdb_writebatch_create_from(const char* rep,
+                                                     size_t size) {
+  rocksdb_writebatch_t* b = new rocksdb_writebatch_t;
+  b->rep = WriteBatch(std::string(rep, size));
+  return b;
+}
+
 void rocksdb_writebatch_destroy(rocksdb_writebatch_t* b) {
  delete b;
 }
@@ -626,6 +887,14 @@ void rocksdb_writebatch_put(
  b->rep.Put(Slice(key, klen), Slice(val, vlen));
 }

+void rocksdb_writebatch_put_cf(
+    rocksdb_writebatch_t* b,
+    rocksdb_column_family_handle_t* column_family,
+    const char* key, size_t klen,
+    const char* val, size_t vlen) {
+  b->rep.Put(column_family->rep, Slice(key, klen), Slice(val, vlen));
+}
+
 void rocksdb_writebatch_merge(
    rocksdb_writebatch_t* b,
    const char* key, size_t klen,
@@ -633,12 +902,27 @@ void rocksdb_writebatch_merge(
  b->rep.Merge(Slice(key, klen), Slice(val, vlen));
 }

+void rocksdb_writebatch_merge_cf(
+    rocksdb_writebatch_t* b,
+    rocksdb_column_family_handle_t* column_family,
+    const char* key, size_t klen,
+    const char* val, size_t vlen) {
+  b->rep.Merge(column_family->rep, Slice(key, klen), Slice(val, vlen));
+}
+
 void rocksdb_writebatch_delete(
    rocksdb_writebatch_t* b,
    const char* key, size_t klen) {
  b->rep.Delete(Slice(key, klen));
 }

+void rocksdb_writebatch_delete_cf(
+    rocksdb_writebatch_t* b,
+    rocksdb_column_family_handle_t* column_family,
+    const char* key, size_t klen) {
+  b->rep.Delete(column_family->rep, Slice(key, klen));
+}
+
 void rocksdb_writebatch_iterate(
    rocksdb_writebatch_t* b,
    void* state,
@@ -676,12 +960,38 @@ void rocksdb_options_destroy(rocksdb_options_t* options) {
  delete options;
 }

+void rocksdb_options_increase_parallelism(
+    rocksdb_options_t* opt, int total_threads) {
+  opt->rep.IncreaseParallelism(total_threads);
+}
+
+void rocksdb_options_optimize_for_point_lookup(
+    rocksdb_options_t* opt) {
+  opt->rep.OptimizeForPointLookup();
+}
+
+void rocksdb_options_optimize_level_style_compaction(
+    rocksdb_options_t* opt, uint64_t memtable_memory_budget) {
+  opt->rep.OptimizeLevelStyleCompaction(memtable_memory_budget);
+}
+
+void rocksdb_options_optimize_universal_style_compaction(
+    rocksdb_options_t* opt, uint64_t memtable_memory_budget) {
+  opt->rep.OptimizeUniversalStyleCompaction(memtable_memory_budget);
+}
+
 void rocksdb_options_set_compaction_filter(
    rocksdb_options_t* opt,
    rocksdb_compactionfilter_t* filter) {
  opt->rep.compaction_filter = filter;
 }

+void rocksdb_options_set_compaction_filter_factory(
+    rocksdb_options_t* opt, rocksdb_compactionfilterfactory_t* factory) {
+  opt->rep.compaction_filter_factory =
+      std::shared_ptr<CompactionFilterFactory>(factory);
+}
+
 void rocksdb_options_set_comparator(
    rocksdb_options_t* opt,
    rocksdb_comparator_t* cmp) {
@@ -705,6 +1015,11 @@ void rocksdb_options_set_create_if_missing(
  opt->rep.create_if_missing = v;
 }

+void rocksdb_options_set_create_missing_column_families(
+    rocksdb_options_t* opt, unsigned char v) {
+  opt->rep.create_missing_column_families = v;
+}
+
 void rocksdb_options_set_error_if_exists(
    rocksdb_options_t* opt, unsigned char v) {
  opt->rep.error_if_exists = v;
@@ -1113,9 +1428,13 @@ void rocksdb_options_set_plain_table_factory(
    double hash_table_ratio, size_t index_sparseness) {
  static rocksdb::TableFactory* factory = 0;
  if (!factory) {
-    factory = rocksdb::NewPlainTableFactory(
-        user_key_len, bloom_bits_per_key,
-        hash_table_ratio, index_sparseness);
+    rocksdb::PlainTableOptions options;
+    options.user_key_len = user_key_len;
+    options.bloom_bits_per_key = bloom_bits_per_key;
+    options.hash_table_ratio = hash_table_ratio;
+    options.index_sparseness = index_sparseness;
+
+    factory = rocksdb::NewPlainTableFactory(options);
  }
  opt->rep.table_factory.reset(factory);
 }
@@ -1158,6 +1477,12 @@ void rocksdb_options_set_universal_compaction_options(rocksdb_options_t *opt, ro
  opt->rep.compaction_options_universal = *(uco->rep);
 }

+void rocksdb_options_set_fifo_compaction_options(
+    rocksdb_options_t* opt,
+    rocksdb_fifo_compaction_options_t* fifo) {
+  opt->rep.compaction_options_fifo = fifo->rep;
+}
+
 /*
 TODO:
 DB::OpenForReadOnly
@@ -1196,6 +1521,35 @@ void rocksdb_compactionfilter_destroy(rocksdb_compactionfilter_t* filter) {
  delete filter;
 }

+unsigned char rocksdb_compactionfiltercontext_is_full_compaction(
+    rocksdb_compactionfiltercontext_t* context) {
+  return context->rep.is_full_compaction;
+}
+
+unsigned char rocksdb_compactionfiltercontext_is_manual_compaction(
+    rocksdb_compactionfiltercontext_t* context) {
+  return context->rep.is_manual_compaction;
+}
+
+rocksdb_compactionfilterfactory_t* rocksdb_compactionfilterfactory_create(
+    void* state, void (*destructor)(void*),
+    rocksdb_compactionfilter_t* (*create_compaction_filter)(
+        void*, rocksdb_compactionfiltercontext_t* context),
+    const char* (*name)(void*)) {
+  rocksdb_compactionfilterfactory_t* result =
+      new rocksdb_compactionfilterfactory_t;
+  result->state_ = state;
+  result->destructor_ = destructor;
+  result->create_compaction_filter_ = create_compaction_filter;
+  result->name_ = name;
+  return result;
+}
+
+void rocksdb_compactionfilterfactory_destroy(
+    rocksdb_compactionfilterfactory_t* factory) {
+  delete factory;
+}
+
 rocksdb_comparator_t* rocksdb_comparator_create(
    void* state,
    void (*destructor)(void*),
@@ -1487,6 +1841,22 @@ void rocksdb_universal_compaction_options_destroy(
  delete uco;
 }

+rocksdb_fifo_compaction_options_t* rocksdb_fifo_compaction_options_create() {
+  rocksdb_fifo_compaction_options_t* result = new rocksdb_fifo_compaction_options_t;
+  result->rep =  CompactionOptionsFIFO();
+  return result;
+}
+
+void rocksdb_fifo_compaction_options_set_max_table_files_size(
+    rocksdb_fifo_compaction_options_t* fifo_opts, uint64_t size) {
+  fifo_opts->rep.max_table_files_size = size;
+}
+
+void rocksdb_fifo_compaction_options_destroy(
+    rocksdb_fifo_compaction_options_t* fifo_opts) {
+  delete fifo_opts;
+}
+
 void rocksdb_options_set_min_level_to_compress(rocksdb_options_t* opt, int level) {
  if (level >= 0) {
    assert(level <= opt->rep.num_levels);
@@ -75,6 +75,22 @@ static void CheckGet(
  Free(&val);
 }

+static void CheckGetCF(
+    rocksdb_t* db,
+    const rocksdb_readoptions_t* options,
+    rocksdb_column_family_handle_t* handle,
+    const char* key,
+    const char* expected) {
+  char* err = NULL;
+  size_t val_len;
+  char* val;
+  val = rocksdb_get_cf(db, options, handle, key, strlen(key), &val_len, &err);
+  CheckNoError(err);
+  CheckEqual(expected, val, val_len);
+  Free(&val);
+}
+
+
 static void CheckIter(rocksdb_iterator_t* iter,
                      const char* key, const char* val) {
  size_t len;
@@ -176,6 +192,39 @@ static unsigned char CFilterFilter(void* arg, int level, const char* key,
  return 0;
 }

+static void CFilterFactoryDestroy(void* arg) {}
+static const char* CFilterFactoryName(void* arg) { return "foo"; }
+static rocksdb_compactionfilter_t* CFilterCreate(
+    void* arg, rocksdb_compactionfiltercontext_t* context) {
+  return rocksdb_compactionfilter_create(NULL, CFilterDestroy, CFilterFilter,
+                                         CFilterName);
+}
+
+static rocksdb_t* CheckCompaction(rocksdb_t* db, rocksdb_options_t* options,
+                                  rocksdb_readoptions_t* roptions,
+                                  rocksdb_writeoptions_t* woptions) {
+  char* err = NULL;
+  db = rocksdb_open(options, dbname, &err);
+  CheckNoError(err);
+  rocksdb_put(db, woptions, "foo", 3, "foovalue", 8, &err);
+  CheckNoError(err);
+  CheckGet(db, roptions, "foo", "foovalue");
+  rocksdb_put(db, woptions, "bar", 3, "barvalue", 8, &err);
+  CheckNoError(err);
+  CheckGet(db, roptions, "bar", "barvalue");
+  rocksdb_put(db, woptions, "baz", 3, "bazvalue", 8, &err);
+  CheckNoError(err);
+  CheckGet(db, roptions, "baz", "bazvalue");
+
+  // Force compaction
+  rocksdb_compact_range(db, NULL, 0, NULL, 0);
+  // should have filtered bar, but not foo
+  CheckGet(db, roptions, "foo", "foovalue");
+  CheckGet(db, roptions, "bar", NULL);
+  CheckGet(db, roptions, "baz", "newbazvalue");
+  return db;
+}
+
 // Custom merge operator
 static void MergeOperatorDestroy(void* arg) { }
 static const char* MergeOperatorName(void* arg) {
@@ -299,6 +348,24 @@ int main(int argc, char** argv) {
    rocksdb_writebatch_destroy(wb);
  }

+  StartPhase("writebatch_rep");
+  {
+    rocksdb_writebatch_t* wb1 = rocksdb_writebatch_create();
+    rocksdb_writebatch_put(wb1, "baz", 3, "d", 1);
+    rocksdb_writebatch_put(wb1, "quux", 4, "e", 1);
+    rocksdb_writebatch_delete(wb1, "quux", 4);
+    size_t repsize1 = 0;
+    const char* rep = rocksdb_writebatch_data(wb1, &repsize1);
+    rocksdb_writebatch_t* wb2 = rocksdb_writebatch_create_from(rep, repsize1);
+    CheckCondition(rocksdb_writebatch_count(wb1) ==
+                   rocksdb_writebatch_count(wb2));
+    size_t repsize2 = 0;
+    CheckCondition(
+        memcmp(rep, rocksdb_writebatch_data(wb2, &repsize2), repsize1) == 0);
+    rocksdb_writebatch_destroy(wb1);
+    rocksdb_writebatch_destroy(wb2);
+  }
+
  StartPhase("iter");
  {
    rocksdb_iterator_t* iter = rocksdb_create_iterator(db, roptions);
@@ -431,6 +498,8 @@ int main(int argc, char** argv) {

  StartPhase("compaction_filter");
  {
+    rocksdb_options_t* options = rocksdb_options_create();
+    rocksdb_options_set_create_if_missing(options, 1);
    rocksdb_compactionfilter_t* cfilter;
    cfilter = rocksdb_compactionfilter_create(NULL, CFilterDestroy,
                                              CFilterFilter, CFilterName);
@@ -438,26 +507,28 @@ int main(int argc, char** argv) {
    rocksdb_close(db);
    rocksdb_destroy_db(options, dbname, &err);
    rocksdb_options_set_compaction_filter(options, cfilter);
-    db = rocksdb_open(options, dbname, &err);
-    CheckNoError(err);
-    rocksdb_put(db, woptions, "foo", 3, "foovalue", 8, &err);
-    CheckNoError(err);
-    CheckGet(db, roptions, "foo", "foovalue");
-    rocksdb_put(db, woptions, "bar", 3, "barvalue", 8, &err);
-    CheckNoError(err);
-    CheckGet(db, roptions, "bar", "barvalue");
-    rocksdb_put(db, woptions, "baz", 3, "bazvalue", 8, &err);
-    CheckNoError(err);
-    CheckGet(db, roptions, "baz", "bazvalue");
-
-    // Force compaction
-    rocksdb_compact_range(db, NULL, 0, NULL, 0);
-    // should have filtered bar, but not foo
-    CheckGet(db, roptions, "foo", "foovalue");
-    CheckGet(db, roptions, "bar", NULL);
-    CheckGet(db, roptions, "baz", "newbazvalue");
+    db = CheckCompaction(db, options, roptions, woptions);

+    rocksdb_options_set_compaction_filter(options, NULL);
    rocksdb_compactionfilter_destroy(cfilter);
+    rocksdb_options_destroy(options);
+  }
+
+  StartPhase("compaction_filter_factory");
+  {
+    rocksdb_options_t* options = rocksdb_options_create();
+    rocksdb_options_set_create_if_missing(options, 1);
+    rocksdb_compactionfilterfactory_t* factory;
+    factory = rocksdb_compactionfilterfactory_create(
+        NULL, CFilterFactoryDestroy, CFilterCreate, CFilterFactoryName);
+    // Create new database
+    rocksdb_close(db);
+    rocksdb_destroy_db(options, dbname, &err);
+    rocksdb_options_set_compaction_filter_factory(options, factory);
+    db = CheckCompaction(db, options, roptions, woptions);
+
+    rocksdb_options_set_compaction_filter_factory(options, NULL);
+    rocksdb_options_destroy(options);
  }

  StartPhase("merge_operator");
@@ -486,16 +557,93 @@ int main(int argc, char** argv) {

  }

+  StartPhase("columnfamilies");
+  {
+    rocksdb_close(db);
+    rocksdb_destroy_db(options, dbname, &err);
+    CheckNoError(err)
+
+    rocksdb_options_t* db_options = rocksdb_options_create();
+    rocksdb_options_set_create_if_missing(db_options, 1);
+    db = rocksdb_open(db_options, dbname, &err);
+    CheckNoError(err)
+    rocksdb_column_family_handle_t* cfh;
+    cfh = rocksdb_create_column_family(db, db_options, "cf1", &err);
+    rocksdb_column_family_handle_destroy(cfh);
+    CheckNoError(err);
+    rocksdb_close(db);
+
+    size_t cflen;
+    char** column_fams = rocksdb_list_column_families(db_options, dbname, &cflen, &err);
+    CheckNoError(err);
+    CheckEqual("default", column_fams[0], 7);
+    CheckEqual("cf1", column_fams[1], 3);
+    CheckCondition(cflen == 2);
+    rocksdb_list_column_families_destroy(column_fams, cflen);
+
+    rocksdb_options_t* cf_options = rocksdb_options_create();
+
+    const char* cf_names[2] = {"default", "cf1"};
+    const rocksdb_options_t* cf_opts[2] = {cf_options, cf_options};
+    rocksdb_column_family_handle_t* handles[2];
+    db = rocksdb_open_column_families(db_options, dbname, 2, cf_names, cf_opts, handles, &err);
+    CheckNoError(err);
+
+    rocksdb_put_cf(db, woptions, handles[1], "foo", 3, "hello", 5, &err);
+    CheckNoError(err);
+
+    CheckGetCF(db, roptions, handles[1], "foo", "hello");
+
+    rocksdb_delete_cf(db, woptions, handles[1], "foo", 3, &err);
+    CheckNoError(err);
+
+    CheckGetCF(db, roptions, handles[1], "foo", NULL);
+
+    rocksdb_writebatch_t* wb = rocksdb_writebatch_create();
+    rocksdb_writebatch_put_cf(wb, handles[1], "baz", 3, "a", 1);
+    rocksdb_writebatch_clear(wb);
+    rocksdb_writebatch_put_cf(wb, handles[1], "bar", 3, "b", 1);
+    rocksdb_writebatch_put_cf(wb, handles[1], "box", 3, "c", 1);
+    rocksdb_writebatch_delete_cf(wb, handles[1], "bar", 3);
+    rocksdb_write(db, woptions, wb, &err);
+    CheckNoError(err);
+    CheckGetCF(db, roptions, handles[1], "baz", NULL);
+    CheckGetCF(db, roptions, handles[1], "bar", NULL);
+    CheckGetCF(db, roptions, handles[1], "box", "c");
+    rocksdb_writebatch_destroy(wb);
+
+    rocksdb_iterator_t* iter = rocksdb_create_iterator_cf(db, roptions, handles[1]);
+    CheckCondition(!rocksdb_iter_valid(iter));
+    rocksdb_iter_seek_to_first(iter);
+    CheckCondition(rocksdb_iter_valid(iter));
+
+    int i;
+    for (i = 0; rocksdb_iter_valid(iter) != 0; rocksdb_iter_next(iter)) {
+      i++;
+    }
+    CheckCondition(i == 1);
+    rocksdb_iter_get_error(iter, &err);
+    CheckNoError(err);
+    rocksdb_iter_destroy(iter);
+
+    rocksdb_drop_column_family(db, handles[1], &err);
+    CheckNoError(err);
+    for (i = 0; i < 2; i++) {
+      rocksdb_column_family_handle_destroy(handles[i]);
+    }
+    rocksdb_close(db);
+    rocksdb_destroy_db(options, dbname, &err);
+    rocksdb_options_destroy(db_options);
+    rocksdb_options_destroy(cf_options);
+  }
+
  StartPhase("prefix");
  {
    // Create new database
-    rocksdb_close(db);
-    rocksdb_destroy_db(options, dbname, &err);
-
    rocksdb_filterpolicy_t* policy = rocksdb_filterpolicy_create_bloom(10);
    rocksdb_options_set_filter_policy(options, policy);
    rocksdb_options_set_prefix_extractor(options, rocksdb_slicetransform_create_fixed_prefix(3));
-    rocksdb_options_set_hash_skip_list_rep(options, 50000, 4, 4);
+    rocksdb_options_set_hash_skip_list_rep(options, 5000, 4, 4);
    rocksdb_options_set_plain_table_factory(options, 4, 10, 0.75, 16);

    db = rocksdb_open(options, dbname, &err);
@@ -533,6 +681,7 @@ int main(int argc, char** argv) {
    rocksdb_filterpolicy_destroy(policy);
  }

+
  StartPhase("cleanup");
  rocksdb_close(db);
  rocksdb_options_destroy(options);
@@ -222,10 +222,9 @@ ColumnFamilyData::ColumnFamilyData(const std::string& dbname, uint32_t id,

  // if dummy_versions is nullptr, then this is a dummy column family.
  if (dummy_versions != nullptr) {
-    internal_stats_.reset(new InternalStats(
-        options_.num_levels, db_options->env, db_options->statistics.get()));
-    table_cache_.reset(
-        new TableCache(dbname, &options_, storage_options, table_cache));
+    internal_stats_.reset(
+        new InternalStats(options_.num_levels, db_options->env, this));
+    table_cache_.reset(new TableCache(&options_, storage_options, table_cache));
    if (options_.compaction_style == kCompactionStyleUniversal) {
      compaction_picker_.reset(
          new UniversalCompactionPicker(&options_, &internal_comparator_));
@@ -243,6 +242,8 @@ ColumnFamilyData::ColumnFamilyData(const std::string& dbname, uint32_t id,
    const ColumnFamilyOptions* cf_options = &options_;
    cf_options->Dump(options_.info_log.get());
  }
+
+  RecalculateWriteStallConditions();
 }

 // DB mutex held
@@ -295,6 +296,35 @@ ColumnFamilyData::~ColumnFamilyData() {
  }
 }

+void ColumnFamilyData::RecalculateWriteStallConditions() {
+  need_wait_for_num_memtables_ =
+    (imm()->size() == options()->max_write_buffer_number - 1);
+
+  if (current_ != nullptr) {
+    need_wait_for_num_level0_files_ =
+      (current_->NumLevelFiles(0) >= options()->level0_stop_writes_trigger);
+  } else {
+    need_wait_for_num_level0_files_ = false;
+  }
+
+  RecalculateWriteStallRateLimitsConditions();
+}
+
+void ColumnFamilyData::RecalculateWriteStallRateLimitsConditions() {
+  if (current_ != nullptr) {
+    exceeds_hard_rate_limit_ =
+        (options()->hard_rate_limit > 1.0 &&
+         current_->MaxCompactionScore() > options()->hard_rate_limit);
+
+    exceeds_soft_rate_limit_ =
+        (options()->soft_rate_limit > 0.0 &&
+         current_->MaxCompactionScore() > options()->soft_rate_limit);
+  } else {
+    exceeds_hard_rate_limit_ = false;
+    exceeds_soft_rate_limit_ = false;
+  }
+}
+
 const EnvOptions* ColumnFamilyData::soptions() const {
  return &(column_family_set_->storage_options_);
 }
@@ -316,15 +346,19 @@ void ColumnFamilyData::CreateNewMemtable() {
 }

 Compaction* ColumnFamilyData::PickCompaction(LogBuffer* log_buffer) {
-  return compaction_picker_->PickCompaction(current_, log_buffer);
+  auto result = compaction_picker_->PickCompaction(current_, log_buffer);
+  RecalculateWriteStallRateLimitsConditions();
+  return result;
 }

 Compaction* ColumnFamilyData::CompactRange(int input_level, int output_level,
+                                           uint32_t output_path_id,
                                           const InternalKey* begin,
                                           const InternalKey* end,
                                           InternalKey** compaction_end) {
  return compaction_picker_->CompactRange(current_, input_level, output_level,
-                                          begin, end, compaction_end);
+                                          output_path_id, begin, end,
+                                          compaction_end);
 }

 SuperVersion* ColumnFamilyData::GetReferencedSuperVersion(
@@ -420,6 +454,9 @@ SuperVersion* ColumnFamilyData::InstallSuperVersion(
  if (column_family_set_->db_options_->allow_thread_local) {
    ResetThreadLocalSuperVersions();
  }
+
+  RecalculateWriteStallConditions();
+
  if (old_superversion != nullptr && old_superversion->Unref()) {
    old_superversion->Cleanup();
    return old_superversion;  // will let caller delete outside of mutex
@@ -185,7 +185,8 @@ class ColumnFamilyData {
  // See documentation in compaction_picker.h
  Compaction* PickCompaction(LogBuffer* log_buffer);
  Compaction* CompactRange(int input_level, int output_level,
-                           const InternalKey* begin, const InternalKey* end,
+                           uint32_t output_path_id, const InternalKey* begin,
+                           const InternalKey* end,
                           InternalKey** compaction_end);

  CompactionPicker* compaction_picker() { return compaction_picker_.get(); }
@@ -229,6 +230,22 @@ class ColumnFamilyData {
    return need_slowdown_for_num_level0_files_;
  }

+  bool NeedWaitForNumLevel0Files() const {
+    return need_wait_for_num_level0_files_;
+  }
+
+  bool NeedWaitForNumMemtables() const {
+    return need_wait_for_num_memtables_;
+  }
+
+  bool ExceedsSoftRateLimit() const {
+    return exceeds_soft_rate_limit_;
+  }
+
+  bool ExceedsHardRateLimit() const {
+    return exceeds_hard_rate_limit_;
+  }
+
 private:
  friend class ColumnFamilySet;
  ColumnFamilyData(const std::string& dbname, uint32_t id,
@@ -238,6 +255,14 @@ class ColumnFamilyData {
                   const EnvOptions& storage_options,
                   ColumnFamilySet* column_family_set);

+  // Recalculate some small conditions, which are changed only during
+  // compaction, adding new memtable and/or
+  // recalculation of compaction score. These values are used in
+  // DBImpl::MakeRoomForWrite function to decide, if it need to make
+  // a write stall
+  void RecalculateWriteStallConditions();
+  void RecalculateWriteStallRateLimitsConditions();
+
  uint32_t id_;
  const std::string name_;
  Version* dummy_versions_;  // Head of circular doubly-linked list of versions.
@@ -282,6 +307,22 @@ class ColumnFamilyData {
  // we have too many level 0 files
  bool need_slowdown_for_num_level0_files_;

+  // These 4 variables are updated only after compaction,
+  // adding new memtable, flushing memtables to files
+  // and/or add recalculation of compaction score.
+  // That's why theirs values are cached in ColumnFamilyData.
+  // Recalculation is made by RecalculateWriteStallConditions and
+  // RecalculateWriteStallRateLimitsConditions function. They are used
+  // in DBImpl::MakeRoomForWrite function to decide, if it need
+  // to sleep during write operation
+  bool need_wait_for_num_memtables_;
+
+  bool need_wait_for_num_level0_files_;
+
+  bool exceeds_hard_rate_limit_;
+
+  bool exceeds_soft_rate_limit_;
+
  // An object that keeps all the compaction stats
  // and picks the next compaction
  std::unique_ptr<CompactionPicker> compaction_picker_;
@@ -749,7 +749,6 @@ TEST(ColumnFamilyTest, DifferentCompactionStyles) {
  default_cf.filter_policy = nullptr;
  default_cf.no_block_cache = true;
  default_cf.source_compaction_factor = 100;
-  default_cf.disable_seek_compaction = false;

  one.compaction_style = kCompactionStyleUniversal;
  // trigger compaction if there are >= 4 files
@@ -764,33 +763,6 @@ TEST(ColumnFamilyTest, DifferentCompactionStyles) {

  Reopen({default_cf, one, two});

-  // SETUP column family "default" - test read compaction
-  ASSERT_EQ("", FilesPerLevel(0));
-  PutRandomData(0, 1, 4096);
-  ASSERT_OK(Flush(0));
-  ASSERT_EQ("0,0,1", FilesPerLevel(0));
-  // write 8MB
-  PutRandomData(0, 2000, 4096);
-  ASSERT_OK(Flush(0));
-  // clear levels 0 and 1
-  dbfull()->TEST_CompactRange(0, nullptr, nullptr, handles_[0]);
-  dbfull()->TEST_CompactRange(1, nullptr, nullptr, handles_[0]);
-  ASSERT_EQ(NumTableFilesAtLevel(0, 0), 0);
-  ASSERT_EQ(NumTableFilesAtLevel(1, 0), 0);
-  // write some new keys into level 0 and 1
-  PutRandomData(0, 1024, 512);
-  ASSERT_OK(Flush(0));
-  WaitForCompaction();
-  PutRandomData(0, 10, 512);
-  ASSERT_OK(Flush(0));
-  // remember number of files in each level
-  int l1 = NumTableFilesAtLevel(0, 0);
-  int l2 = NumTableFilesAtLevel(1, 0);
-  int l3 = NumTableFilesAtLevel(2, 0);
-  ASSERT_NE(l1, 0);
-  ASSERT_NE(l2, 0);
-  ASSERT_NE(l3, 0);
-
  // SETUP column family "one" -- universal style
  for (int i = 0; i < one.level0_file_num_compaction_trigger - 1; ++i) {
    PutRandomData(1, 11, 10000);
@@ -805,12 +777,6 @@ TEST(ColumnFamilyTest, DifferentCompactionStyles) {
    ASSERT_EQ(std::to_string(i + 1), FilesPerLevel(2));
  }

-  // TRIGGER compaction "default"
-  // read a bunch of times, trigger read compaction
-  for (int i = 0; i < 200000; ++i) {
-    Get(0, std::to_string(i));
-  }
-
  // TRIGGER compaction "one"
  PutRandomData(1, 12, 10000);

@@ -820,13 +786,6 @@ TEST(ColumnFamilyTest, DifferentCompactionStyles) {
  // WAIT for compactions
  WaitForCompaction();

-  // VERIFY compaction "default"
-  // verify that the number of files have decreased
-  // in some level, indicating that there was a compaction
-  ASSERT_TRUE(NumTableFilesAtLevel(0, 0) < l1 ||
-              NumTableFilesAtLevel(1, 0) < l2 ||
-              NumTableFilesAtLevel(2, 0) < l3);
-
  // VERIFY compaction "one"
  ASSERT_EQ("1", FilesPerLevel(1));

@@ -898,6 +857,7 @@ TEST(ColumnFamilyTest, NewIteratorsTest) {
 TEST(ColumnFamilyTest, ReadOnlyDBTest) {
  Open();
  CreateColumnFamiliesAndReopen({"one", "two", "three", "four"});
+  ASSERT_OK(Put(0, "a", "b"));
  ASSERT_OK(Put(1, "foo", "bla"));
  ASSERT_OK(Put(2, "foo", "blabla"));
  ASSERT_OK(Put(3, "foo", "blablabla"));
@@ -911,6 +871,29 @@ TEST(ColumnFamilyTest, ReadOnlyDBTest) {
  ASSERT_EQ("bla", Get(1, "foo"));
  ASSERT_EQ("blablablabla", Get(2, "foo"));

+
+  // test newiterators
+  {
+    std::vector<Iterator*> iterators;
+    ASSERT_OK(db_->NewIterators(ReadOptions(), handles_, &iterators));
+    for (auto it : iterators) {
+      it->SeekToFirst();
+    }
+    ASSERT_EQ(IterStatus(iterators[0]), "a->b");
+    ASSERT_EQ(IterStatus(iterators[1]), "foo->bla");
+    ASSERT_EQ(IterStatus(iterators[2]), "foo->blablablabla");
+    for (auto it : iterators) {
+      it->Next();
+    }
+    ASSERT_EQ(IterStatus(iterators[0]), "(invalid)");
+    ASSERT_EQ(IterStatus(iterators[1]), "(invalid)");
+    ASSERT_EQ(IterStatus(iterators[2]), "(invalid)");
+
+    for (auto it : iterators) {
+      delete it;
+    }
+  }
+
  Close();
  // can't open dropped column family
  Status s = OpenReadOnly({"default", "one", "two"});
@@ -26,20 +26,22 @@ static uint64_t TotalFileSize(const std::vector<FileMetaData*>& files) {
  return sum;
 }

-Compaction::Compaction(Version* input_version, int level, int out_level,
+Compaction::Compaction(Version* input_version, int start_level, int out_level,
                       uint64_t target_file_size,
                       uint64_t max_grandparent_overlap_bytes,
-                       bool seek_compaction, bool enable_compression,
+                       uint32_t output_path_id,
+                       CompressionType output_compression, bool seek_compaction,
                       bool deletion_compaction)
-    : level_(level),
-      out_level_(out_level),
+    : start_level_(start_level),
+      output_level_(out_level),
      max_output_file_size_(target_file_size),
      max_grandparent_overlap_bytes_(max_grandparent_overlap_bytes),
      input_version_(input_version),
      number_levels_(input_version_->NumberLevels()),
      cfd_(input_version_->cfd_),
+      output_path_id_(output_path_id),
+      output_compression_(output_compression),
      seek_compaction_(seek_compaction),
-      enable_compression_(enable_compression),
      deletion_compaction_(deletion_compaction),
      grandparent_index_(0),
      seen_key_(false),
@@ -59,6 +61,12 @@ Compaction::Compaction(Version* input_version, int level, int out_level,
  for (int i = 0; i < number_levels_; i++) {
    level_ptrs_[i] = 0;
  }
+  int num_levels = output_level_ - start_level_ + 1;
+  input_levels_.resize(num_levels);
+  inputs_.resize(num_levels);
+  for (int i = 0; i < num_levels; ++i) {
+    inputs_[i].level = start_level_ + i;
+  }
 }

 Compaction::~Compaction() {
@@ -73,43 +81,50 @@ Compaction::~Compaction() {
  }
 }

+void Compaction::GenerateFileLevels() {
+  input_levels_.resize(num_input_levels());
+  for (int which = 0; which < num_input_levels(); which++) {
+    DoGenerateFileLevel(&input_levels_[which], inputs_[which].files, &arena_);
+  }
+}
+
 bool Compaction::IsTrivialMove() const {
  // Avoid a move if there is lots of overlapping grandparent data.
  // Otherwise, the move could create a parent file that will require
  // a very expensive merge later on.
-  // If level_== out_level_, the purpose is to force compaction filter to be
-  // applied to that level, and thus cannot be a trivia move.
-  return (level_ != out_level_ &&
+  // If start_level_== output_level_, the purpose is to force compaction
+  // filter to be applied to that level, and thus cannot be a trivia move.
+  return (start_level_ != output_level_ &&
+          num_input_levels() == 2 &&
          num_input_files(0) == 1 &&
          num_input_files(1) == 0 &&
          TotalFileSize(grandparents_) <= max_grandparent_overlap_bytes_);
 }

-bool Compaction::IsDeletionCompaction() const { return deletion_compaction_; }
-
 void Compaction::AddInputDeletions(VersionEdit* edit) {
-  for (int which = 0; which < 2; which++) {
+  for (int which = 0; which < num_input_levels(); which++) {
    for (size_t i = 0; i < inputs_[which].size(); i++) {
-      edit->DeleteFile(level_ + which, inputs_[which][i]->fd.GetNumber());
+      edit->DeleteFile(level(which), inputs_[which][i]->fd.GetNumber());
    }
  }
 }

-bool Compaction::IsBaseLevelForKey(const Slice& user_key) {
+bool Compaction::KeyNotExistsBeyondOutputLevel(const Slice& user_key) {
  assert(cfd_->options()->compaction_style != kCompactionStyleFIFO);
  if (cfd_->options()->compaction_style == kCompactionStyleUniversal) {
    return bottommost_level_;
  }
  // Maybe use binary search to find right entry instead of linear search?
  const Comparator* user_cmp = cfd_->user_comparator();
-  for (int lvl = level_ + 2; lvl < number_levels_; lvl++) {
+  for (int lvl = output_level_ + 1; lvl < number_levels_; lvl++) {
    const std::vector<FileMetaData*>& files = input_version_->files_[lvl];
    for (; level_ptrs_[lvl] < files.size(); ) {
      FileMetaData* f = files[level_ptrs_[lvl]];
      if (user_cmp->Compare(user_key, f->largest.user_key()) <= 0) {
        // We've advanced far enough
        if (user_cmp->Compare(user_key, f->smallest.user_key()) >= 0) {
-          // Key falls in this file's range, so definitely not base level
+          // Key falls in this file's range, so definitely
+          // exists beyond output level
          return false;
        }
        break;
@@ -147,19 +162,18 @@ bool Compaction::ShouldStopBefore(const Slice& internal_key) {
 }

 // Mark (or clear) each file that is being compacted
-void Compaction::MarkFilesBeingCompacted(bool value) {
-  for (int i = 0; i < 2; i++) {
-    std::vector<FileMetaData*> v = inputs_[i];
+void Compaction::MarkFilesBeingCompacted(bool mark_as_compacted) {
+  for (int i = 0; i < num_input_levels(); i++) {
    for (unsigned int j = 0; j < inputs_[i].size(); j++) {
-      assert(value ? !inputs_[i][j]->being_compacted :
-                      inputs_[i][j]->being_compacted);
-      inputs_[i][j]->being_compacted = value;
+      assert(mark_as_compacted ? !inputs_[i][j]->being_compacted :
+                                  inputs_[i][j]->being_compacted);
+      inputs_[i][j]->being_compacted = mark_as_compacted;
    }
  }
 }

 // Is this compaction producing files at the bottommost level?
-void Compaction::SetupBottomMostLevel(bool isManual) {
+void Compaction::SetupBottomMostLevel(bool is_manual) {
  assert(cfd_->options()->compaction_style != kCompactionStyleFIFO);
  if (cfd_->options()->compaction_style == kCompactionStyleUniversal) {
    // If universal compaction style is used and manual
@@ -168,13 +182,14 @@ void Compaction::SetupBottomMostLevel(bool isManual) {
    // run. We can safely set bottommost_level_ = true.
    // If it is not manual compaction, then bottommost_level_
    // is already set when the Compaction was created.
-    if (isManual) {
+    if (is_manual) {
      bottommost_level_ = true;
    }
    return;
  }
  bottommost_level_ = true;
-  for (int i = output_level() + 1; i < number_levels_; i++) {
+  // checks whether there are files living beyond the output_level.
+  for (int i = output_level_ + 1; i < number_levels_; i++) {
    if (input_version_->NumLevelFiles(i) > 0) {
      bottommost_level_ = false;
      break;
@@ -200,7 +215,7 @@ void Compaction::ReleaseCompactionFiles(Status status) {
 }

 void Compaction::ResetNextCompactionIndex() {
-  input_version_->ResetNextCompactionIndex(level_);
+  input_version_->ResetNextCompactionIndex(start_level_);
 }

 namespace {
@@ -227,24 +242,23 @@ void Compaction::Summary(char* output, int len) {
  int write =
      snprintf(output, len, "Base version %" PRIu64
                            " Base level %d, seek compaction:%d, inputs: [",
-               input_version_->GetVersionNumber(), level_, seek_compaction_);
+               input_version_->GetVersionNumber(),
+               start_level_, seek_compaction_);
  if (write < 0 || write >= len) {
    return;
  }

-  write += InputSummary(inputs_[0], output + write, len - write);
-  if (write < 0 || write >= len) {
-    return;
-  }
-
-  write += snprintf(output + write, len - write, "], [");
-  if (write < 0 || write >= len) {
-    return;
-  }
-
-  write += InputSummary(inputs_[1], output + write, len - write);
-  if (write < 0 || write >= len) {
-    return;
+  for (int level = 0; level < num_input_levels(); ++level) {
+    if (level > 0) {
+      write += snprintf(output + write, len - write, "], [");
+      if (write < 0 || write >= len) {
+        return;
+      }
+    }
+    write += InputSummary(inputs_[level].files, output + write, len - write);
+    if (write < 0 || write >= len) {
+      return;
+    }
  }

  snprintf(output + write, len - write, "]");
@@ -257,8 +271,10 @@ uint64_t Compaction::OutputFilePreallocationSize() {
    preallocation_size =
        cfd_->compaction_picker()->MaxFileSizeForLevel(output_level());
  } else {
-    for (const auto& f : inputs_[0]) {
-      preallocation_size += f->fd.GetFileSize();
+    for (int level = 0; level < num_input_levels(); ++level) {
+      for (const auto& f : inputs_[level].files) {
+        preallocation_size += f->fd.GetFileSize();
+      }
    }
  }
  // Over-estimate slightly so we don't end up just barely crossing
@@ -8,62 +8,119 @@
 // found in the LICENSE file. See the AUTHORS file for names of contributors.

 #pragma once
+#include "util/arena.h"
+#include "util/autovector.h"
 #include "db/version_set.h"

 namespace rocksdb {

+// The structure that manages compaction input files associated
+// with the same physical level.
+struct CompactionInputFiles {
+  int level;
+  std::vector<FileMetaData*> files;
+  inline bool empty() const { return files.empty(); }
+  inline size_t size() const { return files.size(); }
+  inline void clear() { files.clear(); }
+  inline FileMetaData* operator[](int i) const { return files[i]; }
+};
+
 class Version;
 class ColumnFamilyData;

 // A Compaction encapsulates information about a compaction.
 class Compaction {
 public:
+  // No copying allowed
+  Compaction(const Compaction&) = delete;
+  void operator=(const Compaction&) = delete;
+
  ~Compaction();

-  // Return the level that is being compacted.  Inputs from "level"
-  // will be merged.
-  int level() const { return level_; }
+  // Returns the level associated to the specified compaction input level.
+  // If compaction_input_level is not specified, then input_level is set to 0.
+  int level(int compaction_input_level = 0) const {
+    return inputs_[compaction_input_level].level;
+  }

  // Outputs will go to this level
-  int output_level() const { return out_level_; }
+  int output_level() const { return output_level_; }
+
+  // Returns the number of input levels in this compaction.
+  int num_input_levels() const { return inputs_.size(); }

  // Return the object that holds the edits to the descriptor done
  // by this compaction.
-  VersionEdit* edit() { return edit_; }
+  VersionEdit* edit() const { return edit_; }

-  // "which" must be either 0 or 1
-  int num_input_files(int which) const { return inputs_[which].size(); }
+  // Returns the number of input files associated to the specified
+  // compaction input level.
+  // The function will return 0 if when "compaction_input_level" < 0
+  // or "compaction_input_level" >= "num_input_levels()".
+  int num_input_files(size_t compaction_input_level) const {
+    if (compaction_input_level < inputs_.size()) {
+      return inputs_[compaction_input_level].size();
+    }
+    return 0;
+  }

  // Returns input version of the compaction
  Version* input_version() const { return input_version_; }

+  // Returns the ColumnFamilyData associated with the compaction.
  ColumnFamilyData* column_family_data() const { return cfd_; }

-  // Return the ith input file at "level()+which" ("which" must be 0 or 1).
-  FileMetaData* input(int which, int i) const { return inputs_[which][i]; }
+  // Returns the file meta data of the 'i'th input file at the
+  // specified compaction input level.
+  // REQUIREMENT: "compaction_input_level" must be >= 0 and
+  //              < "input_levels()"
+  FileMetaData* input(size_t compaction_input_level, int i) const {
+    assert(compaction_input_level < inputs_.size());
+    return inputs_[compaction_input_level][i];
+  }

-  std::vector<FileMetaData*>* inputs(int which) { return &inputs_[which]; }
+  // Returns the list of file meta data of the specified compaction
+  // input level.
+  // REQUIREMENT: "compaction_input_level" must be >= 0 and
+  //              < "input_levels()"
+  std::vector<FileMetaData*>* const inputs(size_t compaction_input_level) {
+    assert(compaction_input_level < inputs_.size());
+    return &inputs_[compaction_input_level].files;
+  }
+
+  // Returns the FileLevel of the specified compaction input level.
+  FileLevel* input_levels(int compaction_input_level) {
+    return &input_levels_[compaction_input_level];
+  }

  // Maximum size of files to build during this compaction.
  uint64_t MaxOutputFileSize() const { return max_output_file_size_; }

-  // Whether compression will be enabled for compaction outputs
-  bool enable_compression() const { return enable_compression_; }
+  // What compression for output
+  CompressionType OutputCompressionType() const { return output_compression_; }
+
+  // Whether need to write output file to second DB path.
+  uint32_t GetOutputPathId() const { return output_path_id_; }
+
+  // Generate input_levels_ from inputs_
+  // Should be called when inputs_ is stable
+  void GenerateFileLevels();

  // Is this a trivial compaction that can be implemented by just
  // moving a single input file to the next level (no merging or splitting)
  bool IsTrivialMove() const;

-  // If true, just delete all files in inputs_[0]
-  bool IsDeletionCompaction() const;
+  // If true, then the comaction can be done by simply deleting input files.
+  bool IsDeletionCompaction() const {
+    return deletion_compaction_;
+  }

  // Add all inputs to this compaction as delete operations to *edit.
  void AddInputDeletions(VersionEdit* edit);

-  // Returns true if the information we have available guarantees that
-  // the compaction is producing data in "level+1" for which no data exists
-  // in levels greater than "level+1".
-  bool IsBaseLevelForKey(const Slice& user_key);
+  // Returns true if the available information we have guarantees that
+  // the input "user_key" does not exist in any level beyond "output_level()".
+  bool KeyNotExistsBeyondOutputLevel(const Slice& user_key);

  // Returns true iff we should stop building the current output
  // before processing "internal_key".
@@ -77,6 +134,9 @@ class Compaction {
  // Delete this compaction from the list of running compactions.
  void ReleaseCompactionFiles(Status status);

+  // Returns the summary of the compaction in "output" with maximum "len"
+  // in bytes.  The caller is responsible for the memory management of
+  // "output".
  void Summary(char* output, int len);

  // Return the score that was used to pick this compaction run.
@@ -91,9 +151,9 @@ class Compaction {
  // Was this compaction triggered manually by the client?
  bool IsManualCompaction() { return is_manual_compaction_; }

-  // Returns a number of byte that the output file should be preallocated to
+  // Returns the size in bytes that the output file should be preallocated to.
  // In level compaction, that is max_file_size_. In universal compaction, that
-  // is the sum of all input file sizes
+  // is the sum of all input file sizes.
  uint64_t OutputFilePreallocationSize();

 private:
@@ -102,38 +162,45 @@ class Compaction {
  friend class FIFOCompactionPicker;
  friend class LevelCompactionPicker;

-  Compaction(Version* input_version, int level, int out_level,
+  Compaction(Version* input_version, int start_level, int out_level,
             uint64_t target_file_size, uint64_t max_grandparent_overlap_bytes,
-             bool seek_compaction = false, bool enable_compression = true,
-             bool deletion_compaction = false);
+             uint32_t output_path_id, CompressionType output_compression,
+             bool seek_compaction = false, bool deletion_compaction = false);

-  int level_;
-  int out_level_; // levels to which output files are stored
+  const int start_level_;    // the lowest level to be compacted
+  const int output_level_;  // levels to which output files are stored
  uint64_t max_output_file_size_;
  uint64_t max_grandparent_overlap_bytes_;
  Version* input_version_;
  VersionEdit* edit_;
  int number_levels_;
  ColumnFamilyData* cfd_;
+  Arena arena_;          // Arena used to allocate space for file_levels_

+  uint32_t output_path_id_;
+  CompressionType output_compression_;
  bool seek_compaction_;
-  bool enable_compression_;
-  // if true, just delete files in inputs_[0]
+  // If true, then the comaction can be done by simply deleting input files.
  bool deletion_compaction_;

-  // Each compaction reads inputs from "level_" and "level_+1"
-  std::vector<FileMetaData*> inputs_[2];      // The two sets of inputs
+  // Compaction input files organized by level.
+  autovector<CompactionInputFiles> inputs_;
+
+  // A copy of inputs_, organized more closely in memory
+  autovector<FileLevel, 2> input_levels_;

  // State used to check for number of of overlapping grandparent files
-  // (parent == level_ + 1, grandparent == level_ + 2)
+  // (grandparent == "output_level_ + 1")
+  // This vector is updated by Version::GetOverlappingInputs().
  std::vector<FileMetaData*> grandparents_;
-  size_t grandparent_index_;  // Index in grandparent_starts_
-  bool seen_key_;             // Some output key has been seen
+  size_t grandparent_index_;   // Index in grandparent_starts_
+  bool seen_key_;              // Some output key has been seen
  uint64_t overlapped_bytes_;  // Bytes of overlap between current output
-                              // and grandparent files
-  int base_index_;   // index of the file in files_[level_]
-  int parent_index_; // index of some file with same range in files_[level_+1]
-  double score_;     // score that was used to pick this compaction.
+                               // and grandparent files
+  int base_index_;    // index of the file in files_[start_level_]
+  int parent_index_;  // index of some file with same range in
+                      // files_[start_level_+1]
+  double score_;      // score that was used to pick this compaction.

  // Is this compaction creating a file in the bottom most level?
  bool bottommost_level_;
@@ -143,17 +210,21 @@ class Compaction {
  // Is this compaction requested by the client?
  bool is_manual_compaction_;

-  // level_ptrs_ holds indices into input_version_->levels_: our state
-  // is that we are positioned at one of the file ranges for each
-  // higher level than the ones involved in this compaction (i.e. for
-  // all L >= level_ + 2).
+  // "level_ptrs_" holds indices into "input_version_->levels_", where each
+  // index remembers which file of an associated level we are currently used
+  // to check KeyNotExistsBeyondOutputLevel() for deletion operation.
+  // As it is for checking KeyNotExistsBeyondOutputLevel(), it only
+  // records indices for all levels beyond "output_level_".
  std::vector<size_t> level_ptrs_;

  // mark (or clear) all files that are being compacted
-  void MarkFilesBeingCompacted(bool);
+  void MarkFilesBeingCompacted(bool mark_as_compacted);

-  // Initialize whether compaction producing files at the bottommost level
-  void SetupBottomMostLevel(bool isManual);
+  // Initialize whether the compaction is producing files at the
+  // bottommost level.
+  //
+  // @see BottomMostLevel()
+  void SetupBottomMostLevel(bool is_manual);

  // In case of compaction error, reset the nextIndex that is used
  // to pick up the next file to be compacted from files_by_size_
@@ -12,17 +12,43 @@
 #define __STDC_FORMAT_MACROS
 #include <inttypes.h>
 #include <limits>
+#include "db/filename.h"
 #include "util/log_buffer.h"
 #include "util/statistics.h"

 namespace rocksdb {

 namespace {
+// Determine compression type, based on user options, level of the output
+// file and whether compression is disabled.
+// If enable_compression is false, then compression is always disabled no
+// matter what the values of the other two parameters are.
+// Otherwise, the compression type is determined based on options and level.
+CompressionType GetCompressionType(const Options& options, int level,
+                                   const bool enable_compression = true) {
+  if (!enable_compression) {
+    // disable compression
+    return kNoCompression;
+  }
+  // If the use has specified a different compression level for each level,
+  // then pick the compresison for that level.
+  if (!options.compression_per_level.empty()) {
+    const int n = options.compression_per_level.size() - 1;
+    // It is possible for level_ to be -1; in that case, we use level
+    // 0's compression.  This occurs mostly in backwards compatibility
+    // situations when the builder doesn't know what level the file
+    // belongs to.  Likewise, if level_ is beyond the end of the
+    // specified compression levels, use the last value.
+    return options.compression_per_level[std::max(0, std::min(level, n))];
+  } else {
+    return options.compression;
+  }
+}

-uint64_t TotalFileSize(const std::vector<FileMetaData*>& files) {
+uint64_t TotalCompensatedFileSize(const std::vector<FileMetaData*>& files) {
  uint64_t sum = 0;
  for (size_t i = 0; i < files.size() && files[i]; i++) {
-    sum += files[i]->fd.GetFileSize();
+    sum += files[i]->compensated_file_size;
  }
  return sum;
 }
@@ -80,7 +106,7 @@ void CompactionPicker::SizeBeingCompacted(std::vector<uint64_t>& sizes) {
    for (auto c : compactions_in_progress_[level]) {
      assert(c->level() == level);
      for (int i = 0; i < c->num_input_files(0); i++) {
-        total += c->input(0, i)->fd.GetFileSize();
+        total += c->input(0, i)->compensated_file_size;
      }
    }
    sizes[level] = total;
@@ -168,14 +194,15 @@ bool CompactionPicker::ExpandWhileOverlapping(Compaction* c) {
  size_t old_size;
  do {
    old_size = c->inputs_[0].size();
-    GetRange(c->inputs_[0], &smallest, &largest);
+    GetRange(c->inputs_[0].files, &smallest, &largest);
    c->inputs_[0].clear();
    c->input_version_->GetOverlappingInputs(
-        level, &smallest, &largest, &c->inputs_[0], hint_index, &hint_index);
+        level, &smallest, &largest, &c->inputs_[0].files,
+        hint_index, &hint_index);
  } while(c->inputs_[0].size() > old_size);

  // Get the new range
-  GetRange(c->inputs_[0], &smallest, &largest);
+  GetRange(c->inputs_[0].files, &smallest, &largest);

  // If, after the expansion, there are files that are already under
  // compaction, then we must drop/cancel this compaction.
@@ -185,7 +212,7 @@ bool CompactionPicker::ExpandWhileOverlapping(Compaction* c) {
        "[%s] ExpandWhileOverlapping() failure because zero input files",
        c->column_family_data()->GetName().c_str());
  }
-  if (c->inputs_[0].empty() || FilesInCompaction(c->inputs_[0]) ||
+  if (c->inputs_[0].empty() || FilesInCompaction(c->inputs_[0].files) ||
      (c->level() != c->output_level() &&
       ParentRangeInCompaction(c->input_version_, &smallest, &largest, level,
                               &parent_index))) {
@@ -241,16 +268,17 @@ void CompactionPicker::SetupOtherInputs(Compaction* c) {
  InternalKey smallest, largest;

  // Get the range one last time.
-  GetRange(c->inputs_[0], &smallest, &largest);
+  GetRange(c->inputs_[0].files, &smallest, &largest);

  // Populate the set of next-level files (inputs_[1]) to include in compaction
-  c->input_version_->GetOverlappingInputs(level + 1, &smallest, &largest,
-                                          &c->inputs_[1], c->parent_index_,
-                                          &c->parent_index_);
+  c->input_version_->GetOverlappingInputs(
+      level + 1, &smallest, &largest,
+      &c->inputs_[1].files, c->parent_index_,
+      &c->parent_index_);

  // Get entire range covered by compaction
  InternalKey all_start, all_limit;
-  GetRange(c->inputs_[0], c->inputs_[1], &all_start, &all_limit);
+  GetRange(c->inputs_[0].files, c->inputs_[1].files, &all_start, &all_limit);

  // See if we can further grow the number of inputs in "level" without
  // changing the number of "level+1" files we pick up. We also choose NOT
@@ -261,9 +289,9 @@ void CompactionPicker::SetupOtherInputs(Compaction* c) {
    std::vector<FileMetaData*> expanded0;
    c->input_version_->GetOverlappingInputs(
        level, &all_start, &all_limit, &expanded0, c->base_index_, nullptr);
-    const uint64_t inputs0_size = TotalFileSize(c->inputs_[0]);
-    const uint64_t inputs1_size = TotalFileSize(c->inputs_[1]);
-    const uint64_t expanded0_size = TotalFileSize(expanded0);
+    const uint64_t inputs0_size = TotalCompensatedFileSize(c->inputs_[0].files);
+    const uint64_t inputs1_size = TotalCompensatedFileSize(c->inputs_[1].files);
+    const uint64_t expanded0_size = TotalCompensatedFileSize(expanded0);
    uint64_t limit = ExpandedCompactionByteSizeLimit(level);
    if (expanded0.size() > c->inputs_[0].size() &&
        inputs1_size + expanded0_size < limit &&
@@ -278,19 +306,18 @@ void CompactionPicker::SetupOtherInputs(Compaction* c) {
      if (expanded1.size() == c->inputs_[1].size() &&
          !FilesInCompaction(expanded1)) {
        Log(options_->info_log,
-            "[%s] Expanding@%lu %lu+%lu (%lu+%lu bytes) to %lu+%lu (%lu+%lu "
-            "bytes)\n",
-            c->column_family_data()->GetName().c_str(), (unsigned long)level,
-            (unsigned long)(c->inputs_[0].size()),
-            (unsigned long)(c->inputs_[1].size()), (unsigned long)inputs0_size,
-            (unsigned long)inputs1_size, (unsigned long)(expanded0.size()),
-            (unsigned long)(expanded1.size()), (unsigned long)expanded0_size,
-            (unsigned long)inputs1_size);
+            "[%s] Expanding@%d %zu+%zu (%" PRIu64 "+%" PRIu64
+            " bytes) to %zu+%zu (%" PRIu64 "+%" PRIu64 "bytes)\n",
+            c->column_family_data()->GetName().c_str(), level,
+            c->inputs_[0].size(), c->inputs_[1].size(), inputs0_size,
+            inputs1_size, expanded0.size(), expanded1.size(), expanded0_size,
+            inputs1_size);
        smallest = new_start;
        largest = new_limit;
-        c->inputs_[0] = expanded0;
-        c->inputs_[1] = expanded1;
-        GetRange(c->inputs_[0], c->inputs_[1], &all_start, &all_limit);
+        c->inputs_[0].files = expanded0;
+        c->inputs_[1].files = expanded1;
+        GetRange(c->inputs_[0].files, c->inputs_[1].files,
+                 &all_start, &all_limit);
      }
    }
  }
@@ -303,9 +330,9 @@ void CompactionPicker::SetupOtherInputs(Compaction* c) {
  }
 }

-
 Compaction* CompactionPicker::CompactRange(Version* version, int input_level,
                                           int output_level,
+                                           uint32_t output_path_id,
                                           const InternalKey* begin,
                                           const InternalKey* end,
                                           InternalKey** compaction_end) {
@@ -335,7 +362,7 @@ Compaction* CompactionPicker::CompactRange(Version* version, int input_level,
        MaxFileSizeForLevel(input_level) * options_->source_compaction_factor;
    uint64_t total = 0;
    for (size_t i = 0; i + 1 < inputs.size(); ++i) {
-      uint64_t s = inputs[i]->fd.GetFileSize();
+      uint64_t s = inputs[i]->compensated_file_size;
      total += s;
      if (total >= limit) {
        **compaction_end = inputs[i + 1]->smallest;
@@ -345,11 +372,13 @@ Compaction* CompactionPicker::CompactRange(Version* version, int input_level,
      }
    }
  }
-  Compaction* c = new Compaction(version, input_level, output_level,
-                                 MaxFileSizeForLevel(output_level),
-                                 MaxGrandParentOverlapBytes(input_level));
+  assert(output_path_id < static_cast<uint32_t>(options_->db_paths.size()));
+  Compaction* c = new Compaction(
+      version, input_level, output_level, MaxFileSizeForLevel(output_level),
+      MaxGrandParentOverlapBytes(input_level), output_path_id,
+      GetCompressionType(*options_, output_level));

-  c->inputs_[0] = inputs;
+  c->inputs_[0].files = inputs;
  if (ExpandWhileOverlapping(c) == false) {
    delete c;
    Log(options_->info_log,
@@ -407,31 +436,6 @@ Compaction* LevelCompactionPicker::PickCompaction(Version* version,
    }
  }

-  // Find compactions needed by seeks
-  FileMetaData* f = version->file_to_compact_;
-  if (c == nullptr && f != nullptr && !f->being_compacted) {
-
-    level = version->file_to_compact_level_;
-    int parent_index = -1;
-
-    // Only allow one level 0 compaction at a time.
-    // Do not pick this file if its parents at level+1 are being compacted.
-    if (level != 0 || compactions_in_progress_[0].empty()) {
-      if (!ParentRangeInCompaction(version, &f->smallest, &f->largest, level,
-                                   &parent_index)) {
-        c = new Compaction(version, level, level + 1,
-                           MaxFileSizeForLevel(level + 1),
-                           MaxGrandParentOverlapBytes(level), true);
-        c->inputs_[0].push_back(f);
-        c->parent_index_ = parent_index;
-        c->input_version_->file_to_compact_ = nullptr;
-        if (ExpandWhileOverlapping(c) == false) {
-          return nullptr;
-        }
-      }
-    }
-  }
-
  if (c == nullptr) {
    return nullptr;
  }
@@ -441,18 +445,18 @@ Compaction* LevelCompactionPicker::PickCompaction(Version* version,
  if (level == 0) {
    assert(compactions_in_progress_[0].empty());
    InternalKey smallest, largest;
-    GetRange(c->inputs_[0], &smallest, &largest);
+    GetRange(c->inputs_[0].files, &smallest, &largest);
    // Note that the next call will discard the file we placed in
    // c->inputs_[0] earlier and replace it with an overlapping set
    // which will include the picked file.
    c->inputs_[0].clear();
    c->input_version_->GetOverlappingInputs(0, &smallest, &largest,
-                                            &c->inputs_[0]);
+                                            &c->inputs_[0].files);

    // If we include more L0 files in the same compaction run it can
    // cause the 'smallest' and 'largest' key to get extended to a
    // larger range. So, re-invoke GetRange to get the new key range
-    GetRange(c->inputs_[0], &smallest, &largest);
+    GetRange(c->inputs_[0].files, &smallest, &largest);
    if (ParentRangeInCompaction(c->input_version_, &smallest, &largest, level,
                                &c->parent_index_)) {
      delete c;
@@ -492,7 +496,8 @@ Compaction* LevelCompactionPicker::PickCompactionBySize(Version* version,
  assert(level >= 0);
  assert(level + 1 < NumberLevels());
  c = new Compaction(version, level, level + 1, MaxFileSizeForLevel(level + 1),
-                     MaxGrandParentOverlapBytes(level));
+                     MaxGrandParentOverlapBytes(level), 0,
+                     GetCompressionType(*options_, level + 1));
  c->score_ = score;

  // Pick the largest file in this level that is not already
@@ -507,12 +512,12 @@ Compaction* LevelCompactionPicker::PickCompactionBySize(Version* version,
    int index = file_size[i];
    FileMetaData* f = c->input_version_->files_[level][index];

-    // check to verify files are arranged in descending size
-    assert(
-        (i == file_size.size() - 1) ||
-        (i >= Version::number_of_files_to_sort_ - 1) ||
-        (f->fd.GetFileSize() >=
-         c->input_version_->files_[level][file_size[i + 1]]->fd.GetFileSize()));
+    // Check to verify files are arranged in descending compensated size.
+    assert((i == file_size.size() - 1) ||
+           (i >= Version::number_of_files_to_sort_ - 1) ||
+           (f->compensated_file_size >=
+            c->input_version_->files_[level][file_size[i + 1]]->
+                compensated_file_size));

    // do not pick a file to compact if it is being compacted
    // from n-1 level.
@@ -532,7 +537,7 @@ Compaction* LevelCompactionPicker::PickCompactionBySize(Version* version,
                                level, &parent_index)) {
      continue;
    }
-    c->inputs_[0].push_back(f);
+    c->inputs_[0].files.push_back(f);
    c->base_index_ = index;
    c->parent_index_ = parent_index;
    break;
@@ -612,19 +617,13 @@ Compaction* UniversalCompactionPicker::PickCompaction(Version* version,
    newerfile = f;
  }

-  // The files are sorted from newest first to oldest last.
-  std::vector<int>& file_by_time = c->input_version_->files_by_size_[level];
-
  // Is the earliest file part of this compaction?
-  int last_index = file_by_time[file_by_time.size()-1];
-  FileMetaData* last_file = c->input_version_->files_[level][last_index];
-  if (c->inputs_[0][c->inputs_[0].size()-1] == last_file) {
-    c->bottommost_level_ = true;
-  }
+  FileMetaData* last_file = c->input_version_->files_[level].back();
+  c->bottommost_level_ = c->inputs_[0].files.back() == last_file;

  // update statistics
-  MeasureTime(options_->statistics.get(), NUM_FILES_IN_SINGLE_COMPACTION,
-              c->inputs_[0].size());
+  MeasureTime(options_->statistics.get(),
+              NUM_FILES_IN_SINGLE_COMPACTION, c->inputs_[0].size());

  // mark all the files that are being compacted
  c->MarkFilesBeingCompacted(true);
@@ -640,6 +639,37 @@ Compaction* UniversalCompactionPicker::PickCompaction(Version* version,
  return c;
 }

+uint32_t UniversalCompactionPicker::GetPathId(const Options& options,
+                                              uint64_t file_size) {
+  // Two conditions need to be satisfied:
+  // (1) the target path needs to be able to hold the file's size
+  // (2) Total size left in this and previous paths need to be not
+  //     smaller than expected future file size before this new file is
+  //     compacted, which is estimated based on size_ratio.
+  // For example, if now we are compacting files of size (1, 1, 2, 4, 8),
+  // we will make sure the target file, probably with size of 16, will be
+  // placed in a path so that eventually when new files are generated and
+  // compacted to (1, 1, 2, 4, 8, 16), all those files can be stored in or
+  // before the path we chose.
+  //
+  // TODO(sdong): now the case of multiple column families is not
+  // considered in this algorithm. So the target size can be violated in
+  // that case. We need to improve it.
+  uint64_t accumulated_size = 0;
+  uint64_t future_size =
+      file_size * (100 - options.compaction_options_universal.size_ratio) / 100;
+  uint32_t p = 0;
+  for (; p < options.db_paths.size() - 1; p++) {
+    uint64_t target_size = options.db_paths[p].target_size;
+    if (target_size > file_size &&
+        accumulated_size + (target_size - file_size) > future_size) {
+      return p;
+    }
+    accumulated_size += target_size;
+  }
+  return p;
+}
+
 //
 // Consider compaction files based on their size differences with
 // the next file in time order.
@@ -655,12 +685,12 @@ Compaction* UniversalCompactionPicker::PickCompactionUniversalReadAmp(
    options_->compaction_options_universal.max_merge_width;

  // The files are sorted from newest first to oldest last.
-  std::vector<int>& file_by_time = version->files_by_size_[level];
+  const auto& files = version->files_[level];
+
  FileMetaData* f = nullptr;
  bool done = false;
  int start_index = 0;
  unsigned int candidate_count = 0;
-  assert(file_by_time.size() == version->files_[level].size());

  unsigned int max_files_to_compact = std::min(max_merge_width,
                                       max_number_of_files_to_compact);
@@ -668,42 +698,38 @@ Compaction* UniversalCompactionPicker::PickCompactionUniversalReadAmp(

  // Considers a candidate file only if it is smaller than the
  // total size accumulated so far.
-  for (unsigned int loop = 0; loop < file_by_time.size(); loop++) {
+  for (unsigned int loop = 0; loop < files.size(); loop++) {

    candidate_count = 0;

    // Skip files that are already being compacted
-    for (f = nullptr; loop < file_by_time.size(); loop++) {
-      int index = file_by_time[loop];
-      f = version->files_[level][index];
+    for (f = nullptr; loop < files.size(); loop++) {
+      f = files[loop];

      if (!f->being_compacted) {
        candidate_count = 1;
        break;
      }
-      LogToBuffer(log_buffer,
-                  "[%s] Universal: file %lu[%d] being compacted, skipping",
-                  version->cfd_->GetName().c_str(),
-                  (unsigned long)f->fd.GetNumber(), loop);
+      LogToBuffer(log_buffer, "[%s] Universal: file %" PRIu64
+                              "[%d] being compacted, skipping",
+                  version->cfd_->GetName().c_str(), f->fd.GetNumber(), loop);
      f = nullptr;
    }

    // This file is not being compacted. Consider it as the
    // first candidate to be compacted.
-    uint64_t candidate_size = f != nullptr ? f->fd.GetFileSize() : 0;
+    uint64_t candidate_size =  f != nullptr? f->compensated_file_size : 0;
    if (f != nullptr) {
-      LogToBuffer(log_buffer,
-                  "[%s] Universal: Possible candidate file %lu[%d].",
-                  version->cfd_->GetName().c_str(),
-                  (unsigned long)f->fd.GetNumber(), loop);
+      LogToBuffer(
+          log_buffer, "[%s] Universal: Possible candidate file %s[%d].",
+          version->cfd_->GetName().c_str(),
+          FormatFileNumber(f->fd.GetNumber(), f->fd.GetPathId()).c_str(), loop);
    }

    // Check if the suceeding files need compaction.
-    for (unsigned int i = loop+1;
-         candidate_count < max_files_to_compact && i < file_by_time.size();
-         i++) {
-      int index = file_by_time[i];
-      FileMetaData* f = version->files_[level][index];
+    for (unsigned int i = loop + 1;
+         candidate_count < max_files_to_compact && i < files.size(); i++) {
+      FileMetaData* f = files[i];
      if (f->being_compacted) {
        break;
      }
@@ -728,9 +754,9 @@ Compaction* UniversalCompactionPicker::PickCompactionUniversalReadAmp(
          // by the last-resort read amp strategy which disregards size ratios.
          break;
        }
-        candidate_size = f->fd.GetFileSize();
+        candidate_size = f->compensated_file_size;
      } else { // default kCompactionStopStyleTotalSize
-        candidate_size += f->fd.GetFileSize();
+        candidate_size += f->compensated_file_size;
      }
      candidate_count++;
    }
@@ -742,14 +768,14 @@ Compaction* UniversalCompactionPicker::PickCompactionUniversalReadAmp(
      break;
    } else {
      for (unsigned int i = loop;
-           i < loop + candidate_count && i < file_by_time.size(); i++) {
-       int index = file_by_time[i];
-       FileMetaData* f = version->files_[level][index];
-       LogToBuffer(log_buffer,
-                   "[%s] Universal: Skipping file %lu[%d] with size %lu %d\n",
-                   version->cfd_->GetName().c_str(),
-                   (unsigned long)f->fd.GetNumber(), i,
-                   (unsigned long)f->fd.GetFileSize(), f->being_compacted);
+           i < loop + candidate_count && i < files.size(); i++) {
+        FileMetaData* f = files[i];
+        LogToBuffer(log_buffer, "[%s] Universal: Skipping file %" PRIu64
+                                "[%d] with size %" PRIu64
+                                " (compensated size %" PRIu64 ") %d\n",
+                    version->cfd_->GetName().c_str(), f->fd.GetNumber(), i,
+                    f->fd.GetFileSize(), f->compensated_file_size,
+                    f->being_compacted);
      }
    }
  }
@@ -765,29 +791,36 @@ Compaction* UniversalCompactionPicker::PickCompactionUniversalReadAmp(
  if (ratio_to_compress >= 0) {
    uint64_t total_size = version->NumLevelBytes(level);
    uint64_t older_file_size = 0;
-    for (unsigned int i = file_by_time.size() - 1; i >= first_index_after;
-        i--) {
-      older_file_size +=
-          version->files_[level][file_by_time[i]]->fd.GetFileSize();
+    for (unsigned int i = files.size() - 1;
+         i >= first_index_after; i--) {
+      older_file_size += files[i]->fd.GetFileSize();
      if (older_file_size * 100L >= total_size * (long) ratio_to_compress) {
        enable_compression = false;
        break;
      }
    }
  }
-  Compaction* c =
-      new Compaction(version, level, level, MaxFileSizeForLevel(level),
-                     LLONG_MAX, false, enable_compression);
+
+  uint64_t estimated_total_size = 0;
+  for (unsigned int i = 0; i < first_index_after; i++) {
+    estimated_total_size += files[i]->fd.GetFileSize();
+  }
+  uint32_t path_id = GetPathId(*options_, estimated_total_size);
+
+  Compaction* c = new Compaction(
+      version, level, level, MaxFileSizeForLevel(level), LLONG_MAX, path_id,
+      GetCompressionType(*options_, level, enable_compression));
  c->score_ = score;

  for (unsigned int i = start_index; i < first_index_after; i++) {
-    int index = file_by_time[i];
-    FileMetaData* f = c->input_version_->files_[level][index];
-    c->inputs_[0].push_back(f);
-    LogToBuffer(
-        log_buffer, "[%s] Universal: Picking file %lu[%d] with size %lu\n",
-        version->cfd_->GetName().c_str(), (unsigned long)f->fd.GetNumber(), i,
-        (unsigned long)f->fd.GetFileSize());
+    FileMetaData* f = c->input_version_->files_[level][i];
+    c->inputs_[0].files.push_back(f);
+    LogToBuffer(log_buffer,
+                "[%s] Universal: Picking file %s[%d] "
+                "with size %" PRIu64 " (compensated size %" PRIu64 ")\n",
+                version->cfd_->GetName().c_str(),
+                FormatFileNumber(f->fd.GetNumber(), f->fd.GetPathId()).c_str(),
+                i, f->fd.GetFileSize(), f->compensated_file_size);
  }
  return c;
 }
@@ -807,8 +840,7 @@ Compaction* UniversalCompactionPicker::PickCompactionUniversalSizeAmp(
                     max_size_amplification_percent;

  // The files are sorted from newest first to oldest last.
-  std::vector<int>& file_by_time = version->files_by_size_[level];
-  assert(file_by_time.size() == version->files_[level].size());
+  const auto& files = version->files_[level];

  unsigned int candidate_count = 0;
  uint64_t candidate_size = 0;
@@ -816,42 +848,39 @@ Compaction* UniversalCompactionPicker::PickCompactionUniversalSizeAmp(
  FileMetaData* f = nullptr;

  // Skip files that are already being compacted
-  for (unsigned int loop = 0; loop < file_by_time.size() - 1; loop++) {
-    int index = file_by_time[loop];
-    f = version->files_[level][index];
+  for (unsigned int loop = 0; loop < files.size() - 1; loop++) {
+    f = files[loop];
    if (!f->being_compacted) {
      start_index = loop;         // Consider this as the first candidate.
      break;
    }
-    LogToBuffer(
-        log_buffer, "[%s] Universal: skipping file %lu[%d] compacted %s",
-        version->cfd_->GetName().c_str(), (unsigned long)f->fd.GetNumber(),
-        loop, " cannot be a candidate to reduce size amp.\n");
+    LogToBuffer(log_buffer, "[%s] Universal: skipping file %s[%d] compacted %s",
+                version->cfd_->GetName().c_str(),
+                FormatFileNumber(f->fd.GetNumber(), f->fd.GetPathId()).c_str(),
+                loop, " cannot be a candidate to reduce size amp.\n");
    f = nullptr;
  }
  if (f == nullptr) {
    return nullptr;             // no candidate files
  }

-  LogToBuffer(log_buffer, "[%s] Universal: First candidate file %lu[%d] %s",
+  LogToBuffer(log_buffer, "[%s] Universal: First candidate file %s[%d] %s",
              version->cfd_->GetName().c_str(),
-              (unsigned long)f->fd.GetNumber(), start_index,
-              " to reduce size amp.\n");
+              FormatFileNumber(f->fd.GetNumber(), f->fd.GetPathId()).c_str(),
+              start_index, " to reduce size amp.\n");

  // keep adding up all the remaining files
-  for (unsigned int loop = start_index; loop < file_by_time.size() - 1;
-       loop++) {
-    int index = file_by_time[loop];
-    f = version->files_[level][index];
+  for (unsigned int loop = start_index; loop < files.size() - 1; loop++) {
+    f = files[loop];
    if (f->being_compacted) {
      LogToBuffer(
-          log_buffer, "[%s] Universal: Possible candidate file %lu[%d] %s.",
-          version->cfd_->GetName().c_str(), (unsigned long)f->fd.GetNumber(),
-          loop,
+          log_buffer, "[%s] Universal: Possible candidate file %s[%d] %s.",
+          version->cfd_->GetName().c_str(),
+          FormatFileNumber(f->fd.GetNumber(), f->fd.GetPathId()).c_str(), loop,
          " is already being compacted. No size amp reduction possible.\n");
      return nullptr;
    }
-    candidate_size += f->fd.GetFileSize();
+    candidate_size += f->compensated_file_size;
    candidate_count++;
  }
  if (candidate_count == 0) {
@@ -859,42 +888,47 @@ Compaction* UniversalCompactionPicker::PickCompactionUniversalSizeAmp(
  }

  // size of earliest file
-  int index = file_by_time[file_by_time.size() - 1];
-  uint64_t earliest_file_size = version->files_[level][index]->fd.GetFileSize();
+  uint64_t earliest_file_size = files.back()->fd.GetFileSize();

  // size amplification = percentage of additional size
  if (candidate_size * 100 < ratio * earliest_file_size) {
    LogToBuffer(
        log_buffer,
-        "[%s] Universal: size amp not needed. newer-files-total-size %lu "
-        "earliest-file-size %lu",
-        version->cfd_->GetName().c_str(), (unsigned long)candidate_size,
-        (unsigned long)earliest_file_size);
+        "[%s] Universal: size amp not needed. newer-files-total-size %" PRIu64
+        "earliest-file-size %" PRIu64,
+        version->cfd_->GetName().c_str(), candidate_size, earliest_file_size);
    return nullptr;
  } else {
-    LogToBuffer(log_buffer,
-                "[%s] Universal: size amp needed. newer-files-total-size %lu "
-                "earliest-file-size %lu",
-                version->cfd_->GetName().c_str(), (unsigned long)candidate_size,
-                (unsigned long)earliest_file_size);
+    LogToBuffer(
+        log_buffer,
+        "[%s] Universal: size amp needed. newer-files-total-size %" PRIu64
+        "earliest-file-size %" PRIu64,
+        version->cfd_->GetName().c_str(), candidate_size, earliest_file_size);
  }
-  assert(start_index >= 0 && start_index < file_by_time.size() - 1);
+  assert(start_index >= 0 && start_index < files.size() - 1);
+
+  // Estimate total file size
+  uint64_t estimated_total_size = 0;
+  for (unsigned int loop = start_index; loop < files.size(); loop++) {
+    estimated_total_size += files[loop]->fd.GetFileSize();
+  }
+  uint32_t path_id = GetPathId(*options_, estimated_total_size);

  // create a compaction request
  // We always compact all the files, so always compress.
  Compaction* c =
      new Compaction(version, level, level, MaxFileSizeForLevel(level),
-                     LLONG_MAX, false, true);
+                     LLONG_MAX, path_id, GetCompressionType(*options_, level));
  c->score_ = score;
-  for (unsigned int loop = start_index; loop < file_by_time.size(); loop++) {
-    int index = file_by_time[loop];
-    f = c->input_version_->files_[level][index];
-    c->inputs_[0].push_back(f);
+  for (unsigned int loop = start_index; loop < files.size(); loop++) {
+    f = c->input_version_->files_[level][loop];
+    c->inputs_[0].files.push_back(f);
    LogToBuffer(log_buffer,
-                "[%s] Universal: size amp picking file %lu[%d] with size %lu",
-                version->cfd_->GetName().c_str(),
-                (unsigned long)f->fd.GetNumber(), index,
-                (unsigned long)f->fd.GetFileSize());
+        "[%s] Universal: size amp picking file %" PRIu64 "[%d] "
+        "with size %" PRIu64 " (compensated size %" PRIu64 ")",
+        version->cfd_->GetName().c_str(),
+        f->fd.GetNumber(), loop,
+        f->fd.GetFileSize(), f->compensated_file_size);
  }
  return c;
 }
@@ -904,7 +938,7 @@ Compaction* FIFOCompactionPicker::PickCompaction(Version* version,
  assert(version->NumberLevels() == 1);
  uint64_t total_size = 0;
  for (const auto& file : version->files_[0]) {
-    total_size += file->fd.GetFileSize();
+    total_size += file->compensated_file_size;
  }

  if (total_size <= options_->compaction_options_fifo.max_table_files_size ||
@@ -926,14 +960,14 @@ Compaction* FIFOCompactionPicker::PickCompaction(Version* version,
    return nullptr;
  }

-  Compaction* c = new Compaction(version, 0, 0, 0, 0, false, false,
+  Compaction* c = new Compaction(version, 0, 0, 0, 0, 0, kNoCompression, false,
                                 true /* is deletion compaction */);
  // delete old files (FIFO)
  for (auto ritr = version->files_[0].rbegin();
       ritr != version->files_[0].rend(); ++ritr) {
    auto f = *ritr;
-    total_size -= f->fd.GetFileSize();
-    c->inputs_[0].push_back(f);
+    total_size -= f->compensated_file_size;
+    c->inputs_[0].files.push_back(f);
    char tmp_fsize[16];
    AppendHumanBytes(f->fd.GetFileSize(), tmp_fsize, sizeof(tmp_fsize));
    LogToBuffer(log_buffer, "[%s] FIFO compaction: picking file %" PRIu64
@@ -950,17 +984,19 @@ Compaction* FIFOCompactionPicker::PickCompaction(Version* version,
  return c;
 }

-Compaction* FIFOCompactionPicker::CompactRange(Version* version,
-                                               int input_level,
-                                               int output_level,
-                                               const InternalKey* begin,
-                                               const InternalKey* end,
-                                               InternalKey** compaction_end) {
+Compaction* FIFOCompactionPicker::CompactRange(
+    Version* version, int input_level, int output_level,
+    uint32_t output_path_id, const InternalKey* begin, const InternalKey* end,
+    InternalKey** compaction_end) {
  assert(input_level == 0);
  assert(output_level == 0);
  *compaction_end = nullptr;
  LogBuffer log_buffer(InfoLogLevel::INFO_LEVEL, options_->info_log.get());
-  auto c = PickCompaction(version, &log_buffer);
+  Compaction* c = PickCompaction(version, &log_buffer);
+  if (c != nullptr) {
+    assert(output_path_id < static_cast<uint32_t>(options_->db_paths.size()));
+    c->output_path_id_ = output_path_id;
+  }
  log_buffer.FlushBufferToLog();
  return c;
 }
@@ -48,10 +48,15 @@ class CompactionPicker {
  // Client is responsible for compaction_end storage -- when called,
  // *compaction_end should point to valid InternalKey!
  virtual Compaction* CompactRange(Version* version, int input_level,
-                                   int output_level, const InternalKey* begin,
+                                   int output_level, uint32_t output_path_id,
+                                   const InternalKey* begin,
                                   const InternalKey* end,
                                   InternalKey** compaction_end);

+  // Given the current number of levels, returns the lowest allowed level
+  // for compaction input.
+  virtual int MaxInputLevel(int current_num_levels) const = 0;
+
  // Free up the files that participated in a compaction
  void ReleaseCompactionFiles(Compaction* c, Status status);

@@ -135,6 +140,11 @@ class UniversalCompactionPicker : public CompactionPicker {
  virtual Compaction* PickCompaction(Version* version,
                                     LogBuffer* log_buffer) override;

+  // The maxinum allowed input level.  Always return 0.
+  virtual int MaxInputLevel(int current_num_levels) const override {
+    return 0;
+  }
+
 private:
  // Pick Universal compaction to limit read amplification
  Compaction* PickCompactionUniversalReadAmp(Version* version, double score,
@@ -145,6 +155,10 @@ class UniversalCompactionPicker : public CompactionPicker {
  // Pick Universal compaction to limit space amplification.
  Compaction* PickCompactionUniversalSizeAmp(Version* version, double score,
                                             LogBuffer* log_buffer);
+
+  // Pick a path ID to place a newly generated file, with its estimated file
+  // size.
+  static uint32_t GetPathId(const Options& options, uint64_t file_size);
 };

 class LevelCompactionPicker : public CompactionPicker {
@@ -155,6 +169,12 @@ class LevelCompactionPicker : public CompactionPicker {
  virtual Compaction* PickCompaction(Version* version,
                                     LogBuffer* log_buffer) override;

+  // Returns current_num_levels - 2, meaning the last level cannot be
+  // compaction input level.
+  virtual int MaxInputLevel(int current_num_levels) const override {
+    return current_num_levels - 2;
+  }
+
 private:
  // For the specfied level, pick a compaction.
  // Returns nullptr if there is no compaction to be done.
@@ -173,9 +193,15 @@ class FIFOCompactionPicker : public CompactionPicker {
                                     LogBuffer* log_buffer) override;

  virtual Compaction* CompactRange(Version* version, int input_level,
-                                   int output_level, const InternalKey* begin,
+                                   int output_level, uint32_t output_path_id,
+                                   const InternalKey* begin,
                                   const InternalKey* end,
                                   InternalKey** compaction_end) override;
+
+  // The maxinum allowed input level.  Always return 0.
+  virtual int MaxInputLevel(int current_num_levels) const override {
+    return 0;
+  }
 };

 }  // namespace rocksdb
@@ -17,6 +17,11 @@ int main() {
 }
 #else

+#ifdef NUMA
+#include <numa.h>
+#include <numaif.h>
+#endif
+
 #include <inttypes.h>
 #include <cstddef>
 #include <sys/types.h>
@@ -173,6 +178,14 @@ DEFINE_double(compression_ratio, 0.5, "Arrange to generate values that shrink"

 DEFINE_bool(histogram, false, "Print histogram of operation timings");

+DEFINE_bool(enable_numa, false,
+            "Make operations aware of NUMA architecture and bind memory "
+            "and cpus corresponding to nodes together. In NUMA, memory "
+            "in same node as CPUs are closer when compared to memory in "
+            "other nodes. Reads can be faster when the process is bound to "
+            "CPU and memory of same node. Use \"$numactl --hardware\" command "
+            "to see NUMA memory architecture.");
+
 DEFINE_int64(write_buffer_size, rocksdb::Options().write_buffer_size,
             "Number of bytes to buffer in memtable before compacting");

@@ -229,6 +242,10 @@ DEFINE_int64(cache_size, -1, "Number of bytes to use as a cache of uncompressed"
 DEFINE_int32(block_size, rocksdb::Options().block_size,
             "Number of bytes in a block.");

+DEFINE_int32(block_restart_interval, rocksdb::Options().block_restart_interval,
+             "Number of keys between restart points "
+             "for delta encoding of keys.");
+
 DEFINE_int64(compressed_cache_size, -1,
             "Number of bytes to use as a cache of compressed data.");

@@ -333,9 +350,6 @@ DEFINE_int32(deletepercent, 2, "Percentage of deletes out of reads/writes/"
             "deletepercent), so deletepercent must be smaller than (100 - "
             "FLAGS_readwritepercent)");

-DEFINE_int32(disable_seek_compaction, false, "Option to disable compaction"
-             " triggered by read.");
-
 DEFINE_uint64(delete_obsolete_files_period_micros, 0, "Option to delete "
              "obsolete files periodically. 0 means that obsolete files are"
              " deleted after every compaction run.");
@@ -528,6 +542,9 @@ DEFINE_string(memtablerep, "skip_list", "");
 DEFINE_int64(hash_bucket_count, 1024 * 1024, "hash bucket count");
 DEFINE_bool(use_plain_table, false, "if use plain table "
            "instead of block-based table format");
+DEFINE_bool(use_hash_search, false, "if use kHashSearch "
+            "instead of kBinarySearch. "
+            "This is valid if only we use BlockTable");

 DEFINE_string(merge_operator, "", "The merge operator to use with the database."
              "If a new merge operator is specified, be sure to use fresh"
@@ -553,6 +570,9 @@ static const bool FLAGS_cache_numshardbits_dummy __attribute__((unused)) =
 static const bool FLAGS_readwritepercent_dummy __attribute__((unused)) =
    RegisterFlagValidator(&FLAGS_readwritepercent, &ValidateInt32Percent);

+DEFINE_int32(disable_seek_compaction, false,
+             "Not used, left here for backwards compatibility");
+
 static const bool FLAGS_deletepercent_dummy __attribute__((unused)) =
    RegisterFlagValidator(&FLAGS_deletepercent, &ValidateInt32Percent);
 static const bool FLAGS_table_cache_numshardbits_dummy __attribute__((unused)) =
@@ -584,9 +604,9 @@ class RandomGenerator {
  }

  Slice Generate(unsigned int len) {
+    assert(len <= data_.size());
    if (pos_ + len > data_.size()) {
      pos_ = 0;
-      assert(len < data_.size());
    }
    pos_ += len;
    return Slice(data_.data() + pos_ - len, len);
@@ -664,7 +684,7 @@ class Stats {
  void SetId(int id) { id_ = id; }
  void SetExcludeFromMerge() { exclude_from_merge_ = true; }

-  void FinishedSingleOp(DB* db) {
+  void FinishedOps(DB* db, int64_t num_ops) {
    if (FLAGS_histogram) {
      double now = FLAGS_env->NowMicros();
      double micros = now - last_op_finish_;
@@ -676,7 +696,7 @@ class Stats {
      last_op_finish_ = now;
    }

-    done_++;
+    done_ += num_ops;
    if (done_ >= next_report_) {
      if (!FLAGS_stats_interval) {
        if      (next_report_ < 1000)   next_report_ += 100;
@@ -722,7 +742,7 @@ class Stats {

  void Report(const Slice& name) {
    // Pretend at least one op was done in case we are running a benchmark
-    // that does not call FinishedSingleOp().
+    // that does not call FinishedOps().
    if (done_ < 1) done_ = 1;

    std::string extra;
@@ -837,6 +857,15 @@ class Benchmark {
  int64_t writes_;
  int64_t readwrites_;
  int64_t merge_keys_;
+
+  bool SanityCheck() {
+    if (FLAGS_compression_ratio > 1) {
+      fprintf(stderr, "compression_ratio should be between 0 and 1\n");
+      return false;
+    }
+    return true;
+  }
+
  void PrintHeader() {
    PrintEnvironment();
    fprintf(stdout, "Keys:       %d bytes each\n", FLAGS_key_size);
@@ -854,6 +883,18 @@ class Benchmark {
              * num_)
             / 1048576.0));
    fprintf(stdout, "Write rate limit: %d\n", FLAGS_writes_per_second);
+    if (FLAGS_enable_numa) {
+      fprintf(stderr, "Running in NUMA enabled mode.\n");
+#ifndef NUMA
+      fprintf(stderr, "NUMA is not defined in the system.\n");
+      exit(1);
+#else
+      if (numa_available() == -1) {
+        fprintf(stderr, "NUMA is not supported by the system.\n");
+        exit(1);
+      }
+#endif
+    }
    switch (FLAGS_compression_type_e) {
      case rocksdb::kNoCompression:
        fprintf(stdout, "Compression: none\n");
@@ -1116,6 +1157,9 @@ class Benchmark {
  }

  void Run() {
+    if (!SanityCheck()) {
+      exit(1);
+    }
    PrintHeader();
    Open();
    const char* benchmarks = FLAGS_benchmarks.c_str();
@@ -1336,7 +1380,25 @@ class Benchmark {
    shared.start = false;

    ThreadArg* arg = new ThreadArg[n];
+
    for (int i = 0; i < n; i++) {
+#ifdef NUMA
+      if (FLAGS_enable_numa) {
+        // Performs a local allocation of memory to threads in numa node.
+        int n_nodes = numa_num_task_nodes();  // Number of nodes in NUMA.
+        numa_exit_on_error = 1;
+        int numa_node = i % n_nodes;
+        bitmask* nodes = numa_allocate_nodemask();
+        numa_bitmask_clearall(nodes);
+        numa_bitmask_setbit(nodes, numa_node);
+        // numa_bind() call binds the process to the node and these
+        // properties are passed on to the thread that is created in
+        // StartThread method called later in the loop.
+        numa_bind(nodes);
+        numa_set_strict(1);
+        numa_free_nodemask(nodes);
+      }
+#endif
      arg[i].bm = this;
      arg[i].method = method;
      arg[i].shared = &shared;
@@ -1379,7 +1441,7 @@ class Benchmark {
    uint32_t crc = 0;
    while (bytes < 500 * 1048576) {
      crc = crc32c::Value(data.data(), size);
-      thread->stats.FinishedSingleOp(nullptr);
+      thread->stats.FinishedOps(nullptr, 1);
      bytes += size;
    }
    // Print so result is not dead
@@ -1398,7 +1460,7 @@ class Benchmark {
    unsigned int xxh32 = 0;
    while (bytes < 500 * 1048576) {
      xxh32 = XXH32(data.data(), size, 0);
-      thread->stats.FinishedSingleOp(nullptr);
+      thread->stats.FinishedOps(nullptr, 1);
      bytes += size;
    }
    // Print so result is not dead
@@ -1419,7 +1481,7 @@ class Benchmark {
        ptr = ap.Acquire_Load();
      }
      count++;
-      thread->stats.FinishedSingleOp(nullptr);
+      thread->stats.FinishedOps(nullptr, 1);
    }
    if (ptr == nullptr) exit(1); // Disable unused variable warning.
  }
@@ -1460,7 +1522,7 @@ class Benchmark {
      }
      produced += compressed.size();
      bytes += input.size();
-      thread->stats.FinishedSingleOp(nullptr);
+      thread->stats.FinishedOps(nullptr, 1);
    }

    if (!ok) {
@@ -1541,7 +1603,7 @@ class Benchmark {
      }
      delete[] uncompressed;
      bytes += input.size();
-      thread->stats.FinishedSingleOp(nullptr);
+      thread->stats.FinishedOps(nullptr, 1);
    }

    if (!ok) {
@@ -1568,8 +1630,9 @@ class Benchmark {
    options.max_background_flushes = FLAGS_max_background_flushes;
    options.compaction_style = FLAGS_compaction_style_e;
    options.block_size = FLAGS_block_size;
+    options.block_restart_interval = FLAGS_block_restart_interval;
    options.filter_policy = filter_policy_;
-    if (FLAGS_use_plain_table) {
+    if (FLAGS_prefix_size != 0) {
      options.prefix_extractor.reset(
          NewFixedPrefixTransform(FLAGS_prefix_size));
    }
@@ -1630,8 +1693,22 @@ class Benchmark {
      if (bloom_bits_per_key < 0) {
        bloom_bits_per_key = 0;
      }
+
+      PlainTableOptions plain_table_options;
+      plain_table_options.user_key_len = FLAGS_key_size;
+      plain_table_options.bloom_bits_per_key = bloom_bits_per_key;
+      plain_table_options.hash_table_ratio = 0.75;
      options.table_factory = std::shared_ptr<TableFactory>(
-          NewPlainTableFactory(FLAGS_key_size, bloom_bits_per_key, 0.75));
+          NewPlainTableFactory(plain_table_options));
+    } else {
+      BlockBasedTableOptions block_based_options;
+      if (FLAGS_use_hash_search) {
+        block_based_options.index_type = BlockBasedTableOptions::kHashSearch;
+      } else {
+        block_based_options.index_type = BlockBasedTableOptions::kBinarySearch;
+      }
+      options.table_factory.reset(
+          NewBlockBasedTableFactory(block_based_options));
    }
    if (FLAGS_max_bytes_for_level_multiplier_additional_v.size() > 0) {
      if (FLAGS_max_bytes_for_level_multiplier_additional_v.size() !=
@@ -1663,7 +1740,6 @@ class Benchmark {
        options.compression_per_level[i] = FLAGS_compression_type_e;
      }
    }
-    options.disable_seek_compaction = FLAGS_disable_seek_compaction;
    options.delete_obsolete_files_period_micros =
      FLAGS_delete_obsolete_files_period_micros;
    options.soft_rate_limit = FLAGS_soft_rate_limit;
@@ -1851,9 +1927,9 @@ class Benchmark {
        GenerateKeyFromInt(key_gens[id]->Next(), FLAGS_num, &key);
        batch.Put(key, gen.Generate(value_size_));
        bytes += value_size_ + key_size_;
-        thread->stats.FinishedSingleOp(db_to_write);
      }
      s = db_to_write->Write(write_options_, &batch);
+      thread->stats.FinishedOps(db_to_write, entries_per_batch_);
      if (!s.ok()) {
        fprintf(stderr, "put error: %s\n", s.ToString().c_str());
        exit(1);
@@ -1878,7 +1954,7 @@ class Benchmark {
    int64_t bytes = 0;
    for (iter->SeekToFirst(); i < reads_ && iter->Valid(); iter->Next()) {
      bytes += iter->key().size() + iter->value().size();
-      thread->stats.FinishedSingleOp(db);
+      thread->stats.FinishedOps(db, 1);
      ++i;
    }
    delete iter;
@@ -1901,7 +1977,7 @@ class Benchmark {
    int64_t bytes = 0;
    for (iter->SeekToLast(); i < reads_ && iter->Valid(); iter->Prev()) {
      bytes += iter->key().size() + iter->value().size();
-      thread->stats.FinishedSingleOp(db_);
+      thread->stats.FinishedOps(db_, 1);
      ++i;
    }
    delete iter;
@@ -1924,7 +2000,7 @@ class Benchmark {
      if (db->Get(options, key, &value).ok()) {
        found++;
      }
-      thread->stats.FinishedSingleOp(db_);
+      thread->stats.FinishedOps(db_, 1);
    }

    char msg[100];
@@ -1984,7 +2060,7 @@ class Benchmark {
      DB* db = SelectDB(thread);
      Iterator* iter = db->NewIterator(options);
      delete iter;
-      thread->stats.FinishedSingleOp(db);
+      thread->stats.FinishedOps(db, 1);
    }
  }

@@ -2048,7 +2124,7 @@ class Benchmark {
      if (iter_to_use->Valid() && iter_to_use->key().compare(key) == 0) {
        found++;
      }
-      thread->stats.FinishedSingleOp(db_);
+      thread->stats.FinishedOps(db_, 1);
    }
    delete single_iter;
    for (auto iter : multi_iters) {
@@ -2086,9 +2162,9 @@ class Benchmark {
        const int64_t k = seq ? i + j : (thread->rand.Next() % FLAGS_num);
        GenerateKeyFromInt(k, FLAGS_num, &key);
        batch.Delete(key);
-        thread->stats.FinishedSingleOp(db);
      }
      auto s = db->Write(write_options_, &batch);
+      thread->stats.FinishedOps(db, entries_per_batch_);
      if (!s.ok()) {
        fprintf(stderr, "del error: %s\n", s.ToString().c_str());
        exit(1);
@@ -2148,7 +2224,7 @@ class Benchmark {
        fprintf(stderr, "put error: %s\n", s.ToString().c_str());
        exit(1);
      }
-      thread->stats.FinishedSingleOp(db_);
+      thread->stats.FinishedOps(db_, 1);

      ++num_writes;
      if (writes_per_second_by_10 && num_writes >= writes_per_second_by_10) {
@@ -2308,7 +2384,7 @@ class Benchmark {
        deletes_done++;
      }

-      thread->stats.FinishedSingleOp(db_);
+      thread->stats.FinishedOps(db_, 1);
    }
    char msg[100];
    snprintf(msg, sizeof(msg),
@@ -2366,7 +2442,7 @@ class Benchmark {
        put_weight--;
        writes_done++;
      }
-      thread->stats.FinishedSingleOp(db);
+      thread->stats.FinishedOps(db, 1);
    }
    char msg[100];
    snprintf(msg, sizeof(msg), "( reads:%" PRIu64 " writes:%" PRIu64 \
@@ -2400,7 +2476,7 @@ class Benchmark {
        fprintf(stderr, "put error: %s\n", s.ToString().c_str());
        exit(1);
      }
-      thread->stats.FinishedSingleOp(db);
+      thread->stats.FinishedOps(db, 1);
    }
    char msg[100];
    snprintf(msg, sizeof(msg),
@@ -2447,7 +2523,7 @@ class Benchmark {
        fprintf(stderr, "put error: %s\n", s.ToString().c_str());
        exit(1);
      }
-      thread->stats.FinishedSingleOp(db_);
+      thread->stats.FinishedOps(db_, 1);
    }

    char msg[100];
@@ -2483,7 +2559,7 @@ class Benchmark {
        fprintf(stderr, "merge error: %s\n", s.ToString().c_str());
        exit(1);
      }
-      thread->stats.FinishedSingleOp(db_);
+      thread->stats.FinishedOps(db_, 1);
    }

    // Print some statistics
@@ -2544,7 +2620,7 @@ class Benchmark {

      }

-      thread->stats.FinishedSingleOp(db_);
+      thread->stats.FinishedOps(db_, 1);
    }

    char msg[100];
@@ -98,7 +98,7 @@ Status DBImpl::GetLiveFiles(std::vector<std::string>& ret,
  }

  // Make a set of all of the live *.sst files
-  std::set<uint64_t> live;
+  std::vector<FileDescriptor> live;
  for (auto cfd : *versions_->GetColumnFamilySet()) {
    cfd->current()->AddLiveFiles(&live);
  }
@@ -109,7 +109,7 @@ Status DBImpl::GetLiveFiles(std::vector<std::string>& ret,
  // create names of the live files. The names are not absolute
  // paths, instead they are relative to dbname_;
  for (auto live_file : live) {
-    ret.push_back(TableFileName("", live_file));
+    ret.push_back(MakeTableFileName("", live_file.GetNumber()));
  }

  ret.push_back(CurrentFileName(""));
@@ -10,6 +10,7 @@

 #include <atomic>
 #include <deque>
+#include <limits>
 #include <set>
 #include <utility>
 #include <vector>
@@ -27,7 +28,7 @@
 #include "rocksdb/memtablerep.h"
 #include "rocksdb/transaction_log.h"
 #include "util/autovector.h"
-#include "util/stats_logger.h"
+#include "util/stop_watch.h"
 #include "util/thread_local.h"
 #include "db/internal_stats.h"

@@ -95,13 +96,17 @@ class DBImpl : public DB {
  using DB::GetProperty;
  virtual bool GetProperty(ColumnFamilyHandle* column_family,
                           const Slice& property, std::string* value);
+  using DB::GetIntProperty;
+  virtual bool GetIntProperty(ColumnFamilyHandle* column_family,
+                              const Slice& property, uint64_t* value) override;
  using DB::GetApproximateSizes;
  virtual void GetApproximateSizes(ColumnFamilyHandle* column_family,
                                   const Range* range, int n, uint64_t* sizes);
  using DB::CompactRange;
  virtual Status CompactRange(ColumnFamilyHandle* column_family,
                              const Slice* begin, const Slice* end,
-                              bool reduce_level = false, int target_level = -1);
+                              bool reduce_level = false, int target_level = -1,
+                              uint32_t target_path_id = 0);

  using DB::NumberLevels;
  virtual int NumberLevels(ColumnFamilyHandle* column_family);
@@ -144,8 +149,8 @@ class DBImpl : public DB {
  virtual Status GetDbIdentity(std::string& identity);

  Status RunManualCompaction(ColumnFamilyData* cfd, int input_level,
-                             int output_level, const Slice* begin,
-                             const Slice* end);
+                             int output_level, uint32_t output_path_id,
+                             const Slice* begin, const Slice* end);

 #ifndef ROCKSDB_LITE
  // Extra methods (for testing) that are not in the public DB interface
@@ -198,6 +203,17 @@ class DBImpl : public DB {
  Status TEST_ReadFirstLine(const std::string& fname, SequenceNumber* sequence);
 #endif  // NDEBUG

+  // Structure to store information for candidate files to delete.
+  struct CandidateFileInfo {
+    std::string file_name;
+    uint32_t path_id;
+    CandidateFileInfo(std::string name, uint32_t path)
+        : file_name(name), path_id(path) {}
+    bool operator==(const CandidateFileInfo& other) const {
+      return file_name == other.file_name && path_id == other.path_id;
+    }
+  };
+
  // needed for CleanupIteratorState
  struct DeletionState {
    inline bool HaveSomethingToDelete() const {
@@ -209,10 +225,10 @@ class DBImpl : public DB {
    // a list of all files that we'll consider deleting
    // (every once in a while this is filled up with all files
    // in the DB directory)
-    std::vector<std::string> candidate_files;
+    std::vector<CandidateFileInfo> candidate_files;

    // the list of all live sst files that cannot be deleted
-    std::vector<uint64_t> sst_live;
+    std::vector<FileDescriptor> sst_live;

    // a list of sst files that we need to delete
    std::vector<FileMetaData*> sst_delete_files;
@@ -277,6 +293,7 @@ class DBImpl : public DB {
  const std::string dbname_;
  unique_ptr<VersionSet> versions_;
  const DBOptions options_;
+  Statistics* stats_;

  Iterator* NewInternalIterator(const ReadOptions&, ColumnFamilyData* cfd,
                                SuperVersion* super_version,
@@ -334,7 +351,8 @@ class DBImpl : public DB {
  Status MakeRoomForWrite(ColumnFamilyData* cfd,
                          bool force /* flush even if there is room? */,
                          autovector<SuperVersion*>* superversions_to_free,
-                          autovector<log::Writer*>* logs_to_free);
+                          autovector<log::Writer*>* logs_to_free,
+                          uint64_t expiration_time);

  void BuildBatchGroup(Writer** last_writer,
                       autovector<WriteBatch*>* write_batch_group);
@@ -345,12 +363,8 @@ class DBImpl : public DB {
  // Wait for memtable flushed
  Status WaitForFlushMemTable(ColumnFamilyData* cfd);

-  void MaybeScheduleLogDBDeployStats();
-
-#ifndef ROCKSDB_LITE
-  static void BGLogDBDeployStats(void* db);
-  void LogDBDeployStats();
-#endif  // ROCKSDB_LITE
+  void RecordFlushIOStats();
+  void RecordCompactionIOStats();

  void MaybeScheduleFlushOrCompaction();
  static void BGWorkCompaction(void* db);
@@ -461,12 +475,12 @@ class DBImpl : public DB {
  // * whenever bg_flush_scheduled_ value decreases (i.e. whenever a flush is
  // done, even if it didn't make any progress)
  // * whenever there is an error in background flush or compaction
-  // * whenever bg_logstats_scheduled_ turns to false
  port::CondVar bg_cv_;
  uint64_t logfile_number_;
  unique_ptr<log::Writer> log_;
  bool log_empty_;
  ColumnFamilyHandleImpl* default_cf_handle_;
+  InternalStats* default_cf_internal_stats_;
  unique_ptr<ColumnFamilyMemTablesImpl> column_family_memtables_;
  struct LogFileNumberSize {
    explicit LogFileNumberSize(uint64_t _number)
@@ -485,8 +499,6 @@ class DBImpl : public DB {
  // some code-paths
  bool single_column_family_mode_;

-  std::string host_name_;
-
  std::unique_ptr<Directory> db_directory_;

  // Queue of writers.
@@ -501,7 +513,8 @@ class DBImpl : public DB {

  // Set of table files to protect from deletion because they are
  // part of ongoing compactions.
-  std::set<uint64_t> pending_outputs_;
+  // map from pending file number ID to their path IDs.
+  FileNumToPathIdMap pending_outputs_;

  // At least one compaction or flush job is pending but not yet scheduled
  // because of the max background thread limit.
@@ -518,14 +531,12 @@ class DBImpl : public DB {
  // number of background memtable flush jobs, submitted to the HIGH pool
  int bg_flush_scheduled_;

-  // Has a background stats log thread scheduled?
-  bool bg_logstats_scheduled_;
-
  // Information for a manual compaction
  struct ManualCompaction {
    ColumnFamilyData* cfd;
    int input_level;
    int output_level;
+    uint32_t output_path_id;
    bool done;
    Status status;
    bool in_progress;           // compaction request being processed?
@@ -538,10 +549,6 @@ class DBImpl : public DB {
  // Have we encountered a background error in paranoid mode?
  Status bg_error_;

-  std::unique_ptr<StatsLogger> logger_;
-
-  int64_t volatile last_log_ts;
-
  // shall we disable deletion of obsolete files
  // if 0 the deletion is enabled.
  // if non-zero, files will not be getting deleted
@@ -566,6 +573,7 @@ class DBImpl : public DB {
  bool flush_on_destroy_; // Used when disableWAL is true.

  static const int KEEP_LOG_FILE_NUM = 1000;
+  static const uint64_t kNoTimeOut = std::numeric_limits<uint64_t>::max();
  std::string db_absolute_path_;

  // count of the number of contiguous delaying writes
@@ -625,15 +633,4 @@ extern Options SanitizeOptions(const std::string& db,
                               const Options& src);
 extern DBOptions SanitizeOptions(const std::string& db, const DBOptions& src);

-// Determine compression type, based on user options, level of the output
-// file and whether compression is disabled.
-// If enable_compression is false, then compression is always disabled no
-// matter what the values of the other two parameters are.
-// Otherwise, the compression type is determined based on options and level.
-CompressionType GetCompressionType(const Options& options, int level,
-                                   const bool enable_compression);
-
-// Determine compression type for L0 file written by memtable flush.
-CompressionType GetCompressionFlush(const Options& options);
-
 }  // namespace rocksdb
@@ -85,7 +85,7 @@ Status DBImpl::TEST_CompactRange(int level, const Slice* begin,
       cfd->options()->compaction_style == kCompactionStyleFIFO)
          ? level
          : level + 1;
-  return RunManualCompaction(cfd, level, output_level, begin, end);
+  return RunManualCompaction(cfd, level, output_level, 0, begin, end);
 }

 Status DBImpl::TEST_FlushMemTable(bool wait) {
@@ -65,9 +65,7 @@ Status DBImplReadOnly::Get(const ReadOptions& options,
  if (super_version->mem->Get(lkey, value, &s, merge_context,
                              *cfd->options())) {
  } else {
-    Version::GetStats stats;
-    super_version->current->Get(options, lkey, value, &s, &merge_context,
-                                &stats);
+    super_version->current->Get(options, lkey, value, &s, &merge_context);
  }
  return s;
 }
@@ -78,12 +76,42 @@ Iterator* DBImplReadOnly::NewIterator(const ReadOptions& options,
  auto cfd = cfh->cfd();
  SuperVersion* super_version = cfd->GetSuperVersion()->Ref();
  SequenceNumber latest_snapshot = versions_->LastSequence();
-  Iterator* internal_iter = NewInternalIterator(options, cfd, super_version);
-  return NewDBIterator(
-      env_, *cfd->options(), cfd->user_comparator(), internal_iter,
+  auto db_iter = NewArenaWrappedDbIterator(
+      env_, *cfd->options(), cfd->user_comparator(),
      (options.snapshot != nullptr
           ? reinterpret_cast<const SnapshotImpl*>(options.snapshot)->number_
           : latest_snapshot));
+  auto internal_iter =
+      NewInternalIterator(options, cfd, super_version, db_iter->GetArena());
+  db_iter->SetIterUnderDBIter(internal_iter);
+  return db_iter;
+}
+
+Status DBImplReadOnly::NewIterators(
+    const ReadOptions& options,
+    const std::vector<ColumnFamilyHandle*>& column_families,
+    std::vector<Iterator*>* iterators) {
+  if (iterators == nullptr) {
+    return Status::InvalidArgument("iterators not allowed to be nullptr");
+  }
+  iterators->clear();
+  iterators->reserve(column_families.size());
+  SequenceNumber latest_snapshot = versions_->LastSequence();
+
+  for (auto cfh : column_families) {
+    auto cfd = reinterpret_cast<ColumnFamilyHandleImpl*>(cfh)->cfd();
+    auto db_iter = NewArenaWrappedDbIterator(
+        env_, *cfd->options(), cfd->user_comparator(),
+        options.snapshot != nullptr
+            ? reinterpret_cast<const SnapshotImpl*>(options.snapshot)->number_
+            : latest_snapshot);
+    auto internal_iter = NewInternalIterator(
+        options, cfd, cfd->GetSuperVersion()->Ref(), db_iter->GetArena());
+    db_iter->SetIterUnderDBIter(internal_iter);
+    iterators->push_back(db_iter);
+  }
+
+  return Status::OK();
 }

 Status DB::OpenForReadOnly(const Options& options, const std::string& dbname,
@@ -20,7 +20,6 @@
 #include "rocksdb/db.h"
 #include "rocksdb/env.h"
 #include "port/port.h"
-#include "util/stats_logger.h"

 namespace rocksdb {

@@ -43,11 +42,8 @@ class DBImplReadOnly : public DBImpl {

  virtual Status NewIterators(
      const ReadOptions& options,
-      const std::vector<ColumnFamilyHandle*>& column_family,
-      std::vector<Iterator*>* iterators) {
-   // TODO
-    return Status::NotSupported("Not supported yet.");
-  }
+      const std::vector<ColumnFamilyHandle*>& column_families,
+      std::vector<Iterator*>* iterators);

  using DBImpl::Put;
  virtual Status Put(const WriteOptions& options,
@@ -10,6 +10,7 @@
 #include "db/db_iter.h"
 #include <stdexcept>
 #include <deque>
+#include <string>

 #include "db/filename.h"
 #include "db/dbformat.h"
@@ -69,7 +70,7 @@ class DBIter: public Iterator {
        valid_(false),
        current_entry_is_merged_(false),
        statistics_(options.statistics.get()) {
-    RecordTick(statistics_, NO_ITERATORS, 1);
+    RecordTick(statistics_, NO_ITERATORS);
    max_skip_ = options.max_sequential_skip_in_iterations;
  }
  virtual ~DBIter() {
@@ -109,9 +110,14 @@ class DBIter: public Iterator {
  virtual void SeekToLast();

 private:
+  void PrevInternal();
+  void FindParseableKey(ParsedInternalKey* ikey, Direction direction);
+  bool FindValueForCurrentKey();
+  bool FindValueForCurrentKeyUsingSeek();
+  void FindPrevUserKey();
+  void FindNextUserKey();
  inline void FindNextUserEntry(bool skipping);
  void FindNextUserEntryInternal(bool skipping);
-  void FindPrevUserEntry();
  bool ParseKey(ParsedInternalKey* key);
  void MergeValuesNewToOld();

@@ -133,8 +139,8 @@ class DBIter: public Iterator {
  SequenceNumber const sequence_;

  Status status_;
-  IterKey saved_key_;   // == current key when direction_==kReverse
-  std::string saved_value_;   // == current raw value when direction_==kReverse
+  IterKey saved_key_;
+  std::string saved_value_;
  Direction direction_;
  bool valid_;
  bool current_entry_is_merged_;
@@ -160,20 +166,11 @@ inline bool DBIter::ParseKey(ParsedInternalKey* ikey) {
 void DBIter::Next() {
  assert(valid_);

-  if (direction_ == kReverse) {  // Switch directions?
+  if (direction_ == kReverse) {
+    FindNextUserKey();
    direction_ = kForward;
-    // iter_ is pointing just before the entries for this->key(),
-    // so advance into the range of entries for this->key() and then
-    // use the normal skipping code below.
    if (!iter_->Valid()) {
      iter_->SeekToFirst();
-    } else {
-      iter_->Next();
-    }
-    if (!iter_->Valid()) {
-      valid_ = false;
-      saved_key_.Clear();
-      return;
    }
  }

@@ -185,7 +182,6 @@ void DBIter::Next() {
  FindNextUserEntry(true /* skipping the current user key */);
 }

-
 // PRE: saved_key_ has the current user key if skipping
 // POST: saved_key_ should have the next user key if valid_,
 //       if the current entry is a result of merge
@@ -327,100 +323,228 @@ void DBIter::MergeValuesNewToOld() {

 void DBIter::Prev() {
  assert(valid_);
-
-  // Throw an exception now if merge_operator is provided
-  // TODO: support backward iteration
-  if (user_merge_operator_) {
-    Log(logger_, "Prev not supported yet if merge_operator is provided");
-    throw std::logic_error("DBIter::Prev backward iteration not supported"
-                           " if merge_operator is provided");
-  }
-
-  if (direction_ == kForward) {  // Switch directions?
-    // iter_ is pointing at the current entry.  Scan backwards until
-    // the key changes so we can use the normal reverse scanning code.
-    assert(iter_->Valid());  // Otherwise valid_ would have been false
-    saved_key_.SetKey(ExtractUserKey(iter_->key()));
-    while (true) {
-      iter_->Prev();
-      if (!iter_->Valid()) {
-        valid_ = false;
-        saved_key_.Clear();
-        ClearSavedValue();
-        return;
-      }
-      if (user_comparator_->Compare(ExtractUserKey(iter_->key()),
-                                    saved_key_.GetKey()) < 0) {
-        break;
-      }
-    }
+  if (direction_ == kForward) {
+    FindPrevUserKey();
    direction_ = kReverse;
  }
-
-  FindPrevUserEntry();
+  PrevInternal();
 }

-void DBIter::FindPrevUserEntry() {
-  assert(direction_ == kReverse);
-  uint64_t num_skipped = 0;
-
-  ValueType value_type = kTypeDeletion;
-  bool saved_key_valid = true;
-  if (iter_->Valid()) {
-    do {
-      ParsedInternalKey ikey;
-      if (ParseKey(&ikey) && ikey.sequence <= sequence_) {
-        if ((value_type != kTypeDeletion) &&
-            user_comparator_->Compare(ikey.user_key, saved_key_.GetKey()) < 0) {
-          // We encountered a non-deleted value in entries for previous keys,
-          break;
-        }
-        value_type = ikey.type;
-        if (value_type == kTypeDeletion) {
-          saved_key_.Clear();
-          ClearSavedValue();
-          saved_key_valid = false;
-        } else {
-          Slice raw_value = iter_->value();
-          if (saved_value_.capacity() > raw_value.size() + 1048576) {
-            std::string empty;
-            swap(empty, saved_value_);
-          }
-          saved_key_.SetKey(ExtractUserKey(iter_->key()));
-          saved_value_.assign(raw_value.data(), raw_value.size());
-        }
-      } else {
-        // In the case of ikey.sequence > sequence_, we might have already
-        // iterated to a different user key.
-        saved_key_valid = false;
-      }
-      num_skipped++;
-      // If we have sequentially iterated via numerous keys and still not
-      // found the prev user-key, then it is better to seek so that we can
-      // avoid too many key comparisons. We seek to the first occurence of
-      // our current key by looking for max sequence number.
-      if (saved_key_valid && num_skipped > max_skip_) {
-        num_skipped = 0;
-        std::string last_key;
-        AppendInternalKey(&last_key, ParsedInternalKey(saved_key_.GetKey(),
-                                                       kMaxSequenceNumber,
-                                                       kValueTypeForSeek));
-        iter_->Seek(last_key);
-        RecordTick(statistics_, NUMBER_OF_RESEEKS_IN_ITERATION);
-      } else {
-        iter_->Prev();
-      }
-    } while (iter_->Valid());
+void DBIter::PrevInternal() {
+  if (!iter_->Valid()) {
+    valid_ = false;
+    return;
  }

-  if (value_type == kTypeDeletion) {
-    // End
+  ParsedInternalKey ikey;
+
+  while (iter_->Valid()) {
+    saved_key_.SetKey(ExtractUserKey(iter_->key()));
+    if (FindValueForCurrentKey()) {
+      valid_ = true;
+      if (!iter_->Valid()) {
+        return;
+      }
+      FindParseableKey(&ikey, kReverse);
+      if (user_comparator_->Compare(ikey.user_key, saved_key_.GetKey()) == 0) {
+        FindPrevUserKey();
+      }
+      return;
+    }
+    if (!iter_->Valid()) {
+      break;
+    }
+    FindParseableKey(&ikey, kReverse);
+    if (user_comparator_->Compare(ikey.user_key, saved_key_.GetKey()) == 0) {
+
+      FindPrevUserKey();
+    }
+  }
+  // We haven't found any key - iterator is not valid
+  assert(!iter_->Valid());
+  valid_ = false;
+}
+
+// This function checks, if the entry with biggest sequence_number <= sequence_
+// is non kTypeDeletion. If it's not, we save value in saved_value_
+bool DBIter::FindValueForCurrentKey() {
+  assert(iter_->Valid());
+  // Contains operands for merge operator.
+  std::deque<std::string> operands;
+  // last entry before merge (could be kTypeDeletion or kTypeValue)
+  ValueType last_not_merge_type = kTypeDeletion;
+  ValueType last_key_entry_type = kTypeDeletion;
+
+  ParsedInternalKey ikey;
+  FindParseableKey(&ikey, kReverse);
+
+  size_t num_skipped = 0;
+  while (iter_->Valid() && ikey.sequence <= sequence_ &&
+         (user_comparator_->Compare(ikey.user_key, saved_key_.GetKey()) == 0)) {
+    // We iterate too much: let's use Seek() to avoid too much key comparisons
+    if (num_skipped >= max_skip_) {
+      return FindValueForCurrentKeyUsingSeek();
+    }
+
+    last_key_entry_type = ikey.type;
+    switch (last_key_entry_type) {
+      case kTypeValue:
+        operands.clear();
+        saved_value_ = iter_->value().ToString();
+        last_not_merge_type = kTypeValue;
+        break;
+      case kTypeDeletion:
+        operands.clear();
+        last_not_merge_type = kTypeDeletion;
+        break;
+      case kTypeMerge:
+        assert(user_merge_operator_ != nullptr);
+        operands.push_back(iter_->value().ToString());
+        break;
+      default:
+        assert(false);
+    }
+
+    assert(user_comparator_->Compare(ikey.user_key, saved_key_.GetKey()) == 0);
+    iter_->Prev();
+    ++num_skipped;
+    FindParseableKey(&ikey, kReverse);
+  }
+
+  switch (last_key_entry_type) {
+    case kTypeDeletion:
+      valid_ = false;
+      return false;
+    case kTypeMerge:
+      if (last_not_merge_type == kTypeDeletion) {
+        user_merge_operator_->FullMerge(saved_key_.GetKey(), nullptr, operands,
+                                        &saved_value_, logger_);
+      } else {
+        assert(last_not_merge_type == kTypeValue);
+        std::string last_put_value = saved_value_;
+        Slice temp_slice(last_put_value);
+        user_merge_operator_->FullMerge(saved_key_.GetKey(), &temp_slice,
+                                        operands, &saved_value_, logger_);
+      }
+      break;
+    case kTypeValue:
+      // do nothing - we've already has value in saved_value_
+      break;
+    default:
+      assert(false);
+      break;
+  }
+  valid_ = true;
+  return true;
+}
+
+// This function is used in FindValueForCurrentKey.
+// We use Seek() function instead of Prev() to find necessary value
+bool DBIter::FindValueForCurrentKeyUsingSeek() {
+  std::string last_key;
+  AppendInternalKey(&last_key, ParsedInternalKey(saved_key_.GetKey(), sequence_,
+                                                 kValueTypeForSeek));
+  iter_->Seek(last_key);
+  RecordTick(statistics_, NUMBER_OF_RESEEKS_IN_ITERATION);
+
+  // assume there is at least one parseable key for this user key
+  ParsedInternalKey ikey;
+  FindParseableKey(&ikey, kForward);
+
+  if (ikey.type == kTypeValue || ikey.type == kTypeDeletion) {
+    if (ikey.type == kTypeValue) {
+      saved_value_ = iter_->value().ToString();
+      valid_ = true;
+      return true;
+    }
    valid_ = false;
-    saved_key_.Clear();
-    ClearSavedValue();
-    direction_ = kForward;
-  } else {
+    return false;
+  }
+
+  // kTypeMerge. We need to collect all kTypeMerge values and save them
+  // in operands
+  std::deque<std::string> operands;
+  while (iter_->Valid() &&
+         (user_comparator_->Compare(ikey.user_key, saved_key_.GetKey()) == 0) &&
+         ikey.type == kTypeMerge) {
+    operands.push_front(iter_->value().ToString());
+    iter_->Next();
+    FindParseableKey(&ikey, kForward);
+  }
+
+  if (!iter_->Valid() ||
+      (user_comparator_->Compare(ikey.user_key, saved_key_.GetKey()) != 0) ||
+      ikey.type == kTypeDeletion) {
+    user_merge_operator_->FullMerge(saved_key_.GetKey(), nullptr, operands,
+                                    &saved_value_, logger_);
+
+    // Make iter_ valid and point to saved_key_
+    if (!iter_->Valid() ||
+        (user_comparator_->Compare(ikey.user_key, saved_key_.GetKey()) != 0)) {
+      iter_->Seek(last_key);
+      RecordTick(statistics_, NUMBER_OF_RESEEKS_IN_ITERATION);
+    }
    valid_ = true;
+    return true;
+  }
+
+  const Slice& value = iter_->value();
+  user_merge_operator_->FullMerge(saved_key_.GetKey(), &value, operands,
+                                  &saved_value_, logger_);
+  valid_ = true;
+  return true;
+}
+
+// Used in Next to change directions
+// Go to next user key
+// Don't use Seek(),
+// because next user key will be very close
+void DBIter::FindNextUserKey() {
+  if (!iter_->Valid()) {
+    return;
+  }
+  ParsedInternalKey ikey;
+  FindParseableKey(&ikey, kForward);
+  while (iter_->Valid() &&
+         user_comparator_->Compare(ikey.user_key, saved_key_.GetKey()) != 0) {
+    iter_->Next();
+    FindParseableKey(&ikey, kForward);
+  }
+}
+
+// Go to previous user_key
+void DBIter::FindPrevUserKey() {
+  if (!iter_->Valid()) {
+    return;
+  }
+  size_t num_skipped = 0;
+  ParsedInternalKey ikey;
+  FindParseableKey(&ikey, kReverse);
+  while (iter_->Valid() &&
+         user_comparator_->Compare(ikey.user_key, saved_key_.GetKey()) == 0) {
+    if (num_skipped >= max_skip_) {
+      num_skipped = 0;
+      IterKey last_key;
+      last_key.SetInternalKey(ParsedInternalKey(
+          saved_key_.GetKey(), kMaxSequenceNumber, kValueTypeForSeek));
+      iter_->Seek(last_key.GetKey());
+      RecordTick(statistics_, NUMBER_OF_RESEEKS_IN_ITERATION);
+    }
+
+    iter_->Prev();
+    ++num_skipped;
+    FindParseableKey(&ikey, kReverse);
+  }
+}
+
+// Skip all unparseable keys
+void DBIter::FindParseableKey(ParsedInternalKey* ikey, Direction direction) {
+  while (iter_->Valid() && !ParseKey(ikey)) {
+    if (direction == kReverse) {
+      iter_->Prev();
+    } else {
+      iter_->Next();
+    }
  }
 }

@@ -454,20 +578,13 @@ void DBIter::SeekToFirst() {
 }

 void DBIter::SeekToLast() {
-  // Throw an exception for now if merge_operator is provided
-  // TODO: support backward iteration
-  if (user_merge_operator_) {
-    Log(logger_, "SeekToLast not supported yet if merge_operator is provided");
-    throw std::logic_error("DBIter::SeekToLast: backward iteration not"
-                           " supported if merge_operator is provided");
-  }
-
  direction_ = kReverse;
  ClearSavedValue();
  PERF_TIMER_AUTO(seek_internal_seek_time);
  iter_->SeekToLast();
  PERF_TIMER_STOP(seek_internal_seek_time);
-  FindPrevUserEntry();
+
+  PrevInternal();
 }

 Iterator* NewDBIterator(Env* env, const Options& options,
@@ -1,95 +0,0 @@
-//  Copyright (c) 2013, Facebook, Inc.  All rights reserved.
-//  This source code is licensed under the BSD-style license found in the
-//  LICENSE file in the root directory of this source tree. An additional grant
-//  of patent rights can be found in the PATENTS file in the same directory.
-//
-// Copyright (c) 2011 The LevelDB Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style license that can be
-// found in the LICENSE file. See the AUTHORS file for names of contributors.
-
-#include "db/db_impl.h"
-#include <string>
-#include <stdint.h>
-#include <stdio.h>
-#include "db/version_set.h"
-#include "rocksdb/db.h"
-#include "rocksdb/env.h"
-#include "port/port.h"
-#include "util/mutexlock.h"
-
-namespace rocksdb {
-
-void DBImpl::MaybeScheduleLogDBDeployStats() {
-// we did say maybe
-#ifndef ROCKSDB_LITE
-  // There is a lock in the actual logger.
-  if (!logger_ || options_.db_stats_log_interval < 0
-      || host_name_.empty()) {
-    return;
-  }
-
-  if(bg_logstats_scheduled_ || shutting_down_.Acquire_Load()) {
-    // Already scheduled
-  } else {
-    int64_t current_ts = 0;
-    Status st = env_->GetCurrentTime(&current_ts);
-    if (!st.ok()) {
-      return;
-    }
-    if ((current_ts - last_log_ts) < options_.db_stats_log_interval) {
-      return;
-    }
-    last_log_ts = current_ts;
-    bg_logstats_scheduled_ = true;
-    env_->Schedule(&DBImpl::BGLogDBDeployStats, this);
-  }
-}
-
-void DBImpl::BGLogDBDeployStats(void* db) {
-  DBImpl* db_inst = reinterpret_cast<DBImpl*>(db);
-  db_inst->LogDBDeployStats();
-}
-
-void DBImpl::LogDBDeployStats() {
-  mutex_.Lock();
-
-  if (shutting_down_.Acquire_Load()) {
-    bg_logstats_scheduled_ = false;
-    bg_cv_.SignalAll();
-    mutex_.Unlock();
-    return;
-  }
-
-  char tmp_ver[100];
-  sprintf(tmp_ver, "%d.%d", kMajorVersion, kMinorVersion);
-  std::string version_info(tmp_ver);
-
-  uint64_t file_total_size = 0;
-  uint32_t file_total_num = 0;
-  Version* current = default_cf_handle_->cfd()->current();
-  for (int i = 0; i < current->NumberLevels(); i++) {
-    file_total_num += current->NumLevelFiles(i);
-    file_total_size += current->NumLevelBytes(i);
-  }
-
-  Version::LevelSummaryStorage scratch;
-  const char* file_num_summary = current->LevelSummary(&scratch);
-  std::string file_num_per_level(file_num_summary);
-  std::string data_size_per_level(file_num_summary);
-
-  mutex_.Unlock();
-
-  int64_t unix_ts;
-  env_->GetCurrentTime(&unix_ts);
-
-  logger_->Log_Deploy_Stats(version_info, host_name_,
-      db_absolute_path_, file_total_size, file_total_num, file_num_per_level,
-      data_size_per_level, unix_ts);
-
-  mutex_.Lock();
-  bg_logstats_scheduled_ = false;
-  bg_cv_.SignalAll();
-  mutex_.Unlock();
-#endif
-}
-}
@@ -12,6 +12,7 @@
 #include <set>
 #include <unistd.h>
 #include <unordered_set>
+#include <utility>

 #include "db/dbformat.h"
 #include "db/db_impl.h"
@@ -27,6 +28,7 @@
 #include "rocksdb/slice.h"
 #include "rocksdb/slice_transform.h"
 #include "rocksdb/table.h"
+#include "rocksdb/options.h"
 #include "rocksdb/table_properties.h"
 #include "table/block_based_table_factory.h"
 #include "table/plain_table_factory.h"
@@ -35,6 +37,7 @@
 #include "utilities/merge_operators.h"
 #include "util/logging.h"
 #include "util/mutexlock.h"
+#include "util/rate_limiter.h"
 #include "util/statistics.h"
 #include "util/testharness.h"
 #include "util/sync_point.h"
@@ -98,7 +101,12 @@ class AtomicCounter {
    count_ = 0;
  }
 };
+}  // namespace anon

+static std::string Key(int i) {
+  char buf[100];
+  snprintf(buf, sizeof(buf), "key%06d", i);
+  return std::string(buf);
 }

 // Special Env used to delay background operations
@@ -130,6 +138,8 @@ class SpecialEnv : public EnvWrapper {

  anon::AtomicCounter sleep_counter_;

+  std::atomic<int64_t> bytes_written_;
+
  explicit SpecialEnv(Env* base) : EnvWrapper(base) {
    delay_sstable_sync_.Release_Store(nullptr);
    no_space_.Release_Store(nullptr);
@@ -139,7 +149,8 @@ class SpecialEnv : public EnvWrapper {
    manifest_sync_error_.Release_Store(nullptr);
    manifest_write_error_.Release_Store(nullptr);
    log_write_error_.Release_Store(nullptr);
-   }
+    bytes_written_ = 0;
+  }

  Status NewWritableFile(const std::string& f, unique_ptr<WritableFile>* r,
                         const EnvOptions& soptions) {
@@ -158,6 +169,7 @@ class SpecialEnv : public EnvWrapper {
          // Drop writes on the floor
          return Status::OK();
        } else {
+          env_->bytes_written_ += data.size();
          return base_->Append(data);
        }
      }
@@ -169,6 +181,9 @@ class SpecialEnv : public EnvWrapper {
        }
        return base_->Sync();
      }
+      void SetIOPriority(Env::IOPriority pri) {
+        base_->SetIOPriority(pri);
+      }
    };
    class ManifestFile : public WritableFile {
     private:
@@ -355,7 +370,12 @@ class DBTest {

  ~DBTest() {
    Close();
-    ASSERT_OK(DestroyDB(dbname_, Options()));
+    Options options;
+    options.db_paths.emplace_back(dbname_, 0);
+    options.db_paths.emplace_back(dbname_ + "_2", 0);
+    options.db_paths.emplace_back(dbname_ + "_3", 0);
+    options.db_paths.emplace_back(dbname_ + "_4", 0);
+    ASSERT_OK(DestroyDB(dbname_, options));
    delete env_;
    delete filter_policy_;
  }
@@ -436,7 +456,8 @@ class DBTest {
    switch (option_config_) {
      case kHashSkipList:
        options.prefix_extractor.reset(NewFixedPrefixTransform(1));
-        options.memtable_factory.reset(NewHashSkipListRepFactory());
+        options.memtable_factory.reset(
+            NewHashSkipListRepFactory(16));
        break;
      case kPlainTableFirstBytePrefix:
        options.table_factory.reset(new PlainTableFactory());
@@ -466,7 +487,7 @@ class DBTest {
        options.db_log_dir = test::TmpDir();
        break;
      case kWalDir:
-        options.wal_dir = "/tmp/wal";
+        options.wal_dir = test::TmpDir() + "/wal";
        break;
      case kManifestFileSize:
        options.max_manifest_file_size = 50; // 50 bytes
@@ -487,7 +508,8 @@ class DBTest {
        break;
      case kHashLinkList:
        options.prefix_extractor.reset(NewFixedPrefixTransform(1));
-        options.memtable_factory.reset(NewHashLinkListRepFactory(4, 0));
+        options.memtable_factory.reset(
+            NewHashLinkListRepFactory(4, 0, 3, true, 4));
        break;
      case kHashCuckoo:
        options.memtable_factory.reset(
@@ -895,6 +917,30 @@ class DBTest {
    return property;
  }

+  int GetSstFileCount(std::string path) {
+    std::vector<std::string> files;
+    env_->GetChildren(path, &files);
+
+    int sst_count = 0;
+    uint64_t number;
+    FileType type;
+    for (size_t i = 0; i < files.size(); i++) {
+      if (ParseFileName(files[i], &number, &type) && type == kTableFile) {
+        sst_count++;
+      }
+    }
+    return sst_count;
+  }
+
+  void GenerateNewFile(Random* rnd, int* key_idx) {
+    for (int i = 0; i < 11; i++) {
+      ASSERT_OK(Put(Key(*key_idx), RandomString(rnd, (i == 10) ? 1 : 10000)));
+      (*key_idx)++;
+    }
+    dbfull()->TEST_WaitForFlushMemTable();
+    dbfull()->TEST_WaitForCompact();
+  }
+
  std::string IterStatus(Iterator* iter) {
    std::string result;
    if (iter->Valid()) {
@@ -1035,12 +1081,6 @@ class DBTest {

 };

-static std::string Key(int i) {
-  char buf[100];
-  snprintf(buf, sizeof(buf), "key%06d", i);
-  return std::string(buf);
-}
-
 static long TestGetTickerCount(const Options& options, Tickers ticker_type) {
  return options.statistics->getTickerCount(ticker_type);
 }
@@ -2318,6 +2358,7 @@ TEST(DBTest, NumImmutableMemTable) {
    std::string big_value(1000000 * 2, 'x');
    std::string num;
    SetPerfLevel(kEnableTime);;
+    ASSERT_TRUE(GetPerfLevel() == kEnableTime);

    ASSERT_OK(dbfull()->Put(writeOpt, handles_[1], "k1", big_value));
    ASSERT_TRUE(dbfull()->GetProperty(handles_[1],
@@ -2380,6 +2421,7 @@ TEST(DBTest, NumImmutableMemTable) {
    // break if we change the default skiplist implementation
    ASSERT_EQ(num, "200");
    SetPerfLevel(kDisable);
+    ASSERT_TRUE(GetPerfLevel() == kDisable);
  } while (ChangeCompactOptions());
 }

@@ -2444,6 +2486,7 @@ TEST(DBTest, GetProperty) {

  std::string big_value(1000000 * 2, 'x');
  std::string num;
+  uint64_t int_num;
  SetPerfLevel(kEnableTime);

  ASSERT_OK(dbfull()->Put(writeOpt, "k1", big_value));
@@ -2453,11 +2496,14 @@ TEST(DBTest, GetProperty) {
  ASSERT_EQ(num, "0");
  ASSERT_TRUE(dbfull()->GetProperty("rocksdb.compaction-pending", &num));
  ASSERT_EQ(num, "0");
+  ASSERT_TRUE(dbfull()->GetProperty("rocksdb.estimate-num-keys", &num));
+  ASSERT_EQ(num, "1");
  perf_context.Reset();

  ASSERT_OK(dbfull()->Put(writeOpt, "k2", big_value));
  ASSERT_TRUE(dbfull()->GetProperty("rocksdb.num-immutable-mem-table", &num));
  ASSERT_EQ(num, "1");
+  ASSERT_OK(dbfull()->Delete(writeOpt, "k-non-existing"));
  ASSERT_OK(dbfull()->Put(writeOpt, "k3", big_value));
  ASSERT_TRUE(dbfull()->GetProperty("rocksdb.num-immutable-mem-table", &num));
  ASSERT_EQ(num, "2");
@@ -2465,6 +2511,19 @@ TEST(DBTest, GetProperty) {
  ASSERT_EQ(num, "1");
  ASSERT_TRUE(dbfull()->GetProperty("rocksdb.compaction-pending", &num));
  ASSERT_EQ(num, "0");
+  ASSERT_TRUE(dbfull()->GetProperty("rocksdb.estimate-num-keys", &num));
+  ASSERT_EQ(num, "4");
+  // Verify the same set of properties through GetIntProperty
+  ASSERT_TRUE(
+      dbfull()->GetIntProperty("rocksdb.num-immutable-mem-table", &int_num));
+  ASSERT_EQ(int_num, 2U);
+  ASSERT_TRUE(
+      dbfull()->GetIntProperty("rocksdb.mem-table-flush-pending", &int_num));
+  ASSERT_EQ(int_num, 1U);
+  ASSERT_TRUE(dbfull()->GetIntProperty("rocksdb.compaction-pending", &int_num));
+  ASSERT_EQ(int_num, 0);
+  ASSERT_TRUE(dbfull()->GetIntProperty("rocksdb.estimate-num-keys", &int_num));
+  ASSERT_EQ(int_num, 4);

  sleeping_task_high.WakeUp();
  sleeping_task_high.WaitUntilDone();
@@ -2477,6 +2536,8 @@ TEST(DBTest, GetProperty) {
  ASSERT_EQ(num, "0");
  ASSERT_TRUE(dbfull()->GetProperty("rocksdb.compaction-pending", &num));
  ASSERT_EQ(num, "1");
+  ASSERT_TRUE(dbfull()->GetProperty("rocksdb.estimate-num-keys", &num));
+  ASSERT_EQ(num, "4");
  sleeping_task_low.WakeUp();
  sleeping_task_low.WaitUntilDone();
 }
@@ -2726,6 +2787,119 @@ TEST(DBTest, CompactionTrigger) {
  ASSERT_EQ(NumTableFilesAtLevel(1, 1), 1);
 }

+namespace {
+static const int kCDTValueSize = 1000;
+static const int kCDTKeysPerBuffer = 4;
+static const int kCDTNumLevels = 8;
+Options DeletionTriggerOptions() {
+  Options options;
+  options.compression = kNoCompression;
+  options.write_buffer_size = kCDTKeysPerBuffer * (kCDTValueSize + 24);
+  options.min_write_buffer_number_to_merge = 1;
+  options.num_levels = kCDTNumLevels;
+  options.max_mem_compaction_level = 0;
+  options.level0_file_num_compaction_trigger = 1;
+  options.target_file_size_base = options.write_buffer_size * 2;
+  options.target_file_size_multiplier = 2;
+  options.max_bytes_for_level_base =
+      options.target_file_size_base * options.target_file_size_multiplier;
+  options.max_bytes_for_level_multiplier = 2;
+  options.disable_auto_compactions = false;
+  return options;
+}
+}  // anonymous namespace
+
+TEST(DBTest, CompactionDeletionTrigger) {
+  Options options = DeletionTriggerOptions();
+  options.create_if_missing = true;
+
+  for (int tid = 0; tid < 2; ++tid) {
+    uint64_t db_size[2];
+
+    DestroyAndReopen(&options);
+    Random rnd(301);
+
+    const int kTestSize = kCDTKeysPerBuffer * 512;
+    std::vector<std::string> values;
+    for (int k = 0; k < kTestSize; ++k) {
+      values.push_back(RandomString(&rnd, kCDTValueSize));
+      ASSERT_OK(Put(Key(k), values[k]));
+    }
+    dbfull()->TEST_WaitForFlushMemTable();
+    dbfull()->TEST_WaitForCompact();
+    db_size[0] = Size(Key(0), Key(kTestSize - 1));
+
+    for (int k = 0; k < kTestSize; ++k) {
+      ASSERT_OK(Delete(Key(k)));
+    }
+    dbfull()->TEST_WaitForFlushMemTable();
+    dbfull()->TEST_WaitForCompact();
+    db_size[1] = Size(Key(0), Key(kTestSize - 1));
+
+    // must have much smaller db size.
+    ASSERT_GT(db_size[0] / 3, db_size[1]);
+
+    // repeat the test with universal compaction
+    options.compaction_style = kCompactionStyleUniversal;
+    options.num_levels = 1;
+  }
+}
+
+TEST(DBTest, CompactionDeletionTriggerReopen) {
+  for (int tid = 0; tid < 2; ++tid) {
+    uint64_t db_size[3];
+    Options options = DeletionTriggerOptions();
+    options.create_if_missing = true;
+
+    DestroyAndReopen(&options);
+    Random rnd(301);
+
+    // round 1 --- insert key/value pairs.
+    const int kTestSize = kCDTKeysPerBuffer * 512;
+    std::vector<std::string> values;
+    for (int k = 0; k < kTestSize; ++k) {
+      values.push_back(RandomString(&rnd, kCDTValueSize));
+      ASSERT_OK(Put(Key(k), values[k]));
+    }
+    dbfull()->TEST_WaitForFlushMemTable();
+    dbfull()->TEST_WaitForCompact();
+    db_size[0] = Size(Key(0), Key(kTestSize - 1));
+    Close();
+
+    // round 2 --- disable auto-compactions and issue deletions.
+    options.create_if_missing = false;
+    options.disable_auto_compactions = true;
+    Reopen(&options);
+
+    for (int k = 0; k < kTestSize; ++k) {
+      ASSERT_OK(Delete(Key(k)));
+    }
+    db_size[1] = Size(Key(0), Key(kTestSize - 1));
+    Close();
+    // as auto_compaction is off, we shouldn't see too much reduce
+    // in db size.
+    ASSERT_LT(db_size[0] / 3, db_size[1]);
+
+    // round 3 --- reopen db with auto_compaction on and see if
+    // deletion compensation still work.
+    options.disable_auto_compactions = false;
+    Reopen(&options);
+    // insert relatively small amount of data to trigger auto compaction.
+    for (int k = 0; k < kTestSize / 10; ++k) {
+      ASSERT_OK(Put(Key(k), values[k]));
+    }
+    dbfull()->TEST_WaitForFlushMemTable();
+    dbfull()->TEST_WaitForCompact();
+    db_size[2] = Size(Key(0), Key(kTestSize - 1));
+    // this time we're expecting significant drop in size.
+    ASSERT_GT(db_size[0] / 3, db_size[2]);
+
+    // repeat the test with universal compaction
+    options.compaction_style = kCompactionStyleUniversal;
+    options.num_levels = 1;
+  }
+}
+
 // This is a static filter used for filtering
 // kvs during the compaction process.
 static int cfilter_count;
@@ -3319,6 +3493,209 @@ TEST(DBTest, UniversalCompactionCompressRatio2) {
  ASSERT_LT((int)dbfull()->TEST_GetLevel0TotalSize(),
            120000 * 12 * 0.8 + 120000 * 2);
 }
+
+TEST(DBTest, FailMoreDbPaths) {
+  Options options;
+  options.db_paths.emplace_back(dbname_, 10000000);
+  options.db_paths.emplace_back(dbname_ + "_2", 1000000);
+  options.db_paths.emplace_back(dbname_ + "_3", 1000000);
+  options.db_paths.emplace_back(dbname_ + "_4", 1000000);
+  options.db_paths.emplace_back(dbname_ + "_5", 1000000);
+  ASSERT_TRUE(TryReopen(&options).IsNotSupported());
+}
+
+TEST(DBTest, UniversalCompactionSecondPathRatio) {
+  Options options;
+  options.db_paths.emplace_back(dbname_, 500 * 1024);
+  options.db_paths.emplace_back(dbname_ + "_2", 1024 * 1024 * 1024);
+  options.compaction_style = kCompactionStyleUniversal;
+  options.write_buffer_size = 100 << 10;  // 100KB
+  options.level0_file_num_compaction_trigger = 2;
+  options.num_levels = 1;
+  options = CurrentOptions(options);
+
+  std::vector<std::string> filenames;
+  env_->GetChildren(options.db_paths[1].path, &filenames);
+  // Delete archival files.
+  for (size_t i = 0; i < filenames.size(); ++i) {
+    env_->DeleteFile(options.db_paths[1].path + "/" + filenames[i]);
+  }
+  env_->DeleteDir(options.db_paths[1].path);
+  Reopen(&options);
+
+  Random rnd(301);
+  int key_idx = 0;
+
+  // First three 110KB files are not going to second path.
+  // After that, (100K, 200K)
+  for (int num = 0; num < 3; num++) {
+    GenerateNewFile(&rnd, &key_idx);
+  }
+
+  // Another 110KB triggers a compaction to 400K file to second path
+  GenerateNewFile(&rnd, &key_idx);
+  ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path));
+
+  // (1, 4)
+  GenerateNewFile(&rnd, &key_idx);
+  ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path));
+  ASSERT_EQ(1, GetSstFileCount(dbname_));
+
+  // (1,1,4) -> (2, 4)
+  GenerateNewFile(&rnd, &key_idx);
+  ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path));
+  ASSERT_EQ(1, GetSstFileCount(dbname_));
+
+  // (1, 2, 4)
+  GenerateNewFile(&rnd, &key_idx);
+  ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path));
+  ASSERT_EQ(2, GetSstFileCount(dbname_));
+
+  // (1, 1, 2, 4) -> (8)
+  GenerateNewFile(&rnd, &key_idx);
+  ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path));
+  ASSERT_EQ(0, GetSstFileCount(dbname_));
+
+  // (1, 8)
+  GenerateNewFile(&rnd, &key_idx);
+  ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path));
+  ASSERT_EQ(1, GetSstFileCount(dbname_));
+
+  // (1, 1, 8) -> (2, 8)
+  GenerateNewFile(&rnd, &key_idx);
+  ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path));
+  ASSERT_EQ(1, GetSstFileCount(dbname_));
+
+  // (1, 2, 8)
+  GenerateNewFile(&rnd, &key_idx);
+  ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path));
+  ASSERT_EQ(2, GetSstFileCount(dbname_));
+
+  // (1, 1, 2, 8) -> (4, 8)
+  GenerateNewFile(&rnd, &key_idx);
+  ASSERT_EQ(2, GetSstFileCount(options.db_paths[1].path));
+  ASSERT_EQ(0, GetSstFileCount(dbname_));
+
+  // (1, 4, 8)
+  GenerateNewFile(&rnd, &key_idx);
+  ASSERT_EQ(2, GetSstFileCount(options.db_paths[1].path));
+  ASSERT_EQ(1, GetSstFileCount(dbname_));
+
+  for (int i = 0; i < key_idx; i++) {
+    auto v = Get(Key(i));
+    ASSERT_NE(v, "NOT_FOUND");
+    ASSERT_TRUE(v.size() == 1 || v.size() == 10000);
+  }
+
+  Reopen(&options);
+
+  for (int i = 0; i < key_idx; i++) {
+    auto v = Get(Key(i));
+    ASSERT_NE(v, "NOT_FOUND");
+    ASSERT_TRUE(v.size() == 1 || v.size() == 10000);
+  }
+
+  Destroy(&options);
+}
+
+TEST(DBTest, UniversalCompactionFourPaths) {
+  Options options;
+  options.db_paths.emplace_back(dbname_, 300 * 1024);
+  options.db_paths.emplace_back(dbname_ + "_2", 300 * 1024);
+  options.db_paths.emplace_back(dbname_ + "_3", 500 * 1024);
+  options.db_paths.emplace_back(dbname_ + "_4", 1024 * 1024 * 1024);
+  options.compaction_style = kCompactionStyleUniversal;
+  options.write_buffer_size = 100 << 10;  // 100KB
+  options.level0_file_num_compaction_trigger = 2;
+  options.num_levels = 1;
+  options = CurrentOptions(options);
+
+  std::vector<std::string> filenames;
+  env_->GetChildren(options.db_paths[1].path, &filenames);
+  // Delete archival files.
+  for (size_t i = 0; i < filenames.size(); ++i) {
+    env_->DeleteFile(options.db_paths[1].path + "/" + filenames[i]);
+  }
+  env_->DeleteDir(options.db_paths[1].path);
+  Reopen(&options);
+
+  Random rnd(301);
+  int key_idx = 0;
+
+  // First three 110KB files are not going to second path.
+  // After that, (100K, 200K)
+  for (int num = 0; num < 3; num++) {
+    GenerateNewFile(&rnd, &key_idx);
+  }
+
+  // Another 110KB triggers a compaction to 400K file to second path
+  GenerateNewFile(&rnd, &key_idx);
+  ASSERT_EQ(1, GetSstFileCount(options.db_paths[2].path));
+
+  // (1, 4)
+  GenerateNewFile(&rnd, &key_idx);
+  ASSERT_EQ(1, GetSstFileCount(options.db_paths[2].path));
+  ASSERT_EQ(1, GetSstFileCount(dbname_));
+
+  // (1,1,4) -> (2, 4)
+  GenerateNewFile(&rnd, &key_idx);
+  ASSERT_EQ(1, GetSstFileCount(options.db_paths[2].path));
+  ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path));
+  ASSERT_EQ(0, GetSstFileCount(dbname_));
+
+  // (1, 2, 4)
+  GenerateNewFile(&rnd, &key_idx);
+  ASSERT_EQ(1, GetSstFileCount(options.db_paths[2].path));
+  ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path));
+  ASSERT_EQ(1, GetSstFileCount(dbname_));
+
+  // (1, 1, 2, 4) -> (8)
+  GenerateNewFile(&rnd, &key_idx);
+  ASSERT_EQ(1, GetSstFileCount(options.db_paths[3].path));
+
+  // (1, 8)
+  GenerateNewFile(&rnd, &key_idx);
+  ASSERT_EQ(1, GetSstFileCount(options.db_paths[3].path));
+  ASSERT_EQ(1, GetSstFileCount(dbname_));
+
+  // (1, 1, 8) -> (2, 8)
+  GenerateNewFile(&rnd, &key_idx);
+  ASSERT_EQ(1, GetSstFileCount(options.db_paths[3].path));
+  ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path));
+
+  // (1, 2, 8)
+  GenerateNewFile(&rnd, &key_idx);
+  ASSERT_EQ(1, GetSstFileCount(options.db_paths[3].path));
+  ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path));
+  ASSERT_EQ(1, GetSstFileCount(dbname_));
+
+  // (1, 1, 2, 8) -> (4, 8)
+  GenerateNewFile(&rnd, &key_idx);
+  ASSERT_EQ(1, GetSstFileCount(options.db_paths[2].path));
+  ASSERT_EQ(1, GetSstFileCount(options.db_paths[3].path));
+
+  // (1, 4, 8)
+  GenerateNewFile(&rnd, &key_idx);
+  ASSERT_EQ(1, GetSstFileCount(options.db_paths[3].path));
+  ASSERT_EQ(1, GetSstFileCount(options.db_paths[2].path));
+  ASSERT_EQ(1, GetSstFileCount(dbname_));
+
+  for (int i = 0; i < key_idx; i++) {
+    auto v = Get(Key(i));
+    ASSERT_NE(v, "NOT_FOUND");
+    ASSERT_TRUE(v.size() == 1 || v.size() == 10000);
+  }
+
+  Reopen(&options);
+
+  for (int i = 0; i < key_idx; i++) {
+    auto v = Get(Key(i));
+    ASSERT_NE(v, "NOT_FOUND");
+    ASSERT_TRUE(v.size() == 1 || v.size() == 10000);
+  }
+
+  Destroy(&options);
+}
 #endif

 TEST(DBTest, ConvertCompactionStyle) {
@@ -4894,6 +5271,54 @@ TEST(DBTest, ManualCompaction) {

 }

+TEST(DBTest, ManualCompactionOutputPathId) {
+  Options options = CurrentOptions();
+  options.create_if_missing = true;
+  options.db_paths.emplace_back(dbname_, 1000000000);
+  options.db_paths.emplace_back(dbname_ + "_2", 1000000000);
+  options.compaction_style = kCompactionStyleUniversal;
+  options.level0_file_num_compaction_trigger = 10;
+  Destroy(&options);
+  DestroyAndReopen(&options);
+  CreateAndReopenWithCF({"pikachu"}, &options);
+  MakeTables(3, "p", "q", 1);
+  dbfull()->TEST_WaitForCompact();
+  ASSERT_EQ("3", FilesPerLevel(1));
+  ASSERT_EQ(3, GetSstFileCount(options.db_paths[0].path));
+  ASSERT_EQ(0, GetSstFileCount(options.db_paths[1].path));
+
+  // Full compaction to DB path 0
+  db_->CompactRange(handles_[1], nullptr, nullptr, false, -1, 1);
+  ASSERT_EQ("1", FilesPerLevel(1));
+  ASSERT_EQ(0, GetSstFileCount(options.db_paths[0].path));
+  ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path));
+
+  ReopenWithColumnFamilies({kDefaultColumnFamilyName, "pikachu"}, &options);
+  ASSERT_EQ("1", FilesPerLevel(1));
+  ASSERT_EQ(0, GetSstFileCount(options.db_paths[0].path));
+  ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path));
+
+  MakeTables(1, "p", "q", 1);
+  ASSERT_EQ("2", FilesPerLevel(1));
+  ASSERT_EQ(1, GetSstFileCount(options.db_paths[0].path));
+  ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path));
+
+  ReopenWithColumnFamilies({kDefaultColumnFamilyName, "pikachu"}, &options);
+  ASSERT_EQ("2", FilesPerLevel(1));
+  ASSERT_EQ(1, GetSstFileCount(options.db_paths[0].path));
+  ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path));
+
+  // Full compaction to DB path 0
+  db_->CompactRange(handles_[1], nullptr, nullptr, false, -1, 0);
+  ASSERT_EQ("1", FilesPerLevel(1));
+  ASSERT_EQ(1, GetSstFileCount(options.db_paths[0].path));
+  ASSERT_EQ(0, GetSstFileCount(options.db_paths[1].path));
+
+  // Fail when compacting to an invalid path ID
+  ASSERT_TRUE(db_->CompactRange(handles_[1], nullptr, nullptr, false, -1, 2)
+                  .IsInvalidArgument());
+}
+
 TEST(DBTest, DBOpen_Options) {
  std::string dbname = test::TmpDir() + "/db_options_test";
  ASSERT_OK(DestroyDB(dbname, Options()));
@@ -5869,69 +6294,6 @@ TEST(DBTest, ReadFirstRecordCache) {
  ASSERT_EQ(env_->sequential_read_counter_.Read(), 1);
 }

-TEST(DBTest, ReadCompaction) {
-  std::string value(4096, '4'); // a string of size 4K
-  {
-    Options options = CurrentOptions();
-    options.create_if_missing = true;
-    options.max_open_files = 20; // only 10 file in file-cache
-    options.target_file_size_base = 512;
-    options.write_buffer_size = 64 * 1024;
-    options.filter_policy = nullptr;
-    options.block_size = 4096;
-    options.no_block_cache = true;
-    options.disable_seek_compaction = false;
-
-    CreateAndReopenWithCF({"pikachu"}, &options);
-
-    // Write 8MB (2000 values, each 4K)
-    ASSERT_EQ(NumTableFilesAtLevel(0, 1), 0);
-    std::vector<std::string> values;
-    for (int i = 0; i < 2000; i++) {
-      ASSERT_OK(Put(1, Key(i), value));
-    }
-
-    // clear level 0 and 1 if necessary.
-    Flush(1);
-    dbfull()->TEST_CompactRange(0, nullptr, nullptr, handles_[1]);
-    dbfull()->TEST_CompactRange(1, nullptr, nullptr, handles_[1]);
-    ASSERT_EQ(NumTableFilesAtLevel(0, 1), 0);
-    ASSERT_EQ(NumTableFilesAtLevel(1, 1), 0);
-
-    // write some new keys into level 0
-    for (int i = 0; i < 2000; i = i + 16) {
-      ASSERT_OK(Put(1, Key(i), value));
-    }
-    Flush(1);
-
-    // Wait for any write compaction to finish
-    dbfull()->TEST_WaitForCompact();
-
-    // remember number of files in each level
-    int l1 = NumTableFilesAtLevel(0, 1);
-    int l2 = NumTableFilesAtLevel(1, 1);
-    int l3 = NumTableFilesAtLevel(2, 1);
-    ASSERT_NE(NumTableFilesAtLevel(0, 1), 0);
-    ASSERT_NE(NumTableFilesAtLevel(1, 1), 0);
-    ASSERT_NE(NumTableFilesAtLevel(2, 1), 0);
-
-    // read a bunch of times, trigger read compaction
-    for (int j = 0; j < 100; j++) {
-      for (int i = 0; i < 2000; i++) {
-        Get(1, Key(i));
-      }
-    }
-    // wait for read compaction to finish
-    env_->SleepForMicroseconds(1000000);
-
-    // verify that the number of files have decreased
-    // in some level, indicating that there was a compaction
-    ASSERT_TRUE(NumTableFilesAtLevel(0, 1) < l1 ||
-                NumTableFilesAtLevel(1, 1) < l2 ||
-                NumTableFilesAtLevel(2, 1) < l3);
-  }
-}
-
 // Multi-threaded test:
 namespace {

@@ -6254,6 +6616,11 @@ class ModelDB: public DB {
                           const Slice& property, std::string* value) {
    return false;
  }
+  using DB::GetIntProperty;
+  virtual bool GetIntProperty(ColumnFamilyHandle* column_family,
+                              const Slice& property, uint64_t* value) override {
+    return false;
+  }
  using DB::GetApproximateSizes;
  virtual void GetApproximateSizes(ColumnFamilyHandle* column_family,
                                   const Range* range, int n, uint64_t* sizes) {
@@ -6264,7 +6631,8 @@ class ModelDB: public DB {
  using DB::CompactRange;
  virtual Status CompactRange(ColumnFamilyHandle* column_family,
                              const Slice* start, const Slice* end,
-                              bool reduce_level, int target_level) {
+                              bool reduce_level, int target_level,
+                              uint32_t output_path_id) {
    return Status::NotSupported("Not supported operation.");
  }

@@ -6641,8 +7009,7 @@ TEST(DBTest, PrefixScan) {
  options.disable_auto_compactions = true;
  options.max_background_compactions = 2;
  options.create_if_missing = true;
-  options.disable_seek_compaction = true;
-  options.memtable_factory.reset(NewHashSkipListRepFactory());
+  options.memtable_factory.reset(NewHashSkipListRepFactory(16));

  // 11 RAND I/Os
  DestroyAndReopen(&options);
@@ -6799,7 +7166,7 @@ TEST(DBTest, TailingIteratorPrefixSeek) {
  options.create_if_missing = true;
  options.disable_auto_compactions = true;
  options.prefix_extractor.reset(NewFixedPrefixTransform(2));
-  options.memtable_factory.reset(NewHashSkipListRepFactory());
+  options.memtable_factory.reset(NewHashSkipListRepFactory(16));
  DestroyAndReopen(&options);
  CreateAndReopenWithCF({"pikachu"}, &options);

@@ -6822,6 +7189,63 @@ TEST(DBTest, TailingIteratorPrefixSeek) {
  ASSERT_TRUE(!iter->Valid());
 }

+TEST(DBTest, TailingIteratorIncomplete) {
+  CreateAndReopenWithCF({"pikachu"});
+  ReadOptions read_options;
+  read_options.tailing = true;
+  read_options.read_tier = kBlockCacheTier;
+
+  std::string key("key");
+  std::string value("value");
+
+  ASSERT_OK(db_->Put(WriteOptions(), key, value));
+
+  std::unique_ptr<Iterator> iter(db_->NewIterator(read_options));
+  iter->SeekToFirst();
+  // we either see the entry or it's not in cache
+  ASSERT_TRUE(iter->Valid() || iter->status().IsIncomplete());
+
+  ASSERT_OK(db_->CompactRange(nullptr, nullptr));
+  iter->SeekToFirst();
+  // should still be true after compaction
+  ASSERT_TRUE(iter->Valid() || iter->status().IsIncomplete());
+}
+
+TEST(DBTest, TailingIteratorSeekToSame) {
+  Options options = CurrentOptions();
+  options.compaction_style = kCompactionStyleUniversal;
+  options.write_buffer_size = 1000;
+  CreateAndReopenWithCF({"pikachu"}, &options);
+
+  ReadOptions read_options;
+  read_options.tailing = true;
+
+  const int NROWS = 10000;
+  // Write rows with keys 00000, 00002, 00004 etc.
+  for (int i = 0; i < NROWS; ++i) {
+    char buf[100];
+    snprintf(buf, sizeof(buf), "%05d", 2*i);
+    std::string key(buf);
+    std::string value("value");
+    ASSERT_OK(db_->Put(WriteOptions(), key, value));
+  }
+
+  std::unique_ptr<Iterator> iter(db_->NewIterator(read_options));
+  // Seek to 00001.  We expect to find 00002.
+  std::string start_key = "00001";
+  iter->Seek(start_key);
+  ASSERT_TRUE(iter->Valid());
+
+  std::string found = iter->key().ToString();
+  ASSERT_EQ("00002", found);
+
+  // Now seek to the same key.  The iterator should remain in the same
+  // position.
+  iter->Seek(found);
+  ASSERT_TRUE(iter->Valid());
+  ASSERT_EQ(found, iter->key().ToString());
+}
+
 TEST(DBTest, BlockBasedTablePrefixIndexTest) {
  // create a DB with block prefix index
  BlockBasedTableOptions table_options;
@@ -6917,6 +7341,201 @@ TEST(DBTest, FIFOCompactionTest) {
    }
  }
 }
+
+TEST(DBTest, SimpleWriteTimeoutTest) {
+  Options options;
+  options.env = env_;
+  options.create_if_missing = true;
+  options.write_buffer_size = 100000;
+  options.max_background_flushes = 0;
+  options.max_write_buffer_number = 2;
+  options.min_write_buffer_number_to_merge = 3;
+  options.max_total_wal_size = std::numeric_limits<uint64_t>::max();
+  WriteOptions write_opt = WriteOptions();
+  write_opt.timeout_hint_us = 0;
+  DestroyAndReopen(&options);
+  // fill the two write buffer
+  ASSERT_OK(Put(Key(1), Key(1) + std::string(100000, 'v'), write_opt));
+  ASSERT_OK(Put(Key(2), Key(2) + std::string(100000, 'v'), write_opt));
+  // As the only two write buffers are full in this moment, the third
+  // Put is expected to be timed-out.
+  write_opt.timeout_hint_us = 300;
+  ASSERT_TRUE(
+      Put(Key(3), Key(3) + std::string(100000, 'v'), write_opt).IsTimedOut());
+}
+
+// Multi-threaded Timeout Test
+namespace {
+
+static const int kValueSize = 1000;
+static const int kWriteBufferSize = 100000;
+
+struct TimeoutWriterState {
+  int id;
+  DB* db;
+  std::atomic<bool> done;
+  std::map<int, std::string> success_kvs;
+};
+
+static void RandomTimeoutWriter(void* arg) {
+  TimeoutWriterState* state = reinterpret_cast<TimeoutWriterState*>(arg);
+  static const uint64_t kTimerBias = 50;
+  int thread_id = state->id;
+  DB* db = state->db;
+
+  Random rnd(1000 + thread_id);
+  WriteOptions write_opt = WriteOptions();
+  write_opt.timeout_hint_us = 500;
+  int timeout_count = 0;
+  int num_keys = kNumKeys * 5;
+
+  for (int k = 0; k < num_keys; ++k) {
+    int key = k + thread_id * num_keys;
+    std::string value = RandomString(&rnd, kValueSize);
+    // only the second-half is randomized
+    if (k > num_keys / 2) {
+      switch (rnd.Next() % 5) {
+        case 0:
+          write_opt.timeout_hint_us = 500 * thread_id;
+          break;
+        case 1:
+          write_opt.timeout_hint_us = num_keys - k;
+          break;
+        case 2:
+          write_opt.timeout_hint_us = 1;
+          break;
+        default:
+          write_opt.timeout_hint_us = 0;
+          state->success_kvs.insert({key, value});
+      }
+    }
+
+    uint64_t time_before_put = db->GetEnv()->NowMicros();
+    Status s = db->Put(write_opt, Key(key), value);
+    uint64_t put_duration = db->GetEnv()->NowMicros() - time_before_put;
+    if (write_opt.timeout_hint_us == 0 ||
+        put_duration + kTimerBias < write_opt.timeout_hint_us) {
+      ASSERT_OK(s);
+      std::string result;
+    }
+    if (s.IsTimedOut()) {
+      timeout_count++;
+      ASSERT_GT(put_duration + kTimerBias, write_opt.timeout_hint_us);
+    }
+  }
+
+  state->done = true;
+}
+
+TEST(DBTest, MTRandomTimeoutTest) {
+  Options options;
+  options.env = env_;
+  options.create_if_missing = true;
+  options.max_write_buffer_number = 2;
+  options.compression = kNoCompression;
+  options.level0_slowdown_writes_trigger = 10;
+  options.level0_stop_writes_trigger = 20;
+  options.write_buffer_size = kWriteBufferSize;
+  DestroyAndReopen(&options);
+
+  TimeoutWriterState thread_states[kNumThreads];
+  for (int tid = 0; tid < kNumThreads; ++tid) {
+    thread_states[tid].id = tid;
+    thread_states[tid].db = db_;
+    thread_states[tid].done = false;
+    env_->StartThread(RandomTimeoutWriter, &thread_states[tid]);
+  }
+
+  for (int tid = 0; tid < kNumThreads; ++tid) {
+    while (thread_states[tid].done == false) {
+      env_->SleepForMicroseconds(100000);
+    }
+  }
+
+  Flush();
+
+  for (int tid = 0; tid < kNumThreads; ++tid) {
+    auto& success_kvs = thread_states[tid].success_kvs;
+    for (auto it = success_kvs.begin(); it != success_kvs.end(); ++it) {
+      ASSERT_EQ(Get(Key(it->first)), it->second);
+    }
+  }
+}
+
+}  // anonymous namespace
+
+/*
+ * This test is not reliable enough as it heavily depends on disk behavior.
+ */
+TEST(DBTest, RateLimitingTest) {
+  Options options = CurrentOptions();
+  options.write_buffer_size = 1 << 20;         // 1MB
+  options.level0_file_num_compaction_trigger = 2;
+  options.target_file_size_base = 1 << 20;     // 1MB
+  options.max_bytes_for_level_base = 4 << 20;  // 4MB
+  options.max_bytes_for_level_multiplier = 4;
+  options.compression = kNoCompression;
+  options.create_if_missing = true;
+  options.env = env_;
+  options.IncreaseParallelism(4);
+  DestroyAndReopen(&options);
+
+  WriteOptions wo;
+  wo.disableWAL = true;
+
+  // # no rate limiting
+  Random rnd(301);
+  uint64_t start = env_->NowMicros();
+  // Write ~96M data
+  for (int64_t i = 0; i < (96 << 10); ++i) {
+    ASSERT_OK(Put(RandomString(&rnd, 32),
+                  RandomString(&rnd, (1 << 10) + 1), wo));
+  }
+  uint64_t elapsed = env_->NowMicros() - start;
+  double raw_rate = env_->bytes_written_ * 1000000 / elapsed;
+  Close();
+
+  // # rate limiting with 0.7 x threshold
+  options.rate_limiter.reset(
+    NewGenericRateLimiter(static_cast<int64_t>(0.7 * raw_rate)));
+  env_->bytes_written_ = 0;
+  DestroyAndReopen(&options);
+
+  start = env_->NowMicros();
+  // Write ~96M data
+  for (int64_t i = 0; i < (96 << 10); ++i) {
+    ASSERT_OK(Put(RandomString(&rnd, 32),
+                  RandomString(&rnd, (1 << 10) + 1), wo));
+  }
+  elapsed = env_->NowMicros() - start;
+  Close();
+  ASSERT_TRUE(options.rate_limiter->GetTotalBytesThrough() ==
+              env_->bytes_written_);
+  double ratio = env_->bytes_written_ * 1000000 / elapsed / raw_rate;
+  fprintf(stderr, "write rate ratio = %.2lf, expected 0.7\n", ratio);
+  ASSERT_TRUE(ratio < 0.8);
+
+  // # rate limiting with half of the raw_rate
+  options.rate_limiter.reset(
+    NewGenericRateLimiter(static_cast<int64_t>(raw_rate / 2)));
+  env_->bytes_written_ = 0;
+  DestroyAndReopen(&options);
+
+  start = env_->NowMicros();
+  // Write ~96M data
+  for (int64_t i = 0; i < (96 << 10); ++i) {
+    ASSERT_OK(Put(RandomString(&rnd, 32),
+                  RandomString(&rnd, (1 << 10) + 1), wo));
+  }
+  elapsed = env_->NowMicros() - start;
+  Close();
+  ASSERT_TRUE(options.rate_limiter->GetTotalBytesThrough() ==
+              env_->bytes_written_);
+  ratio = env_->bytes_written_ * 1000000 / elapsed / raw_rate;
+  fprintf(stderr, "write rate ratio = %.2lf, expected 0.5\n", ratio);
+  ASSERT_TRUE(ratio < 0.6);
+}
+
 }  // namespace rocksdb

 int main(int argc, char** argv) {
@@ -9,6 +9,7 @@

 #pragma once
 #include <stdio.h>
+#include <string>
 #include "rocksdb/comparator.h"
 #include "rocksdb/db.h"
 #include "rocksdb/filter_policy.h"
@@ -254,8 +255,38 @@ class IterKey {

  Slice GetKey() const { return Slice(key_, key_size_); }

+  const size_t Size() { return key_size_; }
+
  void Clear() { key_size_ = 0; }

+  // Append "non_shared_data" to its back, from "shared_len"
+  // This function is used in Block::Iter::ParseNextKey
+  // shared_len: bytes in [0, shard_len-1] would be remained
+  // non_shared_data: data to be append, its length must be >= non_shared_len
+  void TrimAppend(const size_t shared_len, const char* non_shared_data,
+                  const size_t non_shared_len) {
+    assert(shared_len <= key_size_);
+
+    size_t total_size = shared_len + non_shared_len;
+    if (total_size <= buf_size_) {
+      key_size_ = total_size;
+    } else {
+      // Need to allocate space, delete previous space
+      char* p = new char[total_size];
+      memcpy(p, key_, shared_len);
+
+      if (key_ != nullptr && key_ != space_) {
+        delete[] key_;
+      }
+
+      key_ = p;
+      key_size_ = total_size;
+      buf_size_ = total_size;
+    }
+
+    memcpy(key_ + shared_len, non_shared_data, non_shared_len);
+  }
+
  void SetKey(const Slice& key) {
    size_t size = key.size();
    EnlargeBufferIfNeeded(size);
@@ -297,6 +328,13 @@ class IterKey {
                   parsed_key_suffix.sequence, parsed_key_suffix.type);
  }

+  void EncodeLengthPrefixedKey(const Slice& key) {
+    auto size = key.size();
+    EnlargeBufferIfNeeded(size + VarintLength(size));
+    char* ptr = EncodeVarint32(key_, size);
+    memcpy(ptr, key.data(), size);
+  }
+
 private:
  char* key_;
  size_t buf_size_;
@@ -110,6 +110,45 @@ TEST(FormatTest, InternalKeyShortestSuccessor) {
            ShortSuccessor(IKey("\xff\xff", 100, kTypeValue)));
 }

+TEST(FormatTest, IterKeyOperation) {
+  IterKey k;
+  const char p[] = "abcdefghijklmnopqrstuvwxyz";
+  const char q[] = "0123456789";
+
+  ASSERT_EQ(std::string(k.GetKey().data(), k.GetKey().size()),
+            std::string(""));
+
+  k.TrimAppend(0, p, 3);
+  ASSERT_EQ(std::string(k.GetKey().data(), k.GetKey().size()),
+            std::string("abc"));
+
+  k.TrimAppend(1, p, 3);
+  ASSERT_EQ(std::string(k.GetKey().data(), k.GetKey().size()),
+            std::string("aabc"));
+
+  k.TrimAppend(0, p, 26);
+  ASSERT_EQ(std::string(k.GetKey().data(), k.GetKey().size()),
+            std::string("abcdefghijklmnopqrstuvwxyz"));
+
+  k.TrimAppend(26, q, 10);
+  ASSERT_EQ(std::string(k.GetKey().data(), k.GetKey().size()),
+            std::string("abcdefghijklmnopqrstuvwxyz0123456789"));
+
+  k.TrimAppend(36, q, 1);
+  ASSERT_EQ(std::string(k.GetKey().data(), k.GetKey().size()),
+            std::string("abcdefghijklmnopqrstuvwxyz01234567890"));
+
+  k.TrimAppend(26, q, 1);
+  ASSERT_EQ(std::string(k.GetKey().data(), k.GetKey().size()),
+            std::string("abcdefghijklmnopqrstuvwxyz0"));
+
+  // Size going up, memory allocation is triggered
+  k.TrimAppend(27, p, 26);
+  ASSERT_EQ(std::string(k.GetKey().data(), k.GetKey().size()),
+            std::string("abcdefghijklmnopqrstuvwxyz0"
+              "abcdefghijklmnopqrstuvwxyz"));
+}
+
 }  // namespace rocksdb

 int main(int argc, char** argv) {
@@ -14,21 +14,15 @@

 namespace rocksdb {

-FileIndexer::FileIndexer(const uint32_t num_levels,
-                         const Comparator* ucmp)
-  : num_levels_(num_levels),
-    ucmp_(ucmp),
-    next_level_index_(num_levels),
-    level_rb_(num_levels, -1) {
-}
-
+FileIndexer::FileIndexer(const Comparator* ucmp)
+    : num_levels_(0), ucmp_(ucmp), level_rb_(nullptr) {}

 uint32_t FileIndexer::NumLevelIndex() {
  return next_level_index_.size();
 }

 uint32_t FileIndexer::LevelIndexSize(uint32_t level) {
-  return next_level_index_[level].size();
+  return next_level_index_[level].num_index;
 }

 void FileIndexer::GetNextLevelIndex(
@@ -46,11 +40,13 @@ void FileIndexer::GetNextLevelIndex(
  assert(level < num_levels_ - 1);
  assert(static_cast<int32_t>(file_index) <= level_rb_[level]);

-  const auto& index = next_level_index_[level][file_index];
+  const IndexUnit* index_units = next_level_index_[level].index_units;
+  const auto& index = index_units[file_index];

  if (cmp_smallest < 0) {
-    *left_bound = (level > 0 && file_index > 0) ?
-      next_level_index_[level][file_index - 1].largest_lb : 0;
+    *left_bound = (level > 0 && file_index > 0)
+                      ? index_units[file_index - 1].largest_lb
+                      : 0;
    *right_bound = index.smallest_rb;
  } else if (cmp_smallest == 0) {
    *left_bound = index.smallest_lb;
@@ -73,16 +69,25 @@ void FileIndexer::GetNextLevelIndex(
  assert(*right_bound <= level_rb_[level + 1]);
 }

-void FileIndexer::ClearIndex() {
-  for (uint32_t level = 1; level < num_levels_; ++level) {
-    next_level_index_[level].clear();
-  }
-}
-
-void FileIndexer::UpdateIndex(std::vector<FileMetaData*>* const files) {
+void FileIndexer::UpdateIndex(Arena* arena, const uint32_t num_levels,
+                              std::vector<FileMetaData*>* const files) {
  if (files == nullptr) {
    return;
  }
+  if (num_levels == 0) {  // uint_32 0-1 would cause bad behavior
+    num_levels_ = num_levels;
+    return;
+  }
+  assert(level_rb_ == nullptr);  // level_rb_ should be init here
+
+  num_levels_ = num_levels;
+  next_level_index_.resize(num_levels);
+
+  char* mem = arena->AllocateAligned(num_levels_ * sizeof(int32_t));
+  level_rb_ = new (mem) int32_t[num_levels_];
+  for (size_t i = 0; i < num_levels_; i++) {
+    level_rb_[i] = -1;
+  }

  // L1 - Ln-1
  for (uint32_t level = 1; level < num_levels_ - 1; ++level) {
@@ -93,55 +98,56 @@ void FileIndexer::UpdateIndex(std::vector<FileMetaData*>* const files) {
    if (upper_size == 0) {
      continue;
    }
-    auto& index = next_level_index_[level];
-    index.resize(upper_size);
+    IndexLevel& index_level = next_level_index_[level];
+    index_level.num_index = upper_size;
+    char* mem = arena->AllocateAligned(upper_size * sizeof(IndexUnit));
+    index_level.index_units = new (mem) IndexUnit[upper_size];

-    CalculateLB(upper_files, lower_files, &index,
-        [this](const FileMetaData* a, const FileMetaData* b) -> int {
+    CalculateLB(
+        upper_files, lower_files, &index_level,
+        [this](const FileMetaData * a, const FileMetaData * b)->int {
          return ucmp_->Compare(a->smallest.user_key(), b->largest.user_key());
        },
-        [](IndexUnit* index, int32_t f_idx) {
-          index->smallest_lb = f_idx;
-        });
-    CalculateLB(upper_files, lower_files, &index,
-        [this](const FileMetaData* a, const FileMetaData* b) -> int {
+        [](IndexUnit* index, int32_t f_idx) { index->smallest_lb = f_idx; });
+    CalculateLB(
+        upper_files, lower_files, &index_level,
+        [this](const FileMetaData * a, const FileMetaData * b)->int {
          return ucmp_->Compare(a->largest.user_key(), b->largest.user_key());
        },
-        [](IndexUnit* index, int32_t f_idx) {
-          index->largest_lb = f_idx;
-        });
-    CalculateRB(upper_files, lower_files, &index,
-        [this](const FileMetaData* a, const FileMetaData* b) -> int {
+        [](IndexUnit* index, int32_t f_idx) { index->largest_lb = f_idx; });
+    CalculateRB(
+        upper_files, lower_files, &index_level,
+        [this](const FileMetaData * a, const FileMetaData * b)->int {
          return ucmp_->Compare(a->smallest.user_key(), b->smallest.user_key());
        },
-        [](IndexUnit* index, int32_t f_idx) {
-          index->smallest_rb = f_idx;
-        });
-    CalculateRB(upper_files, lower_files, &index,
-        [this](const FileMetaData* a, const FileMetaData* b) -> int {
+        [](IndexUnit* index, int32_t f_idx) { index->smallest_rb = f_idx; });
+    CalculateRB(
+        upper_files, lower_files, &index_level,
+        [this](const FileMetaData * a, const FileMetaData * b)->int {
          return ucmp_->Compare(a->largest.user_key(), b->smallest.user_key());
        },
-        [](IndexUnit* index, int32_t f_idx) {
-          index->largest_rb = f_idx;
-        });
+        [](IndexUnit* index, int32_t f_idx) { index->largest_rb = f_idx; });
  }
+
  level_rb_[num_levels_ - 1] = files[num_levels_ - 1].size() - 1;
 }

-void FileIndexer::CalculateLB(const std::vector<FileMetaData*>& upper_files,
-    const std::vector<FileMetaData*>& lower_files,
-    std::vector<IndexUnit>* index,
+void FileIndexer::CalculateLB(
+    const std::vector<FileMetaData*>& upper_files,
+    const std::vector<FileMetaData*>& lower_files, IndexLevel* index_level,
    std::function<int(const FileMetaData*, const FileMetaData*)> cmp_op,
    std::function<void(IndexUnit*, int32_t)> set_index) {
  const int32_t upper_size = upper_files.size();
  const int32_t lower_size = lower_files.size();
  int32_t upper_idx = 0;
  int32_t lower_idx = 0;
+
+  IndexUnit* index = index_level->index_units;
  while (upper_idx < upper_size && lower_idx < lower_size) {
    int cmp = cmp_op(upper_files[upper_idx], lower_files[lower_idx]);

    if (cmp == 0) {
-      set_index(&(*index)[upper_idx], lower_idx);
+      set_index(&index[upper_idx], lower_idx);
      ++upper_idx;
      ++lower_idx;
    } else if (cmp > 0) {
@@ -151,7 +157,7 @@ void FileIndexer::CalculateLB(const std::vector<FileMetaData*>& upper_files,
    } else {
      // Lower level's file becomes larger, update the index, and
      // move to the next upper file
-      set_index(&(*index)[upper_idx], lower_idx);
+      set_index(&index[upper_idx], lower_idx);
      ++upper_idx;
    }
  }
@@ -159,25 +165,27 @@ void FileIndexer::CalculateLB(const std::vector<FileMetaData*>& upper_files,
  while (upper_idx < upper_size) {
    // Lower files are exhausted, that means the remaining upper files are
    // greater than any lower files. Set the index to be the lower level size.
-    set_index(&(*index)[upper_idx], lower_size);
+    set_index(&index[upper_idx], lower_size);
    ++upper_idx;
  }
 }

-void FileIndexer::CalculateRB(const std::vector<FileMetaData*>& upper_files,
-    const std::vector<FileMetaData*>& lower_files,
-    std::vector<IndexUnit>* index,
+void FileIndexer::CalculateRB(
+    const std::vector<FileMetaData*>& upper_files,
+    const std::vector<FileMetaData*>& lower_files, IndexLevel* index_level,
    std::function<int(const FileMetaData*, const FileMetaData*)> cmp_op,
    std::function<void(IndexUnit*, int32_t)> set_index) {
  const int32_t upper_size = upper_files.size();
  const int32_t lower_size = lower_files.size();
  int32_t upper_idx = upper_size - 1;
  int32_t lower_idx = lower_size - 1;
+
+  IndexUnit* index = index_level->index_units;
  while (upper_idx >= 0 && lower_idx >= 0) {
    int cmp = cmp_op(upper_files[upper_idx], lower_files[lower_idx]);

    if (cmp == 0) {
-      set_index(&(*index)[upper_idx], lower_idx);
+      set_index(&index[upper_idx], lower_idx);
      --upper_idx;
      --lower_idx;
    } else if (cmp < 0) {
@@ -187,14 +195,14 @@ void FileIndexer::CalculateRB(const std::vector<FileMetaData*>& upper_files,
    } else {
      // Lower level's file becomes smaller, update the index, and move to
      // the next the upper file
-      set_index(&(*index)[upper_idx], lower_idx);
+      set_index(&index[upper_idx], lower_idx);
      --upper_idx;
    }
  }
  while (upper_idx >= 0) {
    // Lower files are exhausted, that means the remaining upper files are
    // smaller than any lower files. Set it to -1.
-    set_index(&(*index)[upper_idx], -1);
+    set_index(&index[upper_idx], -1);
    --upper_idx;
  }
 }
@@ -12,11 +12,15 @@
 #include <functional>
 #include <limits>
 #include <vector>
+#include "util/arena.h"
+#include "util/autovector.h"

 namespace rocksdb {

 class Comparator;
 struct FileMetaData;
+struct FdWithKeyRange;
+struct FileLevel;

 // The file tree structure in Version is prebuilt and the range of each file
 // is known. On Version::Get(), it uses binary search to find a potential file
@@ -36,7 +40,7 @@ struct FileMetaData;
 // naive approach.
 class FileIndexer {
 public:
-  FileIndexer(const uint32_t num_levels, const Comparator* ucmp);
+  explicit FileIndexer(const Comparator* ucmp);

  uint32_t NumLevelIndex();

@@ -50,16 +54,15 @@ class FileIndexer {
    const uint32_t level, const uint32_t file_index, const int cmp_smallest,
    const int cmp_largest, int32_t* left_bound, int32_t* right_bound);

-  void ClearIndex();
-
-  void UpdateIndex(std::vector<FileMetaData*>* const files);
+  void UpdateIndex(Arena* arena, const uint32_t num_levels,
+                   std::vector<FileMetaData*>* const files);

  enum {
    kLevelMaxIndex = std::numeric_limits<int32_t>::max()
  };

 private:
-  const uint32_t num_levels_;
+  uint32_t num_levels_;
  const Comparator* ucmp_;

  struct IndexUnit {
@@ -110,20 +113,28 @@ class FileIndexer {
    int32_t largest_rb;
  };

-  void CalculateLB(const std::vector<FileMetaData*>& upper_files,
-    const std::vector<FileMetaData*>& lower_files,
-    std::vector<IndexUnit>* index,
-    std::function<int(const FileMetaData*, const FileMetaData*)> cmp_op,
-    std::function<void(IndexUnit*, int32_t)> set_index);
+  // Data structure to store IndexUnits in a whole level
+  struct IndexLevel {
+    size_t num_index;
+    IndexUnit* index_units;

-  void CalculateRB(const std::vector<FileMetaData*>& upper_files,
-    const std::vector<FileMetaData*>& lower_files,
-    std::vector<IndexUnit>* index,
-    std::function<int(const FileMetaData*, const FileMetaData*)> cmp_op,
-    std::function<void(IndexUnit*, int32_t)> set_index);
+    IndexLevel() : num_index(0), index_units(nullptr) {}
+  };

-  std::vector<std::vector<IndexUnit>> next_level_index_;
-  std::vector<int32_t> level_rb_;
+  void CalculateLB(
+      const std::vector<FileMetaData*>& upper_files,
+      const std::vector<FileMetaData*>& lower_files, IndexLevel* index_level,
+      std::function<int(const FileMetaData*, const FileMetaData*)> cmp_op,
+      std::function<void(IndexUnit*, int32_t)> set_index);
+
+  void CalculateRB(
+      const std::vector<FileMetaData*>& upper_files,
+      const std::vector<FileMetaData*>& lower_files, IndexLevel* index_level,
+      std::function<int(const FileMetaData*, const FileMetaData*)> cmp_op,
+      std::function<void(IndexUnit*, int32_t)> set_index);
+
+  autovector<IndexLevel> next_level_index_;
+  int32_t* level_rb_;
 };

 }  // namespace rocksdb
@@ -35,16 +35,13 @@ class IntComparator : public Comparator {
  void FindShortSuccessor(std::string* key) const {}
 };

-
 struct FileIndexerTest {
 public:
-  FileIndexerTest() :
-    kNumLevels(4), indexer(kNumLevels, &ucmp),
-    files(new std::vector<FileMetaData*>[kNumLevels]) {
-  }
+  FileIndexerTest()
+      : kNumLevels(4), files(new std::vector<FileMetaData*>[kNumLevels]) {}

  ~FileIndexerTest() {
-    Reset();
+    ClearFiles();
    delete[] files;
  }

@@ -59,14 +56,13 @@ struct FileIndexerTest {
    return InternalKey(Slice(reinterpret_cast<char*>(&v), 8), 0, kTypeValue);
  }

-  void Reset() {
+  void ClearFiles() {
    for (uint32_t i = 0; i < kNumLevels; ++i) {
      for (auto* f : files[i]) {
        delete f;
      }
      files[i].clear();
    }
-    indexer.ClearIndex();
  }

  void GetNextLevelIndex(const uint32_t level, const uint32_t file_index,
@@ -74,23 +70,32 @@ struct FileIndexerTest {
      int32_t* right_index) {
    *left_index = 100;
    *right_index = 100;
-    indexer.GetNextLevelIndex(level, file_index, cmp_smallest, cmp_largest,
-        left_index, right_index);
+    indexer->GetNextLevelIndex(level, file_index, cmp_smallest, cmp_largest,
+                               left_index, right_index);
  }

+  int32_t left = 100;
+  int32_t right = 100;
  const uint32_t kNumLevels;
  IntComparator ucmp;
-  FileIndexer indexer;
+  FileIndexer* indexer;

  std::vector<FileMetaData*>* files;
 };

-TEST(FileIndexerTest, next_level_hint) {
-  for (uint32_t i = 0; i < kNumLevels; ++i) {
-    ASSERT_EQ(0U, indexer.LevelIndexSize(i));
-  }
+// Case 0: Empty
+TEST(FileIndexerTest, Empty) {
+  Arena arena;
+  indexer = new FileIndexer(&ucmp);
+  indexer->UpdateIndex(&arena, 0, files);
+  delete indexer;
+}

-  // Case 1: no overlap, files are on the left of next level files
+// Case 1: no overlap, files are on the left of next level files
+TEST(FileIndexerTest, no_overlap_left) {
+  Arena arena;
+  uint32_t kNumLevels = 4;
+  indexer = new FileIndexer(&ucmp);
  // level 1
  AddFile(1, 100, 200);
  AddFile(1, 300, 400);
@@ -103,9 +108,7 @@ TEST(FileIndexerTest, next_level_hint) {
  AddFile(3, 2500, 2600);
  AddFile(3, 2601, 2699);
  AddFile(3, 2700, 2800);
-  indexer.UpdateIndex(files);
-  int32_t left = 100;
-  int32_t right = 100;
+  indexer->UpdateIndex(&arena, kNumLevels, files);
  for (uint32_t level = 1; level < 3; ++level) {
    for (uint32_t f = 0; f < 3; ++f) {
      GetNextLevelIndex(level, f, -1, -1, &left, &right);
@@ -125,12 +128,15 @@ TEST(FileIndexerTest, next_level_hint) {
      ASSERT_EQ(2, right);
    }
  }
+  delete indexer;
+  ClearFiles();
+}

-  // Case 2: no overlap, files are on the right of next level files
-  Reset();
-  for (uint32_t i = 1; i < kNumLevels; ++i) {
-    ASSERT_EQ(0U, indexer.LevelIndexSize(i));
-  }
+// Case 2: no overlap, files are on the right of next level files
+TEST(FileIndexerTest, no_overlap_right) {
+  Arena arena;
+  uint32_t kNumLevels = 4;
+  indexer = new FileIndexer(&ucmp);
  // level 1
  AddFile(1, 2100, 2200);
  AddFile(1, 2300, 2400);
@@ -143,7 +149,7 @@ TEST(FileIndexerTest, next_level_hint) {
  AddFile(3, 500, 600);
  AddFile(3, 501, 699);
  AddFile(3, 700, 800);
-  indexer.UpdateIndex(files);
+  indexer->UpdateIndex(&arena, kNumLevels, files);
  for (uint32_t level = 1; level < 3; ++level) {
    for (uint32_t f = 0; f < 3; ++f) {
      GetNextLevelIndex(level, f, -1, -1, &left, &right);
@@ -166,11 +172,16 @@ TEST(FileIndexerTest, next_level_hint) {
      ASSERT_EQ(2, right);
    }
  }
+  delete indexer;
+}

-  // Case 3: empty L2
-  Reset();
+// Case 3: empty L2
+TEST(FileIndexerTest, empty_L2) {
+  Arena arena;
+  uint32_t kNumLevels = 4;
+  indexer = new FileIndexer(&ucmp);
  for (uint32_t i = 1; i < kNumLevels; ++i) {
-    ASSERT_EQ(0U, indexer.LevelIndexSize(i));
+    ASSERT_EQ(0U, indexer->LevelIndexSize(i));
  }
  // level 1
  AddFile(1, 2100, 2200);
@@ -180,7 +191,7 @@ TEST(FileIndexerTest, next_level_hint) {
  AddFile(3, 500, 600);
  AddFile(3, 501, 699);
  AddFile(3, 700, 800);
-  indexer.UpdateIndex(files);
+  indexer->UpdateIndex(&arena, kNumLevels, files);
  for (uint32_t f = 0; f < 3; ++f) {
    GetNextLevelIndex(1, f, -1, -1, &left, &right);
    ASSERT_EQ(0, left);
@@ -201,13 +212,14 @@ TEST(FileIndexerTest, next_level_hint) {
    ASSERT_EQ(0, left);
    ASSERT_EQ(-1, right);
  }
+  delete indexer;
+  ClearFiles();
+}

-
-  // Case 4: mixed
-  Reset();
-  for (uint32_t i = 1; i < kNumLevels; ++i) {
-    ASSERT_EQ(0U, indexer.LevelIndexSize(i));
-  }
+// Case 4: mixed
+TEST(FileIndexerTest, mixed) {
+  Arena arena;
+  indexer = new FileIndexer(&ucmp);
  // level 1
  AddFile(1, 100, 200);
  AddFile(1, 250, 400);
@@ -222,7 +234,7 @@ TEST(FileIndexerTest, next_level_hint) {
  AddFile(3, 0, 50);
  AddFile(3, 100, 200);
  AddFile(3, 201, 250);
-  indexer.UpdateIndex(files);
+  indexer->UpdateIndex(&arena, kNumLevels, files);
  // level 1, 0
  GetNextLevelIndex(1, 0, -1, -1, &left, &right);
  ASSERT_EQ(0, left);
@@ -321,6 +333,8 @@ TEST(FileIndexerTest, next_level_hint) {
    ASSERT_EQ(3, left);
    ASSERT_EQ(2, right);
  }
+  delete indexer;
+  ClearFiles();
 }

 }  // namespace rocksdb
@@ -11,6 +11,7 @@

 #include <ctype.h>
 #include <stdio.h>
+#include <vector>
 #include "db/dbformat.h"
 #include "rocksdb/env.h"
 #include "util/logging.h"
@@ -66,9 +67,28 @@ std::string ArchivedLogFileName(const std::string& name, uint64_t number) {
  return MakeFileName(name + "/" + ARCHIVAL_DIR, number, "log");
 }

-std::string TableFileName(const std::string& name, uint64_t number) {
+std::string MakeTableFileName(const std::string& path, uint64_t number) {
+  return MakeFileName(path, number, "sst");
+}
+
+std::string TableFileName(const std::vector<DbPath>& db_paths, uint64_t number,
+                          uint32_t path_id) {
  assert(number > 0);
-  return MakeFileName(name, number, "sst");
+  std::string path;
+  if (path_id >= db_paths.size()) {
+    path = db_paths.back().path;
+  } else {
+    path = db_paths[path_id].path;
+  }
+  return MakeTableFileName(path, number);
+}
+
+std::string FormatFileNumber(uint64_t number, uint32_t path_id) {
+  if (path_id == 0) {
+    return std::to_string(number);
+  } else {
+    return std::to_string(number) + "(path " + std::to_string(path_id) + ")";
+  }
 }

 std::string DescriptorFileName(const std::string& dbname, uint64_t number) {
@@ -11,7 +11,9 @@

 #pragma once
 #include <stdint.h>
+#include <unordered_map>
 #include <string>
+#include <vector>
 #include "rocksdb/slice.h"
 #include "rocksdb/status.h"
 #include "rocksdb/transaction_log.h"
@@ -34,6 +36,9 @@ enum FileType {
  kIdentityFile
 };

+// map from file number to path ID.
+typedef std::unordered_map<uint64_t, uint32_t> FileNumToPathIdMap;
+
 // Return the name of the log file with the specified number
 // in the db named by "dbname".  The result will be prefixed with
 // "dbname".
@@ -48,10 +53,15 @@ extern std::string ArchivalDirectory(const std::string& dbname);
 extern std::string ArchivedLogFileName(const std::string& dbname,
                                       uint64_t num);

+extern std::string MakeTableFileName(const std::string& name, uint64_t number);
+
 // Return the name of the sstable with the specified number
 // in the db named by "dbname".  The result will be prefixed with
 // "dbname".
-extern std::string TableFileName(const std::string& dbname, uint64_t number);
+extern std::string TableFileName(const std::vector<DbPath>& db_paths,
+                                 uint64_t number, uint32_t path_id);
+
+extern std::string FormatFileNumber(uint64_t number, uint32_t path_id);

 // Return the name of the descriptor file for the db named by
 // "dbname" and the specified incarnation number.  The result will be
@@ -108,7 +108,10 @@ TEST(FileNameTest, Construction) {
  ASSERT_EQ(192U, number);
  ASSERT_EQ(kLogFile, type);

-  fname = TableFileName("bar", 200);
+  fname = TableFileName({DbPath("bar", 0)}, 200, 0);
+  std::string fname1 =
+      TableFileName({DbPath("foo", 0), DbPath("bar", 0)}, 200, 1);
+  ASSERT_EQ(fname, fname1);
  ASSERT_EQ("bar/", std::string(fname.data(), 4));
  ASSERT_TRUE(ParseFileName(fname.c_str() + 4, &number, &type));
  ASSERT_EQ(200U, number);
@@ -87,7 +87,12 @@ class LevelIterator : public Iterator {
    return file_iter_->value();
  }
  Status status() const override {
-    return status_;
+    if (!status_.ok()) {
+      return status_;
+    } else if (file_iter_ && !file_iter_->status().ok()) {
+      return file_iter_->status();
+    }
+    return Status::OK();
  }

 private:
@@ -189,6 +194,10 @@ void ForwardIterator::SeekInternal(const Slice& internal_key,
      }
    }

+    Slice user_key;
+    if (!seek_to_first) {
+      user_key = ExtractUserKey(internal_key);
+    }
    auto* files = sv_->current->files_;
    for (uint32_t i = 0; i < files[0].size(); ++i) {
      if (seek_to_first) {
@@ -196,7 +205,7 @@ void ForwardIterator::SeekInternal(const Slice& internal_key,
      } else {
        // If the target key passes over the larget key, we are sure Next()
        // won't go over this file.
-        if (user_comparator_->Compare(ExtractUserKey(internal_key),
+        if (user_comparator_->Compare(user_key,
              files[0][i]->largest.user_key()) > 0) {
          continue;
        }
@@ -206,16 +215,67 @@ void ForwardIterator::SeekInternal(const Slice& internal_key,
        immutable_min_heap_.push(l0_iters_[i]);
      }
    }
+
+    int32_t search_left_bound = 0;
+    int32_t search_right_bound = FileIndexer::kLevelMaxIndex;
    for (int32_t level = 1; level < sv_->current->NumberLevels(); ++level) {
      if (files[level].empty()) {
+        search_left_bound = 0;
+        search_right_bound = FileIndexer::kLevelMaxIndex;
        continue;
      }
      assert(level_iters_[level - 1] != nullptr);
      uint32_t f_idx = 0;
      if (!seek_to_first) {
-        f_idx = FindFileInRange(
-            files[level], internal_key, 0, files[level].size());
+        // TODO(ljin): remove before committing
+        // f_idx = FindFileInRange(
+        //    files[level], internal_key, 0, files[level].size());
+
+        if (search_left_bound == search_right_bound) {
+          f_idx = search_left_bound;
+        } else if (search_left_bound < search_right_bound) {
+          f_idx = FindFileInRange(
+              files[level], internal_key, search_left_bound,
+              search_right_bound == FileIndexer::kLevelMaxIndex ?
+                files[level].size() : search_right_bound);
+        } else {
+          // search_left_bound > search_right_bound
+          // There are only 2 cases this can happen:
+          // (1) target key is smaller than left most file
+          // (2) target key is larger than right most file
+          assert(search_left_bound == (int32_t)files[level].size() ||
+                 search_right_bound == -1);
+          if (search_right_bound == -1) {
+            assert(search_left_bound == 0);
+            f_idx = 0;
+          } else {
+            sv_->current->file_indexer_.GetNextLevelIndex(
+                level, files[level].size() - 1,
+                1, 1, &search_left_bound, &search_right_bound);
+            continue;
+          }
+        }
+
+        // Prepare hints for the next level
+        if (f_idx < files[level].size()) {
+          int cmp_smallest = user_comparator_->Compare(
+              user_key, files[level][f_idx]->smallest.user_key());
+          int cmp_largest = -1;
+          if (cmp_smallest >= 0) {
+            cmp_smallest = user_comparator_->Compare(
+                user_key, files[level][f_idx]->smallest.user_key());
+          }
+          sv_->current->file_indexer_.GetNextLevelIndex(level, f_idx,
+              cmp_smallest, cmp_largest,
+              &search_left_bound, &search_right_bound);
+        } else {
+          sv_->current->file_indexer_.GetNextLevelIndex(
+              level, files[level].size() - 1,
+              1, 1, &search_left_bound, &search_right_bound);
+        }
      }
+
+      // Seek
      if (f_idx < files[level].size()) {
        level_iters_[level - 1]->SetFileIndex(f_idx);
        seek_to_first ? level_iters_[level - 1]->SeekToFirst() :
@@ -232,6 +292,9 @@ void ForwardIterator::SeekInternal(const Slice& internal_key,
      prev_key_.SetKey(internal_key);
      is_prev_set_ = true;
    }
+  } else if (current_ && current_ != mutable_iter_) {
+    // current_ is one of immutable iterators, push it back to the heap
+    immutable_min_heap_.push(current_);
  }

  UpdateCurrent();
@@ -279,6 +342,23 @@ Status ForwardIterator::status() const {
  } else if (!mutable_iter_->status().ok()) {
    return mutable_iter_->status();
  }
+
+  for (auto *it : imm_iters_) {
+    if (it && !it->status().ok()) {
+      return it->status();
+    }
+  }
+  for (auto *it : l0_iters_) {
+    if (it && !it->status().ok()) {
+      return it->status();
+    }
+  }
+  for (auto *it : level_iters_) {
+    if (it && !it->status().ok()) {
+      return it->status();
+    }
+  }
+
  return Status::OK();
 }

@@ -7,12 +7,85 @@
 // found in the LICENSE file. See the AUTHORS file for names of contributors.

 #include "db/internal_stats.h"
+#define __STDC_FORMAT_MACROS
+#include <inttypes.h>
+#include <vector>
 #include "db/column_family.h"

-#include <vector>
-
 namespace rocksdb {

+namespace {
+const double kMB = 1048576.0;
+const double kGB = kMB * 1024;
+
+void PrintLevelStatsHeader(char* buf, size_t len, const std::string& cf_name) {
+  snprintf(
+      buf, len,
+      "\n** Compaction Stats [%s] **\n"
+      "Level   Files   Size(MB) Score Read(GB)  Rn(GB) Rnp1(GB) "
+      "Write(GB) Wnew(GB) RW-Amp W-Amp Rd(MB/s) Wr(MB/s)  Rn(cnt) "
+      "Rnp1(cnt) Wnp1(cnt) Wnew(cnt)  Comp(sec) Comp(cnt) Avg(sec) "
+      "Stall(sec) Stall(cnt) Avg(ms)\n"
+      "--------------------------------------------------------------------"
+      "--------------------------------------------------------------------"
+      "--------------------------------------------------------------------\n",
+      cf_name.c_str());
+}
+
+void PrintLevelStats(char* buf, size_t len, const std::string& name,
+    int num_files, int being_compacted, double total_file_size, double score,
+    double rw_amp, double w_amp, double stall_us, uint64_t stalls,
+    const InternalStats::CompactionStats& stats) {
+  uint64_t bytes_read = stats.bytes_readn + stats.bytes_readnp1;
+  uint64_t bytes_new = stats.bytes_written - stats.bytes_readnp1;
+  double elapsed = (stats.micros + 1) / 1000000.0;
+
+  snprintf(buf, len,
+    "%4s %5d/%-3d %8.0f %5.1f " /* Level, Files, Size(MB), Score */
+    "%8.1f " /* Read(GB) */
+    "%7.1f " /* Rn(GB) */
+    "%8.1f " /* Rnp1(GB) */
+    "%9.1f " /* Write(GB) */
+    "%8.1f " /* Wnew(GB) */
+    "%6.1f " /* RW-Amp */
+    "%5.1f " /* W-Amp */
+    "%8.1f " /* Rd(MB/s) */
+    "%8.1f " /* Wr(MB/s) */
+    "%8d " /* Rn(cnt) */
+    "%9d " /* Rnp1(cnt) */
+    "%9d " /* Wnp1(cnt) */
+    "%9d " /* Wnew(cnt) */
+    "%10.0f " /* Comp(sec) */
+    "%9d " /* Comp(cnt) */
+    "%8.3f " /* Avg(sec) */
+    "%10.2f " /* Stall(sec) */
+    "%10" PRIu64 " " /* Stall(cnt) */
+    "%7.2f\n" /* Avg(ms) */,
+    name.c_str(), num_files, being_compacted, total_file_size / kMB, score,
+    bytes_read / kGB,
+    stats.bytes_readn / kGB,
+    stats.bytes_readnp1 / kGB,
+    stats.bytes_written / kGB,
+    bytes_new / kGB,
+    rw_amp,
+    w_amp,
+    bytes_read / kMB / elapsed,
+    stats.bytes_written / kMB / elapsed,
+    stats.files_in_leveln,
+    stats.files_in_levelnp1,
+    stats.files_out_levelnp1,
+    stats.files_out_levelnp1 - stats.files_in_levelnp1,
+    stats.micros / 1000000.0,
+    stats.count,
+    stats.count == 0 ? 0 : stats.micros / 1000000.0 / stats.count,
+    stall_us / 1000000.0,
+    stalls,
+    stalls == 0 ? 0 : stall_us / 1000.0 / stalls);
+}
+
+
+}
+
 DBPropertyType GetPropertyType(const Slice& property) {
  Slice in = property;
  Slice prefix("rocksdb.");
@@ -25,6 +98,10 @@ DBPropertyType GetPropertyType(const Slice& property) {
    return kLevelStats;
  } else if (in == "stats") {
    return kStats;
+  } else if (in == "cfstats") {
+    return kCFStats;
+  } else if (in == "dbstats") {
+    return kDBStats;
  } else if (in == "sstables") {
    return kSsTables;
  } else if (in == "num-immutable-mem-table") {
@@ -41,14 +118,24 @@ DBPropertyType GetPropertyType(const Slice& property) {
    return kNumEntriesInMutableMemtable;
  } else if (in == "num-entries-imm-mem-tables") {
    return kNumEntriesInImmutableMemtable;
+  } else if (in == "estimate-num-keys") {
+    return kEstimatedNumKeys;
  }
  return kUnknown;
 }

 bool InternalStats::GetProperty(DBPropertyType property_type,
-                                const Slice& property, std::string* value,
-                                ColumnFamilyData* cfd) {
-  Version* current = cfd->current();
+                                const Slice& property, std::string* value) {
+  if (property_type > kStartIntTypes) {
+    uint64_t int_value;
+    bool ret_value = GetIntProperty(property_type, property, &int_value);
+    if (ret_value) {
+      *value = std::to_string(int_value);
+    }
+    return ret_value;
+  }
+
+  Version* current = cfd_->current();
  Slice in = property;

  switch (property_type) {
@@ -76,294 +163,288 @@ bool InternalStats::GetProperty(DBPropertyType property_type,
      for (int level = 0; level < number_levels_; level++) {
        snprintf(buf, sizeof(buf), "%3d %8d %8.0f\n", level,
                 current->NumLevelFiles(level),
-                 current->NumLevelBytes(level) / 1048576.0);
+                 current->NumLevelBytes(level) / kMB);
        value->append(buf);
      }
      return true;
    }
    case kStats: {
-      char buf[1000];
-
-      uint64_t wal_bytes = 0;
-      uint64_t wal_synced = 0;
-      uint64_t user_bytes_written = 0;
-      uint64_t write_other = 0;
-      uint64_t write_self = 0;
-      uint64_t write_with_wal = 0;
-      uint64_t total_bytes_written = 0;
-      uint64_t total_bytes_read = 0;
-      uint64_t micros_up = env_->NowMicros() - started_at_;
-      // Add "+1" to make sure seconds_up is > 0 and avoid NaN later
-      double seconds_up = (micros_up + 1) / 1000000.0;
-      uint64_t total_slowdown = 0;
-      uint64_t total_slowdown_count = 0;
-      uint64_t interval_bytes_written = 0;
-      uint64_t interval_bytes_read = 0;
-      uint64_t interval_bytes_new = 0;
-      double interval_seconds_up = 0;
-
-      if (statistics_) {
-        wal_bytes = statistics_->getTickerCount(WAL_FILE_BYTES);
-        wal_synced = statistics_->getTickerCount(WAL_FILE_SYNCED);
-        user_bytes_written = statistics_->getTickerCount(BYTES_WRITTEN);
-        write_other = statistics_->getTickerCount(WRITE_DONE_BY_OTHER);
-        write_self = statistics_->getTickerCount(WRITE_DONE_BY_SELF);
-        write_with_wal = statistics_->getTickerCount(WRITE_WITH_WAL);
+      if (!GetProperty(kCFStats, "rocksdb.cfstats", value)) {
+        return false;
      }
-
-      snprintf(
-          buf, sizeof(buf),
-          "                               Compactions\n"
-          "Level  Files Size(MB) Score Time(sec)  Read(MB) Write(MB)    Rn(MB) "
-          " "
-          "Rnp1(MB)  Wnew(MB) RW-Amplify Read(MB/s) Write(MB/s)      Rn     "
-          "Rnp1 "
-          "    Wnp1     NewW    Count   msComp   msStall  Ln-stall Stall-cnt\n"
-          "--------------------------------------------------------------------"
-          "--"
-          "--------------------------------------------------------------------"
-          "--"
-          "----------------------------------------------------------------\n");
-      value->append(buf);
-      for (int level = 0; level < number_levels_; level++) {
-        int files = current->NumLevelFiles(level);
-        if (compaction_stats_[level].micros > 0 || files > 0) {
-          int64_t bytes_read = compaction_stats_[level].bytes_readn +
-                               compaction_stats_[level].bytes_readnp1;
-          int64_t bytes_new = compaction_stats_[level].bytes_written -
-                              compaction_stats_[level].bytes_readnp1;
-          double amplify =
-              (compaction_stats_[level].bytes_readn == 0)
-                  ? 0.0
-                  : (compaction_stats_[level].bytes_written +
-                     compaction_stats_[level].bytes_readnp1 +
-                     compaction_stats_[level].bytes_readn) /
-                        (double)compaction_stats_[level].bytes_readn;
-
-          total_bytes_read += bytes_read;
-          total_bytes_written += compaction_stats_[level].bytes_written;
-
-          uint64_t stalls = level == 0 ? (stall_counts_[LEVEL0_SLOWDOWN] +
-                                          stall_counts_[LEVEL0_NUM_FILES] +
-                                          stall_counts_[MEMTABLE_COMPACTION])
-                                       : stall_leveln_slowdown_count_[level];
-
-          double stall_us = level == 0 ? (stall_micros_[LEVEL0_SLOWDOWN] +
-                                          stall_micros_[LEVEL0_NUM_FILES] +
-                                          stall_micros_[MEMTABLE_COMPACTION])
-                                       : stall_leveln_slowdown_[level];
-
-          snprintf(buf, sizeof(buf),
-                   "%3d %8d %8.0f %5.1f %9.0f %9.0f %9.0f %9.0f %9.0f %9.0f "
-                   "%10.1f %9.1f %11.1f %8d %8d %8d %8d %8d %8d %9.1f %9.1f "
-                   "%9lu\n",
-                   level, files, current->NumLevelBytes(level) / 1048576.0,
-                   current->NumLevelBytes(level) /
-                       cfd->compaction_picker()->MaxBytesForLevel(level),
-                   compaction_stats_[level].micros / 1e6,
-                   bytes_read / 1048576.0,
-                   compaction_stats_[level].bytes_written / 1048576.0,
-                   compaction_stats_[level].bytes_readn / 1048576.0,
-                   compaction_stats_[level].bytes_readnp1 / 1048576.0,
-                   bytes_new / 1048576.0, amplify,
-                   // +1 to avoid division by 0
-                   (bytes_read / 1048576.0) /
-                       ((compaction_stats_[level].micros + 1) / 1000000.0),
-                   (compaction_stats_[level].bytes_written / 1048576.0) /
-                       ((compaction_stats_[level].micros + 1) / 1000000.0),
-                   compaction_stats_[level].files_in_leveln,
-                   compaction_stats_[level].files_in_levelnp1,
-                   compaction_stats_[level].files_out_levelnp1,
-                   compaction_stats_[level].files_out_levelnp1 -
-                       compaction_stats_[level].files_in_levelnp1,
-                   compaction_stats_[level].count,
-                   (int)((double)compaction_stats_[level].micros / 1000.0 /
-                         (compaction_stats_[level].count + 1)),
-                   (double)stall_us / 1000.0 / (stalls + 1),
-                   stall_us / 1000000.0, (unsigned long)stalls);
-          total_slowdown += stall_leveln_slowdown_[level];
-          total_slowdown_count += stall_leveln_slowdown_count_[level];
-          value->append(buf);
-        }
+      if (!GetProperty(kDBStats, "rocksdb.dbstats", value)) {
+        return false;
      }
-
-      interval_bytes_new = user_bytes_written - last_stats_.ingest_bytes_;
-      interval_bytes_read =
-          total_bytes_read - last_stats_.compaction_bytes_read_;
-      interval_bytes_written =
-          total_bytes_written - last_stats_.compaction_bytes_written_;
-      interval_seconds_up = seconds_up - last_stats_.seconds_up_;
-
-      snprintf(buf, sizeof(buf), "Uptime(secs): %.1f total, %.1f interval\n",
-               seconds_up, interval_seconds_up);
-      value->append(buf);
-
-      snprintf(buf, sizeof(buf),
-               "Writes cumulative: %llu total, %llu batches, "
-               "%.1f per batch, %.2f ingest GB\n",
-               (unsigned long long)(write_other + write_self),
-               (unsigned long long)write_self,
-               (write_other + write_self) / (double)(write_self + 1),
-               user_bytes_written / (1048576.0 * 1024));
-      value->append(buf);
-
-      snprintf(buf, sizeof(buf),
-               "WAL cumulative: %llu WAL writes, %llu WAL syncs, "
-               "%.2f writes per sync, %.2f GB written\n",
-               (unsigned long long)write_with_wal,
-               (unsigned long long)wal_synced,
-               write_with_wal / (double)(wal_synced + 1),
-               wal_bytes / (1048576.0 * 1024));
-      value->append(buf);
-
-      snprintf(buf, sizeof(buf),
-               "Compaction IO cumulative (GB): "
-               "%.2f new, %.2f read, %.2f write, %.2f read+write\n",
-               user_bytes_written / (1048576.0 * 1024),
-               total_bytes_read / (1048576.0 * 1024),
-               total_bytes_written / (1048576.0 * 1024),
-               (total_bytes_read + total_bytes_written) / (1048576.0 * 1024));
-      value->append(buf);
-
-      snprintf(
-          buf, sizeof(buf),
-          "Compaction IO cumulative (MB/sec): "
-          "%.1f new, %.1f read, %.1f write, %.1f read+write\n",
-          user_bytes_written / 1048576.0 / seconds_up,
-          total_bytes_read / 1048576.0 / seconds_up,
-          total_bytes_written / 1048576.0 / seconds_up,
-          (total_bytes_read + total_bytes_written) / 1048576.0 / seconds_up);
-      value->append(buf);
-
-      // +1 to avoid divide by 0 and NaN
-      snprintf(
-          buf, sizeof(buf),
-          "Amplification cumulative: %.1f write, %.1f compaction\n",
-          (double)(total_bytes_written + wal_bytes) / (user_bytes_written + 1),
-          (double)(total_bytes_written + total_bytes_read + wal_bytes) /
-              (user_bytes_written + 1));
-      value->append(buf);
-
-      uint64_t interval_write_other = write_other - last_stats_.write_other_;
-      uint64_t interval_write_self = write_self - last_stats_.write_self_;
-
-      snprintf(buf, sizeof(buf),
-               "Writes interval: %llu total, %llu batches, "
-               "%.1f per batch, %.1f ingest MB\n",
-               (unsigned long long)(interval_write_other + interval_write_self),
-               (unsigned long long)interval_write_self,
-               (double)(interval_write_other + interval_write_self) /
-                   (interval_write_self + 1),
-               (user_bytes_written - last_stats_.ingest_bytes_) / 1048576.0);
-      value->append(buf);
-
-      uint64_t interval_write_with_wal =
-          write_with_wal - last_stats_.write_with_wal_;
-
-      uint64_t interval_wal_synced = wal_synced - last_stats_.wal_synced_;
-      uint64_t interval_wal_bytes = wal_bytes - last_stats_.wal_bytes_;
-
-      snprintf(buf, sizeof(buf),
-               "WAL interval: %llu WAL writes, %llu WAL syncs, "
-               "%.2f writes per sync, %.2f MB written\n",
-               (unsigned long long)interval_write_with_wal,
-               (unsigned long long)interval_wal_synced,
-               interval_write_with_wal / (double)(interval_wal_synced + 1),
-               interval_wal_bytes / (1048576.0 * 1024));
-      value->append(buf);
-
-      snprintf(buf, sizeof(buf),
-               "Compaction IO interval (MB): "
-               "%.2f new, %.2f read, %.2f write, %.2f read+write\n",
-               interval_bytes_new / 1048576.0, interval_bytes_read / 1048576.0,
-               interval_bytes_written / 1048576.0,
-               (interval_bytes_read + interval_bytes_written) / 1048576.0);
-      value->append(buf);
-
-      snprintf(buf, sizeof(buf),
-               "Compaction IO interval (MB/sec): "
-               "%.1f new, %.1f read, %.1f write, %.1f read+write\n",
-               interval_bytes_new / 1048576.0 / interval_seconds_up,
-               interval_bytes_read / 1048576.0 / interval_seconds_up,
-               interval_bytes_written / 1048576.0 / interval_seconds_up,
-               (interval_bytes_read + interval_bytes_written) / 1048576.0 /
-                   interval_seconds_up);
-      value->append(buf);
-
-      // +1 to avoid divide by 0 and NaN
-      snprintf(
-          buf, sizeof(buf),
-          "Amplification interval: %.1f write, %.1f compaction\n",
-          (double)(interval_bytes_written + wal_bytes) /
-              (interval_bytes_new + 1),
-          (double)(interval_bytes_written + interval_bytes_read + wal_bytes) /
-              (interval_bytes_new + 1));
-      value->append(buf);
-
-      snprintf(buf, sizeof(buf),
-               "Stalls(secs): %.3f level0_slowdown, %.3f level0_numfiles, "
-               "%.3f memtable_compaction, %.3f leveln_slowdown\n",
-               stall_micros_[LEVEL0_SLOWDOWN] / 1000000.0,
-               stall_micros_[LEVEL0_NUM_FILES] / 1000000.0,
-               stall_micros_[MEMTABLE_COMPACTION] / 1000000.0,
-               total_slowdown / 1000000.0);
-      value->append(buf);
-
-      snprintf(buf, sizeof(buf),
-               "Stalls(count): %lu level0_slowdown, %lu level0_numfiles, "
-               "%lu memtable_compaction, %lu leveln_slowdown\n",
-               (unsigned long)stall_counts_[LEVEL0_SLOWDOWN],
-               (unsigned long)stall_counts_[LEVEL0_NUM_FILES],
-               (unsigned long)stall_counts_[MEMTABLE_COMPACTION],
-               (unsigned long)total_slowdown_count);
-      value->append(buf);
-
-      last_stats_.compaction_bytes_read_ = total_bytes_read;
-      last_stats_.compaction_bytes_written_ = total_bytes_written;
-      last_stats_.ingest_bytes_ = user_bytes_written;
-      last_stats_.seconds_up_ = seconds_up;
-      last_stats_.wal_bytes_ = wal_bytes;
-      last_stats_.wal_synced_ = wal_synced;
-      last_stats_.write_with_wal_ = write_with_wal;
-      last_stats_.write_other_ = write_other;
-      last_stats_.write_self_ = write_self;
-
+      return true;
+    }
+    case kCFStats: {
+      DumpCFStats(value);
+      return true;
+    }
+    case kDBStats: {
+      DumpDBStats(value);
      return true;
    }
    case kSsTables:
      *value = current->DebugString();
      return true;
-    case kNumImmutableMemTable:
-      *value = std::to_string(cfd->imm()->size());
-      return true;
-    case kMemtableFlushPending:
-      // Return number of mem tables that are ready to flush (made immutable)
-      *value = std::to_string(cfd->imm()->IsFlushPending() ? 1 : 0);
-      return true;
-    case kCompactionPending:
-      // 1 if the system already determines at least one compacdtion is needed.
-      // 0 otherwise,
-      *value = std::to_string(current->NeedsCompaction() ? 1 : 0);
-      return true;
-    case kBackgroundErrors:
-      // Accumulated number of  errors in background flushes or compactions.
-      *value = std::to_string(GetBackgroundErrorCount());
-      return true;
-    case kCurSizeActiveMemTable:
-      // Current size of the active memtable
-      *value = std::to_string(cfd->mem()->ApproximateMemoryUsage());
-      return true;
-    case kNumEntriesInMutableMemtable:
-      // Current size of the active memtable
-      *value = std::to_string(cfd->mem()->GetNumEntries());
-      return true;
-    case kNumEntriesInImmutableMemtable:
-      // Current size of the active memtable
-      *value = std::to_string(cfd->imm()->current()->GetTotalNumEntries());
-      return true;
    default:
      return false;
  }
 }

+bool InternalStats::GetIntProperty(DBPropertyType property_type,
+                                   const Slice& property,
+                                   uint64_t* value) const {
+  Version* current = cfd_->current();
+
+  switch (property_type) {
+    case kNumImmutableMemTable:
+      *value = cfd_->imm()->size();
+      return true;
+    case kMemtableFlushPending:
+      // Return number of mem tables that are ready to flush (made immutable)
+      *value = (cfd_->imm()->IsFlushPending() ? 1 : 0);
+      return true;
+    case kCompactionPending:
+      // 1 if the system already determines at least one compacdtion is needed.
+      // 0 otherwise,
+      *value = (current->NeedsCompaction() ? 1 : 0);
+      return true;
+    case kBackgroundErrors:
+      // Accumulated number of  errors in background flushes or compactions.
+      *value = GetBackgroundErrorCount();
+      return true;
+    case kCurSizeActiveMemTable:
+      // Current size of the active memtable
+      *value = cfd_->mem()->ApproximateMemoryUsage();
+      return true;
+    case kNumEntriesInMutableMemtable:
+      // Current size of the active memtable
+      *value = cfd_->mem()->GetNumEntries();
+      return true;
+    case kNumEntriesInImmutableMemtable:
+      // Current size of the active memtable
+      *value = cfd_->imm()->current()->GetTotalNumEntries();
+      return true;
+    case kEstimatedNumKeys:
+      // Estimate number of entries in the column family:
+      // Use estimated entries in tables + total entries in memtables.
+      *value = cfd_->mem()->GetNumEntries() +
+               cfd_->imm()->current()->GetTotalNumEntries() +
+               current->GetEstimatedActiveKeys();
+      return true;
+    default:
+      return false;
+  }
+}
+
+void InternalStats::DumpDBStats(std::string* value) {
+  char buf[1000];
+  // DB-level stats, only available from default column family
+  double seconds_up = (env_->NowMicros() - started_at_ + 1) / 1000000.0;
+  double interval_seconds_up = seconds_up - db_stats_snapshot_.seconds_up;
+  snprintf(buf, sizeof(buf),
+           "\n** DB Stats **\nUptime(secs): %.1f total, %.1f interval\n",
+           seconds_up, interval_seconds_up);
+  value->append(buf);
+  // Cumulative
+  uint64_t user_bytes_written = db_stats_[InternalStats::BYTES_WRITTEN];
+  uint64_t write_other = db_stats_[InternalStats::WRITE_DONE_BY_OTHER];
+  uint64_t write_self = db_stats_[InternalStats::WRITE_DONE_BY_SELF];
+  uint64_t wal_bytes = db_stats_[InternalStats::WAL_FILE_BYTES];
+  uint64_t wal_synced = db_stats_[InternalStats::WAL_FILE_SYNCED];
+  uint64_t write_with_wal = db_stats_[InternalStats::WRITE_WITH_WAL];
+  // Data
+  snprintf(buf, sizeof(buf),
+           "Cumulative writes: %" PRIu64 " writes, %" PRIu64 " batches, "
+           "%.1f writes per batch, %.2f GB user ingest\n",
+           write_other + write_self, write_self,
+           (write_other + write_self) / static_cast<double>(write_self + 1),
+           user_bytes_written / kGB);
+  value->append(buf);
+  // WAL
+  snprintf(buf, sizeof(buf),
+           "Cumulative WAL: %" PRIu64 " writes, %" PRIu64 " syncs, "
+           "%.2f writes per sync, %.2f GB written\n",
+           write_with_wal, wal_synced,
+           write_with_wal / static_cast<double>(wal_synced + 1),
+           wal_bytes / kGB);
+  value->append(buf);
+
+  // Interval
+  uint64_t interval_write_other = write_other - db_stats_snapshot_.write_other;
+  uint64_t interval_write_self = write_self - db_stats_snapshot_.write_self;
+  snprintf(buf, sizeof(buf),
+           "Interval writes: %" PRIu64 " writes, %" PRIu64 " batches, "
+           "%.1f writes per batch, %.1f MB user ingest\n",
+           interval_write_other + interval_write_self,
+           interval_write_self,
+           static_cast<double>(interval_write_other + interval_write_self) /
+               (interval_write_self + 1),
+           (user_bytes_written - db_stats_snapshot_.ingest_bytes) / kMB);
+  value->append(buf);
+
+  uint64_t interval_write_with_wal =
+      write_with_wal - db_stats_snapshot_.write_with_wal;
+  uint64_t interval_wal_synced = wal_synced - db_stats_snapshot_.wal_synced;
+  uint64_t interval_wal_bytes = wal_bytes - db_stats_snapshot_.wal_bytes;
+
+  snprintf(buf, sizeof(buf),
+           "Interval WAL: %" PRIu64 " writes, %" PRIu64 " syncs, "
+           "%.2f writes per sync, %.2f MB written\n",
+           interval_write_with_wal,
+           interval_wal_synced,
+           interval_write_with_wal /
+              static_cast<double>(interval_wal_synced + 1),
+           interval_wal_bytes / kGB);
+  value->append(buf);
+
+  db_stats_snapshot_.seconds_up = seconds_up;
+  db_stats_snapshot_.ingest_bytes = user_bytes_written;
+  db_stats_snapshot_.write_other = write_other;
+  db_stats_snapshot_.write_self = write_self;
+  db_stats_snapshot_.wal_bytes = wal_bytes;
+  db_stats_snapshot_.wal_synced = wal_synced;
+  db_stats_snapshot_.write_with_wal = write_with_wal;
+}
+
+void InternalStats::DumpCFStats(std::string* value) {
+  Version* current = cfd_->current();
+
+  int num_levels_to_check =
+      (cfd_->options()->compaction_style != kCompactionStyleUniversal &&
+       cfd_->options()->compaction_style != kCompactionStyleFIFO)
+          ? current->NumberLevels() - 1
+          : 1;
+  // Compaction scores are sorted base on its value. Restore them to the
+  // level order
+  std::vector<double> compaction_score(number_levels_, 0);
+  for (int i = 0; i < num_levels_to_check; ++i) {
+    compaction_score[current->compaction_level_[i]] =
+      current->compaction_score_[i];
+  }
+  // Count # of files being compacted for each level
+  std::vector<int> files_being_compacted(number_levels_, 0);
+  for (int level = 0; level < num_levels_to_check; ++level) {
+    for (auto* f : current->files_[level]) {
+      if (f->being_compacted) {
+        ++files_being_compacted[level];
+      }
+    }
+  }
+
+  char buf[1000];
+  // Per-ColumnFamily stats
+  PrintLevelStatsHeader(buf, sizeof(buf), cfd_->GetName());
+  value->append(buf);
+
+  CompactionStats stats_sum(0);
+  int total_files = 0;
+  int total_files_being_compacted = 0;
+  double total_file_size = 0;
+  uint64_t total_slowdown_soft = 0;
+  uint64_t total_slowdown_count_soft = 0;
+  uint64_t total_slowdown_hard = 0;
+  uint64_t total_slowdown_count_hard = 0;
+  uint64_t total_stall_count = 0;
+  double total_stall_us = 0;
+  for (int level = 0; level < number_levels_; level++) {
+    int files = current->NumLevelFiles(level);
+    total_files += files;
+    total_files_being_compacted += files_being_compacted[level];
+    if (comp_stats_[level].micros > 0 || files > 0) {
+      uint64_t stalls = level == 0 ?
+        (cf_stats_count_[LEVEL0_SLOWDOWN] +
+         cf_stats_count_[LEVEL0_NUM_FILES] +
+         cf_stats_count_[MEMTABLE_COMPACTION])
+        : (stall_leveln_slowdown_count_soft_[level] +
+           stall_leveln_slowdown_count_hard_[level]);
+
+      double stall_us = level == 0 ?
+         (cf_stats_value_[LEVEL0_SLOWDOWN] +
+          cf_stats_value_[LEVEL0_NUM_FILES] +
+          cf_stats_value_[MEMTABLE_COMPACTION])
+         : (stall_leveln_slowdown_soft_[level] +
+            stall_leveln_slowdown_hard_[level]);
+
+      stats_sum.Add(comp_stats_[level]);
+      total_file_size += current->NumLevelBytes(level);
+      total_stall_us += stall_us;
+      total_stall_count += stalls;
+      total_slowdown_soft += stall_leveln_slowdown_soft_[level];
+      total_slowdown_count_soft += stall_leveln_slowdown_count_soft_[level];
+      total_slowdown_hard += stall_leveln_slowdown_hard_[level];
+      total_slowdown_count_hard += stall_leveln_slowdown_count_hard_[level];
+      int64_t bytes_read = comp_stats_[level].bytes_readn +
+                           comp_stats_[level].bytes_readnp1;
+      double rw_amp = (comp_stats_[level].bytes_readn == 0) ? 0.0
+          : (comp_stats_[level].bytes_written + bytes_read) /
+            static_cast<double>(comp_stats_[level].bytes_readn);
+      double w_amp = (comp_stats_[level].bytes_readn == 0) ? 0.0
+          : comp_stats_[level].bytes_written /
+            static_cast<double>(comp_stats_[level].bytes_readn);
+      PrintLevelStats(buf, sizeof(buf), "L" + std::to_string(level),
+          files, files_being_compacted[level], current->NumLevelBytes(level),
+          compaction_score[level], rw_amp, w_amp, stall_us, stalls,
+          comp_stats_[level]);
+      value->append(buf);
+    }
+  }
+  uint64_t curr_ingest = cf_stats_value_[BYTES_FLUSHED];
+  // Cumulative summary
+  double rw_amp = (stats_sum.bytes_written + stats_sum.bytes_readn +
+      stats_sum.bytes_readnp1) / static_cast<double>(curr_ingest + 1);
+  double w_amp = stats_sum.bytes_written / static_cast<double>(curr_ingest + 1);
+  // Stats summary across levels
+  PrintLevelStats(buf, sizeof(buf), "Sum", total_files,
+      total_files_being_compacted, total_file_size, 0, rw_amp, w_amp,
+      total_stall_us, total_stall_count, stats_sum);
+  value->append(buf);
+  // Interval summary
+  uint64_t interval_ingest =
+      curr_ingest - cf_stats_snapshot_.ingest_bytes + 1;
+  CompactionStats interval_stats(stats_sum);
+  interval_stats.Subtract(cf_stats_snapshot_.comp_stats);
+  rw_amp = (interval_stats.bytes_written +
+      interval_stats.bytes_readn + interval_stats.bytes_readnp1) /
+      static_cast<double>(interval_ingest);
+  w_amp = interval_stats.bytes_written / static_cast<double>(interval_ingest);
+  PrintLevelStats(buf, sizeof(buf), "Int", 0, 0, 0, 0,
+      rw_amp, w_amp, total_stall_us - cf_stats_snapshot_.stall_us,
+      total_stall_count - cf_stats_snapshot_.stall_count, interval_stats);
+  value->append(buf);
+
+  snprintf(buf, sizeof(buf),
+           "Flush(GB): accumulative %.3f, interval %.3f\n",
+           curr_ingest / kGB, interval_ingest / kGB);
+  value->append(buf);
+  snprintf(buf, sizeof(buf),
+           "Stalls(secs): %.3f level0_slowdown, %.3f level0_numfiles, "
+           "%.3f memtable_compaction, %.3f leveln_slowdown_soft, "
+           "%.3f leveln_slowdown_hard\n",
+           cf_stats_value_[LEVEL0_SLOWDOWN] / 1000000.0,
+           cf_stats_value_[LEVEL0_NUM_FILES] / 1000000.0,
+           cf_stats_value_[MEMTABLE_COMPACTION] / 1000000.0,
+           total_slowdown_soft / 1000000.0,
+           total_slowdown_hard / 1000000.0);
+  value->append(buf);
+
+  snprintf(buf, sizeof(buf),
+           "Stalls(count): %" PRIu64 " level0_slowdown, "
+           "%" PRIu64 " level0_numfiles, %" PRIu64 " memtable_compaction, "
+           "%" PRIu64 " leveln_slowdown_soft, "
+           "%" PRIu64 " leveln_slowdown_hard\n",
+           cf_stats_count_[LEVEL0_SLOWDOWN],
+           cf_stats_count_[LEVEL0_NUM_FILES],
+           cf_stats_count_[MEMTABLE_COMPACTION],
+           total_slowdown_count_soft, total_slowdown_count_hard);
+  value->append(buf);
+
+  cf_stats_snapshot_.ingest_bytes = curr_ingest;
+  cf_stats_snapshot_.comp_stats = stats_sum;
+  cf_stats_snapshot_.stall_us = total_stall_us;
+  cf_stats_snapshot_.stall_count = total_stall_count;
+}
+
 }  // namespace rocksdb
@@ -9,8 +9,6 @@
 //

 #pragma once
-#include "rocksdb/statistics.h"
-#include "util/statistics.h"
 #include "db/version_set.h"

 #include <vector>
@@ -23,11 +21,15 @@ namespace rocksdb {
 class MemTableList;
 class DBImpl;

-enum DBPropertyType {
+enum DBPropertyType : uint32_t {
+  kUnknown,
  kNumFilesAtLevel,  // Number of files at a specific level
  kLevelStats,       // Return number of files and total sizes of each level
-  kStats,            // Return general statitistics of DB
+  kCFStats,          // Return general statitistics of CF
+  kDBStats,          // Return general statitistics of DB
+  kStats,            // Return general statitistics of both DB and CF
  kSsTables,         // Return a human readable string of current SST files
+  kStartIntTypes,    // ---- Dummy value to indicate the start of integer values
  kNumImmutableMemTable,   // Return number of immutable mem tables
  kMemtableFlushPending,   // Return 1 if mem table flushing is pending,
                           // otherwise 0.
@@ -38,45 +40,74 @@ enum DBPropertyType {
                                   // memtable.
  kNumEntriesInImmutableMemtable,  // Return sum of number of entries in all
                                   // the immutable mem tables.
-  kUnknown,
+  kEstimatedNumKeys,  // Estimated total number of keys in the database.
 };

 extern DBPropertyType GetPropertyType(const Slice& property);

 class InternalStats {
 public:
-  enum WriteStallType {
+  enum InternalCFStatsType {
    LEVEL0_SLOWDOWN,
    MEMTABLE_COMPACTION,
    LEVEL0_NUM_FILES,
    WRITE_STALLS_ENUM_MAX,
+    BYTES_FLUSHED,
+    INTERNAL_CF_STATS_ENUM_MAX,
  };

-  InternalStats(int num_levels, Env* env, Statistics* statistics)
-      : compaction_stats_(num_levels),
-        stall_micros_(WRITE_STALLS_ENUM_MAX, 0),
-        stall_counts_(WRITE_STALLS_ENUM_MAX, 0),
-        stall_leveln_slowdown_(num_levels, 0),
-        stall_leveln_slowdown_count_(num_levels, 0),
+  enum InternalDBStatsType {
+    WAL_FILE_BYTES,
+    WAL_FILE_SYNCED,
+    BYTES_WRITTEN,
+    WRITE_DONE_BY_OTHER,
+    WRITE_DONE_BY_SELF,
+    WRITE_WITH_WAL,
+    INTERNAL_DB_STATS_ENUM_MAX,
+  };
+
+  InternalStats(int num_levels, Env* env, ColumnFamilyData* cfd)
+      : db_stats_(INTERNAL_DB_STATS_ENUM_MAX),
+        cf_stats_value_(INTERNAL_CF_STATS_ENUM_MAX),
+        cf_stats_count_(INTERNAL_CF_STATS_ENUM_MAX),
+        comp_stats_(num_levels),
+        stall_leveln_slowdown_hard_(num_levels),
+        stall_leveln_slowdown_count_hard_(num_levels),
+        stall_leveln_slowdown_soft_(num_levels),
+        stall_leveln_slowdown_count_soft_(num_levels),
        bg_error_count_(0),
        number_levels_(num_levels),
-        statistics_(statistics),
        env_(env),
-        started_at_(env->NowMicros()) {}
+        cfd_(cfd),
+        started_at_(env->NowMicros()) {
+    for (int i = 0; i< INTERNAL_DB_STATS_ENUM_MAX; ++i) {
+      db_stats_[i] = 0;
+    }
+    for (int i = 0; i< INTERNAL_CF_STATS_ENUM_MAX; ++i) {
+      cf_stats_value_[i] = 0;
+      cf_stats_count_[i] = 0;
+    }
+    for (int i = 0; i < num_levels; ++i) {
+      stall_leveln_slowdown_hard_[i] = 0;
+      stall_leveln_slowdown_count_hard_[i] = 0;
+      stall_leveln_slowdown_soft_[i] = 0;
+      stall_leveln_slowdown_count_soft_[i] = 0;
+    }
+  }

-  // Per level compaction stats.  compaction_stats_[level] stores the stats for
+  // Per level compaction stats.  comp_stats_[level] stores the stats for
  // compactions that produced data for the specified "level".
  struct CompactionStats {
    uint64_t micros;

    // Bytes read from level N during compaction between levels N and N+1
-    int64_t bytes_readn;
+    uint64_t bytes_readn;

    // Bytes read from level N+1 during compaction between levels N and N+1
-    int64_t bytes_readnp1;
+    uint64_t bytes_readnp1;

    // Total bytes written during compaction between levels N and N+1
-    int64_t bytes_written;
+    uint64_t bytes_written;

    // Files read from level N during compaction between levels N and N+1
    int files_in_leveln;
@@ -90,7 +121,7 @@ class InternalStats {
    // Number of compactions done
    int count;

-    CompactionStats()
+    explicit CompactionStats(int count = 0)
        : micros(0),
          bytes_readn(0),
          bytes_readnp1(0),
@@ -98,7 +129,17 @@ class InternalStats {
          files_in_leveln(0),
          files_in_levelnp1(0),
          files_out_levelnp1(0),
-          count(0) {}
+          count(count) {}
+
+    explicit CompactionStats(const CompactionStats& c)
+        : micros(c.micros),
+          bytes_readn(c.bytes_readn),
+          bytes_readnp1(c.bytes_readnp1),
+          bytes_written(c.bytes_written),
+          files_in_leveln(c.files_in_leveln),
+          files_in_levelnp1(c.files_in_levelnp1),
+          files_out_levelnp1(c.files_out_levelnp1),
+          count(c.count) {}

    void Add(const CompactionStats& c) {
      this->micros += c.micros;
@@ -108,22 +149,42 @@ class InternalStats {
      this->files_in_leveln += c.files_in_leveln;
      this->files_in_levelnp1 += c.files_in_levelnp1;
      this->files_out_levelnp1 += c.files_out_levelnp1;
-      this->count += 1;
+      this->count += c.count;
+    }
+
+    void Subtract(const CompactionStats& c) {
+      this->micros -= c.micros;
+      this->bytes_readn -= c.bytes_readn;
+      this->bytes_readnp1 -= c.bytes_readnp1;
+      this->bytes_written -= c.bytes_written;
+      this->files_in_leveln -= c.files_in_leveln;
+      this->files_in_levelnp1 -= c.files_in_levelnp1;
+      this->files_out_levelnp1 -= c.files_out_levelnp1;
+      this->count -= c.count;
    }
  };

  void AddCompactionStats(int level, const CompactionStats& stats) {
-    compaction_stats_[level].Add(stats);
+    comp_stats_[level].Add(stats);
  }

-  void RecordWriteStall(WriteStallType write_stall_type, uint64_t micros) {
-    stall_micros_[write_stall_type] += micros;
-    stall_counts_[write_stall_type]++;
+  void RecordLevelNSlowdown(int level, uint64_t micros, bool soft) {
+    if (soft) {
+      stall_leveln_slowdown_soft_[level] += micros;
+      ++stall_leveln_slowdown_count_soft_[level];
+    } else {
+      stall_leveln_slowdown_hard_[level] += micros;
+      ++stall_leveln_slowdown_count_hard_[level];
+    }
  }

-  void RecordLevelNSlowdown(int level, uint64_t micros) {
-    stall_leveln_slowdown_[level] += micros;
-    stall_leveln_slowdown_count_[level] += micros;
+  void AddCFStats(InternalCFStatsType type, uint64_t value) {
+    cf_stats_value_[type] += value;
+    ++cf_stats_count_[type];
+  }
+
+  void AddDBStats(InternalDBStatsType type, uint64_t value) {
+    db_stats_[type] += value;
  }

  uint64_t GetBackgroundErrorCount() const { return bg_error_count_; }
@@ -131,45 +192,64 @@ class InternalStats {
  uint64_t BumpAndGetBackgroundErrorCount() { return ++bg_error_count_; }

  bool GetProperty(DBPropertyType property_type, const Slice& property,
-                   std::string* value, ColumnFamilyData* cfd);
+                   std::string* value);
+
+  bool GetIntProperty(DBPropertyType property_type, const Slice& property,
+                      uint64_t* value) const;

 private:
-  std::vector<CompactionStats> compaction_stats_;
+  void DumpDBStats(std::string* value);
+  void DumpCFStats(std::string* value);
+
+  // Per-DB stats
+  std::vector<uint64_t> db_stats_;
+  // Per-ColumnFamily stats
+  std::vector<uint64_t> cf_stats_value_;
+  std::vector<uint64_t> cf_stats_count_;
+  // Per-ColumnFamily/level compaction stats
+  std::vector<CompactionStats> comp_stats_;
+  // These count the number of microseconds for which MakeRoomForWrite stalls.
+  std::vector<uint64_t> stall_leveln_slowdown_hard_;
+  std::vector<uint64_t> stall_leveln_slowdown_count_hard_;
+  std::vector<uint64_t> stall_leveln_slowdown_soft_;
+  std::vector<uint64_t> stall_leveln_slowdown_count_soft_;

  // Used to compute per-interval statistics
-  struct StatsSnapshot {
-    uint64_t compaction_bytes_read_;     // Bytes read by compaction
-    uint64_t compaction_bytes_written_;  // Bytes written by compaction
-    uint64_t ingest_bytes_;              // Bytes written by user
-    uint64_t wal_bytes_;                 // Bytes written to WAL
-    uint64_t wal_synced_;                // Number of times WAL is synced
-    uint64_t write_with_wal_;            // Number of writes that request WAL
+  struct CFStatsSnapshot {
+    // ColumnFamily-level stats
+    CompactionStats comp_stats;
+    uint64_t ingest_bytes;            // Bytes written to L0
+    uint64_t stall_us;                // Stall time in micro-seconds
+    uint64_t stall_count;             // Stall count
+
+    CFStatsSnapshot()
+        : comp_stats(0),
+          ingest_bytes(0),
+          stall_us(0),
+          stall_count(0) {}
+  } cf_stats_snapshot_;
+
+  struct DBStatsSnapshot {
+    // DB-level stats
+    uint64_t ingest_bytes;            // Bytes written by user
+    uint64_t wal_bytes;               // Bytes written to WAL
+    uint64_t wal_synced;              // Number of times WAL is synced
+    uint64_t write_with_wal;          // Number of writes that request WAL
    // These count the number of writes processed by the calling thread or
    // another thread.
-    uint64_t write_other_;
-    uint64_t write_self_;
-    double seconds_up_;
+    uint64_t write_other;
+    uint64_t write_self;
+    double seconds_up;

-    StatsSnapshot()
-        : compaction_bytes_read_(0),
-          compaction_bytes_written_(0),
-          ingest_bytes_(0),
-          wal_bytes_(0),
-          wal_synced_(0),
-          write_with_wal_(0),
-          write_other_(0),
-          write_self_(0),
-          seconds_up_(0) {}
-  };
-
-  // Counters from the previous time per-interval stats were computed
-  StatsSnapshot last_stats_;
-
-  // These count the number of microseconds for which MakeRoomForWrite stalls.
-  std::vector<uint64_t> stall_micros_;
-  std::vector<uint64_t> stall_counts_;
-  std::vector<uint64_t> stall_leveln_slowdown_;
-  std::vector<uint64_t> stall_leveln_slowdown_count_;
+    DBStatsSnapshot()
+        : ingest_bytes(0),
+          wal_bytes(0),
+          wal_synced(0),
+          write_with_wal(0),
+          write_other(0),
+          write_self(0),
+          seconds_up(0) {}
+  } db_stats_snapshot_;

  // Total number of background errors encountered. Every time a flush task
  // or compaction task fails, this counter is incremented. The failure can
@@ -178,10 +258,10 @@ class InternalStats {
  // or compaction will cause the counter to increase too.
  uint64_t bg_error_count_;

-  int number_levels_;
-  Statistics* statistics_;
+  const int number_levels_;
  Env* env_;
-  uint64_t started_at_;
+  ColumnFamilyData* cfd_;
+  const uint64_t started_at_;
 };

 }  // namespace rocksdb
@@ -51,7 +51,7 @@ void BM_LogAndApply(int iters, int num_base_files) {
    for (int i = 0; i < num_base_files; i++) {
      InternalKey start(MakeKey(2 * fnum), 1, kTypeValue);
      InternalKey limit(MakeKey(2 * fnum + 1), 1, kTypeDeletion);
-      vbase.AddFile(2, ++fnum, 1 /* file size */, start, limit, 1, 1);
+      vbase.AddFile(2, ++fnum, 0, 1 /* file size */, start, limit, 1, 1);
    }
    ASSERT_OK(vset->LogAndApply(default_cfd, &vbase, &mu));
  }
@@ -61,7 +61,7 @@ void BM_LogAndApply(int iters, int num_base_files) {
    vedit.DeleteFile(2, fnum);
    InternalKey start(MakeKey(2 * fnum), 1, kTypeValue);
    InternalKey limit(MakeKey(2 * fnum + 1), 1, kTypeDeletion);
-    vedit.AddFile(2, ++fnum, 1 /* file size */, start, limit, 1, 1);
+    vedit.AddFile(2, ++fnum, 0, 1 /* file size */, start, limit, 1, 1);
    vset->LogAndApply(default_cfd, &vedit, &mu);
  }
 }
@@ -55,6 +55,7 @@ MemTable::MemTable(const InternalKeyComparator& cmp, const Options& options)
  assert(!should_flush_);
  if (prefix_extractor_ && options.memtable_prefix_bloom_bits > 0) {
    prefix_bloom_.reset(new DynamicBloom(
+        &arena_,
        options.memtable_prefix_bloom_bits, options.bloom_locality,
        options.memtable_prefix_bloom_probes, nullptr,
        options.memtable_prefix_bloom_huge_page_tlb_size,
@@ -449,7 +450,7 @@ void MemTable::Update(SequenceNumber seq,
  Slice mem_key = lkey.memtable_key();

  std::unique_ptr<MemTableRep::Iterator> iter(
-    table_->GetIterator(lkey.user_key()));
+      table_->GetDynamicPrefixIterator());
  iter->Seek(lkey.internal_key(), mem_key.data());

  if (iter->Valid()) {
@@ -508,7 +509,7 @@ bool MemTable::UpdateCallback(SequenceNumber seq,
  Slice memkey = lkey.memtable_key();

  std::unique_ptr<MemTableRep::Iterator> iter(
-    table_->GetIterator(lkey.user_key()));
+      table_->GetDynamicPrefixIterator());
  iter->Seek(lkey.internal_key(), memkey.data());

  if (iter->Valid()) {
@@ -583,7 +584,7 @@ size_t MemTable::CountSuccessiveMergeEntries(const LookupKey& key) {
  // reps). By passing in the user key, we allow efficient iterator creation.
  // The iterator only needs to be ordered within the same user key.
  std::unique_ptr<MemTableRep::Iterator> iter(
-      table_->GetIterator(key.user_key()));
+      table_->GetDynamicPrefixIterator());
  iter->Seek(key.internal_key(), memkey.data());

  size_t num_successive_merges = 0;
@@ -610,7 +611,7 @@ size_t MemTable::CountSuccessiveMergeEntries(const LookupKey& key) {

 void MemTableRep::Get(const LookupKey& k, void* callback_args,
                      bool (*callback_func)(void* arg, const char* entry)) {
-  auto iter = GetIterator(k.user_key());
+  auto iter = GetDynamicPrefixIterator();
  for (iter->Seek(k.internal_key(), k.memtable_key().data());
       iter->Valid() && callback_func(callback_args, iter->key());
       iter->Next()) {
@@ -140,7 +140,7 @@ void MemTableList::PickMemtablesToFlush(autovector<MemTable*>* ret) {

 void MemTableList::RollbackMemtableFlush(const autovector<MemTable*>& mems,
                                         uint64_t file_number,
-                                         std::set<uint64_t>* pending_outputs) {
+                                         FileNumToPathIdMap* pending_outputs) {
  assert(!mems.empty());

  // If the flush was not successful, then just reset state.
@@ -162,7 +162,7 @@ void MemTableList::RollbackMemtableFlush(const autovector<MemTable*>& mems,
 Status MemTableList::InstallMemtableFlushResults(
    ColumnFamilyData* cfd, const autovector<MemTable*>& mems, VersionSet* vset,
    port::Mutex* mu, Logger* info_log, uint64_t file_number,
-    std::set<uint64_t>& pending_outputs, autovector<MemTable*>* to_delete,
+    FileNumToPathIdMap* pending_outputs, autovector<MemTable*>* to_delete,
    Directory* db_directory, LogBuffer* log_buffer) {
  mu->AssertHeld();

@@ -219,7 +219,7 @@ Status MemTableList::InstallMemtableFlushResults(
        // has been written to a committed version so that other concurrently
        // executing compaction threads do not mistakenly assume that this
        // file is not live.
-        pending_outputs.erase(m->file_number_);
+        pending_outputs->erase(m->file_number_);
        if (m->Unref() != nullptr) {
          to_delete->push_back(m);
        }
@@ -233,7 +233,7 @@ Status MemTableList::InstallMemtableFlushResults(
        m->flush_in_progress_ = false;
        m->edit_.Clear();
        num_flush_not_started_++;
-        pending_outputs.erase(m->file_number_);
+        pending_outputs->erase(m->file_number_);
        m->file_number_ = 0;
        imm_flush_needed.Release_Store((void *)1);
      }
@@ -15,6 +15,7 @@
 #include "rocksdb/iterator.h"

 #include "db/dbformat.h"
+#include "db/filename.h"
 #include "db/skiplist.h"
 #include "db/memtable.h"
 #include "rocksdb/db.h"
@@ -108,17 +109,14 @@ class MemTableList {
  // they can get picked up again on the next round of flush.
  void RollbackMemtableFlush(const autovector<MemTable*>& mems,
                             uint64_t file_number,
-                             std::set<uint64_t>* pending_outputs);
+                             FileNumToPathIdMap* pending_outputs);

  // Commit a successful flush in the manifest file
-  Status InstallMemtableFlushResults(ColumnFamilyData* cfd,
-                                     const autovector<MemTable*>& m,
-                                     VersionSet* vset, port::Mutex* mu,
-                                     Logger* info_log, uint64_t file_number,
-                                     std::set<uint64_t>& pending_outputs,
-                                     autovector<MemTable*>* to_delete,
-                                     Directory* db_directory,
-                                     LogBuffer* log_buffer);
+  Status InstallMemtableFlushResults(
+      ColumnFamilyData* cfd, const autovector<MemTable*>& m, VersionSet* vset,
+      port::Mutex* mu, Logger* info_log, uint64_t file_number,
+      FileNumToPathIdMap* pending_outputs, autovector<MemTable*>* to_delete,
+      Directory* db_directory, LogBuffer* log_buffer);

  // New memtables are inserted at the front of the list.
  // Takes ownership of the referenced held on *m by the caller of Add().
@@ -12,12 +12,12 @@
 #include "rocksdb/db.h"
 #include "rocksdb/env.h"
 #include "rocksdb/merge_operator.h"
+#include "rocksdb/utilities/db_ttl.h"
 #include "db/dbformat.h"
 #include "db/db_impl.h"
 #include "db/write_batch_internal.h"
 #include "utilities/merge_operators.h"
 #include "util/testharness.h"
-#include "utilities/db_ttl.h"

 using namespace std;
 using namespace rocksdb;
@@ -23,6 +23,7 @@
 #include "rocksdb/slice_transform.h"
 #include "rocksdb/table.h"
 #include "table/meta_blocks.h"
+#include "table/bloom_block.h"
 #include "table/plain_table_factory.h"
 #include "table/plain_table_reader.h"
 #include "util/hash.h"
@@ -61,8 +62,20 @@ class PlainTableDBTest {
  // Return the current option configuration.
  Options CurrentOptions() {
    Options options;
-    options.table_factory.reset(NewPlainTableFactory(0, 2, 0.8, 3, 0, kPrefix));
+
+    PlainTableOptions plain_table_options;
+    plain_table_options.user_key_len = 0;
+    plain_table_options.bloom_bits_per_key = 2;
+    plain_table_options.hash_table_ratio = 0.8;
+    plain_table_options.index_sparseness = 3;
+    plain_table_options.huge_page_tlb_size = 0;
+    plain_table_options.encoding_type = kPrefix;
+    plain_table_options.full_scan_mode = false;
+    plain_table_options.store_index_in_file = false;
+
+    options.table_factory.reset(NewPlainTableFactory(plain_table_options));
    options.memtable_factory.reset(NewHashLinkListRepFactory(4, 0, 3, true));
+
    options.prefix_extractor.reset(NewFixedPrefixTransform(8));
    options.allow_mmap_reads = true;
    return options;
@@ -175,6 +188,8 @@ TEST(PlainTableDBTest, Empty) {
  ASSERT_EQ("NOT_FOUND", Get("0000000000000foo"));
 }

+extern const uint64_t kPlainTableMagicNumber;
+
 class TestPlainTableReader : public PlainTableReader {
 public:
  TestPlainTableReader(const EnvOptions& storage_options,
@@ -184,7 +199,8 @@ class TestPlainTableReader : public PlainTableReader {
                       size_t index_sparseness,
                       const TableProperties* table_properties,
                       unique_ptr<RandomAccessFile>&& file,
-                       const Options& options, bool* expect_bloom_not_match)
+                       const Options& options, bool* expect_bloom_not_match,
+                       bool store_index_in_file)
      : PlainTableReader(options, std::move(file), storage_options, icomparator,
                         encoding_type, file_size, table_properties),
        expect_bloom_not_match_(expect_bloom_not_match) {
@@ -195,6 +211,19 @@ class TestPlainTableReader : public PlainTableReader {
                      bloom_bits_per_key, hash_table_ratio, index_sparseness,
                      2 * 1024 * 1024);
    ASSERT_TRUE(s.ok());
+
+    TableProperties* props = const_cast<TableProperties*>(table_properties);
+    if (store_index_in_file) {
+      auto bloom_version_ptr = props->user_collected_properties.find(
+          PlainTablePropertyNames::kBloomVersion);
+      ASSERT_TRUE(bloom_version_ptr != props->user_collected_properties.end());
+      ASSERT_EQ(bloom_version_ptr->second, std::string("1"));
+      if (options.bloom_locality > 0) {
+        auto num_blocks_ptr = props->user_collected_properties.find(
+            PlainTablePropertyNames::kNumBloomBlocks);
+        ASSERT_TRUE(num_blocks_ptr != props->user_collected_properties.end());
+      }
+    }
  }

  virtual ~TestPlainTableReader() {}
@@ -202,7 +231,11 @@ class TestPlainTableReader : public PlainTableReader {
 private:
  virtual bool MatchBloom(uint32_t hash) const override {
    bool ret = PlainTableReader::MatchBloom(hash);
-    ASSERT_TRUE(!*expect_bloom_not_match_ || !ret);
+    if (*expect_bloom_not_match_) {
+      ASSERT_TRUE(!ret);
+    } else {
+      ASSERT_TRUE(ret);
+    }
    return ret;
  }
  bool* expect_bloom_not_match_;
@@ -212,16 +245,12 @@ extern const uint64_t kPlainTableMagicNumber;
 class TestPlainTableFactory : public PlainTableFactory {
 public:
  explicit TestPlainTableFactory(bool* expect_bloom_not_match,
-                                 uint32_t user_key_len, int bloom_bits_per_key,
-                                 double hash_table_ratio,
-                                 size_t index_sparseness,
-                                 size_t huge_page_tlb_size,
-                                 EncodingType encoding_type)
-      : PlainTableFactory(user_key_len, bloom_bits_per_key, hash_table_ratio,
-                          index_sparseness, huge_page_tlb_size, encoding_type),
-        bloom_bits_per_key_(bloom_bits_per_key),
-        hash_table_ratio_(hash_table_ratio),
-        index_sparseness_(index_sparseness),
+                                 const PlainTableOptions& options)
+      : PlainTableFactory(options),
+        bloom_bits_per_key_(options.bloom_bits_per_key),
+        hash_table_ratio_(options.hash_table_ratio),
+        index_sparseness_(options.index_sparseness),
+        store_index_in_file_(options.store_index_in_file),
        expect_bloom_not_match_(expect_bloom_not_match) {}

  Status NewTableReader(const Options& options, const EnvOptions& soptions,
@@ -233,6 +262,20 @@ class TestPlainTableFactory : public PlainTableFactory {
                                 options.env, options.info_log.get(), &props);
    ASSERT_TRUE(s.ok());

+    if (store_index_in_file_) {
+      BlockHandle bloom_block_handle;
+      s = FindMetaBlock(file.get(), file_size, kPlainTableMagicNumber,
+                        options.env, BloomBlockBuilder::kBloomBlock,
+                        &bloom_block_handle);
+      ASSERT_TRUE(s.ok());
+
+      BlockHandle index_block_handle;
+      s = FindMetaBlock(
+          file.get(), file_size, kPlainTableMagicNumber, options.env,
+          PlainTableIndexBuilder::kPlainTableIndexBlock, &index_block_handle);
+      ASSERT_TRUE(s.ok());
+    }
+
    auto& user_props = props->user_collected_properties;
    auto encoding_type_prop =
        user_props.find(PlainTablePropertyNames::kEncodingType);
@@ -243,7 +286,8 @@ class TestPlainTableFactory : public PlainTableFactory {
    std::unique_ptr<PlainTableReader> new_reader(new TestPlainTableReader(
        soptions, internal_comparator, encoding_type, file_size,
        bloom_bits_per_key_, hash_table_ratio_, index_sparseness_, props,
-        std::move(file), options, expect_bloom_not_match_));
+        std::move(file), options, expect_bloom_not_match_,
+        store_index_in_file_));

    *table = std::move(new_reader);
    return s;
@@ -253,6 +297,7 @@ class TestPlainTableFactory : public PlainTableFactory {
  int bloom_bits_per_key_;
  double hash_table_ratio_;
  size_t index_sparseness_;
+  bool store_index_in_file_;
  bool* expect_bloom_not_match_;
 };

@@ -262,42 +307,75 @@ TEST(PlainTableDBTest, Flush) {
    for (EncodingType encoding_type : {kPlain, kPrefix}) {
    for (int bloom_bits = 0; bloom_bits <= 117; bloom_bits += 117) {
      for (int total_order = 0; total_order <= 1; total_order++) {
-        if (encoding_type == kPrefix && total_order == 1) {
-          continue;
+        for (int store_index_in_file = 0; store_index_in_file <= 1;
+             ++store_index_in_file) {
+          if (!bloom_bits && store_index_in_file) {
+            continue;
+          }
+
+          Options options = CurrentOptions();
+          options.create_if_missing = true;
+          // Set only one bucket to force bucket conflict.
+          // Test index interval for the same prefix to be 1, 2 and 4
+          if (total_order) {
+            options.prefix_extractor.reset();
+
+            PlainTableOptions plain_table_options;
+            plain_table_options.user_key_len = 0;
+            plain_table_options.bloom_bits_per_key = bloom_bits;
+            plain_table_options.hash_table_ratio = 0;
+            plain_table_options.index_sparseness = 2;
+            plain_table_options.huge_page_tlb_size = huge_page_tlb_size;
+            plain_table_options.encoding_type = encoding_type;
+            plain_table_options.full_scan_mode = false;
+            plain_table_options.store_index_in_file = store_index_in_file;
+
+            options.table_factory.reset(
+                NewPlainTableFactory(plain_table_options));
+          } else {
+            PlainTableOptions plain_table_options;
+            plain_table_options.user_key_len = 0;
+            plain_table_options.bloom_bits_per_key = bloom_bits;
+            plain_table_options.hash_table_ratio = 0.75;
+            plain_table_options.index_sparseness = 16;
+            plain_table_options.huge_page_tlb_size = huge_page_tlb_size;
+            plain_table_options.encoding_type = encoding_type;
+            plain_table_options.full_scan_mode = false;
+            plain_table_options.store_index_in_file = store_index_in_file;
+
+            options.table_factory.reset(
+                NewPlainTableFactory(plain_table_options));
+          }
+          DestroyAndReopen(&options);
+          ASSERT_OK(Put("1000000000000foo", "v1"));
+          ASSERT_OK(Put("0000000000000bar", "v2"));
+          ASSERT_OK(Put("1000000000000foo", "v3"));
+          dbfull()->TEST_FlushMemTable();
+
+          TablePropertiesCollection ptc;
+          reinterpret_cast<DB*>(dbfull())->GetPropertiesOfAllTables(&ptc);
+          ASSERT_EQ(1U, ptc.size());
+          auto row = ptc.begin();
+          auto tp = row->second;
+
+          if (!store_index_in_file) {
+            ASSERT_EQ(total_order ? "4" : "12",
+                      (tp->user_collected_properties)
+                          .at("plain_table_hash_table_size"));
+            ASSERT_EQ("0", (tp->user_collected_properties)
+                               .at("plain_table_sub_index_size"));
+          } else {
+            ASSERT_EQ("0", (tp->user_collected_properties)
+                               .at("plain_table_hash_table_size"));
+            ASSERT_EQ("0", (tp->user_collected_properties)
+                               .at("plain_table_sub_index_size"));
+          }
+          ASSERT_EQ("v3", Get("1000000000000foo"));
+          ASSERT_EQ("v2", Get("0000000000000bar"));
        }
-        Options options = CurrentOptions();
-        options.create_if_missing = true;
-        // Set only one bucket to force bucket conflict.
-        // Test index interval for the same prefix to be 1, 2 and 4
-        if (total_order) {
-          options.table_factory.reset(NewTotalOrderPlainTableFactory(
-              0, bloom_bits, 2, huge_page_tlb_size));
-        } else {
-          options.table_factory.reset(NewPlainTableFactory(
-              0, bloom_bits, 0.75, 16, huge_page_tlb_size, encoding_type));
        }
-        DestroyAndReopen(&options);
-
-        ASSERT_OK(Put("1000000000000foo", "v1"));
-        ASSERT_OK(Put("0000000000000bar", "v2"));
-        ASSERT_OK(Put("1000000000000foo", "v3"));
-        dbfull()->TEST_FlushMemTable();
-
-        TablePropertiesCollection ptc;
-        reinterpret_cast<DB*>(dbfull())->GetPropertiesOfAllTables(&ptc);
-        ASSERT_EQ(1U, ptc.size());
-        auto row = ptc.begin();
-        auto tp = row->second;
-        ASSERT_EQ(total_order ? "4" : "12", (tp->user_collected_properties).at(
-                                                "plain_table_hash_table_size"));
-        ASSERT_EQ(total_order ? "9" : "0", (tp->user_collected_properties).at(
-                                               "plain_table_sub_index_size"));
-
-        ASSERT_EQ("v3", Get("1000000000000foo"));
-        ASSERT_EQ("v2", Get("0000000000000bar"));
      }
    }
-    }
  }
 }

@@ -307,7 +385,15 @@ TEST(PlainTableDBTest, Flush2) {
    for (EncodingType encoding_type : {kPlain, kPrefix}) {
    for (int bloom_bits = 0; bloom_bits <= 117; bloom_bits += 117) {
      for (int total_order = 0; total_order <= 1; total_order++) {
-        if (encoding_type == kPrefix && total_order == 1) {
+        for (int store_index_in_file = 0; store_index_in_file <= 1;
+             ++store_index_in_file) {
+          if (encoding_type == kPrefix && total_order) {
+            continue;
+          }
+          if (!bloom_bits && store_index_in_file) {
+            continue;
+          }
+          if (total_order && store_index_in_file) {
          continue;
        }
        bool expect_bloom_not_match = false;
@@ -315,16 +401,23 @@ TEST(PlainTableDBTest, Flush2) {
        options.create_if_missing = true;
        // Set only one bucket to force bucket conflict.
        // Test index interval for the same prefix to be 1, 2 and 4
+        PlainTableOptions plain_table_options;
        if (total_order) {
          options.prefix_extractor = nullptr;
-          options.table_factory.reset(new TestPlainTableFactory(
-              &expect_bloom_not_match, 0, bloom_bits, 0, 2, huge_page_tlb_size,
-              encoding_type));
+          plain_table_options.hash_table_ratio = 0;
+          plain_table_options.index_sparseness = 2;
        } else {
-          options.table_factory.reset(new TestPlainTableFactory(
-              &expect_bloom_not_match, 0, bloom_bits, 0.75, 16,
-              huge_page_tlb_size, encoding_type));
+          plain_table_options.hash_table_ratio = 0.75;
+          plain_table_options.index_sparseness = 16;
        }
+        plain_table_options.user_key_len = kPlainTableVariableLength;
+        plain_table_options.bloom_bits_per_key = bloom_bits;
+        plain_table_options.huge_page_tlb_size = huge_page_tlb_size;
+        plain_table_options.encoding_type = encoding_type;
+        plain_table_options.store_index_in_file = store_index_in_file;
+        options.table_factory.reset(new TestPlainTableFactory(
+            &expect_bloom_not_match, plain_table_options));
+
        DestroyAndReopen(&options);
        ASSERT_OK(Put("0000000000000bar", "b"));
        ASSERT_OK(Put("1000000000000foo", "v1"));
@@ -352,7 +445,6 @@ TEST(PlainTableDBTest, Flush2) {
          // Neither key nor value should exist.
          expect_bloom_not_match = true;
          ASSERT_EQ("NOT_FOUND", Get("5_not00000000bar"));
-
          // Key doesn't exist any more but prefix exists.
          if (total_order) {
            ASSERT_EQ("NOT_FOUND", Get("1000000000000not"));
@@ -361,6 +453,7 @@ TEST(PlainTableDBTest, Flush2) {
          expect_bloom_not_match = false;
        }
      }
+      }
    }
    }
  }
@@ -382,13 +475,28 @@ TEST(PlainTableDBTest, Iterator) {
        // Test index interval for the same prefix to be 1, 2 and 4
        if (total_order) {
          options.prefix_extractor = nullptr;
+
+          PlainTableOptions plain_table_options;
+          plain_table_options.user_key_len = 16;
+          plain_table_options.bloom_bits_per_key = bloom_bits;
+          plain_table_options.hash_table_ratio = 0;
+          plain_table_options.index_sparseness = 2;
+          plain_table_options.huge_page_tlb_size = huge_page_tlb_size;
+          plain_table_options.encoding_type = encoding_type;
+
          options.table_factory.reset(new TestPlainTableFactory(
-              &expect_bloom_not_match, 16, bloom_bits, 0, 2, huge_page_tlb_size,
-              encoding_type));
+              &expect_bloom_not_match, plain_table_options));
        } else {
+          PlainTableOptions plain_table_options;
+          plain_table_options.user_key_len = 16;
+          plain_table_options.bloom_bits_per_key = bloom_bits;
+          plain_table_options.hash_table_ratio = 0.75;
+          plain_table_options.index_sparseness = 16;
+          plain_table_options.huge_page_tlb_size = huge_page_tlb_size;
+          plain_table_options.encoding_type = encoding_type;
+
          options.table_factory.reset(new TestPlainTableFactory(
-              &expect_bloom_not_match, 16, bloom_bits, 0.75, 16,
-              huge_page_tlb_size, encoding_type));
+              &expect_bloom_not_match, plain_table_options));
        }
        DestroyAndReopen(&options);

@@ -487,7 +595,13 @@ std::string MakeLongKey(size_t length, char c) {

 TEST(PlainTableDBTest, IteratorLargeKeys) {
  Options options = CurrentOptions();
-  options.table_factory.reset(NewTotalOrderPlainTableFactory(0, 0, 16, 0));
+
+  PlainTableOptions plain_table_options;
+  plain_table_options.user_key_len = 0;
+  plain_table_options.bloom_bits_per_key = 0;
+  plain_table_options.hash_table_ratio = 0;
+
+  options.table_factory.reset(NewPlainTableFactory(plain_table_options));
  options.create_if_missing = true;
  options.prefix_extractor.reset();
  DestroyAndReopen(&options);
@@ -531,7 +645,16 @@ std::string MakeLongKeyWithPrefix(size_t length, char c) {

 TEST(PlainTableDBTest, IteratorLargeKeysWithPrefix) {
  Options options = CurrentOptions();
-  options.table_factory.reset(NewPlainTableFactory(16, 0, 0.8, 3, 0, kPrefix));
+
+  PlainTableOptions plain_table_options;
+  plain_table_options.user_key_len = 16;
+  plain_table_options.bloom_bits_per_key = 0;
+  plain_table_options.hash_table_ratio = 0.8;
+  plain_table_options.index_sparseness = 3;
+  plain_table_options.huge_page_tlb_size = 0;
+  plain_table_options.encoding_type = kPrefix;
+
+  options.table_factory.reset(NewPlainTableFactory(plain_table_options));
  options.create_if_missing = true;
  DestroyAndReopen(&options);

@@ -667,8 +790,16 @@ TEST(PlainTableDBTest, HashBucketConflict) {
      options.create_if_missing = true;
      // Set only one bucket to force bucket conflict.
      // Test index interval for the same prefix to be 1, 2 and 4
-      options.table_factory.reset(
-          NewTotalOrderPlainTableFactory(16, 0, 2 ^ i, huge_page_tlb_size));
+
+      PlainTableOptions plain_table_options;
+      plain_table_options.user_key_len = 16;
+      plain_table_options.bloom_bits_per_key = 0;
+      plain_table_options.hash_table_ratio = 0;
+      plain_table_options.index_sparseness = 2 ^ i;
+      plain_table_options.huge_page_tlb_size = huge_page_tlb_size;
+
+      options.table_factory.reset(NewPlainTableFactory(plain_table_options));
+
      DestroyAndReopen(&options);
      ASSERT_OK(Put("5000000000000fo0", "v1"));
      ASSERT_OK(Put("5000000000000fo1", "v2"));
@@ -754,8 +885,15 @@ TEST(PlainTableDBTest, HashBucketConflictReverseSuffixComparator) {
      options.comparator = &comp;
      // Set only one bucket to force bucket conflict.
      // Test index interval for the same prefix to be 1, 2 and 4
-      options.table_factory.reset(
-          NewTotalOrderPlainTableFactory(16, 0, 2 ^ i, huge_page_tlb_size));
+
+      PlainTableOptions plain_table_options;
+      plain_table_options.user_key_len = 16;
+      plain_table_options.bloom_bits_per_key = 0;
+      plain_table_options.hash_table_ratio = 0;
+      plain_table_options.index_sparseness = 2 ^ i;
+      plain_table_options.huge_page_tlb_size = huge_page_tlb_size;
+
+      options.table_factory.reset(NewPlainTableFactory(plain_table_options));
      DestroyAndReopen(&options);
      ASSERT_OK(Put("5000000000000fo0", "v1"));
      ASSERT_OK(Put("5000000000000fo1", "v2"));
@@ -835,7 +973,13 @@ TEST(PlainTableDBTest, NonExistingKeyToNonEmptyBucket) {
  options.create_if_missing = true;
  // Set only one bucket to force bucket conflict.
  // Test index interval for the same prefix to be 1, 2 and 4
-  options.table_factory.reset(NewTotalOrderPlainTableFactory(16, 0, 5));
+  PlainTableOptions plain_table_options;
+  plain_table_options.user_key_len = 16;
+  plain_table_options.bloom_bits_per_key = 0;
+  plain_table_options.hash_table_ratio = 0;
+  plain_table_options.index_sparseness = 5;
+
+  options.table_factory.reset(NewPlainTableFactory(plain_table_options));
  DestroyAndReopen(&options);
  ASSERT_OK(Put("5000000000000fo0", "v1"));
  ASSERT_OK(Put("5000000000000fo1", "v2"));
@@ -189,6 +189,10 @@ class PrefixTest {
          options.memtable_factory.reset(
              NewHashLinkListRepFactory(bucket_count, 2 * 1024 * 1024));
          return true;
+        case kHashLinkListTriggerSkipList:
+          options.memtable_factory.reset(
+              NewHashLinkListRepFactory(bucket_count, 0, 3));
+          return true;
        default:
          return false;
      }
@@ -208,6 +212,7 @@ class PrefixTest {
    kHashSkipList,
    kHashLinkList,
    kHashLinkListHugePageTlb,
+    kHashLinkListTriggerSkipList,
    kEnd
  };
  int option_config_;
@@ -31,6 +31,8 @@

 #ifndef ROCKSDB_LITE

+#define __STDC_FORMAT_MACROS
+#include <inttypes.h>
 #include "db/builder.h"
 #include "db/db_impl.h"
 #include "db/dbformat.h"
@@ -63,8 +65,8 @@ class Repairer {
            NewLRUCache(10, options_.table_cache_numshardbits,
                        options_.table_cache_remove_scan_count_limit)),
        next_file_number_(1) {
-    table_cache_ = new TableCache(dbname_, &options_, storage_options_,
-                                  raw_table_cache_.get());
+    table_cache_ =
+        new TableCache(&options_, storage_options_, raw_table_cache_.get());
    edit_ = new VersionEdit();
  }

@@ -82,18 +84,17 @@ class Repairer {
      status = WriteDescriptor();
    }
    if (status.ok()) {
-      unsigned long long bytes = 0;
+      uint64_t bytes = 0;
      for (size_t i = 0; i < tables_.size(); i++) {
        bytes += tables_[i].meta.fd.GetFileSize();
      }
      Log(options_.info_log,
          "**** Repaired rocksdb %s; "
-          "recovered %d files; %llu bytes. "
+          "recovered %zu files; %" PRIu64
+          "bytes. "
          "Some data may have been lost. "
          "****",
-          dbname_.c_str(),
-          static_cast<int>(tables_.size()),
-          bytes);
+          dbname_.c_str(), tables_.size(), bytes);
    }
    return status;
  }
@@ -115,7 +116,7 @@ class Repairer {
  VersionEdit* edit_;

  std::vector<std::string> manifests_;
-  std::vector<uint64_t> table_numbers_;
+  std::vector<FileDescriptor> table_fds_;
  std::vector<uint64_t> logs_;
  std::vector<TableInfo> tables_;
  uint64_t next_file_number_;
@@ -123,35 +124,44 @@ class Repairer {

  Status FindFiles() {
    std::vector<std::string> filenames;
-    Status status = env_->GetChildren(dbname_, &filenames);
-    if (!status.ok()) {
-      return status;
-    }
-    if (filenames.empty()) {
-      return Status::Corruption(dbname_, "repair found no files");
-    }
+    bool found_file = false;
+    for (uint32_t path_id = 0; path_id < options_.db_paths.size(); path_id++) {
+      Status status =
+          env_->GetChildren(options_.db_paths[path_id].path, &filenames);
+      if (!status.ok()) {
+        return status;
+      }
+      if (!filenames.empty()) {
+        found_file = true;
+      }

-    uint64_t number;
-    FileType type;
-    for (size_t i = 0; i < filenames.size(); i++) {
-      if (ParseFileName(filenames[i], &number, &type)) {
-        if (type == kDescriptorFile) {
-          manifests_.push_back(filenames[i]);
-        } else {
-          if (number + 1 > next_file_number_) {
-            next_file_number_ = number + 1;
-          }
-          if (type == kLogFile) {
-            logs_.push_back(number);
-          } else if (type == kTableFile) {
-            table_numbers_.push_back(number);
+      uint64_t number;
+      FileType type;
+      for (size_t i = 0; i < filenames.size(); i++) {
+        if (ParseFileName(filenames[i], &number, &type)) {
+          if (type == kDescriptorFile) {
+            assert(path_id == 0);
+            manifests_.push_back(filenames[i]);
          } else {
-            // Ignore other files
+            if (number + 1 > next_file_number_) {
+              next_file_number_ = number + 1;
+            }
+            if (type == kLogFile) {
+              assert(path_id == 0);
+              logs_.push_back(number);
+            } else if (type == kTableFile) {
+              table_fds_.emplace_back(number, path_id, 0);
+            } else {
+              // Ignore other files
+            }
          }
        }
      }
    }
-    return status;
+    if (!found_file) {
+      return Status::Corruption(dbname_, "repair found no files");
+    }
+    return Status::OK();
  }

  void ConvertLogFilesToTables() {
@@ -159,8 +169,8 @@ class Repairer {
      std::string logname = LogFileName(dbname_, logs_[i]);
      Status status = ConvertLogToTable(logs_[i]);
      if (!status.ok()) {
-        Log(options_.info_log, "Log #%llu: ignoring conversion error: %s",
-            (unsigned long long) logs_[i],
+        Log(options_.info_log,
+            "Log #%" PRIu64 ": ignoring conversion error: %s", logs_[i],
            status.ToString().c_str());
      }
      ArchiveFile(logname);
@@ -174,10 +184,8 @@ class Repairer {
      uint64_t lognum;
      virtual void Corruption(size_t bytes, const Status& s) {
        // We print error messages for corruption, but continue repairing.
-        Log(info_log, "Log #%llu: dropping %d bytes; %s",
-            (unsigned long long) lognum,
-            static_cast<int>(bytes),
-            s.ToString().c_str());
+        Log(info_log, "Log #%" PRIu64 ": dropping %d bytes; %s", lognum,
+            static_cast<int>(bytes), s.ToString().c_str());
      }
    };

@@ -220,8 +228,7 @@ class Repairer {
      if (status.ok()) {
        counter += WriteBatchInternal::Count(&batch);
      } else {
-        Log(options_.info_log, "Log #%llu: ignoring %s",
-            (unsigned long long) log,
+        Log(options_.info_log, "Log #%" PRIu64 ": ignoring %s", log,
            status.ToString().c_str());
        status = Status::OK();  // Keep going with rest of file
      }
@@ -230,7 +237,7 @@ class Repairer {
    // Do not record a version edit for this conversion to a Table
    // since ExtractMetaData() will also generate edits.
    FileMetaData meta;
-    meta.fd.number = next_file_number_++;
+    meta.fd = FileDescriptor(next_file_number_++, 0, 0);
    ReadOptions ro;
    Iterator* iter = mem->NewIterator(ro, true /* enforce_total_order */);
    status = BuildTable(dbname_, env_, options_, storage_options_, table_cache_,
@@ -241,24 +248,26 @@ class Repairer {
    mem = nullptr;
    if (status.ok()) {
      if (meta.fd.GetFileSize() > 0) {
-        table_numbers_.push_back(meta.fd.GetNumber());
+        table_fds_.push_back(meta.fd);
      }
    }
-    Log(options_.info_log, "Log #%llu: %d ops saved to Table #%llu %s",
-        (unsigned long long)log, counter,
-        (unsigned long long)meta.fd.GetNumber(), status.ToString().c_str());
+    Log(options_.info_log,
+        "Log #%" PRIu64 ": %d ops saved to Table #%" PRIu64 " %s", log, counter,
+        meta.fd.GetNumber(), status.ToString().c_str());
    return status;
  }

  void ExtractMetaData() {
-    for (size_t i = 0; i < table_numbers_.size(); i++) {
+    for (size_t i = 0; i < table_fds_.size(); i++) {
      TableInfo t;
-      t.meta.fd.number = table_numbers_[i];
+      t.meta.fd = table_fds_[i];
      Status status = ScanTable(&t);
      if (!status.ok()) {
-        std::string fname = TableFileName(dbname_, table_numbers_[i]);
-        Log(options_.info_log, "Table #%llu: ignoring %s",
-            (unsigned long long) table_numbers_[i],
+        std::string fname = TableFileName(
+            options_.db_paths, t.meta.fd.GetNumber(), t.meta.fd.GetPathId());
+        Log(options_.info_log, "Table #%s: ignoring %s",
+            FormatFileNumber(t.meta.fd.GetNumber(), t.meta.fd.GetPathId())
+                .c_str(),
            status.ToString().c_str());
        ArchiveFile(fname);
      } else {
@@ -268,9 +277,13 @@ class Repairer {
  }

  Status ScanTable(TableInfo* t) {
-    std::string fname = TableFileName(dbname_, t->meta.fd.GetNumber());
+    std::string fname = TableFileName(options_.db_paths, t->meta.fd.GetNumber(),
+                                      t->meta.fd.GetPathId());
    int counter = 0;
-    Status status = env_->GetFileSize(fname, &t->meta.fd.file_size);
+    uint64_t file_size;
+    Status status = env_->GetFileSize(fname, &file_size);
+    t->meta.fd = FileDescriptor(t->meta.fd.GetNumber(), t->meta.fd.GetPathId(),
+                                file_size);
    if (status.ok()) {
      Iterator* iter = table_cache_->NewIterator(
          ReadOptions(), storage_options_, icmp_, t->meta.fd);
@@ -281,9 +294,8 @@ class Repairer {
      for (iter->SeekToFirst(); iter->Valid(); iter->Next()) {
        Slice key = iter->key();
        if (!ParseInternalKey(key, &parsed)) {
-          Log(options_.info_log, "Table #%llu: unparsable key %s",
-              (unsigned long long)t->meta.fd.GetNumber(),
-              EscapeString(key).c_str());
+          Log(options_.info_log, "Table #%" PRIu64 ": unparsable key %s",
+              t->meta.fd.GetNumber(), EscapeString(key).c_str());
          continue;
        }

@@ -305,9 +317,8 @@ class Repairer {
      }
      delete iter;
    }
-    Log(options_.info_log, "Table #%llu: %d entries %s",
-        (unsigned long long)t->meta.fd.GetNumber(), counter,
-        status.ToString().c_str());
+    Log(options_.info_log, "Table #%" PRIu64 ": %d entries %s",
+        t->meta.fd.GetNumber(), counter, status.ToString().c_str());
    return status;
  }

@@ -335,9 +346,9 @@ class Repairer {
    for (size_t i = 0; i < tables_.size(); i++) {
      // TODO(opt): separate out into multiple levels
      const TableInfo& t = tables_[i];
-      edit_->AddFile(0, t.meta.fd.GetNumber(), t.meta.fd.GetFileSize(),
-                     t.meta.smallest, t.meta.largest, t.min_sequence,
-                     t.max_sequence);
+      edit_->AddFile(0, t.meta.fd.GetNumber(), t.meta.fd.GetPathId(),
+                     t.meta.fd.GetFileSize(), t.meta.smallest, t.meta.largest,
+                     t.min_sequence, t.max_sequence);
    }

    //fprintf(stderr, "NewDescriptor:\n%s\n", edit_.DebugString().c_str());
@@ -87,7 +87,7 @@ public:

  Status Get(const ReadOptions&, const Slice& key, void* arg,
             bool (*handle_result)(void* arg, const ParsedInternalKey& k,
-                                   const Slice& v, bool),
+                                   const Slice& v),
             void (*mark_key_may_exist)(void*) = nullptr) override;

  uint64_t ApproximateOffsetOf(const Slice& key) override;
@@ -285,7 +285,7 @@ Status SimpleTableReader::GetOffset(const Slice& target, uint64_t* offset) {
 Status SimpleTableReader::Get(const ReadOptions& options, const Slice& k,
                              void* arg,
                              bool (*saver)(void*, const ParsedInternalKey&,
-                                            const Slice&, bool),
+                                            const Slice&),
                              void (*mark_key_may_exist)(void*)) {
  Status s;
  SimpleTableIterator* iter = new SimpleTableIterator(this);
@@ -295,7 +295,7 @@ Status SimpleTableReader::Get(const ReadOptions& options, const Slice& k,
      return Status::Corruption(Slice());
    }

-    if (!(*saver)(arg, parsed_key, iter->value(), true)) {
+    if (!(*saver)(arg, parsed_key, iter->value())) {
      break;
    }
  }
@@ -36,10 +36,10 @@ static Slice GetSliceForFileNumber(const uint64_t* file_number) {
               sizeof(*file_number));
 }

-TableCache::TableCache(const std::string& dbname, const Options* options,
+TableCache::TableCache(const Options* options,
                       const EnvOptions& storage_options, Cache* const cache)
    : env_(options->env),
-      dbname_(dbname),
+      db_paths_(options->db_paths),
      options_(options),
      storage_options_(storage_options),
      cache_(cache) {}
@@ -58,18 +58,17 @@ void TableCache::ReleaseHandle(Cache::Handle* handle) {
 Status TableCache::FindTable(const EnvOptions& toptions,
                             const InternalKeyComparator& internal_comparator,
                             const FileDescriptor& fd, Cache::Handle** handle,
-                             bool* table_io, const bool no_io) {
+                             const bool no_io) {
  Status s;
-  Slice key = GetSliceForFileNumber(&fd.number);
+  uint64_t number = fd.GetNumber();
+  Slice key = GetSliceForFileNumber(&number);
  *handle = cache_->Lookup(key);
  if (*handle == nullptr) {
    if (no_io) { // Dont do IO and return a not-found status
      return Status::Incomplete("Table not found in table_cache, no_io is set");
    }
-    if (table_io != nullptr) {
-      *table_io = true;    // we had to do IO from storage
-    }
-    std::string fname = TableFileName(dbname_, fd.GetNumber());
+    std::string fname =
+        TableFileName(db_paths_, fd.GetNumber(), fd.GetPathId());
    unique_ptr<RandomAccessFile> file;
    unique_ptr<TableReader> table_reader;
    s = env_->NewRandomAccessFile(fname, &file, toptions);
@@ -110,7 +109,7 @@ Iterator* TableCache::NewIterator(const ReadOptions& options,
  Cache::Handle* handle = nullptr;
  Status s;
  if (table_reader == nullptr) {
-    s = FindTable(toptions, icomparator, fd, &handle, nullptr,
+    s = FindTable(toptions, icomparator, fd, &handle,
                  options.read_tier == kBlockCacheTier);
    if (!s.ok()) {
      return NewErrorIterator(s, arena);
@@ -137,13 +136,13 @@ Status TableCache::Get(const ReadOptions& options,
                       const InternalKeyComparator& internal_comparator,
                       const FileDescriptor& fd, const Slice& k, void* arg,
                       bool (*saver)(void*, const ParsedInternalKey&,
-                                     const Slice&, bool),
-                       bool* table_io, void (*mark_key_may_exist)(void*)) {
+                                     const Slice&),
+                       void (*mark_key_may_exist)(void*)) {
  TableReader* t = fd.table_reader;
  Status s;
  Cache::Handle* handle = nullptr;
  if (!t) {
-    s = FindTable(storage_options_, internal_comparator, fd, &handle, table_io,
+    s = FindTable(storage_options_, internal_comparator, fd, &handle,
                  options.read_tier == kBlockCacheTier);
    if (s.ok()) {
      t = GetTableReaderFromHandle(handle);
@@ -174,10 +173,8 @@ Status TableCache::GetTableProperties(
    return s;
  }

-  bool table_io;
  Cache::Handle* table_handle = nullptr;
-  s = FindTable(toptions, internal_comparator, fd, &table_handle, &table_io,
-                no_io);
+  s = FindTable(toptions, internal_comparator, fd, &table_handle, no_io);
  if (!s.ok()) {
    return s;
  }
@@ -11,6 +11,7 @@

 #pragma once
 #include <string>
+#include <vector>
 #include <stdint.h>

 #include "db/dbformat.h"
@@ -28,8 +29,8 @@ struct FileDescriptor;

 class TableCache {
 public:
-  TableCache(const std::string& dbname, const Options* options,
-             const EnvOptions& storage_options, Cache* cache);
+  TableCache(const Options* options, const EnvOptions& storage_options,
+             Cache* cache);
  ~TableCache();

  // Return an iterator for the specified file number (the corresponding
@@ -52,8 +53,8 @@ class TableCache {
             const InternalKeyComparator& internal_comparator,
             const FileDescriptor& file_fd, const Slice& k, void* arg,
             bool (*handle_result)(void*, const ParsedInternalKey&,
-                                   const Slice&, bool),
-             bool* table_io, void (*mark_key_may_exist)(void*) = nullptr);
+                                   const Slice&),
+             void (*mark_key_may_exist)(void*) = nullptr);

  // Evict any entry for the specified file number
  static void Evict(Cache* cache, uint64_t file_number);
@@ -62,7 +63,7 @@ class TableCache {
  Status FindTable(const EnvOptions& toptions,
                   const InternalKeyComparator& internal_comparator,
                   const FileDescriptor& file_fd, Cache::Handle**,
-                   bool* table_io = nullptr, const bool no_io = false);
+                   const bool no_io = false);

  // Get TableReader from a cache handle.
  TableReader* GetTableReaderFromHandle(Cache::Handle* handle);
@@ -84,7 +85,7 @@ class TableCache {

 private:
  Env* const env_;
-  const std::string dbname_;
+  const std::vector<DbPath> db_paths_;
  const Options* options_;
  const EnvOptions& storage_options_;
  Cache* const cache_;
@@ -213,7 +213,14 @@ TEST(TablePropertiesTest, CustomizedTablePropertiesCollector) {
  Options options;
  options.table_properties_collector_factories.emplace_back(
      new RegularKeysStartWithAFactory());
-  options.table_factory = std::make_shared<PlainTableFactory>(8, 8, 0);
+
+  PlainTableOptions plain_table_options;
+  plain_table_options.user_key_len = 8;
+  plain_table_options.bloom_bits_per_key = 8;
+  plain_table_options.hash_table_ratio = 0;
+
+  options.table_factory =
+      std::make_shared<PlainTableFactory>(plain_table_options);
  test::PlainInternalKeyComparator ikc(options.comparator);
  TestCustomizedTablePropertiesCollector(kPlainTableMagicNumber, true, options,
                                         ikc);
@@ -299,11 +306,15 @@ TEST(TablePropertiesTest, InternalKeyPropertiesCollector) {
      true /* not sanitize */,
      std::make_shared<BlockBasedTableFactory>()
  );
+
+  PlainTableOptions plain_table_options;
+  plain_table_options.user_key_len = 8;
+  plain_table_options.bloom_bits_per_key = 8;
+  plain_table_options.hash_table_ratio = 0;
+
  TestInternalKeyPropertiesCollector(
-      kPlainTableMagicNumber,
-      false /* not sanitize */,
-      std::make_shared<PlainTableFactory>(8, 8, 0)
-  );
+      kPlainTableMagicNumber, false /* not sanitize */,
+      std::make_shared<PlainTableFactory>(plain_table_options));
 }

 }  // namespace rocksdb
@@ -18,25 +18,30 @@ namespace rocksdb {
 // Tag numbers for serialized VersionEdit.  These numbers are written to
 // disk and should not be changed.
 enum Tag {
-  kComparator           = 1,
-  kLogNumber            = 2,
-  kNextFileNumber       = 3,
-  kLastSequence         = 4,
-  kCompactPointer       = 5,
-  kDeletedFile          = 6,
-  kNewFile              = 7,
+  kComparator = 1,
+  kLogNumber = 2,
+  kNextFileNumber = 3,
+  kLastSequence = 4,
+  kCompactPointer = 5,
+  kDeletedFile = 6,
+  kNewFile = 7,
  // 8 was used for large value refs
-  kPrevLogNumber        = 9,
+  kPrevLogNumber = 9,

  // these are new formats divergent from open source leveldb
-  kNewFile2             = 100,  // store smallest & largest seqno
-
-  kColumnFamily         = 200,  // specify column family for version edit
-  kColumnFamilyAdd      = 201,
-  kColumnFamilyDrop     = 202,
-  kMaxColumnFamily      = 203,
+  kNewFile2 = 100,
+  kNewFile3 = 102,
+  kColumnFamily = 200,  // specify column family for version edit
+  kColumnFamilyAdd = 201,
+  kColumnFamilyDrop = 202,
+  kMaxColumnFamily = 203,
 };

+uint64_t PackFileNumberAndPathId(uint64_t number, uint64_t path_id) {
+  assert(number <= kFileNumberMask);
+  return number | (path_id * (kFileNumberMask + 1));
+}
+
 void VersionEdit::Clear() {
  comparator_.clear();
  max_level_ = 0;
@@ -93,9 +98,18 @@ void VersionEdit::EncodeTo(std::string* dst) const {

  for (size_t i = 0; i < new_files_.size(); i++) {
    const FileMetaData& f = new_files_[i].second;
-    PutVarint32(dst, kNewFile2);
+    if (f.fd.GetPathId() == 0) {
+      // Use older format to make sure user can roll back the build if they
+      // don't config multiple DB paths.
+      PutVarint32(dst, kNewFile2);
+    } else {
+      PutVarint32(dst, kNewFile3);
+    }
    PutVarint32(dst, new_files_[i].first);  // level
    PutVarint64(dst, f.fd.GetNumber());
+    if (f.fd.GetPathId() != 0) {
+      PutVarint32(dst, f.fd.GetPathId());
+    }
    PutVarint64(dst, f.fd.GetFileSize());
    PutLengthPrefixedSlice(dst, f.smallest.Encode());
    PutLengthPrefixedSlice(dst, f.largest.Encode());
@@ -237,7 +251,7 @@ Status VersionEdit::DecodeFrom(const Slice& src) {
            GetVarint64(&input, &file_size) &&
            GetInternalKey(&input, &f.smallest) &&
            GetInternalKey(&input, &f.largest)) {
-          f.fd = FileDescriptor(number, file_size);
+          f.fd = FileDescriptor(number, 0, file_size);
          new_files_.push_back(std::make_pair(level, f));
        } else {
          if (!msg) {
@@ -255,7 +269,27 @@ Status VersionEdit::DecodeFrom(const Slice& src) {
            GetInternalKey(&input, &f.largest) &&
            GetVarint64(&input, &f.smallest_seqno) &&
            GetVarint64(&input, &f.largest_seqno)) {
-          f.fd = FileDescriptor(number, file_size);
+          f.fd = FileDescriptor(number, 0, file_size);
+          new_files_.push_back(std::make_pair(level, f));
+        } else {
+          if (!msg) {
+            msg = "new-file2 entry";
+          }
+        }
+        break;
+      }
+
+      case kNewFile3: {
+        uint64_t number;
+        uint32_t path_id;
+        uint64_t file_size;
+        if (GetLevel(&input, &level, &msg) && GetVarint64(&input, &number) &&
+            GetVarint32(&input, &path_id) && GetVarint64(&input, &file_size) &&
+            GetInternalKey(&input, &f.smallest) &&
+            GetInternalKey(&input, &f.largest) &&
+            GetVarint64(&input, &f.smallest_seqno) &&
+            GetVarint64(&input, &f.largest_seqno)) {
+          f.fd = FileDescriptor(number, path_id, file_size);
          new_files_.push_back(std::make_pair(level, f));
        } else {
          if (!msg) {
@@ -14,48 +14,115 @@
 #include <string>
 #include "rocksdb/cache.h"
 #include "db/dbformat.h"
+#include "util/arena.h"
+#include "util/autovector.h"

 namespace rocksdb {

 class VersionSet;

+const uint64_t kFileNumberMask = 0x3FFFFFFFFFFFFFFF;
+
+extern uint64_t PackFileNumberAndPathId(uint64_t number, uint64_t path_id);
+
 // A copyable structure contains information needed to read data from an SST
 // file. It can contains a pointer to a table reader opened for the file, or
 // file number and size, which can be used to create a new table reader for it.
 // The behavior is undefined when a copied of the structure is used when the
 // file is not in any live version any more.
 struct FileDescriptor {
-  uint64_t number;
-  uint64_t file_size;  // File size in bytes
  // Table reader in table_reader_handle
  TableReader* table_reader;
+  uint64_t packed_number_and_path_id;
+  uint64_t file_size;  // File size in bytes

-  FileDescriptor(uint64_t number, uint64_t file_size)
-      : number(number), file_size(file_size), table_reader(nullptr) {}
+  FileDescriptor() : FileDescriptor(0, 0, 0) {}

-  uint64_t GetNumber() const { return number; }
+  FileDescriptor(uint64_t number, uint32_t path_id, uint64_t file_size)
+      : table_reader(nullptr),
+        packed_number_and_path_id(PackFileNumberAndPathId(number, path_id)),
+        file_size(file_size) {}
+
+  FileDescriptor& operator=(const FileDescriptor& fd) {
+    table_reader = fd.table_reader;
+    packed_number_and_path_id = fd.packed_number_and_path_id;
+    file_size = fd.file_size;
+    return *this;
+  }
+
+  uint64_t GetNumber() const {
+    return packed_number_and_path_id & kFileNumberMask;
+  }
+  uint32_t GetPathId() const {
+    return packed_number_and_path_id / (kFileNumberMask + 1);
+  }
  uint64_t GetFileSize() const { return file_size; }
 };

 struct FileMetaData {
  int refs;
  FileDescriptor fd;
-  int allowed_seeks;          // Seeks allowed until compaction
-  InternalKey smallest;       // Smallest internal key served by table
-  InternalKey largest;        // Largest internal key served by table
-  bool being_compacted;       // Is this file undergoing compaction?
-  SequenceNumber smallest_seqno;// The smallest seqno in this file
-  SequenceNumber largest_seqno; // The largest seqno in this file
+  InternalKey smallest;            // Smallest internal key served by table
+  InternalKey largest;             // Largest internal key served by table
+  bool being_compacted;            // Is this file undergoing compaction?
+  SequenceNumber smallest_seqno;   // The smallest seqno in this file
+  SequenceNumber largest_seqno;    // The largest seqno in this file

  // Needs to be disposed when refs becomes 0.
  Cache::Handle* table_reader_handle;

+  // Stats for compensating deletion entries during compaction
+
+  // File size compensated by deletion entry.
+  // This is updated in Version::UpdateTemporaryStats() first time when the
+  // file is created or loaded.  After it is updated, it is immutable.
+  uint64_t compensated_file_size;
+  uint64_t num_entries;            // the number of entries.
+  uint64_t num_deletions;          // the number of deletion entries.
+  uint64_t raw_key_size;           // total uncompressed key size.
+  uint64_t raw_value_size;         // total uncompressed value size.
+
  FileMetaData()
      : refs(0),
-        fd(0, 0),
-        allowed_seeks(1 << 30),
        being_compacted(false),
-        table_reader_handle(nullptr) {}
+        table_reader_handle(nullptr),
+        compensated_file_size(0),
+        num_entries(0),
+        num_deletions(0),
+        raw_key_size(0),
+        raw_value_size(0) {}
+};
+
+// A compressed copy of file meta data that just contain
+// smallest and largest key's slice
+struct FdWithKeyRange {
+  FileDescriptor fd;
+  Slice smallest_key;    // slice that contain smallest key
+  Slice largest_key;     // slice that contain largest key
+
+  FdWithKeyRange()
+      : fd(),
+        smallest_key(),
+        largest_key() {
+  }
+
+  FdWithKeyRange(FileDescriptor fd,
+      Slice smallest_key, Slice largest_key)
+      : fd(fd),
+        smallest_key(smallest_key),
+        largest_key(largest_key) {
+  }
+};
+
+// Data structure to store an array of FdWithKeyRange in one level
+// Actual data is guaranteed to be stored closely
+struct FileLevel {
+  size_t num_files;
+  FdWithKeyRange* files;
+  FileLevel() {
+    num_files = 0;
+    files = nullptr;
+  }
 };

 class VersionEdit {
@@ -93,15 +160,13 @@ class VersionEdit {
  // Add the specified file at the specified number.
  // REQUIRES: This version has not been saved (see VersionSet::SaveTo)
  // REQUIRES: "smallest" and "largest" are smallest and largest keys in file
-  void AddFile(int level, uint64_t file,
-               uint64_t file_size,
-               const InternalKey& smallest,
-               const InternalKey& largest,
-               const SequenceNumber& smallest_seqno,
+  void AddFile(int level, uint64_t file, uint64_t file_size,
+               uint64_t file_path_id, const InternalKey& smallest,
+               const InternalKey& largest, const SequenceNumber& smallest_seqno,
               const SequenceNumber& largest_seqno) {
    assert(smallest_seqno <= largest_seqno);
    FileMetaData f;
-    f.fd = FileDescriptor(file, file_size);
+    f.fd = FileDescriptor(file, file_size, file_path_id);
    f.smallest = smallest;
    f.largest = largest;
    f.smallest_seqno = smallest_seqno;
@@ -151,6 +216,7 @@ class VersionEdit {

 private:
  friend class VersionSet;
+  friend class Version;

  typedef std::set< std::pair<int, uint64_t>> DeletedFileSet;

@@ -30,11 +30,10 @@ TEST(VersionEditTest, EncodeDecode) {
  VersionEdit edit;
  for (int i = 0; i < 4; i++) {
    TestEncodeDecode(edit);
-    edit.AddFile(3, kBig + 300 + i, kBig + 400 + i,
+    edit.AddFile(3, kBig + 300 + i, kBig + 400 + i, 0,
                 InternalKey("foo", kBig + 500 + i, kTypeValue),
                 InternalKey("zoo", kBig + 600 + i, kTypeDeletion),
-                 kBig + 500 + i,
-                 kBig + 600 + i);
+                 kBig + 500 + i, kBig + 600 + i);
    edit.DeleteFile(4, kBig + 700 + i);
  }

@@ -53,26 +53,34 @@ class ColumnFamilySet;
 class TableCache;
 class MergeIteratorBuilder;

-// Return the smallest index i such that files[i]->largest >= key.
-// Return files.size() if there is no such file.
-// REQUIRES: "files" contains a sorted list of non-overlapping files.
+// Return the smallest index i such that file_level.files[i]->largest >= key.
+// Return file_level.num_files if there is no such file.
+// REQUIRES: "file_level.files" contains a sorted list of
+// non-overlapping files.
 extern int FindFile(const InternalKeyComparator& icmp,
-                    const std::vector<FileMetaData*>& files,
+                    const FileLevel& file_level,
                    const Slice& key);

 // Returns true iff some file in "files" overlaps the user key range
 // [*smallest,*largest].
 // smallest==nullptr represents a key smaller than all keys in the DB.
 // largest==nullptr represents a key largest than all keys in the DB.
-// REQUIRES: If disjoint_sorted_files, files[] contains disjoint ranges
-//           in sorted order.
+// REQUIRES: If disjoint_sorted_files, file_level.files[]
+// contains disjoint ranges in sorted order.
 extern bool SomeFileOverlapsRange(
    const InternalKeyComparator& icmp,
    bool disjoint_sorted_files,
-    const std::vector<FileMetaData*>& files,
+    const FileLevel& file_level,
    const Slice* smallest_user_key,
    const Slice* largest_user_key);

+// Generate FileLevel from vector<FdWithKeyRange*>
+// Would copy smallest_key and largest_key data to sequential memory
+// arena: Arena used to allocate the memory
+extern void DoGenerateFileLevel(FileLevel* file_level,
+        const std::vector<FileMetaData*>& files,
+        Arena* arena);
+
 class Version {
 public:
  // Append to *iters a sequence of iterators that will
@@ -85,29 +93,22 @@ class Version {
                    MergeIteratorBuilder* merger_iter_builder);

  // Lookup the value for key.  If found, store it in *val and
-  // return OK.  Else return a non-OK status.  Fills *stats.
+  // return OK.  Else return a non-OK status.
  // Uses *operands to store merge_operator operations to apply later
  // REQUIRES: lock is not held
-  struct GetStats {
-    FileMetaData* seek_file;
-    int seek_file_level;
-  };
-
  void Get(const ReadOptions&, const LookupKey& key, std::string* val,
-           Status* status, MergeContext* merge_context, GetStats* stats,
+           Status* status, MergeContext* merge_context,
           bool* value_found = nullptr);

-  // Adds "stats" into the current state.  Returns true if a new
-  // compaction may need to be triggered, false otherwise.
-  // REQUIRES: lock is held
-  bool UpdateStats(const GetStats& stats);
-
  // Updates internal structures that keep track of compaction scores
  // We use compaction scores to figure out which compaction to do next
  // REQUIRES: If Version is not yet saved to current_, it can be called without
  // a lock. Once a version is saved to current_, call only with mutex held
  void ComputeCompactionScore(std::vector<uint64_t>& size_being_compacted);

+  // Generate file_levels_ from files_
+  void GenerateFileLevels();
+
  // Update scores, pre-calculated variables. It needs to be called before
  // applying the version to the version set.
  void PrepareApply(std::vector<uint64_t>& size_being_compacted);
@@ -198,7 +199,7 @@ class Version {
  int64_t MaxNextLevelOverlappingBytes();

  // Add all files listed in the current version to *live.
-  void AddLiveFiles(std::set<uint64_t>* live);
+  void AddLiveFiles(std::vector<FileDescriptor>* live);

  // Return a human readable string that describes this version's contents.
  std::string DebugString(bool hex = false) const;
@@ -206,12 +207,33 @@ class Version {
  // Returns the version nuber of this version
  uint64_t GetVersionNumber() const { return version_number_; }

+  uint64_t GetAverageValueSize() const {
+    if (num_non_deletions_ == 0) {
+      return 0;
+    }
+    assert(total_raw_key_size_ + total_raw_value_size_ > 0);
+    assert(total_file_size_ > 0);
+    return total_raw_value_size_ / num_non_deletions_ * total_file_size_ /
+           (total_raw_key_size_ + total_raw_value_size_);
+  }
+
+  // REQUIRES: lock is held
+  // On success, "tp" will contains the table properties of the file
+  // specified in "file_meta".  If the file name of "file_meta" is
+  // known ahread, passing it by a non-null "fname" can save a
+  // file-name conversion.
+  Status GetTableProperties(std::shared_ptr<const TableProperties>* tp,
+                            const FileMetaData* file_meta,
+                            const std::string* fname = nullptr);
+
  // REQUIRES: lock is held
  // On success, *props will be populated with all SSTables' table properties.
  // The keys of `props` are the sst file name, the values of `props` are the
  // tables' propertis, represented as shared_ptr.
  Status GetPropertiesOfAllTables(TablePropertiesCollection* props);

+  uint64_t GetEstimatedActiveKeys();
+
  // used to sort files by size
  struct Fsize {
    int index;
@@ -228,6 +250,7 @@ class Version {
  friend class UniversalCompactionPicker;
  friend class FIFOCompactionPicker;
  friend class ForwardIterator;
+  friend class InternalStats;

  class LevelFileNumIterator;
  class LevelFileIteratorState;
@@ -238,6 +261,15 @@ class Version {
  // Update num_non_empty_levels_.
  void UpdateNumNonEmptyLevels();

+  // The helper function of UpdateTemporaryStats, which may fill the missing
+  // fields of file_mata from its associated TableProperties.
+  // Returns true if it does initialize FileMetaData.
+  bool MaybeInitializeFileMetaData(FileMetaData* file_meta);
+
+  // Update the temporary stats associated with the current version.
+  // This temporary stats will be used in compaction.
+  void UpdateTemporaryStats();
+
  // Sort all files for this version based on their file size and
  // record results in files_by_size_. The largest files are listed first.
  void UpdateFilesBySize();
@@ -247,12 +279,16 @@ class Version {
  const Comparator* user_comparator_;
  TableCache* table_cache_;
  const MergeOperator* merge_operator_;
+
+  autovector<FileLevel> file_levels_;   // A copy of list of files per level
  Logger* info_log_;
  Statistics* db_statistics_;
  int num_levels_;              // Number of levels
  int num_non_empty_levels_;    // Number of levels. Any level larger than it
                                // is guaranteed to be empty.
+  FileIndexer file_indexer_;
  VersionSet* vset_;            // VersionSet to which this Version belongs
+  Arena arena_;                 // Used to allocate space for file_levels_
  Version* next_;               // Next version in linked list
  Version* prev_;               // Previous version in linked list
  int refs_;                    // Number of live refs to this version
@@ -276,11 +312,7 @@ class Version {
  // that on a running system, we need to look at only the first
  // few largest files because a new version is created every few
  // seconds/minutes (because of concurrent compactions).
-  static const int number_of_files_to_sort_ = 50;
-
-  // Next file to compact based on seek stats.
-  FileMetaData* file_to_compact_;
-  int file_to_compact_level_;
+  static const size_t number_of_files_to_sort_ = 50;

  // Level that should be compacted next and its compaction score.
  // Score < 1 means compaction is not strictly needed.  These fields
@@ -297,7 +329,17 @@ class Version {
  uint64_t version_number_;

  Version(ColumnFamilyData* cfd, VersionSet* vset, uint64_t version_number = 0);
-  FileIndexer file_indexer_;
+
+  // total file size
+  uint64_t total_file_size_;
+  // the total size of all raw keys.
+  uint64_t total_raw_key_size_;
+  // the total size of all raw values.
+  uint64_t total_raw_value_size_;
+  // total number of non-deletion entries
+  uint64_t num_non_deletions_;
+  // total number of deletion entries
+  uint64_t num_deletions_;

  ~Version();

@@ -375,7 +417,7 @@ class VersionSet {
  // Arrange to reuse "file_number" unless a newer file number has
  // already been allocated.
  // REQUIRES: "file_number" was returned by a call to NewFileNumber().
-  void ReuseFileNumber(uint64_t file_number) {
+  void ReuseLogFileNumber(uint64_t file_number) {
    if (next_file_number_ == file_number + 1) {
      next_file_number_ = file_number;
    }
@@ -416,7 +458,7 @@ class VersionSet {
  Iterator* MakeInputIterator(Compaction* c);

  // Add all files listed in any live version to *live.
-  void AddLiveFiles(std::vector<uint64_t>* live_list);
+  void AddLiveFiles(std::vector<FileDescriptor>* live_list);

  // Return the approximate offset in the database of the data for
  // "key" as of version "v".
@@ -14,14 +14,15 @@

 namespace rocksdb {

-class FindFileTest {
+class GenerateFileLevelTest {
 public:
  std::vector<FileMetaData*> files_;
-  bool disjoint_sorted_files_;
+  FileLevel file_level_;
+  Arena arena_;

-  FindFileTest() : disjoint_sorted_files_(true) { }
+  GenerateFileLevelTest() { }

-  ~FindFileTest() {
+  ~GenerateFileLevelTest() {
    for (unsigned int i = 0; i < files_.size(); i++) {
      delete files_[i];
    }
@@ -31,29 +32,108 @@ class FindFileTest {
           SequenceNumber smallest_seq = 100,
           SequenceNumber largest_seq = 100) {
    FileMetaData* f = new FileMetaData;
-    f->fd = FileDescriptor(files_.size() + 1, 0);
+    f->fd = FileDescriptor(files_.size() + 1, 0, 0);
    f->smallest = InternalKey(smallest, smallest_seq, kTypeValue);
    f->largest = InternalKey(largest, largest_seq, kTypeValue);
    files_.push_back(f);
  }

+  int Compare() {
+    int diff = 0;
+    for (size_t i = 0; i < files_.size(); i++) {
+      if (file_level_.files[i].fd.GetNumber() != files_[i]->fd.GetNumber()) {
+        diff++;
+      }
+    }
+    return diff;
+  }
+};
+
+TEST(GenerateFileLevelTest, Empty) {
+  DoGenerateFileLevel(&file_level_, files_, &arena_);
+  ASSERT_EQ(0u, file_level_.num_files);
+  ASSERT_EQ(0, Compare());
+}
+
+TEST(GenerateFileLevelTest, Single) {
+  Add("p", "q");
+  DoGenerateFileLevel(&file_level_, files_, &arena_);
+  ASSERT_EQ(1u, file_level_.num_files);
+  ASSERT_EQ(0, Compare());
+}
+
+
+TEST(GenerateFileLevelTest, Multiple) {
+  Add("150", "200");
+  Add("200", "250");
+  Add("300", "350");
+  Add("400", "450");
+  DoGenerateFileLevel(&file_level_, files_, &arena_);
+  ASSERT_EQ(4u, file_level_.num_files);
+  ASSERT_EQ(0, Compare());
+}
+
+class FindLevelFileTest {
+ public:
+  FileLevel file_level_;
+  bool disjoint_sorted_files_;
+  Arena arena_;
+
+  FindLevelFileTest() : disjoint_sorted_files_(true) { }
+
+  ~FindLevelFileTest() {
+  }
+
+  void LevelFileInit(size_t num = 0) {
+    char* mem = arena_.AllocateAligned(num * sizeof(FdWithKeyRange));
+    file_level_.files = new (mem)FdWithKeyRange[num];
+    file_level_.num_files = 0;
+  }
+
+  void Add(const char* smallest, const char* largest,
+           SequenceNumber smallest_seq = 100,
+           SequenceNumber largest_seq = 100) {
+    InternalKey smallest_key = InternalKey(smallest, smallest_seq, kTypeValue);
+    InternalKey largest_key = InternalKey(largest, largest_seq, kTypeValue);
+
+    Slice smallest_slice = smallest_key.Encode();
+    Slice largest_slice = largest_key.Encode();
+
+    char* mem = arena_.AllocateAligned(
+        smallest_slice.size() + largest_slice.size());
+    memcpy(mem, smallest_slice.data(), smallest_slice.size());
+    memcpy(mem + smallest_slice.size(), largest_slice.data(),
+        largest_slice.size());
+
+    // add to file_level_
+    size_t num = file_level_.num_files;
+    auto& file = file_level_.files[num];
+    file.fd = FileDescriptor(num + 1, 0, 0);
+    file.smallest_key = Slice(mem, smallest_slice.size());
+    file.largest_key = Slice(mem + smallest_slice.size(),
+        largest_slice.size());
+    file_level_.num_files++;
+  }
+
  int Find(const char* key) {
    InternalKey target(key, 100, kTypeValue);
    InternalKeyComparator cmp(BytewiseComparator());
-    return FindFile(cmp, files_, target.Encode());
+    return FindFile(cmp, file_level_, target.Encode());
  }

  bool Overlaps(const char* smallest, const char* largest) {
    InternalKeyComparator cmp(BytewiseComparator());
    Slice s(smallest != nullptr ? smallest : "");
    Slice l(largest != nullptr ? largest : "");
-    return SomeFileOverlapsRange(cmp, disjoint_sorted_files_, files_,
+    return SomeFileOverlapsRange(cmp, disjoint_sorted_files_, file_level_,
                                 (smallest != nullptr ? &s : nullptr),
                                 (largest != nullptr ? &l : nullptr));
  }
 };

-TEST(FindFileTest, Empty) {
+TEST(FindLevelFileTest, LevelEmpty) {
+  LevelFileInit(0);
+
  ASSERT_EQ(0, Find("foo"));
  ASSERT_TRUE(! Overlaps("a", "z"));
  ASSERT_TRUE(! Overlaps(nullptr, "z"));
@@ -61,7 +141,9 @@ TEST(FindFileTest, Empty) {
  ASSERT_TRUE(! Overlaps(nullptr, nullptr));
 }

-TEST(FindFileTest, Single) {
+TEST(FindLevelFileTest, LevelSingle) {
+  LevelFileInit(1);
+
  Add("p", "q");
  ASSERT_EQ(0, Find("a"));
  ASSERT_EQ(0, Find("p"));
@@ -91,8 +173,9 @@ TEST(FindFileTest, Single) {
  ASSERT_TRUE(Overlaps(nullptr, nullptr));
 }

+TEST(FindLevelFileTest, LevelMultiple) {
+  LevelFileInit(4);

-TEST(FindFileTest, Multiple) {
  Add("150", "200");
  Add("200", "250");
  Add("300", "350");
@@ -130,7 +213,9 @@ TEST(FindFileTest, Multiple) {
  ASSERT_TRUE(Overlaps("450", "500"));
 }

-TEST(FindFileTest, MultipleNullBoundaries) {
+TEST(FindLevelFileTest, LevelMultipleNullBoundaries) {
+  LevelFileInit(4);
+
  Add("150", "200");
  Add("200", "250");
  Add("300", "350");
@@ -150,7 +235,9 @@ TEST(FindFileTest, MultipleNullBoundaries) {
  ASSERT_TRUE(Overlaps("450", nullptr));
 }

-TEST(FindFileTest, OverlapSequenceChecks) {
+TEST(FindLevelFileTest, LevelOverlapSequenceChecks) {
+  LevelFileInit(1);
+
  Add("200", "200", 5000, 3000);
  ASSERT_TRUE(! Overlaps("199", "199"));
  ASSERT_TRUE(! Overlaps("201", "300"));
@@ -159,7 +246,9 @@ TEST(FindFileTest, OverlapSequenceChecks) {
  ASSERT_TRUE(Overlaps("200", "210"));
 }

-TEST(FindFileTest, OverlappingFiles) {
+TEST(FindLevelFileTest, LevelOverlappingFiles) {
+  LevelFileInit(2);
+
  Add("150", "600");
  Add("400", "500");
  disjoint_sorted_files_ = false;
@@ -236,6 +236,23 @@ void WriteBatch::Delete(ColumnFamilyHandle* column_family, const Slice& key) {
  WriteBatchInternal::Delete(this, GetColumnFamilyID(column_family), key);
 }

+void WriteBatchInternal::Delete(WriteBatch* b, uint32_t column_family_id,
+                                const SliceParts& key) {
+  WriteBatchInternal::SetCount(b, WriteBatchInternal::Count(b) + 1);
+  if (column_family_id == 0) {
+    b->rep_.push_back(static_cast<char>(kTypeDeletion));
+  } else {
+    b->rep_.push_back(static_cast<char>(kTypeColumnFamilyDeletion));
+    PutVarint32(&b->rep_, column_family_id);
+  }
+  PutLengthPrefixedSliceParts(&b->rep_, key);
+}
+
+void WriteBatch::Delete(ColumnFamilyHandle* column_family,
+                        const SliceParts& key) {
+  WriteBatchInternal::Delete(this, GetColumnFamilyID(column_family), key);
+}
+
 void WriteBatchInternal::Merge(WriteBatch* b, uint32_t column_family_id,
                               const Slice& key, const Slice& value) {
  WriteBatchInternal::SetCount(b, WriteBatchInternal::Count(b) + 1);
@@ -419,8 +436,8 @@ class MemTableInserter : public WriteBatch::Handler {
          // Failed to merge!
        RecordTick(options->statistics.get(), NUMBER_MERGE_FAILURES);

-          // Store the delta in memtable
-          perform_merge = false;
+        // Store the delta in memtable
+        perform_merge = false;
      } else {
        // 3) Add value to memtable
        mem->Add(sequence_, kTypeValue, key, new_value);
@@ -71,6 +71,9 @@ class WriteBatchInternal {
  static void Put(WriteBatch* batch, uint32_t column_family_id,
                  const SliceParts& key, const SliceParts& value);

+  static void Delete(WriteBatch* batch, uint32_t column_family_id,
+                     const SliceParts& key);
+
  static void Delete(WriteBatch* batch, uint32_t column_family_id,
                     const Slice& key);

@@ -642,10 +642,6 @@ Default:1, i.e. pick maxfilesize amount of data as the source of a compaction.
 <li> <code>Options::max_grandparent_overlap_factor</code> -   Control maximum bytes of overlaps in grandparent (i.e., level+2) before we
 stop building a single file in a level->level+1 compaction.
 <p>
-<li> <code>Options::disable_seek_compaction</code> -  Disable compaction triggered by seek.
-With bloomfilter and fast storage, a miss on one level is very cheap if the file handle is cached in table cache
-(which is true if max_open_files is large).
-<p>
 <li> <code>Options::max_background_compactions</code> - Maximum number of concurrent background jobs, submitted to
 the default LOW priority thread pool
 </ul>
@@ -57,8 +57,13 @@ extern "C" {
 typedef struct rocksdb_t                 rocksdb_t;
 typedef struct rocksdb_cache_t           rocksdb_cache_t;
 typedef struct rocksdb_compactionfilter_t rocksdb_compactionfilter_t;
+typedef struct rocksdb_compactionfiltercontext_t
+    rocksdb_compactionfiltercontext_t;
+typedef struct rocksdb_compactionfilterfactory_t
+    rocksdb_compactionfilterfactory_t;
 typedef struct rocksdb_comparator_t      rocksdb_comparator_t;
 typedef struct rocksdb_env_t             rocksdb_env_t;
+typedef struct rocksdb_fifo_compaction_options_t rocksdb_fifo_compaction_options_t;
 typedef struct rocksdb_filelock_t        rocksdb_filelock_t;
 typedef struct rocksdb_filterpolicy_t    rocksdb_filterpolicy_t;
 typedef struct rocksdb_flushoptions_t    rocksdb_flushoptions_t;
@@ -76,6 +81,7 @@ typedef struct rocksdb_writebatch_t      rocksdb_writebatch_t;
 typedef struct rocksdb_writeoptions_t    rocksdb_writeoptions_t;
 typedef struct rocksdb_universal_compaction_options_t rocksdb_universal_compaction_options_t;
 typedef struct rocksdb_livefiles_t     rocksdb_livefiles_t;
+typedef struct rocksdb_column_family_handle_t rocksdb_column_family_handle_t;

 /* DB operations */

@@ -90,6 +96,45 @@ extern rocksdb_t* rocksdb_open_for_read_only(
    unsigned char error_if_log_file_exist,
    char** errptr);

+extern rocksdb_t* rocksdb_open_column_families(
+    const rocksdb_options_t* options,
+    const char* name,
+    int num_column_families,
+    const char** column_family_names,
+    const rocksdb_options_t** column_family_options,
+    rocksdb_column_family_handle_t** column_family_handles,
+    char** errptr);
+
+extern rocksdb_t* rocksdb_open_for_read_only_column_families(
+    const rocksdb_options_t* options,
+    const char* name,
+    int num_column_families,
+    const char** column_family_names,
+    const rocksdb_options_t** column_family_options,
+    rocksdb_column_family_handle_t** column_family_handles,
+    unsigned char error_if_log_file_exist,
+    char** errptr);
+
+char** rocksdb_list_column_families(
+    const rocksdb_options_t* options,
+    const char* name,
+    size_t* lencf,
+    char** errptr);
+void rocksdb_list_column_families_destroy(char** list, size_t len);
+
+extern rocksdb_column_family_handle_t* rocksdb_create_column_family(
+    rocksdb_t* db,
+    const rocksdb_options_t* column_family_options,
+    const char* column_family_name,
+    char** errptr);
+
+extern void rocksdb_drop_column_family(
+    rocksdb_t* db,
+    rocksdb_column_family_handle_t* handle,
+    char** errptr);
+
+extern void rocksdb_column_family_handle_destroy(rocksdb_column_family_handle_t*);
+
 extern void rocksdb_close(rocksdb_t* db);

 extern void rocksdb_put(
@@ -99,12 +144,27 @@ extern void rocksdb_put(
    const char* val, size_t vallen,
    char** errptr);

+extern void rocksdb_put_cf(
+    rocksdb_t* db,
+    const rocksdb_writeoptions_t* options,
+    rocksdb_column_family_handle_t* column_family,
+    const char* key, size_t keylen,
+    const char* val, size_t vallen,
+    char** errptr);
+
 extern void rocksdb_delete(
    rocksdb_t* db,
    const rocksdb_writeoptions_t* options,
    const char* key, size_t keylen,
    char** errptr);

+void rocksdb_delete_cf(
+    rocksdb_t* db,
+    const rocksdb_writeoptions_t* options,
+    rocksdb_column_family_handle_t* column_family,
+    const char* key, size_t keylen,
+    char** errptr);
+
 extern void rocksdb_merge(
    rocksdb_t* db,
    const rocksdb_writeoptions_t* options,
@@ -112,6 +172,14 @@ extern void rocksdb_merge(
    const char* val, size_t vallen,
    char** errptr);

+extern void rocksdb_merge_cf(
+    rocksdb_t* db,
+    const rocksdb_writeoptions_t* options,
+    rocksdb_column_family_handle_t* column_family,
+    const char* key, size_t keylen,
+    const char* val, size_t vallen,
+    char** errptr);
+
 extern void rocksdb_write(
    rocksdb_t* db,
    const rocksdb_writeoptions_t* options,
@@ -127,10 +195,23 @@ extern char* rocksdb_get(
    size_t* vallen,
    char** errptr);

+extern char* rocksdb_get_cf(
+    rocksdb_t* db,
+    const rocksdb_readoptions_t* options,
+    rocksdb_column_family_handle_t* column_family,
+    const char* key, size_t keylen,
+    size_t* vallen,
+    char** errptr);
+
 extern rocksdb_iterator_t* rocksdb_create_iterator(
    rocksdb_t* db,
    const rocksdb_readoptions_t* options);

+extern rocksdb_iterator_t* rocksdb_create_iterator_cf(
+    rocksdb_t* db,
+    const rocksdb_readoptions_t* options,
+    rocksdb_column_family_handle_t* column_family);
+
 extern const rocksdb_snapshot_t* rocksdb_create_snapshot(
    rocksdb_t* db);

@@ -144,6 +225,11 @@ extern char* rocksdb_property_value(
    rocksdb_t* db,
    const char* propname);

+extern char* rocksdb_property_value_cf(
+    rocksdb_t* db,
+    rocksdb_column_family_handle_t* column_family,
+    const char* propname);
+
 extern void rocksdb_approximate_sizes(
    rocksdb_t* db,
    int num_ranges,
@@ -151,11 +237,25 @@ extern void rocksdb_approximate_sizes(
    const char* const* range_limit_key, const size_t* range_limit_key_len,
    uint64_t* sizes);

+extern void rocksdb_approximate_sizes_cf(
+    rocksdb_t* db,
+    rocksdb_column_family_handle_t* column_family,
+    int num_ranges,
+    const char* const* range_start_key, const size_t* range_start_key_len,
+    const char* const* range_limit_key, const size_t* range_limit_key_len,
+    uint64_t* sizes);
+
 extern void rocksdb_compact_range(
    rocksdb_t* db,
    const char* start_key, size_t start_key_len,
    const char* limit_key, size_t limit_key_len);

+extern void rocksdb_compact_range_cf(
+    rocksdb_t* db,
+    rocksdb_column_family_handle_t* column_family,
+    const char* start_key, size_t start_key_len,
+    const char* limit_key, size_t limit_key_len);
+
 extern void rocksdb_delete_file(
    rocksdb_t* db,
    const char* name);
@@ -205,6 +305,8 @@ extern void rocksdb_iter_get_error(const rocksdb_iterator_t*, char** errptr);
 /* Write batch */

 extern rocksdb_writebatch_t* rocksdb_writebatch_create();
+extern rocksdb_writebatch_t* rocksdb_writebatch_create_from(const char* rep,
+                                                            size_t size);
 extern void rocksdb_writebatch_destroy(rocksdb_writebatch_t*);
 extern void rocksdb_writebatch_clear(rocksdb_writebatch_t*);
 extern int rocksdb_writebatch_count(rocksdb_writebatch_t*);
@@ -212,13 +314,27 @@ extern void rocksdb_writebatch_put(
    rocksdb_writebatch_t*,
    const char* key, size_t klen,
    const char* val, size_t vlen);
+extern void rocksdb_writebatch_put_cf(
+    rocksdb_writebatch_t*,
+    rocksdb_column_family_handle_t* column_family,
+    const char* key, size_t klen,
+    const char* val, size_t vlen);
 extern void rocksdb_writebatch_merge(
    rocksdb_writebatch_t*,
    const char* key, size_t klen,
    const char* val, size_t vlen);
+extern void rocksdb_writebatch_merge_cf(
+    rocksdb_writebatch_t*,
+    rocksdb_column_family_handle_t* column_family,
+    const char* key, size_t klen,
+    const char* val, size_t vlen);
 extern void rocksdb_writebatch_delete(
    rocksdb_writebatch_t*,
    const char* key, size_t klen);
+extern void rocksdb_writebatch_delete_cf(
+    rocksdb_writebatch_t*,
+    rocksdb_column_family_handle_t* column_family,
+    const char* key, size_t klen);
 extern void rocksdb_writebatch_iterate(
    rocksdb_writebatch_t*,
    void* state,
@@ -230,9 +346,19 @@ extern const char* rocksdb_writebatch_data(rocksdb_writebatch_t*, size_t *size);

 extern rocksdb_options_t* rocksdb_options_create();
 extern void rocksdb_options_destroy(rocksdb_options_t*);
+extern void rocksdb_options_increase_parallelism(
+    rocksdb_options_t* opt, int total_threads);
+extern void rocksdb_options_optimize_for_point_lookup(
+    rocksdb_options_t* opt);
+extern void rocksdb_options_optimize_level_style_compaction(
+    rocksdb_options_t* opt, uint64_t memtable_memory_budget);
+extern void rocksdb_options_optimize_universal_style_compaction(
+    rocksdb_options_t* opt, uint64_t memtable_memory_budget);
 extern void rocksdb_options_set_compaction_filter(
    rocksdb_options_t*,
    rocksdb_compactionfilter_t*);
+extern void rocksdb_options_set_compaction_filter_factory(
+    rocksdb_options_t*, rocksdb_compactionfilterfactory_t*);
 extern void rocksdb_options_set_comparator(
    rocksdb_options_t*,
    rocksdb_comparator_t*);
@@ -247,6 +373,8 @@ extern void rocksdb_options_set_filter_policy(
    rocksdb_filterpolicy_t*);
 extern void rocksdb_options_set_create_if_missing(
    rocksdb_options_t*, unsigned char);
+extern void rocksdb_options_set_create_missing_column_families(
+    rocksdb_options_t*, unsigned char);
 extern void rocksdb_options_set_error_if_exists(
    rocksdb_options_t*, unsigned char);
 extern void rocksdb_options_set_paranoid_checks(
@@ -401,10 +529,13 @@ extern void rocksdb_options_set_compression(rocksdb_options_t*, int);

 enum {
  rocksdb_level_compaction = 0,
-  rocksdb_universal_compaction = 1
+  rocksdb_universal_compaction = 1,
+  rocksdb_fifo_compaction = 2
 };
 extern void rocksdb_options_set_compaction_style(rocksdb_options_t*, int);
 extern void rocksdb_options_set_universal_compaction_options(rocksdb_options_t*, rocksdb_universal_compaction_options_t*);
+extern void rocksdb_options_set_fifo_compaction_options(rocksdb_options_t* opt,
+    rocksdb_fifo_compaction_options_t* fifo);

 /* Compaction Filter */

@@ -421,6 +552,25 @@ extern rocksdb_compactionfilter_t* rocksdb_compactionfilter_create(
    const char* (*name)(void*));
 extern void rocksdb_compactionfilter_destroy(rocksdb_compactionfilter_t*);

+/* Compaction Filter Context */
+
+extern unsigned char rocksdb_compactionfiltercontext_is_full_compaction(
+    rocksdb_compactionfiltercontext_t* context);
+
+extern unsigned char rocksdb_compactionfiltercontext_is_manual_compaction(
+    rocksdb_compactionfiltercontext_t* context);
+
+/* Compaction Filter Factory */
+
+extern rocksdb_compactionfilterfactory_t*
+    rocksdb_compactionfilterfactory_create(
+        void* state, void (*destructor)(void*),
+        rocksdb_compactionfilter_t* (*create_compaction_filter)(
+            void*, rocksdb_compactionfiltercontext_t* context),
+        const char* (*name)(void*));
+extern void rocksdb_compactionfilterfactory_destroy(
+    rocksdb_compactionfilterfactory_t*);
+
 /* Comparator */

 extern rocksdb_comparator_t* rocksdb_comparator_create(
@@ -566,6 +716,12 @@ extern void rocksdb_universal_compaction_options_set_stop_style(
 extern void rocksdb_universal_compaction_options_destroy(
  rocksdb_universal_compaction_options_t*);

+extern rocksdb_fifo_compaction_options_t* rocksdb_fifo_compaction_options_create();
+extern void rocksdb_fifo_compaction_options_set_max_table_files_size(
+    rocksdb_fifo_compaction_options_t* fifo_opts, uint64_t size);
+extern void rocksdb_fifo_compaction_options_destroy(
+    rocksdb_fifo_compaction_options_t* fifo_opts);
+
 extern int rocksdb_livefiles_count(
  const rocksdb_livefiles_t*);
 extern const char* rocksdb_livefiles_name(
@@ -86,7 +86,7 @@ class CompactionFilterV2 {
  //
  // Each entry in the return vector indicates if the corresponding kv should
  // be preserved in the output of this compaction run. The application can
-  // inspect the exisitng values of the keys and make decision based on it.
+  // inspect the existing values of the keys and make decision based on it.
  //
  // When a value is to be preserved, the application has the option
  // to modify the entry in existing_values and pass it back through an entry
@@ -108,7 +108,7 @@ class CompactionFilterV2 {
 };

 // Each compaction will create a new CompactionFilter allowing the
-// application to know about different campactions
+// application to know about different compactions
 class CompactionFilterFactory {
 public:
  virtual ~CompactionFilterFactory() { }
@@ -120,7 +120,7 @@ class CompactionFilterFactory {
  virtual const char* Name() const = 0;
 };

-// Default implementaion of CompactionFilterFactory which does not
+// Default implementation of CompactionFilterFactory which does not
 // return any filter
 class DefaultCompactionFilterFactory : public CompactionFilterFactory {
 public:
@@ -175,7 +175,7 @@ class CompactionFilterFactoryV2 {
  const SliceTransform* prefix_extractor_;
 };

-// Default implementaion of CompactionFilterFactoryV2 which does not
+// Default implementation of CompactionFilterFactoryV2 which does not
 // return any filter
 class DefaultCompactionFilterFactoryV2 : public CompactionFilterFactoryV2 {
 public:
@@ -55,6 +55,7 @@ class Env;
 // Metadata associated with each SST file.
 struct LiveFileMetaData {
  std::string column_family_name;  // Name of the column family
+  std::string db_path;
  std::string name;                // Name of the file
  int level;               // Level at which this file resides.
  size_t size;             // File size in bytes.
@@ -159,6 +160,7 @@ class DB {
  virtual Status DropColumnFamily(ColumnFamilyHandle* column_family);

  // Set the database entry for "key" to "value".
+  // If "key" already exists, it will be overwritten.
  // Returns OK on success, and a non-OK status on error.
  // Note: consider setting options.sync = true.
  virtual Status Put(const WriteOptions& options,
@@ -305,6 +307,14 @@ class DB {
    return GetProperty(DefaultColumnFamily(), property, value);
  }

+  // Similar to GetProperty(), but only works for a subset of properties whose
+  // return value is an integer. Return the value by integer.
+  virtual bool GetIntProperty(ColumnFamilyHandle* column_family,
+                              const Slice& property, uint64_t* value) = 0;
+  virtual bool GetIntProperty(const Slice& property, uint64_t* value) {
+    return GetIntProperty(DefaultColumnFamily(), property, value);
+  }
+
  // For each i in [0,n-1], store in "sizes[i]", the approximate
  // file system space used by keys in "[range[i].start .. range[i].limit)".
  //
@@ -337,15 +347,17 @@ class DB {
  // hosting all the files. In this case, client could set reduce_level
  // to true, to move the files back to the minimum level capable of holding
  // the data set or a given level (specified by non-negative target_level).
+  // Compaction outputs should be placed in options.db_paths[target_path_id].
+  // Behavior is undefined if target_path_id is out of range.
  virtual Status CompactRange(ColumnFamilyHandle* column_family,
                              const Slice* begin, const Slice* end,
-                              bool reduce_level = false,
-                              int target_level = -1) = 0;
+                              bool reduce_level = false, int target_level = -1,
+                              uint32_t target_path_id = 0) = 0;
  virtual Status CompactRange(const Slice* begin, const Slice* end,
-                              bool reduce_level = false,
-                              int target_level = -1) {
+                              bool reduce_level = false, int target_level = -1,
+                              uint32_t target_path_id = 0) {
    return CompactRange(DefaultColumnFamily(), begin, end, reduce_level,
-                        target_level);
+                        target_level, target_path_id);
  }

  // Number of levels used for this DB.
@@ -35,6 +35,7 @@ class WritableFile;
 class RandomRWFile;
 class Directory;
 struct DBOptions;
+class RateLimiter;

 using std::unique_ptr;
 using std::shared_ptr;
@@ -74,6 +75,9 @@ struct EnvOptions {
  // write. By default, we set it to true for MANIFEST writes and false for
  // WAL writes
  bool fallocate_with_keep_size = true;
+
+  // If not nullptr, write rate limiting is enabled for flush and compaction
+  RateLimiter* rate_limiter = nullptr;
 };

 class Env {
@@ -194,8 +198,16 @@ class Env {
  // REQUIRES: lock has not already been unlocked.
  virtual Status UnlockFile(FileLock* lock) = 0;

+  // Priority for scheduling job in thread pool
  enum Priority { LOW, HIGH, TOTAL };

+  // Priority for scheduling job in thread pool
+  enum IOPriority {
+    IO_LOW = 0,
+    IO_HIGH = 1,
+    IO_TOTAL = 2
+  };
+
  // Arrange to run "(*function)(arg)" once in a background thread, in
  // the thread pool specified by pri. By default, jobs go to the 'LOW'
  // priority thread pool.
@@ -371,7 +383,10 @@ class RandomAccessFile {
 // at a time to the file.
 class WritableFile {
 public:
-  WritableFile() : last_preallocated_block_(0), preallocation_block_size_ (0) {
+  WritableFile()
+    : last_preallocated_block_(0),
+      preallocation_block_size_(0),
+      io_priority_(Env::IO_TOTAL) {
  }
  virtual ~WritableFile();

@@ -390,6 +405,14 @@ class WritableFile {
    return Sync();
  }

+  /*
+   * Change the priority in rate limiter if rate limiting is enabled.
+   * If rate limiting is not enabled, this call has no effect.
+   */
+  virtual void SetIOPriority(Env::IOPriority pri) {
+    io_priority_ = pri;
+  }
+
  /*
   * Get the size of valid data in the file.
   */
@@ -474,6 +497,9 @@ class WritableFile {
  // No copying allowed
  WritableFile(const WritableFile&);
  void operator=(const WritableFile&);
+
+ protected:
+  Env::IOPriority io_priority_;
 };

 // A file abstraction for random reading and writing.
@@ -0,0 +1,34 @@
+// Copyright (c) 2014, Facebook, Inc.  All rights reserved.
+// This source code is licensed under the BSD-style license found in the
+// LICENSE file in the root directory of this source tree. An additional grant
+// of patent rights can be found in the PATENTS file in the same directory.
+
+#ifndef INCLUDE_ROCKSDB_IOSTATS_CONTEXT_H_
+#define INCLUDE_ROCKSDB_IOSTATS_CONTEXT_H_
+
+#include <stdint.h>
+#include <string>
+
+// A thread local context for gathering io-stats efficiently and transparently.
+namespace rocksdb {
+
+struct IOStatsContext {
+  // reset all io-stats counter to zero
+  void Reset();
+
+  std::string ToString() const;
+
+  // the thread pool id
+  uint64_t thread_pool_id;
+
+  // number of bytes that has been written.
+  uint64_t bytes_written;
+  // number of bytes that has been read.
+  uint64_t bytes_read;
+};
+
+extern __thread IOStatsContext iostats_context;
+
+}  // namespace rocksdb
+
+#endif  // INCLUDE_ROCKSDB_IOSTATS_CONTEXT_H_
@@ -4,13 +4,32 @@
 // of patent rights can be found in the PATENTS file in the same directory.
 #ifndef ROCKSDB_LITE
 #pragma once
+#include <string>
 #include "rocksdb/options.h"

 namespace rocksdb {

+// An interface for converting a slice to a readable string
+class SliceFormatter {
+ public:
+  virtual ~SliceFormatter() {}
+  virtual std::string Format(const Slice& s) const = 0;
+};
+
+// Options for customizing ldb tool (beyond the DB Options)
+struct LDBOptions {
+  // Create LDBOptions with default values for all fields
+  LDBOptions();
+
+  // Key formatter that converts a slice to a readable string.
+  // Default: Slice::ToString()
+  std::shared_ptr<SliceFormatter> key_formatter;
+};
+
 class LDBTool {
 public:
-  void Run(int argc, char** argv, Options = Options());
+  void Run(int argc, char** argv, Options db_options= Options(),
+           const LDBOptions& ldb_options = LDBOptions());
 };

 } // namespace rocksdb
@@ -148,13 +148,6 @@ class MemTableRep {
  //        all the states but those allocated in arena.
  virtual Iterator* GetIterator(Arena* arena = nullptr) = 0;

-  // Return an iterator over at least the keys with the specified user key. The
-  // iterator may also allow access to other keys, but doesn't have to. Default:
-  // GetIterator().
-  virtual Iterator* GetIterator(const Slice& user_key) {
-    return GetIterator(nullptr);
-  }
-
  // Return an iterator that has a special Seek semantics. The result of
  // a Seek might only include keys with the same prefix as the target key.
  // arena: If not null, the arena needs to be used to allocate the Iterator.
@@ -234,9 +227,10 @@ extern MemTableRepFactory* NewHashSkipListRepFactory(
    int32_t skiplist_branching_factor = 4
 );

-// The factory is to create memtables with a hashed linked list:
-// it contains a fixed array of buckets, each pointing to a sorted single
-// linked list (null if the bucket is empty).
+// The factory is to create memtables based on a hash table:
+// it contains a fixed array of buckets, each pointing to either a linked list
+// or a skip list if number of entries inside the bucket exceeds
+// threshold_use_skiplist.
 // @bucket_count: number of fixed array buckets
 // @huge_page_tlb_size: if <=0, allocate the hash table bytes from malloc.
 //                      Otherwise from huge page TLB. The user needs to reserve
@@ -247,10 +241,13 @@ extern MemTableRepFactory* NewHashSkipListRepFactory(
 //                                    exceeds this number, log about it.
 // @if_log_bucket_dist_when_flash: if true, log distribution of number of
 //                                 entries when flushing.
+// @threshold_use_skiplist: a bucket switches to skip list if number of
+//                          entries exceed this parameter.
 extern MemTableRepFactory* NewHashLinkListRepFactory(
    size_t bucket_count = 50000, size_t huge_page_tlb_size = 0,
    int bucket_entries_logging_threshold = 4096,
-    bool if_log_bucket_dist_when_flash = true);
+    bool if_log_bucket_dist_when_flash = true,
+    uint32_t threshold_use_skiplist = 256);

 // This factory creates a cuckoo-hashing based mem-table representation.
 // Cuckoo-hash is a closed-hash strategy, in which all key/value pairs
@@ -34,13 +34,12 @@ class Snapshot;
 class TableFactory;
 class MemTableRepFactory;
 class TablePropertiesCollectorFactory;
+class RateLimiter;
 class Slice;
 class SliceTransform;
 class Statistics;
 class InternalKeyComparator;

-using std::shared_ptr;
-
 // DB contents are stored in a set of blocks, each of which holds a
 // sequence of key,value pairs.  Each block may be compressed before
 // being stored in a file.  The following enum describes which
@@ -83,6 +82,14 @@ enum UpdateStatus {    // Return status For inplace update callback
  UPDATED         = 2, // No inplace update. Merged value set
 };

+struct DbPath {
+  std::string path;
+  uint64_t target_size;  // Target size of total files under the path, in byte.
+
+  DbPath() : target_size(0) {}
+  DbPath(const std::string& p, uint64_t t) : path(p), target_size(t) {}
+};
+
 struct Options;

 struct ColumnFamilyOptions {
@@ -132,7 +139,7 @@ struct ColumnFamilyOptions {
  // for the first time. It's necessary to specify a merge operator when
  // openning the DB in this case.
  // Default: nullptr
-  shared_ptr<MergeOperator> merge_operator;
+  std::shared_ptr<MergeOperator> merge_operator;

  // A single CompactionFilter instance to call into during compaction.
  // Allows an application to modify/delete a key-value during background
@@ -205,12 +212,12 @@ struct ColumnFamilyOptions {
  // If non-NULL use the specified cache for blocks.
  // If NULL, rocksdb will automatically create and use an 8MB internal cache.
  // Default: nullptr
-  shared_ptr<Cache> block_cache;
+  std::shared_ptr<Cache> block_cache;

  // If non-NULL use the specified cache for compressed blocks.
  // If NULL, rocksdb will not use a compressed block cache.
  // Default: nullptr
-  shared_ptr<Cache> block_cache_compressed;
+  std::shared_ptr<Cache> block_cache_compressed;

  // Approximate size of user data packed per block.  Note that the
  // block size specified here corresponds to uncompressed data.  The
@@ -368,10 +375,15 @@ struct ColumnFamilyOptions {
  // stop building a single file in a level->level+1 compaction.
  int max_grandparent_overlap_factor;

-  // Disable compaction triggered by seek.
-  // With bloomfilter and fast storage, a miss on one level
-  // is very cheap if the file handle is cached in table cache
-  // (which is true if max_open_files is large).
+  // We decided to remove seek compaction from RocksDB because:
+  // 1) It makes more sense for spinning disk workloads, while RocksDB is
+  // primarily designed for flash and memory,
+  // 2) It added some complexity to the important code-paths,
+  // 3) None of our internal customers were really using it.
+  //
+  // Since we removed seek compaction, this option is now obsolete.
+  // We left it here for backwards compatiblity (otherwise it would break the
+  // build), but we'll remove it at some point.
  // Default: true
  bool disable_seek_compaction;

@@ -620,11 +632,16 @@ struct DBOptions {
  // Default: Env::Default()
  Env* env;

+  // Use to control write rate of flush and compaction. Flush has higher
+  // priority than compaction. Rate limiting is disabled if nullptr.
+  // Default: nullptr
+  std::shared_ptr<RateLimiter> rate_limiter;
+
  // Any internal progress/error information generated by the db will
  // be written to info_log if it is non-nullptr, or to a file stored
  // in the same directory as the DB contents if info_log is nullptr.
  // Default: nullptr
-  shared_ptr<Logger> info_log;
+  std::shared_ptr<Logger> info_log;

  InfoLogLevel info_log_level;

@@ -647,7 +664,7 @@ struct DBOptions {
  // If non-null, then we should collect metrics about database operations
  // Statistics objects should not be shared between DB instances as
  // it does not use any locks to prevent concurrent updates.
-  shared_ptr<Statistics> statistics;
+  std::shared_ptr<Statistics> statistics;

  // If true, then the contents of data files are not synced
  // to stable storage. Their contents remain in the OS buffers till the
@@ -664,12 +681,35 @@ struct DBOptions {
  // Default: false
  bool use_fsync;

-  // This number controls how often a new scribe log about
-  // db deploy stats is written out.
-  // -1 indicates no logging at all.
-  // Default value is 1800 (half an hour).
+  // This options is not used!!
  int db_stats_log_interval;

+  // A list of paths where SST files can be put into, with its target size.
+  // Newer data is placed into paths specified earlier in the vector while
+  // older data gradually moves to paths specified later in the vector.
+  //
+  // For example, you have a flash device with 10GB allocated for the DB,
+  // as well as a hard drive of 2TB, you should config it to be:
+  //   [{"/flash_path", 10GB}, {"/hard_drive", 2TB}]
+  //
+  // The system will try to guarantee data under each path is close to but
+  // not larger than the target size. But current and future file sizes used
+  // by determining where to place a file are based on best-effort estimation,
+  // which means there is a chance that the actual size under the directory
+  // is slightly more than target size under some workloads. User should give
+  // some buffer room for those cases.
+  //
+  // If none of the paths has sufficient room to place a file, the file will
+  // be placed to the last path anyway, despite to the target size.
+  //
+  // Placing newer data to ealier paths is also best-efforts. User should
+  // expect user files to be placed in higher levels in some extreme cases.
+  //
+  // If left empty, only one path will be used, which is db_name passed when
+  // opening the DB.
+  // Default: empty
+  std::vector<DbPath> db_paths;
+
  // This specifies the info LOG dir.
  // If it is empty, the log files will be in the same dir as data.
  // If it is non empty, the log files will be in the specified dir,
@@ -963,7 +1003,18 @@ struct WriteOptions {
  // and the write may got lost after a crash.
  bool disableWAL;

-  WriteOptions() : sync(false), disableWAL(false) {}
+  // If non-zero, then associated write waiting longer than the specified
+  // time MAY be aborted and returns Status::TimedOut. A write that takes
+  // less than the specified time is guaranteed to not fail with
+  // Status::TimedOut.
+  //
+  // The number of times a write call encounters a timeout is recorded in
+  // Statistics.WRITE_TIMEDOUT
+  //
+  // Default: 0
+  uint64_t timeout_hint_us;
+
+  WriteOptions() : sync(false), disableWAL(false), timeout_hint_us(0) {}
 };

 // Options that control flush operations
@@ -975,6 +1026,16 @@ struct FlushOptions {
  FlushOptions() : wait(true) {}
 };

+// Get options based on some guidelines. Now only tune parameter based on
+// flush/compaction and fill default parameters for other parameters.
+// total_write_buffer_limit: budget for memory spent for mem tables
+// read_amplification_threshold: comfortable value of read amplification
+// write_amplification_threshold: comfortable value of write amplification.
+// target_db_size: estimated total DB size.
+extern Options GetOptions(size_t total_write_buffer_limit,
+                          int read_amplification_threshold = 8,
+                          int write_amplification_threshold = 32,
+                          uint64_t target_db_size = 68719476736 /* 64GB */);
 }  // namespace rocksdb

 #endif  // STORAGE_ROCKSDB_INCLUDE_OPTIONS_H_
@@ -20,6 +20,9 @@ enum PerfLevel {
 // set the perf stats level
 void SetPerfLevel(PerfLevel level);

+// get current perf stats level
+PerfLevel GetPerfLevel();
+
 // A thread local context for gathering performance counter efficiently
 // and transparently.

@@ -0,0 +1,60 @@
+//  Copyright (c) 2014, Facebook, Inc.  All rights reserved.
+//  This source code is licensed under the BSD-style license found in the
+//  LICENSE file in the root directory of this source tree. An additional grant
+//  of patent rights can be found in the PATENTS file in the same directory.
+//
+// Copyright (c) 2011 The LevelDB Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file. See the AUTHORS file for names of contributors.
+
+#pragma once
+
+#include "rocksdb/env.h"
+
+namespace rocksdb {
+
+class RateLimiter {
+ public:
+  virtual ~RateLimiter() {}
+
+  // Request for token to write bytes. If this request can not be satisfied,
+  // the call is blocked. Caller is responsible to make sure
+  // bytes < GetSingleBurstBytes()
+  virtual void Request(const int64_t bytes, const Env::IOPriority pri) = 0;
+
+  // Max bytes can be granted in a single burst
+  virtual int64_t GetSingleBurstBytes() const = 0;
+
+  // Total bytes that go though rate limiter
+  virtual int64_t GetTotalBytesThrough(
+      const Env::IOPriority pri = Env::IO_TOTAL) const = 0;
+
+  // Total # of requests that go though rate limiter
+  virtual int64_t GetTotalRequests(
+      const Env::IOPriority pri = Env::IO_TOTAL) const = 0;
+};
+
+// Create a RateLimiter object, which can be shared among RocksDB instances to
+// control write rate of flush and compaction.
+// @rate_bytes_per_sec: this is the only parameter you want to set most of the
+// time. It controls the total write rate of compaction and flush in bytes per
+// second. Currently, RocksDB does not enforce rate limit for anything other
+// than flush and compaction, e.g. write to WAL.
+// @refill_period_us: this controls how often tokens are refilled. For example,
+// when rate_bytes_per_sec is set to 10MB/s and refill_period_us is set to
+// 100ms, then 1MB is refilled every 100ms internally. Larger value can lead to
+// burstier writes while smaller value introduces more CPU overhead.
+// The default should work for most cases.
+// @fairness: RateLimiter accepts high-pri requests and low-pri requests.
+// A low-pri request is usually blocked in favor of hi-pri request. Currently,
+// RocksDB assigns low-pri to request from compaciton and high-pri to request
+// from flush. Low-pri requests can get blocked if flush requests come in
+// continuouly. This fairness parameter grants low-pri requests permission by
+// 1/fairness chance even though high-pri requests exist to avoid starvation.
+// You should be good by leaving it at default 10.
+extern RateLimiter* NewGenericRateLimiter(
+    int64_t rate_bytes_per_sec,
+    int64_t refill_period_us = 100 * 1000,
+    int32_t fairness = 10);
+
+}  // namespace rocksdb
@@ -107,6 +107,7 @@ class Slice {
 struct SliceParts {
  SliceParts(const Slice* _parts, int _num_parts) :
      parts(_parts), num_parts(_num_parts) { }
+  SliceParts() : parts(nullptr), num_parts(0) {}

  const Slice* parts;
  int num_parts;
@@ -20,12 +20,12 @@ namespace rocksdb {
 *  1. Any ticker should be added before TICKER_ENUM_MAX.
 *  2. Add a readable string in TickersNameMap below for the newly added ticker.
 */
-enum Tickers {
+enum Tickers : uint32_t {
  // total block cache misses
  // REQUIRES: BLOCK_CACHE_MISS == BLOCK_CACHE_INDEX_MISS +
  //                               BLOCK_CACHE_FILTER_MISS +
  //                               BLOCK_CACHE_DATA_MISS;
-  BLOCK_CACHE_MISS,
+  BLOCK_CACHE_MISS = 0,
  // total block cache hit
  // REQUIRES: BLOCK_CACHE_HIT == BLOCK_CACHE_INDEX_HIT +
  //                              BLOCK_CACHE_FILTER_HIT +
@@ -115,9 +115,11 @@ enum Tickers {
  // head of the writers queue.
  WRITE_DONE_BY_SELF,
  WRITE_DONE_BY_OTHER,
+  WRITE_TIMEDOUT,        // Number of writes ending up with timed-out.
  WRITE_WITH_WAL,       // Number of Write calls that request WAL
  COMPACT_READ_BYTES,   // Bytes read during compaction
  COMPACT_WRITE_BYTES,  // Bytes written during compaction
+  FLUSH_WRITE_BYTES,    // Bytes written during flush

  // Number of table's properties loaded directly from file, without creating
  // table reader object.
@@ -176,7 +178,9 @@ const std::vector<std::pair<Tickers, std::string>> TickersNameMap = {
    {WAL_FILE_BYTES, "rocksdb.wal.bytes"},
    {WRITE_DONE_BY_SELF, "rocksdb.write.self"},
    {WRITE_DONE_BY_OTHER, "rocksdb.write.other"},
+    {WRITE_TIMEDOUT, "rocksdb.write.timedout"},
    {WRITE_WITH_WAL, "rocksdb.write.wal"},
+    {FLUSH_WRITE_BYTES, "rocksdb.flush.write.bytes"},
    {COMPACT_READ_BYTES, "rocksdb.compact.read.bytes"},
    {COMPACT_WRITE_BYTES, "rocksdb.compact.write.bytes"},
    {NUMBER_DIRECT_LOAD_TABLE_PROPERTIES,
@@ -194,8 +198,8 @@ const std::vector<std::pair<Tickers, std::string>> TickersNameMap = {
 * Add a string representation in HistogramsNameMap below
 * And increment HISTOGRAM_ENUM_MAX
 */
-enum Histograms {
-  DB_GET,
+enum Histograms : uint32_t {
+  DB_GET = 0,
  DB_WRITE,
  COMPACTION_TIME,
  TABLE_SYNC_MICROS,
@@ -252,14 +256,21 @@ class Statistics {
 public:
  virtual ~Statistics() {}

-  virtual long getTickerCount(Tickers tickerType) = 0;
-  virtual void recordTick(Tickers tickerType, uint64_t count = 0) = 0;
-  virtual void setTickerCount(Tickers tickerType, uint64_t count) = 0;
-  virtual void measureTime(Histograms histogramType, uint64_t time) = 0;
+  virtual uint64_t getTickerCount(uint32_t tickerType) const = 0;
+  virtual void histogramData(uint32_t type,
+                             HistogramData* const data) const = 0;
+
+  virtual void recordTick(uint32_t tickerType, uint64_t count = 0) = 0;
+  virtual void setTickerCount(uint32_t tickerType, uint64_t count) = 0;
+  virtual void measureTime(uint32_t histogramType, uint64_t time) = 0;

-  virtual void histogramData(Histograms type, HistogramData* const data) = 0;
  // String representation of the statistic object.
-  std::string ToString();
+  virtual std::string ToString() const = 0;
+
+  // Override this function to disable particular histogram collection
+  virtual bool HistEnabledForType(uint32_t type) const {
+    return type < HISTOGRAM_ENUM_MAX;
+  }
 };

 // Create a concrete DBStatistics object
@@ -65,6 +65,12 @@ class Status {
                                   const Slice& msg2 = Slice()) {
    return Status(kShutdownInProgress, msg, msg2);
  }
+  static Status TimedOut() {
+    return Status(kTimedOut);
+  }
+  static Status TimedOut(const Slice& msg, const Slice& msg2 = Slice()) {
+    return Status(kTimedOut, msg, msg2);
+  }

  // Returns true iff the status indicates success.
  bool ok() const { return code() == kOk; }
@@ -93,6 +99,8 @@ class Status {
  // Returns true iff the status indicates Incomplete
  bool IsShutdownInProgress() const { return code() == kShutdownInProgress; }

+  bool IsTimedOut() const { return code() == kTimedOut; }
+
  // Return a string representation of this status suitable for printing.
  // Returns the string "OK" for success.
  std::string ToString() const;
@@ -106,7 +114,8 @@ class Status {
    kIOError = 5,
    kMergeInProgress = 6,
    kIncomplete = 7,
-    kShutdownInProgress = 8
+    kShutdownInProgress = 8,
+    kTimedOut = 9
  };

  Code code() const {
@@ -119,33 +119,43 @@ enum EncodingType : char {
 struct PlainTablePropertyNames {
  static const std::string kPrefixExtractorName;
  static const std::string kEncodingType;
+  static const std::string kBloomVersion;
+  static const std::string kNumBloomBlocks;
 };

-// -- Plain Table with prefix-only seek
-// For this factory, you need to set Options.prefix_extrator properly to make it
-// work. Look-up will starts with prefix hash lookup for key prefix. Inside the
-// hash bucket found, a binary search is executed for hash conflicts. Finally,
-// a linear search is used.
+const uint32_t kPlainTableVariableLength = 0;
+
+struct PlainTableOptions {
 // @user_key_len: plain table has optimization for fix-sized keys, which can be
 //                specified via user_key_len.  Alternatively, you can pass
 //                `kPlainTableVariableLength` if your keys have variable
 //                lengths.
+uint32_t user_key_len = kPlainTableVariableLength;
+
 // @bloom_bits_per_key: the number of bits used for bloom filer per prefix. You
 //                      may disable it by passing a zero.
+int bloom_bits_per_key = 10;
+
 // @hash_table_ratio: the desired utilization of the hash table used for prefix
 //                    hashing. hash_table_ratio = number of prefixes / #buckets
 //                    in the hash table
+double hash_table_ratio = 0.75;
+
 // @index_sparseness: inside each prefix, need to build one index record for how
 //                    many keys for binary search inside each hash bucket.
 //                    For encoding type kPrefix, the value will be used when
 //                    writing to determine an interval to rewrite the full key.
 //                    It will also be used as a suggestion and satisfied when
 //                    possible.
+size_t index_sparseness = 16;
+
 // @huge_page_tlb_size: if <=0, allocate hash indexes and blooms from malloc.
 //                      Otherwise from huge page TLB. The user needs to reserve
 //                      huge pages for it to be allocated, like:
 //                          sysctl -w vm.nr_hugepages=20
 //                      See linux doc Documentation/vm/hugetlbpage.txt
+size_t huge_page_tlb_size = 0;
+
 // @encoding_type: how to encode the keys. See enum EncodingType above for
 //                 the choices. The value will determine how to encode keys
 //                 when writing to a new SST file. This value will be stored
@@ -153,37 +163,34 @@ struct PlainTablePropertyNames {
 //                 file, which makes it possible for users to choose different
 //                 encoding type when reopening a DB. Files with different
 //                 encoding types can co-exist in the same DB and can be read.
+EncodingType encoding_type = kPlain;

-const uint32_t kPlainTableVariableLength = 0;
-extern TableFactory* NewPlainTableFactory(uint32_t user_key_len =
-                                              kPlainTableVariableLength,
-                                          int bloom_bits_per_prefix = 10,
-                                          double hash_table_ratio = 0.75,
-                                          size_t index_sparseness = 16,
-                                          size_t huge_page_tlb_size = 0,
-                                          EncodingType encoding_type = kPlain);
+// @full_scan_mode: mode for reading the whole file one record by one without
+//                  using the index.
+  bool full_scan_mode = false;

-// -- Plain Table
-// This factory of plain table ignores Options.prefix_extractor and assumes no
-// hashable prefix available to the key structure. Lookup will be based on
-// binary search index only. Total order seek() can be issued.
-// @user_key_len: plain table has optimization for fix-sized keys, which can be
-//                specified via user_key_len.  Alternatively, you can pass
-//                `kPlainTableVariableLength` if your keys have variable
-//                lengths.
-// @bloom_bits_per_key: the number of bits used for bloom filer per key. You may
-//                  disable it by passing a zero.
-// @index_sparseness: need to build one index record for how many keys for
-//                    binary search.
-// @huge_page_tlb_size: if <=0, allocate hash indexes and blooms from malloc.
-//                      Otherwise from huge page TLB. The user needs to reserve
-//                      huge pages for it to be allocated, like:
-//                          sysctl -w vm.nr_hugepages=20
-//                      See linux doc Documentation/vm/hugetlbpage.txt
-extern TableFactory* NewTotalOrderPlainTableFactory(
-    uint32_t user_key_len = kPlainTableVariableLength,
-    int bloom_bits_per_key = 0, size_t index_sparseness = 16,
-    size_t huge_page_tlb_size = 0, bool full_scan_mode = false);
+  // @store_index_in_file: compute plain table index and bloom filter during
+  //                       file building and store it in file. When reading
+  //                       file, index will be mmaped instead of recomputation.
+  bool store_index_in_file = false;
+};
+
+// -- Plain Table with prefix-only seek
+// For this factory, you need to set Options.prefix_extrator properly to make it
+// work. Look-up will starts with prefix hash lookup for key prefix. Inside the
+// hash bucket found, a binary search is executed for hash conflicts. Finally,
+// a linear search is used.
+
+extern TableFactory* NewPlainTableFactory(const PlainTableOptions& options =
+                                              PlainTableOptions());
+
+struct CuckooTablePropertyNames {
+  static const std::string kEmptyKey;
+  static const std::string kValueLength;
+  static const std::string kNumHashTable;
+  static const std::string kMaxNumBuckets;
+  static const std::string kIsLastLevel;
+};

 #endif  // ROCKSDB_LITE

@@ -8,6 +8,7 @@

 #include <stdint.h>
 #include <climits>
+#include <vector>

 namespace rocksdb {

@@ -61,6 +62,7 @@ class CompactionOptionsUniversal {
  // well as  the total size of C1...Ct as total_C, the compaction output file
  // will be compressed iff
  //   total_C / total_size < this percentage
+  // Default: -1
  int compression_size_percent;

  // The algorithm used to stop picking files into a single compaction run
@@ -68,14 +70,13 @@ class CompactionOptionsUniversal {
  CompactionStopStyle stop_style;

  // Default set of parameters
-  CompactionOptionsUniversal() :
-    size_ratio(1),
-    min_merge_width(2),
-    max_merge_width(UINT_MAX),
-    max_size_amplification_percent(200),
-    compression_size_percent(-1),
-    stop_style(kCompactionStopStyleTotalSize) {
-  }
+  CompactionOptionsUniversal()
+      : size_ratio(1),
+        min_merge_width(2),
+        max_merge_width(UINT_MAX),
+        max_size_amplification_percent(200),
+        compression_size_percent(-1),
+        stop_style(kCompactionStopStyleTotalSize) {}
 };

 }  // namespace rocksdb
@@ -0,0 +1,252 @@
+//  Copyright (c) 2013, Facebook, Inc.  All rights reserved.
+//  This source code is licensed under the BSD-style license found in the
+//  LICENSE file in the root directory of this source tree. An additional grant
+//  of patent rights can be found in the PATENTS file in the same directory.
+//
+// Copyright (c) 2011 The LevelDB Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file. See the AUTHORS file for names of contributors.
+
+#pragma once
+#ifndef ROCKSDB_LITE
+
+#define __STDC_FORMAT_MACROS
+#include <inttypes.h>
+#include <string>
+#include <map>
+#include <vector>
+
+#include "rocksdb/utilities/stackable_db.h"
+
+#include "rocksdb/env.h"
+#include "rocksdb/status.h"
+
+namespace rocksdb {
+
+struct BackupableDBOptions {
+  // Where to keep the backup files. Has to be different than dbname_
+  // Best to set this to dbname_ + "/backups"
+  // Required
+  std::string backup_dir;
+
+  // Backup Env object. It will be used for backup file I/O. If it's
+  // nullptr, backups will be written out using DBs Env. If it's
+  // non-nullptr, backup's I/O will be performed using this object.
+  // If you want to have backups on HDFS, use HDFS Env here!
+  // Default: nullptr
+  Env* backup_env;
+
+  // If share_table_files == true, backup will assume that table files with
+  // same name have the same contents. This enables incremental backups and
+  // avoids unnecessary data copies.
+  // If share_table_files == false, each backup will be on its own and will
+  // not share any data with other backups.
+  // default: true
+  bool share_table_files;
+
+  // Backup info and error messages will be written to info_log
+  // if non-nullptr.
+  // Default: nullptr
+  Logger* info_log;
+
+  // If sync == true, we can guarantee you'll get consistent backup even
+  // on a machine crash/reboot. Backup process is slower with sync enabled.
+  // If sync == false, we don't guarantee anything on machine reboot. However,
+  // chances are some of the backups are consistent.
+  // Default: true
+  bool sync;
+
+  // If true, it will delete whatever backups there are already
+  // Default: false
+  bool destroy_old_data;
+
+  // If false, we won't backup log files. This option can be useful for backing
+  // up in-memory databases where log file are persisted, but table files are in
+  // memory.
+  // Default: true
+  bool backup_log_files;
+
+  // Max bytes that can be transferred in a second during backup.
+  // If 0, go as fast as you can
+  // Default: 0
+  uint64_t backup_rate_limit;
+
+  // Max bytes that can be transferred in a second during restore.
+  // If 0, go as fast as you can
+  // Default: 0
+  uint64_t restore_rate_limit;
+
+  // Only used if share_table_files is set to true. If true, will consider that
+  // backups can come from different databases, hence a sst is not uniquely
+  // identifed by its name, but by the triple (file name, crc32, file length)
+  // Default: false
+  // Note: this is an experimental option, and you'll need to set it manually
+  // *turn it on only if you know what you're doing*
+  bool share_files_with_checksum;
+
+  void Dump(Logger* logger) const;
+
+  explicit BackupableDBOptions(const std::string& _backup_dir,
+                               Env* _backup_env = nullptr,
+                               bool _share_table_files = true,
+                               Logger* _info_log = nullptr, bool _sync = true,
+                               bool _destroy_old_data = false,
+                               bool _backup_log_files = true,
+                               uint64_t _backup_rate_limit = 0,
+                               uint64_t _restore_rate_limit = 0)
+      : backup_dir(_backup_dir),
+        backup_env(_backup_env),
+        share_table_files(_share_table_files),
+        info_log(_info_log),
+        sync(_sync),
+        destroy_old_data(_destroy_old_data),
+        backup_log_files(_backup_log_files),
+        backup_rate_limit(_backup_rate_limit),
+        restore_rate_limit(_restore_rate_limit),
+        share_files_with_checksum(false) {
+    assert(share_table_files || !share_files_with_checksum);
+  }
+};
+
+struct RestoreOptions {
+  // If true, restore won't overwrite the existing log files in wal_dir. It will
+  // also move all log files from archive directory to wal_dir. Use this option
+  // in combination with BackupableDBOptions::backup_log_files = false for
+  // persisting in-memory databases.
+  // Default: false
+  bool keep_log_files;
+
+  explicit RestoreOptions(bool _keep_log_files = false)
+      : keep_log_files(_keep_log_files) {}
+};
+
+typedef uint32_t BackupID;
+
+struct BackupInfo {
+  BackupID backup_id;
+  int64_t timestamp;
+  uint64_t size;
+
+  BackupInfo() {}
+  BackupInfo(BackupID _backup_id, int64_t _timestamp, uint64_t _size)
+      : backup_id(_backup_id), timestamp(_timestamp), size(_size) {}
+};
+
+class BackupEngineReadOnly {
+ public:
+  virtual ~BackupEngineReadOnly() {}
+
+  static BackupEngineReadOnly* NewReadOnlyBackupEngine(
+      Env* db_env, const BackupableDBOptions& options);
+
+  // You can GetBackupInfo safely, even with other BackupEngine performing
+  // backups on the same directory
+  virtual void GetBackupInfo(std::vector<BackupInfo>* backup_info) = 0;
+
+  // Restoring DB from backup is NOT safe when there is another BackupEngine
+  // running that might call DeleteBackup() or PurgeOldBackups(). It is caller's
+  // responsibility to synchronize the operation, i.e. don't delete the backup
+  // when you're restoring from it
+  virtual Status RestoreDBFromBackup(
+      BackupID backup_id, const std::string& db_dir, const std::string& wal_dir,
+      const RestoreOptions& restore_options = RestoreOptions()) = 0;
+  virtual Status RestoreDBFromLatestBackup(
+      const std::string& db_dir, const std::string& wal_dir,
+      const RestoreOptions& restore_options = RestoreOptions()) = 0;
+};
+
+// Please see the documentation in BackupableDB and RestoreBackupableDB
+class BackupEngine {
+ public:
+  virtual ~BackupEngine() {}
+
+  static BackupEngine* NewBackupEngine(Env* db_env,
+                                       const BackupableDBOptions& options);
+
+  virtual Status CreateNewBackup(DB* db, bool flush_before_backup = false) = 0;
+  virtual Status PurgeOldBackups(uint32_t num_backups_to_keep) = 0;
+  virtual Status DeleteBackup(BackupID backup_id) = 0;
+  virtual void StopBackup() = 0;
+
+  virtual void GetBackupInfo(std::vector<BackupInfo>* backup_info) = 0;
+  virtual Status RestoreDBFromBackup(
+      BackupID backup_id, const std::string& db_dir, const std::string& wal_dir,
+      const RestoreOptions& restore_options = RestoreOptions()) = 0;
+  virtual Status RestoreDBFromLatestBackup(
+      const std::string& db_dir, const std::string& wal_dir,
+      const RestoreOptions& restore_options = RestoreOptions()) = 0;
+};
+
+// Stack your DB with BackupableDB to be able to backup the DB
+class BackupableDB : public StackableDB {
+ public:
+  // BackupableDBOptions have to be the same as the ones used in a previous
+  // incarnation of the DB
+  //
+  // BackupableDB ownes the pointer `DB* db` now. You should not delete it or
+  // use it after the invocation of BackupableDB
+  BackupableDB(DB* db, const BackupableDBOptions& options);
+  virtual ~BackupableDB();
+
+  // Captures the state of the database in the latest backup
+  // NOT a thread safe call
+  Status CreateNewBackup(bool flush_before_backup = false);
+  // Returns info about backups in backup_info
+  void GetBackupInfo(std::vector<BackupInfo>* backup_info);
+  // deletes old backups, keeping latest num_backups_to_keep alive
+  Status PurgeOldBackups(uint32_t num_backups_to_keep);
+  // deletes a specific backup
+  Status DeleteBackup(BackupID backup_id);
+  // Call this from another thread if you want to stop the backup
+  // that is currently happening. It will return immediatelly, will
+  // not wait for the backup to stop.
+  // The backup will stop ASAP and the call to CreateNewBackup will
+  // return Status::Incomplete(). It will not clean up after itself, but
+  // the state will remain consistent. The state will be cleaned up
+  // next time you create BackupableDB or RestoreBackupableDB.
+  void StopBackup();
+
+ private:
+  BackupEngine* backup_engine_;
+};
+
+// Use this class to access information about backups and restore from them
+class RestoreBackupableDB {
+ public:
+  RestoreBackupableDB(Env* db_env, const BackupableDBOptions& options);
+  ~RestoreBackupableDB();
+
+  // Returns info about backups in backup_info
+  void GetBackupInfo(std::vector<BackupInfo>* backup_info);
+
+  // restore from backup with backup_id
+  // IMPORTANT -- if options_.share_table_files == true and you restore DB
+  // from some backup that is not the latest, and you start creating new
+  // backups from the new DB, they will probably fail
+  //
+  // Example: Let's say you have backups 1, 2, 3, 4, 5 and you restore 3.
+  // If you add new data to the DB and try creating a new backup now, the
+  // database will diverge from backups 4 and 5 and the new backup will fail.
+  // If you want to create new backup, you will first have to delete backups 4
+  // and 5.
+  Status RestoreDBFromBackup(BackupID backup_id, const std::string& db_dir,
+                             const std::string& wal_dir,
+                             const RestoreOptions& restore_options =
+                                 RestoreOptions());
+
+  // restore from the latest backup
+  Status RestoreDBFromLatestBackup(const std::string& db_dir,
+                                   const std::string& wal_dir,
+                                   const RestoreOptions& restore_options =
+                                       RestoreOptions());
+  // deletes old backups, keeping latest num_backups_to_keep alive
+  Status PurgeOldBackups(uint32_t num_backups_to_keep);
+  // deletes a specific backup
+  Status DeleteBackup(BackupID backup_id);
+
+ private:
+  BackupEngine* backup_engine_;
+};
+
+}  // namespace rocksdb
+#endif  // ROCKSDB_LITE
@@ -0,0 +1,68 @@
+//  Copyright (c) 2013, Facebook, Inc.  All rights reserved.
+//  This source code is licensed under the BSD-style license found in the
+//  LICENSE file in the root directory of this source tree. An additional grant
+//  of patent rights can be found in the PATENTS file in the same directory.
+
+#pragma once
+#ifndef ROCKSDB_LITE
+
+#include <string>
+#include <vector>
+
+#include "rocksdb/utilities/stackable_db.h"
+#include "rocksdb/db.h"
+
+namespace rocksdb {
+
+// Database with TTL support.
+//
+// USE-CASES:
+// This API should be used to open the db when key-values inserted are
+//  meant to be removed from the db in a non-strict 'ttl' amount of time
+//  Therefore, this guarantees that key-values inserted will remain in the
+//  db for >= ttl amount of time and the db will make efforts to remove the
+//  key-values as soon as possible after ttl seconds of their insertion.
+//
+// BEHAVIOUR:
+// TTL is accepted in seconds
+// (int32_t)Timestamp(creation) is suffixed to values in Put internally
+// Expired TTL values deleted in compaction only:(Timestamp+ttl<time_now)
+// Get/Iterator may return expired entries(compaction not run on them yet)
+// Different TTL may be used during different Opens
+// Example: Open1 at t=0 with ttl=4 and insert k1,k2, close at t=2
+//          Open2 at t=3 with ttl=5. Now k1,k2 should be deleted at t>=5
+// read_only=true opens in the usual read-only mode. Compactions will not be
+//  triggered(neither manual nor automatic), so no expired entries removed
+//
+// CONSTRAINTS:
+// Not specifying/passing or non-positive TTL behaves like TTL = infinity
+//
+// !!!WARNING!!!:
+// Calling DB::Open directly to re-open a db created by this API will get
+//  corrupt values(timestamp suffixed) and no ttl effect will be there
+//  during the second Open, so use this API consistently to open the db
+// Be careful when passing ttl with a small positive value because the
+//  whole database may be deleted in a small amount of time
+
+class DBWithTTL : public StackableDB {
+ public:
+  virtual Status CreateColumnFamilyWithTtl(
+      const ColumnFamilyOptions& options, const std::string& column_family_name,
+      ColumnFamilyHandle** handle, int ttl) = 0;
+
+  static Status Open(const Options& options, const std::string& dbname,
+                     DBWithTTL** dbptr, int32_t ttl = 0,
+                     bool read_only = false);
+
+  static Status Open(const DBOptions& db_options, const std::string& dbname,
+                     const std::vector<ColumnFamilyDescriptor>& column_families,
+                     std::vector<ColumnFamilyHandle*>* handles,
+                     DBWithTTL** dbptr, std::vector<int32_t> ttls,
+                     bool read_only = false);
+
+ protected:
+  explicit DBWithTTL(DB* db) : StackableDB(db) {}
+};
+
+}  // namespace rocksdb
+#endif  // ROCKSDB_LITE
@@ -0,0 +1,149 @@
+//  Copyright (c) 2013, Facebook, Inc.  All rights reserved.
+//  This source code is licensed under the BSD-style license found in the
+//  LICENSE file in the root directory of this source tree. An additional grant
+//  of patent rights can be found in the PATENTS file in the same directory.
+
+#pragma once
+#ifndef ROCKSDB_LITE
+
+#include <string>
+#include <vector>
+
+#include "rocksdb/utilities/stackable_db.h"
+#include "rocksdb/utilities/json_document.h"
+#include "rocksdb/db.h"
+
+namespace rocksdb {
+
+// IMPORTANT: DocumentDB is a work in progress. It is unstable and we might
+// change the API without warning. Talk to RocksDB team before using this in
+// production ;)
+
+// DocumentDB is a layer on top of RocksDB that provides a very simple JSON API.
+// When creating a DB, you specify a list of indexes you want to keep on your
+// data. You can insert a JSON document to the DB, which is automatically
+// indexed. Every document added to the DB needs to have "_id" field which is
+// automatically indexed and is an unique primary key. All other indexes are
+// non-unique.
+
+// NOTE: field names in the JSON are NOT allowed to start with '$' or
+// contain '.'. We don't currently enforce that rule, but will start behaving
+// badly.
+
+// Cursor is what you get as a result of executing query. To get all
+// results from a query, call Next() on a Cursor while  Valid() returns true
+class Cursor {
+ public:
+  Cursor() = default;
+  virtual ~Cursor() {}
+
+  virtual bool Valid() const = 0;
+  virtual void Next() = 0;
+  // Lifecycle of the returned JSONDocument is until the next Next() call
+  virtual const JSONDocument& document() const = 0;
+  virtual Status status() const = 0;
+
+ private:
+  // No copying allowed
+  Cursor(const Cursor&);
+  void operator=(const Cursor&);
+};
+
+struct DocumentDBOptions {
+  int background_threads = 4;
+  uint64_t memtable_size = 128 * 1024 * 1024;    // 128 MB
+  uint64_t cache_size = 1 * 1024 * 1024 * 1024;  // 1 GB
+};
+
+// TODO(icanadi) Add `JSONDocument* info` parameter to all calls that can be
+// used by the caller to get more information about the call execution (number
+// of dropped records, number of updated records, etc.)
+class DocumentDB : public StackableDB {
+ public:
+  struct IndexDescriptor {
+    // Currently, you can only define an index on a single field. To specify an
+    // index on a field X, set index description to JSON "{X: 1}"
+    // Currently the value needs to be 1, which means ascending.
+    // In the future, we plan to also support indexes on multiple keys, where
+    // you could mix ascending sorting (1) with descending sorting indexes (-1)
+    JSONDocument* description;
+    std::string name;
+  };
+
+  // Open DocumentDB with specified indexes. The list of indexes has to be
+  // complete, i.e. include all indexes present in the DB, except the primary
+  // key index.
+  // Otherwise, Open() will return an error
+  static Status Open(const DocumentDBOptions& options, const std::string& name,
+                     const std::vector<IndexDescriptor>& indexes,
+                     DocumentDB** db, bool read_only = false);
+
+  explicit DocumentDB(DB* db) : StackableDB(db) {}
+
+  // Create a new index. It will stop all writes for the duration of the call.
+  // All current documents in the DB are scanned and corresponding index entries
+  // are created
+  virtual Status CreateIndex(const WriteOptions& write_options,
+                             const IndexDescriptor& index) = 0;
+
+  // Drop an index. Client is responsible to make sure that index is not being
+  // used by currently executing queries
+  virtual Status DropIndex(const std::string& name) = 0;
+
+  // Insert a document to the DB. The document needs to have a primary key "_id"
+  // which can either be a string or an integer. Otherwise the write will fail
+  // with InvalidArgument.
+  virtual Status Insert(const WriteOptions& options,
+                        const JSONDocument& document) = 0;
+
+  // Deletes all documents matching a filter atomically
+  virtual Status Remove(const ReadOptions& read_options,
+                        const WriteOptions& write_options,
+                        const JSONDocument& query) = 0;
+
+  // Does this sequence of operations:
+  // 1. Find all documents matching a filter
+  // 2. For all documents, atomically:
+  // 2.1. apply the update operators
+  // 2.2. update the secondary indexes
+  //
+  // Currently only $set update operator is supported.
+  // Syntax is: {$set: {key1: value1, key2: value2, etc...}}
+  // This operator will change a document's key1 field to value1, key2 to
+  // value2, etc. New values will be set even if a document didn't have an entry
+  // for the specified key.
+  //
+  // You can not change a primary key of a document.
+  //
+  // Update example: Update({id: {$gt: 5}, $index: id}, {$set: {enabled: true}})
+  virtual Status Update(const ReadOptions& read_options,
+                        const WriteOptions& write_options,
+                        const JSONDocument& filter,
+                        const JSONDocument& updates) = 0;
+
+  // query has to be an array in which every element is an operator. Currently
+  // only $filter operator is supported. Syntax of $filter operator is:
+  // {$filter: {key1: condition1, key2: condition2, etc.}} where conditions can
+  // be either:
+  // 1) a single value in which case the condition is equality condition, or
+  // 2) a defined operators, like {$gt: 4}, which will match all documents that
+  // have key greater than 4.
+  //
+  // Supported operators are:
+  // 1) $gt -- greater than
+  // 2) $gte -- greater than or equal
+  // 3) $lt -- less than
+  // 4) $lte -- less than or equal
+  // If you want the filter to use an index, you need to specify it like this:
+  // {$filter: {...(conditions)..., $index: index_name}}
+  //
+  // Example query:
+  // * [{$filter: {name: John, age: {$gte: 18}, $index: age}}]
+  // will return all Johns whose age is greater or equal to 18 and it will use
+  // index "age" to satisfy the query.
+  virtual Cursor* Query(const ReadOptions& read_options,
+                        const JSONDocument& query) = 0;
+};
+
+}  // namespace rocksdb
+#endif  // ROCKSDB_LITE
@@ -0,0 +1,105 @@
+//  Copyright (c) 2013, Facebook, Inc.  All rights reserved.
+//  This source code is licensed under the BSD-style license found in the
+//  LICENSE file in the root directory of this source tree. An additional grant
+//  of patent rights can be found in the PATENTS file in the same directory.
+//
+
+#ifndef ROCKSDB_LITE
+#pragma once
+#include <string>
+#include <vector>
+
+#include "rocksdb/utilities/stackable_db.h"
+#include "rocksdb/status.h"
+
+namespace rocksdb {
+
+//
+// Configurable options needed for setting up a Geo database
+//
+struct GeoDBOptions {
+  // Backup info and error messages will be written to info_log
+  // if non-nullptr.
+  // Default: nullptr
+  Logger* info_log;
+
+  explicit GeoDBOptions(Logger* _info_log = nullptr):info_log(_info_log) { }
+};
+
+//
+// A position in the earth's geoid
+//
+class GeoPosition {
+ public:
+  double latitude;
+  double longitude;
+
+  explicit GeoPosition(double la = 0, double lo = 0) :
+    latitude(la), longitude(lo) {
+  }
+};
+
+//
+// Description of an object on the Geoid. It is located by a GPS location,
+// and is identified by the id. The value associated with this object is
+// an opaque string 'value'. Different objects identified by unique id's
+// can have the same gps-location associated with them.
+//
+class GeoObject {
+ public:
+  GeoPosition position;
+  std::string id;
+  std::string value;
+
+  GeoObject() {}
+
+  GeoObject(const GeoPosition& pos, const std::string& i,
+            const std::string& val) :
+    position(pos), id(i), value(val) {
+  }
+};
+
+//
+// Stack your DB with GeoDB to be able to get geo-spatial support
+//
+class GeoDB : public StackableDB {
+ public:
+  // GeoDBOptions have to be the same as the ones used in a previous
+  // incarnation of the DB
+  //
+  // GeoDB owns the pointer `DB* db` now. You should not delete it or
+  // use it after the invocation of GeoDB
+  // GeoDB(DB* db, const GeoDBOptions& options) : StackableDB(db) {}
+  GeoDB(DB* db, const GeoDBOptions& options) : StackableDB(db) {}
+  virtual ~GeoDB() {}
+
+  // Insert a new object into the location database. The object is
+  // uniquely identified by the id. If an object with the same id already
+  // exists in the db, then the old one is overwritten by the new
+  // object being inserted here.
+  virtual Status Insert(const GeoObject& object) = 0;
+
+  // Retrieve the value of the object located at the specified GPS
+  // location and is identified by the 'id'.
+  virtual Status GetByPosition(const GeoPosition& pos,
+                               const Slice& id, std::string* value) = 0;
+
+  // Retrieve the value of the object identified by the 'id'. This method
+  // could be potentially slower than GetByPosition
+  virtual Status GetById(const Slice& id, GeoObject*  object) = 0;
+
+  // Delete the specified object
+  virtual Status Remove(const Slice& id) = 0;
+
+  // Returns a list of all items within a circular radius from the
+  // specified gps location. If 'number_of_values' is specified,
+  // then this call returns at most that many number of objects.
+  // The radius is specified in 'meters'.
+  virtual Status SearchRadial(const GeoPosition& pos,
+                              double radius,
+                              std::vector<GeoObject>* values,
+                              int number_of_values = INT_MAX) = 0;
+};
+
+}  // namespace rocksdb
+#endif  // ROCKSDB_LITE
@@ -0,0 +1,174 @@
+//  Copyright (c) 2013, Facebook, Inc.  All rights reserved.
+//  This source code is licensed under the BSD-style license found in the
+//  LICENSE file in the root directory of this source tree. An additional grant
+//  of patent rights can be found in the PATENTS file in the same directory.
+#pragma once
+#ifndef ROCKSDB_LITE
+
+#include <string>
+#include <map>
+#include <unordered_map>
+#include <vector>
+
+#include "rocksdb/slice.h"
+
+// We use JSONDocument for DocumentDB API
+// Implementation inspired by folly::dynamic and rapidjson
+
+namespace rocksdb {
+
+// NOTE: none of this is thread-safe
+class JSONDocument {
+ public:
+  // return nullptr on parse failure
+  static JSONDocument* ParseJSON(const char* json);
+
+  enum Type {
+    kNull,
+    kArray,
+    kBool,
+    kDouble,
+    kInt64,
+    kObject,
+    kString,
+  };
+
+  JSONDocument();  // null
+  /* implicit */ JSONDocument(bool b);
+  /* implicit */ JSONDocument(double d);
+  /* implicit */ JSONDocument(int64_t i);
+  /* implicit */ JSONDocument(const std::string& s);
+  /* implicit */ JSONDocument(const char* s);
+  // constructs JSONDocument of specific type with default value
+  explicit JSONDocument(Type type);
+
+  // copy constructor
+  JSONDocument(const JSONDocument& json_document);
+
+  ~JSONDocument();
+
+  Type type() const;
+
+  // REQUIRES: IsObject()
+  bool Contains(const std::string& key) const;
+  // Returns nullptr if !Contains()
+  // don't delete the returned pointer
+  // REQUIRES: IsObject()
+  const JSONDocument* Get(const std::string& key) const;
+  // REQUIRES: IsObject()
+  JSONDocument& operator[](const std::string& key);
+  // REQUIRES: IsObject()
+  const JSONDocument& operator[](const std::string& key) const;
+  // returns `this`, so you can chain operations.
+  // Copies value
+  // REQUIRES: IsObject()
+  JSONDocument* Set(const std::string& key, const JSONDocument& value);
+
+  // REQUIRES: IsArray() == true || IsObject() == true
+  size_t Count() const;
+
+  // REQUIRES: IsArray()
+  const JSONDocument* GetFromArray(size_t i) const;
+  // REQUIRES: IsArray()
+  JSONDocument& operator[](size_t i);
+  // REQUIRES: IsArray()
+  const JSONDocument& operator[](size_t i) const;
+  // returns `this`, so you can chain operations.
+  // Copies the value
+  // REQUIRES: IsArray() && i < Count()
+  JSONDocument* SetInArray(size_t i, const JSONDocument& value);
+  // REQUIRES: IsArray()
+  JSONDocument* PushBack(const JSONDocument& value);
+
+  bool IsNull() const;
+  bool IsArray() const;
+  bool IsBool() const;
+  bool IsDouble() const;
+  bool IsInt64() const;
+  bool IsObject() const;
+  bool IsString() const;
+
+  // REQUIRES: IsBool() == true
+  bool GetBool() const;
+  // REQUIRES: IsDouble() == true
+  double GetDouble() const;
+  // REQUIRES: IsInt64() == true
+  int64_t GetInt64() const;
+  // REQUIRES: IsString() == true
+  const std::string& GetString() const;
+
+  bool operator==(const JSONDocument& rhs) const;
+
+  std::string DebugString() const;
+
+ private:
+  class ItemsIteratorGenerator;
+
+ public:
+  // REQUIRES: IsObject()
+  ItemsIteratorGenerator Items() const;
+
+  // appends serialized object to dst
+  void Serialize(std::string* dst) const;
+  // returns nullptr if Slice doesn't represent valid serialized JSONDocument
+  static JSONDocument* Deserialize(const Slice& src);
+
+ private:
+  void SerializeInternal(std::string* dst, bool type_prefix) const;
+  // returns false if Slice doesn't represent valid serialized JSONDocument.
+  // Otherwise, true
+  bool DeserializeInternal(Slice* input);
+
+  typedef std::vector<JSONDocument*> Array;
+  typedef std::unordered_map<std::string, JSONDocument*> Object;
+
+  // iteration on objects
+  class const_item_iterator {
+   public:
+    typedef Object::const_iterator It;
+    typedef Object::value_type value_type;
+    /* implicit */ const_item_iterator(It it) : it_(it) {}
+    It& operator++() { return ++it_; }
+    bool operator!=(const const_item_iterator& other) {
+      return it_ != other.it_;
+    }
+    value_type operator*() { return *it_; }
+
+   private:
+    It it_;
+  };
+  class ItemsIteratorGenerator {
+   public:
+    /* implicit */ ItemsIteratorGenerator(const Object& object)
+        : object_(object) {}
+    const_item_iterator begin() { return object_.begin(); }
+    const_item_iterator end() { return object_.end(); }
+
+   private:
+    const Object& object_;
+  };
+
+  union Data {
+    Data() : n(nullptr) {}
+    ~Data() {}
+
+    void* n;
+    Array a;
+    bool b;
+    double d;
+    int64_t i;
+    std::string s;
+    Object o;
+  } data_;
+  const Type type_;
+
+  // Our serialization format's first byte specifies the encoding version. That
+  // way, we can easily change our format while providing backwards
+  // compatibility. This constant specifies the current version of the
+  // serialization format
+  static const char kSerializationFormatVersion;
+};
+
+}  // namespace rocksdb
+
+#endif  // ROCKSDB_LITE
@@ -0,0 +1,233 @@
+//  Copyright (c) 2013, Facebook, Inc.  All rights reserved.
+//  This source code is licensed under the BSD-style license found in the
+//  LICENSE file in the root directory of this source tree. An additional grant
+//  of patent rights can be found in the PATENTS file in the same directory.
+
+#pragma once
+#ifndef ROCKSDB_LITE
+
+#include <string>
+#include <vector>
+
+#include "rocksdb/db.h"
+#include "rocksdb/slice.h"
+#include "rocksdb/utilities/stackable_db.h"
+
+namespace rocksdb {
+namespace spatial {
+
+// NOTE: SpatialDB is experimental and we might change its API without warning.
+// Please talk to us before developing against SpatialDB API.
+//
+// SpatialDB is a support for spatial indexes built on top of RocksDB.
+// When creating a new SpatialDB, clients specifies a list of spatial indexes to
+// build on their data. Each spatial index is defined by the area and
+// granularity. If you're storing map data, different spatial index
+// granularities can be used for different zoom levels.
+//
+// Each element inserted into SpatialDB has:
+// * a bounding box, which determines how will the element be indexed
+// * string blob, which will usually be WKB representation of the polygon
+// (http://en.wikipedia.org/wiki/Well-known_text)
+// * feature set, which is a map of key-value pairs, where value can be null,
+// int, double, bool, string
+// * a list of indexes to insert the element in
+//
+// Each query is executed on a single spatial index. Query guarantees that it
+// will return all elements intersecting the specified bounding box, but it
+// might also return some extra non-intersecting elements.
+
+// Variant is a class that can be many things: null, bool, int, double or string
+// It is used to store different value types in FeatureSet (see below)
+struct Variant {
+  // Don't change the values here, they are persisted on disk
+  enum Type {
+    kNull = 0x0,
+    kBool = 0x1,
+    kInt = 0x2,
+    kDouble = 0x3,
+    kString = 0x4,
+  };
+
+  Variant() : type_(kNull) {}
+  /* implicit */ Variant(bool b) : type_(kBool) { data_.b = b; }
+  /* implicit */ Variant(uint64_t i) : type_(kInt) { data_.i = i; }
+  /* implicit */ Variant(double d) : type_(kDouble) { data_.d = d; }
+  /* implicit */ Variant(const std::string& s) : type_(kString) {
+    new (&data_.s) std::string(s);
+  }
+
+  Variant(const Variant& v);
+
+  ~Variant() {
+    if (type_ == kString) {
+      using std::string;
+      (&data_.s)->~string();
+    }
+  }
+
+  Type type() const { return type_; }
+  bool get_bool() const { return data_.b; }
+  uint64_t get_int() const { return data_.i; }
+  double get_double() const { return data_.d; }
+  const std::string& get_string() const { return data_.s; }
+
+  bool operator==(const Variant& other);
+  bool operator!=(const Variant& other);
+
+ private:
+  Type type_;
+  union Data {
+    Data() {}
+    ~Data() {}
+    bool b;
+    uint64_t i;
+    double d;
+    std::string s;
+  } data_;
+};
+
+// FeatureSet is a map of key-value pairs. One feature set is associated with
+// each element in SpatialDB. It can be used to add rich data about the element.
+class FeatureSet {
+ private:
+  typedef std::unordered_map<std::string, Variant> map;
+
+ public:
+  class iterator {
+   public:
+    /* implicit */ iterator(const map::const_iterator itr) : itr_(itr) {}
+    iterator& operator++() {
+      ++itr_;
+      return *this;
+    }
+    bool operator!=(const iterator& other) { return itr_ != other.itr_; }
+    bool operator==(const iterator& other) { return itr_ == other.itr_; }
+    map::value_type operator*() { return *itr_; }
+
+   private:
+    map::const_iterator itr_;
+  };
+  FeatureSet() = default;
+
+  FeatureSet* Set(const std::string& key, const Variant& value);
+  bool Contains(const std::string& key) const;
+  // REQUIRES: Contains(key)
+  const Variant& Get(const std::string& key) const;
+  iterator Find(const std::string& key) const;
+
+  iterator begin() const { return map_.begin(); }
+  iterator end() const { return map_.end(); }
+
+  void Clear();
+
+  void Serialize(std::string* output) const;
+  // REQUIRED: empty FeatureSet
+  bool Deserialize(const Slice& input);
+
+ private:
+  map map_;
+};
+
+// BoundingBox is a helper structure for defining rectangles representing
+// bounding boxes of spatial elements.
+template <typename T>
+struct BoundingBox {
+  T min_x, min_y, max_x, max_y;
+  BoundingBox() = default;
+  BoundingBox(T _min_x, T _min_y, T _max_x, T _max_y)
+      : min_x(_min_x), min_y(_min_y), max_x(_max_x), max_y(_max_y) {}
+
+  bool Intersects(const BoundingBox<T>& a) const {
+    return !(min_x > a.max_x || min_y > a.max_y || a.min_x > max_x ||
+             a.min_y > max_y);
+  }
+};
+
+struct SpatialDBOptions {
+  uint64_t cache_size = 1 * 1024 * 1024 * 1024LL;  // 1GB
+  int num_threads = 16;
+  bool bulk_load = true;
+};
+
+// Cursor is used to return data from the query to the client. To get all the
+// data from the query, just call Next() while Valid() is true
+class Cursor {
+ public:
+  Cursor() = default;
+  virtual ~Cursor() {}
+
+  virtual bool Valid() const = 0;
+  // REQUIRES: Valid()
+  virtual void Next() = 0;
+
+  // Lifetime of the underlying storage until the next call to Next()
+  // REQUIRES: Valid()
+  virtual const Slice blob() = 0;
+  // Lifetime of the underlying storage until the next call to Next()
+  // REQUIRES: Valid()
+  virtual const FeatureSet& feature_set() = 0;
+
+  virtual Status status() const = 0;
+
+ private:
+  // No copying allowed
+  Cursor(const Cursor&);
+  void operator=(const Cursor&);
+};
+
+// SpatialIndexOptions defines a spatial index that will be built on the data
+struct SpatialIndexOptions {
+  // Spatial indexes are referenced by names
+  std::string name;
+  // An area that is indexed. If the element is not intersecting with spatial
+  // index's bbox, it will not be inserted into the index
+  BoundingBox<double> bbox;
+  // tile_bits control the granularity of the spatial index. Each dimension of
+  // the bbox will be split into (1 << tile_bits) tiles, so there will be a
+  // total of (1 << tile_bits)^2 tiles. It is recommended to configure a size of
+  // each  tile to be approximately the size of the query on that spatial index
+  uint32_t tile_bits;
+  SpatialIndexOptions() {}
+  SpatialIndexOptions(const std::string& _name,
+                      const BoundingBox<double>& _bbox, uint32_t _tile_bits)
+      : name(_name), bbox(_bbox), tile_bits(_tile_bits) {}
+};
+
+class SpatialDB : public StackableDB {
+ public:
+  // Creates the SpatialDB with specified list of indexes.
+  // REQUIRED: db doesn't exist
+  static Status Create(const SpatialDBOptions& options, const std::string& name,
+                       const std::vector<SpatialIndexOptions>& spatial_indexes);
+
+  // Open the existing SpatialDB.  The resulting db object will be returned
+  // through db parameter.
+  // REQUIRED: db was created using SpatialDB::Create
+  static Status Open(const SpatialDBOptions& options, const std::string& name,
+                     SpatialDB** db, bool read_only = false);
+
+  explicit SpatialDB(DB* db) : StackableDB(db) {}
+
+  // Insert the element into the DB. Element will be inserted into specified
+  // spatial_indexes, based on specified bbox.
+  // REQUIRES: spatial_indexes.size() > 0
+  virtual Status Insert(const WriteOptions& write_options,
+                        const BoundingBox<double>& bbox, const Slice& blob,
+                        const FeatureSet& feature_set,
+                        const std::vector<std::string>& spatial_indexes) = 0;
+
+  // Calling Compact() after inserting a bunch of elements should speed up
+  // reading. This is especially useful if you use SpatialDBOptions::bulk_load
+  virtual Status Compact() = 0;
+
+  // Query the specified spatial_index. Query will return all elements that
+  // intersect bbox, but it may also return some extra elements.
+  virtual Cursor* Query(const ReadOptions& read_options,
+                        const BoundingBox<double>& bbox,
+                        const std::string& spatial_index) = 0;
+};
+
+}  // namespace spatial
+}  // namespace rocksdb
+#endif  // ROCKSDB_LITE
@@ -0,0 +1,226 @@
+// Copyright (c) 2011 The LevelDB Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file. See the AUTHORS file for names of contributors.
+
+#pragma once
+#include "rocksdb/db.h"
+
+namespace rocksdb {
+
+// This class contains APIs to stack rocksdb wrappers.Eg. Stack TTL over base d
+class StackableDB : public DB {
+ public:
+  // StackableDB is the owner of db now!
+  explicit StackableDB(DB* db) : db_(db) {}
+
+  ~StackableDB() {
+    delete db_;
+  }
+
+  virtual DB* GetBaseDB() {
+    return db_;
+  }
+
+  virtual Status CreateColumnFamily(const ColumnFamilyOptions& options,
+                                    const std::string& column_family_name,
+                                    ColumnFamilyHandle** handle) {
+    return db_->CreateColumnFamily(options, column_family_name, handle);
+  }
+
+  virtual Status DropColumnFamily(ColumnFamilyHandle* column_family) {
+    return db_->DropColumnFamily(column_family);
+  }
+
+  using DB::Put;
+  virtual Status Put(const WriteOptions& options,
+                     ColumnFamilyHandle* column_family, const Slice& key,
+                     const Slice& val) override {
+    return db_->Put(options, column_family, key, val);
+  }
+
+  using DB::Get;
+  virtual Status Get(const ReadOptions& options,
+                     ColumnFamilyHandle* column_family, const Slice& key,
+                     std::string* value) override {
+    return db_->Get(options, column_family, key, value);
+  }
+
+  using DB::MultiGet;
+  virtual std::vector<Status> MultiGet(
+      const ReadOptions& options,
+      const std::vector<ColumnFamilyHandle*>& column_family,
+      const std::vector<Slice>& keys,
+      std::vector<std::string>* values) override {
+    return db_->MultiGet(options, column_family, keys, values);
+  }
+
+  using DB::KeyMayExist;
+  virtual bool KeyMayExist(const ReadOptions& options,
+                           ColumnFamilyHandle* column_family, const Slice& key,
+                           std::string* value,
+                           bool* value_found = nullptr) override {
+    return db_->KeyMayExist(options, column_family, key, value, value_found);
+  }
+
+  using DB::Delete;
+  virtual Status Delete(const WriteOptions& wopts,
+                        ColumnFamilyHandle* column_family,
+                        const Slice& key) override {
+    return db_->Delete(wopts, column_family, key);
+  }
+
+  using DB::Merge;
+  virtual Status Merge(const WriteOptions& options,
+                       ColumnFamilyHandle* column_family, const Slice& key,
+                       const Slice& value) override {
+    return db_->Merge(options, column_family, key, value);
+  }
+
+
+  virtual Status Write(const WriteOptions& opts, WriteBatch* updates)
+    override {
+      return db_->Write(opts, updates);
+  }
+
+  using DB::NewIterator;
+  virtual Iterator* NewIterator(const ReadOptions& opts,
+                                ColumnFamilyHandle* column_family) override {
+    return db_->NewIterator(opts, column_family);
+  }
+
+  virtual Status NewIterators(
+      const ReadOptions& options,
+      const std::vector<ColumnFamilyHandle*>& column_families,
+      std::vector<Iterator*>* iterators) {
+    return db_->NewIterators(options, column_families, iterators);
+  }
+
+
+  virtual const Snapshot* GetSnapshot() override {
+    return db_->GetSnapshot();
+  }
+
+  virtual void ReleaseSnapshot(const Snapshot* snapshot) override {
+    return db_->ReleaseSnapshot(snapshot);
+  }
+
+  using DB::GetProperty;
+  virtual bool GetProperty(ColumnFamilyHandle* column_family,
+                           const Slice& property, std::string* value) override {
+    return db_->GetProperty(column_family, property, value);
+  }
+
+  using DB::GetIntProperty;
+  virtual bool GetIntProperty(ColumnFamilyHandle* column_family,
+                              const Slice& property, uint64_t* value) override {
+    return db_->GetIntProperty(column_family, property, value);
+  }
+
+  using DB::GetApproximateSizes;
+  virtual void GetApproximateSizes(ColumnFamilyHandle* column_family,
+                                   const Range* r, int n,
+                                   uint64_t* sizes) override {
+      return db_->GetApproximateSizes(column_family, r, n, sizes);
+  }
+
+  using DB::CompactRange;
+  virtual Status CompactRange(ColumnFamilyHandle* column_family,
+                              const Slice* begin, const Slice* end,
+                              bool reduce_level = false, int target_level = -1,
+                              uint32_t target_path_id = 0) override {
+    return db_->CompactRange(column_family, begin, end, reduce_level,
+                             target_level, target_path_id);
+  }
+
+  using DB::NumberLevels;
+  virtual int NumberLevels(ColumnFamilyHandle* column_family) override {
+    return db_->NumberLevels(column_family);
+  }
+
+  using DB::MaxMemCompactionLevel;
+  virtual int MaxMemCompactionLevel(ColumnFamilyHandle* column_family)
+      override {
+    return db_->MaxMemCompactionLevel(column_family);
+  }
+
+  using DB::Level0StopWriteTrigger;
+  virtual int Level0StopWriteTrigger(ColumnFamilyHandle* column_family)
+      override {
+    return db_->Level0StopWriteTrigger(column_family);
+  }
+
+  virtual const std::string& GetName() const override {
+    return db_->GetName();
+  }
+
+  virtual Env* GetEnv() const override {
+    return db_->GetEnv();
+  }
+
+  using DB::GetOptions;
+  virtual const Options& GetOptions(ColumnFamilyHandle* column_family) const
+      override {
+    return db_->GetOptions(column_family);
+  }
+
+  using DB::Flush;
+  virtual Status Flush(const FlushOptions& fopts,
+                       ColumnFamilyHandle* column_family) override {
+    return db_->Flush(fopts, column_family);
+  }
+
+  virtual Status DisableFileDeletions() override {
+    return db_->DisableFileDeletions();
+  }
+
+  virtual Status EnableFileDeletions(bool force) override {
+    return db_->EnableFileDeletions(force);
+  }
+
+  virtual void GetLiveFilesMetaData(
+      std::vector<LiveFileMetaData>* metadata) override {
+    db_->GetLiveFilesMetaData(metadata);
+  }
+
+  virtual Status GetLiveFiles(std::vector<std::string>& vec, uint64_t* mfs,
+                              bool flush_memtable = true) override {
+      return db_->GetLiveFiles(vec, mfs, flush_memtable);
+  }
+
+  virtual SequenceNumber GetLatestSequenceNumber() const override {
+    return db_->GetLatestSequenceNumber();
+  }
+
+  virtual Status GetSortedWalFiles(VectorLogPtr& files) override {
+    return db_->GetSortedWalFiles(files);
+  }
+
+  virtual Status DeleteFile(std::string name) override {
+    return db_->DeleteFile(name);
+  }
+
+  virtual Status GetDbIdentity(std::string& identity) {
+    return db_->GetDbIdentity(identity);
+  }
+
+  using DB::GetPropertiesOfAllTables;
+  virtual Status GetPropertiesOfAllTables(ColumnFamilyHandle* column_family,
+                                          TablePropertiesCollection* props) {
+    return db_->GetPropertiesOfAllTables(column_family, props);
+  }
+
+  virtual Status GetUpdatesSince(
+      SequenceNumber seq_number, unique_ptr<TransactionLogIterator>* iter,
+      const TransactionLogIterator::ReadOptions& read_options) override {
+    return db_->GetUpdatesSince(seq_number, iter, read_options);
+  }
+
+  virtual ColumnFamilyHandle* DefaultColumnFamily() const override {
+    return db_->DefaultColumnFamily();
+  }
+
+ protected:
+  DB* db_;
+};
+
+} //  namespace rocksdb
@@ -0,0 +1,30 @@
+// Copyright (c) 2011 The LevelDB Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file. See the AUTHORS file for names of contributors.
+
+#pragma once
+#ifndef ROCKSDB_LITE
+#include <vector>
+#include <string>
+
+#include "rocksdb/utilities/stackable_db.h"
+#include "rocksdb/utilities/db_ttl.h"
+#include "rocksdb/db.h"
+
+namespace rocksdb {
+
+// Please don't use this class. It's deprecated
+class UtilityDB {
+ public:
+  // This function is here only for backwards compatibility. Please use the
+  // functions defined in DBWithTTl (rocksdb/utilities/db_ttl.h)
+  // (deprecated)
+  __attribute__((deprecated)) static Status OpenTtlDB(const Options& options,
+                                                      const std::string& name,
+                                                      StackableDB** dbptr,
+                                                      int32_t ttl = 0,
+                                                      bool read_only = false);
+};
+
+} //  namespace rocksdb
+#endif  // ROCKSDB_LITE
@@ -6,7 +6,7 @@

 // Also update Makefile if you change these
 #define ROCKSDB_MAJOR 3
-#define ROCKSDB_MINOR 2
+#define ROCKSDB_MINOR 4
 #define ROCKSDB_PATCH 0

 // Do not use these. We made the mistake of declaring macros starting with
@@ -67,6 +67,10 @@ class WriteBatch {
  void Delete(ColumnFamilyHandle* column_family, const Slice& key);
  void Delete(const Slice& key) { Delete(nullptr, key); }

+  // variant that takes SliceParts
+  void Delete(ColumnFamilyHandle* column_family, const SliceParts& key);
+  void Delete(const SliceParts& key) { Delete(nullptr, key); }
+
  // Append a blob of arbitrary size to the records in this batch. The blob will
  // be stored in the transaction log but not in any other file. In particular,
  // it will not be persisted to the SST files. When iterating over this
@@ -8,244 +8,5 @@
 // found in the LICENSE file. See the AUTHORS file for names of contributors.

 #pragma once
-#ifndef ROCKSDB_LITE
-
-#define __STDC_FORMAT_MACROS
-#include <inttypes.h>
-#include <string>
-#include <map>
-#include <vector>
-
-#include "utilities/stackable_db.h"
-#include "rocksdb/env.h"
-#include "rocksdb/status.h"
-
-namespace rocksdb {
-
-struct BackupableDBOptions {
-  // Where to keep the backup files. Has to be different than dbname_
-  // Best to set this to dbname_ + "/backups"
-  // Required
-  std::string backup_dir;
-
-  // Backup Env object. It will be used for backup file I/O. If it's
-  // nullptr, backups will be written out using DBs Env. If it's
-  // non-nullptr, backup's I/O will be performed using this object.
-  // If you want to have backups on HDFS, use HDFS Env here!
-  // Default: nullptr
-  Env* backup_env;
-
-  // If share_table_files == true, backup will assume that table files with
-  // same name have the same contents. This enables incremental backups and
-  // avoids unnecessary data copies.
-  // If share_table_files == false, each backup will be on its own and will
-  // not share any data with other backups.
-  // default: true
-  bool share_table_files;
-
-  // Backup info and error messages will be written to info_log
-  // if non-nullptr.
-  // Default: nullptr
-  Logger* info_log;
-
-  // If sync == true, we can guarantee you'll get consistent backup even
-  // on a machine crash/reboot. Backup process is slower with sync enabled.
-  // If sync == false, we don't guarantee anything on machine reboot. However,
-  // chances are some of the backups are consistent.
-  // Default: true
-  bool sync;
-
-  // If true, it will delete whatever backups there are already
-  // Default: false
-  bool destroy_old_data;
-
-  // If false, we won't backup log files. This option can be useful for backing
-  // up in-memory databases where log file are persisted, but table files are in
-  // memory.
-  // Default: true
-  bool backup_log_files;
-
-  // Max bytes that can be transferred in a second during backup.
-  // If 0, go as fast as you can
-  // Default: 0
-  uint64_t backup_rate_limit;
-
-  // Max bytes that can be transferred in a second during restore.
-  // If 0, go as fast as you can
-  // Default: 0
-  uint64_t restore_rate_limit;
-
-  // Only used if share_table_files is set to true. If true, will consider that
-  // backups can come from different databases, hence a sst is not uniquely
-  // identifed by its name, but by the triple (file name, crc32, file length)
-  // Default: false
-  // Note: this is an experimental option, and you'll need to set it manually
-  // *turn it on only if you know what you're doing*
-  bool share_files_with_checksum;
-
-  void Dump(Logger* logger) const;
-
-  explicit BackupableDBOptions(const std::string& _backup_dir,
-                               Env* _backup_env = nullptr,
-                               bool _share_table_files = true,
-                               Logger* _info_log = nullptr, bool _sync = true,
-                               bool _destroy_old_data = false,
-                               bool _backup_log_files = true,
-                               uint64_t _backup_rate_limit = 0,
-                               uint64_t _restore_rate_limit = 0)
-      : backup_dir(_backup_dir),
-        backup_env(_backup_env),
-        share_table_files(_share_table_files),
-        info_log(_info_log),
-        sync(_sync),
-        destroy_old_data(_destroy_old_data),
-        backup_log_files(_backup_log_files),
-        backup_rate_limit(_backup_rate_limit),
-        restore_rate_limit(_restore_rate_limit),
-        share_files_with_checksum(false) {
-    assert(share_table_files || !share_files_with_checksum);
-  }
-};
-
-struct RestoreOptions {
-  // If true, restore won't overwrite the existing log files in wal_dir. It will
-  // also move all log files from archive directory to wal_dir. Use this option
-  // in combination with BackupableDBOptions::backup_log_files = false for
-  // persisting in-memory databases.
-  // Default: false
-  bool keep_log_files;
-
-  explicit RestoreOptions(bool _keep_log_files = false)
-      : keep_log_files(_keep_log_files) {}
-};
-
-typedef uint32_t BackupID;
-
-struct BackupInfo {
-  BackupID backup_id;
-  int64_t timestamp;
-  uint64_t size;
-
-  BackupInfo() {}
-  BackupInfo(BackupID _backup_id, int64_t _timestamp, uint64_t _size)
-      : backup_id(_backup_id), timestamp(_timestamp), size(_size) {}
-};
-
-class BackupEngineReadOnly {
- public:
-  virtual ~BackupEngineReadOnly() {}
-
-  static BackupEngineReadOnly* NewReadOnlyBackupEngine(
-      Env* db_env, const BackupableDBOptions& options);
-
-  // You can GetBackupInfo safely, even with other BackupEngine performing
-  // backups on the same directory
-  virtual void GetBackupInfo(std::vector<BackupInfo>* backup_info) = 0;
-
-  // Restoring DB from backup is NOT safe when there is another BackupEngine
-  // running that might call DeleteBackup() or PurgeOldBackups(). It is caller's
-  // responsibility to synchronize the operation, i.e. don't delete the backup
-  // when you're restoring from it
-  virtual Status RestoreDBFromBackup(
-      BackupID backup_id, const std::string& db_dir, const std::string& wal_dir,
-      const RestoreOptions& restore_options = RestoreOptions()) = 0;
-  virtual Status RestoreDBFromLatestBackup(
-      const std::string& db_dir, const std::string& wal_dir,
-      const RestoreOptions& restore_options = RestoreOptions()) = 0;
-};
-
-// Please see the documentation in BackupableDB and RestoreBackupableDB
-class BackupEngine {
- public:
-  virtual ~BackupEngine() {}
-
-  static BackupEngine* NewBackupEngine(Env* db_env,
-                                       const BackupableDBOptions& options);
-
-  virtual Status CreateNewBackup(DB* db, bool flush_before_backup = false) = 0;
-  virtual Status PurgeOldBackups(uint32_t num_backups_to_keep) = 0;
-  virtual Status DeleteBackup(BackupID backup_id) = 0;
-  virtual void StopBackup() = 0;
-
-  virtual void GetBackupInfo(std::vector<BackupInfo>* backup_info) = 0;
-  virtual Status RestoreDBFromBackup(
-      BackupID backup_id, const std::string& db_dir, const std::string& wal_dir,
-      const RestoreOptions& restore_options = RestoreOptions()) = 0;
-  virtual Status RestoreDBFromLatestBackup(
-      const std::string& db_dir, const std::string& wal_dir,
-      const RestoreOptions& restore_options = RestoreOptions()) = 0;
-};
-
-// Stack your DB with BackupableDB to be able to backup the DB
-class BackupableDB : public StackableDB {
- public:
-  // BackupableDBOptions have to be the same as the ones used in a previous
-  // incarnation of the DB
-  //
-  // BackupableDB ownes the pointer `DB* db` now. You should not delete it or
-  // use it after the invocation of BackupableDB
-  BackupableDB(DB* db, const BackupableDBOptions& options);
-  virtual ~BackupableDB();
-
-  // Captures the state of the database in the latest backup
-  // NOT a thread safe call
-  Status CreateNewBackup(bool flush_before_backup = false);
-  // Returns info about backups in backup_info
-  void GetBackupInfo(std::vector<BackupInfo>* backup_info);
-  // deletes old backups, keeping latest num_backups_to_keep alive
-  Status PurgeOldBackups(uint32_t num_backups_to_keep);
-  // deletes a specific backup
-  Status DeleteBackup(BackupID backup_id);
-  // Call this from another thread if you want to stop the backup
-  // that is currently happening. It will return immediatelly, will
-  // not wait for the backup to stop.
-  // The backup will stop ASAP and the call to CreateNewBackup will
-  // return Status::Incomplete(). It will not clean up after itself, but
-  // the state will remain consistent. The state will be cleaned up
-  // next time you create BackupableDB or RestoreBackupableDB.
-  void StopBackup();
-
- private:
-  BackupEngine* backup_engine_;
-};
-
-// Use this class to access information about backups and restore from them
-class RestoreBackupableDB {
- public:
-  RestoreBackupableDB(Env* db_env, const BackupableDBOptions& options);
-  ~RestoreBackupableDB();
-
-  // Returns info about backups in backup_info
-  void GetBackupInfo(std::vector<BackupInfo>* backup_info);
-
-  // restore from backup with backup_id
-  // IMPORTANT -- if options_.share_table_files == true and you restore DB
-  // from some backup that is not the latest, and you start creating new
-  // backups from the new DB, they will probably fail
-  //
-  // Example: Let's say you have backups 1, 2, 3, 4, 5 and you restore 3.
-  // If you add new data to the DB and try creating a new backup now, the
-  // database will diverge from backups 4 and 5 and the new backup will fail.
-  // If you want to create new backup, you will first have to delete backups 4
-  // and 5.
-  Status RestoreDBFromBackup(BackupID backup_id, const std::string& db_dir,
-                             const std::string& wal_dir,
-                             const RestoreOptions& restore_options =
-                                 RestoreOptions());
-
-  // restore from the latest backup
-  Status RestoreDBFromLatestBackup(const std::string& db_dir,
-                                   const std::string& wal_dir,
-                                   const RestoreOptions& restore_options =
-                                       RestoreOptions());
-  // deletes old backups, keeping latest num_backups_to_keep alive
-  Status PurgeOldBackups(uint32_t num_backups_to_keep);
-  // deletes a specific backup
-  Status DeleteBackup(BackupID backup_id);
-
- private:
-  BackupEngine* backup_engine_;
-};
-
-}  // namespace rocksdb
-#endif  // ROCKSDB_LITE
+#warning This file was moved to rocksdb/utilities/backupable_db.h
+#include "rocksdb/utilities/backupable_db.h"
@@ -4,65 +4,5 @@
 //  of patent rights can be found in the PATENTS file in the same directory.

 #pragma once
-#ifndef ROCKSDB_LITE
-
-#include <string>
-#include <vector>
-
-#include "utilities/stackable_db.h"
-#include "rocksdb/db.h"
-
-namespace rocksdb {
-
-// Database with TTL support.
-//
-// USE-CASES:
-// This API should be used to open the db when key-values inserted are
-//  meant to be removed from the db in a non-strict 'ttl' amount of time
-//  Therefore, this guarantees that key-values inserted will remain in the
-//  db for >= ttl amount of time and the db will make efforts to remove the
-//  key-values as soon as possible after ttl seconds of their insertion.
-//
-// BEHAVIOUR:
-// TTL is accepted in seconds
-// (int32_t)Timestamp(creation) is suffixed to values in Put internally
-// Expired TTL values deleted in compaction only:(Timestamp+ttl<time_now)
-// Get/Iterator may return expired entries(compaction not run on them yet)
-// Different TTL may be used during different Opens
-// Example: Open1 at t=0 with ttl=4 and insert k1,k2, close at t=2
-//          Open2 at t=3 with ttl=5. Now k1,k2 should be deleted at t>=5
-// read_only=true opens in the usual read-only mode. Compactions will not be
-//  triggered(neither manual nor automatic), so no expired entries removed
-//
-// CONSTRAINTS:
-// Not specifying/passing or non-positive TTL behaves like TTL = infinity
-//
-// !!!WARNING!!!:
-// Calling DB::Open directly to re-open a db created by this API will get
-//  corrupt values(timestamp suffixed) and no ttl effect will be there
-//  during the second Open, so use this API consistently to open the db
-// Be careful when passing ttl with a small positive value because the
-//  whole database may be deleted in a small amount of time
-
-class DBWithTTL : public StackableDB {
- public:
-  virtual Status CreateColumnFamilyWithTtl(
-      const ColumnFamilyOptions& options, const std::string& column_family_name,
-      ColumnFamilyHandle** handle, int ttl) = 0;
-
-  static Status Open(const Options& options, const std::string& dbname,
-                     DBWithTTL** dbptr, int32_t ttl = 0,
-                     bool read_only = false);
-
-  static Status Open(const DBOptions& db_options, const std::string& dbname,
-                     const std::vector<ColumnFamilyDescriptor>& column_families,
-                     std::vector<ColumnFamilyHandle*>* handles,
-                     DBWithTTL** dbptr, std::vector<int32_t> ttls,
-                     bool read_only = false);
-
- protected:
-  explicit DBWithTTL(DB* db) : StackableDB(db) {}
-};
-
-}  // namespace rocksdb
-#endif  // ROCKSDB_LITE
+#warning This file was moved to rocksdb/utilities/db_ttl.h
+#include "rocksdb/utilities/db_ttl.h"
@@ -0,0 +1,8 @@
+//  Copyright (c) 2013, Facebook, Inc.  All rights reserved.
+//  This source code is licensed under the BSD-style license found in the
+//  LICENSE file in the root directory of this source tree. An additional grant
+//  of patent rights can be found in the PATENTS file in the same directory.
+
+#pragma once
+#warning This file was moved to rocksdb/utilities/document_db.h
+#include "rocksdb/utilities/document_db.h"
@@ -2,104 +2,7 @@
 //  This source code is licensed under the BSD-style license found in the
 //  LICENSE file in the root directory of this source tree. An additional grant
 //  of patent rights can be found in the PATENTS file in the same directory.
-//

-#ifndef ROCKSDB_LITE
 #pragma once
-#include <string>
-#include <vector>
-
-#include "utilities/stackable_db.h"
-#include "rocksdb/status.h"
-
-namespace rocksdb {
-
-//
-// Configurable options needed for setting up a Geo database
-//
-struct GeoDBOptions {
-  // Backup info and error messages will be written to info_log
-  // if non-nullptr.
-  // Default: nullptr
-  Logger* info_log;
-
-  explicit GeoDBOptions(Logger* _info_log = nullptr):info_log(_info_log) { }
-};
-
-//
-// A position in the earth's geoid
-//
-class GeoPosition {
- public:
-  double latitude;
-  double longitude;
-
-  explicit GeoPosition(double la = 0, double lo = 0) :
-    latitude(la), longitude(lo) {
-  }
-};
-
-//
-// Description of an object on the Geoid. It is located by a GPS location,
-// and is identified by the id. The value associated with this object is
-// an opaque string 'value'. Different objects identified by unique id's
-// can have the same gps-location associated with them.
-//
-class GeoObject {
- public:
-  GeoPosition position;
-  std::string id;
-  std::string value;
-
-  GeoObject() {}
-
-  GeoObject(const GeoPosition& pos, const std::string& i,
-            const std::string& val) :
-    position(pos), id(i), value(val) {
-  }
-};
-
-//
-// Stack your DB with GeoDB to be able to get geo-spatial support
-//
-class GeoDB : public StackableDB {
- public:
-  // GeoDBOptions have to be the same as the ones used in a previous
-  // incarnation of the DB
-  //
-  // GeoDB owns the pointer `DB* db` now. You should not delete it or
-  // use it after the invocation of GeoDB
-  // GeoDB(DB* db, const GeoDBOptions& options) : StackableDB(db) {}
-  GeoDB(DB* db, const GeoDBOptions& options) : StackableDB(db) {}
-  virtual ~GeoDB() {}
-
-  // Insert a new object into the location database. The object is
-  // uniquely identified by the id. If an object with the same id already
-  // exists in the db, then the old one is overwritten by the new
-  // object being inserted here.
-  virtual Status Insert(const GeoObject& object) = 0;
-
-  // Retrieve the value of the object located at the specified GPS
-  // location and is identified by the 'id'.
-  virtual Status GetByPosition(const GeoPosition& pos,
-                               const Slice& id, std::string* value) = 0;
-
-  // Retrieve the value of the object identified by the 'id'. This method
-  // could be potentially slower than GetByPosition
-  virtual Status GetById(const Slice& id, GeoObject*  object) = 0;
-
-  // Delete the specified object
-  virtual Status Remove(const Slice& id) = 0;
-
-  // Returns a list of all items within a circular radius from the
-  // specified gps location. If 'number_of_values' is specified,
-  // then this call returns at most that many number of objects.
-  // The radius is specified in 'meters'.
-  virtual Status SearchRadial(const GeoPosition& pos,
-                              double radius,
-                              std::vector<GeoObject>* values,
-                              int number_of_values = INT_MAX) = 0;
-};
-
-}  // namespace rocksdb
-#endif  // ROCKSDB_LITE
+#warning This file was moved to rocksdb/utilities/geo_db.h
+#include "rocksdb/utilities/geo_db.h"
@@ -0,0 +1,7 @@
+//  Copyright (c) 2013, Facebook, Inc.  All rights reserved.
+//  This source code is licensed under the BSD-style license found in the
+//  LICENSE file in the root directory of this source tree. An additional grant
+//  of patent rights can be found in the PATENTS file in the same directory.
+#pragma once
+#warning This file was moved to rocksdb/utilities/json_document.h
+#include "rocksdb/utilities/json_document.h"
@@ -3,213 +3,5 @@
 // found in the LICENSE file. See the AUTHORS file for names of contributors.

 #pragma once
-#include "rocksdb/db.h"
-
-namespace rocksdb {
-
-// This class contains APIs to stack rocksdb wrappers.Eg. Stack TTL over base d
-class StackableDB : public DB {
- public:
-  // StackableDB is the owner of db now!
-  explicit StackableDB(DB* db) : db_(db) {}
-
-  ~StackableDB() {
-    delete db_;
-  }
-
-  virtual DB* GetBaseDB() {
-    return db_;
-  }
-
-  virtual Status CreateColumnFamily(const ColumnFamilyOptions& options,
-                                    const std::string& column_family_name,
-                                    ColumnFamilyHandle** handle) {
-    return db_->CreateColumnFamily(options, column_family_name, handle);
-  }
-
-  virtual Status DropColumnFamily(ColumnFamilyHandle* column_family) {
-    return db_->DropColumnFamily(column_family);
-  }
-
-  using DB::Put;
-  virtual Status Put(const WriteOptions& options,
-                     ColumnFamilyHandle* column_family, const Slice& key,
-                     const Slice& val) override {
-    return db_->Put(options, column_family, key, val);
-  }
-
-  using DB::Get;
-  virtual Status Get(const ReadOptions& options,
-                     ColumnFamilyHandle* column_family, const Slice& key,
-                     std::string* value) override {
-    return db_->Get(options, column_family, key, value);
-  }
-
-  using DB::MultiGet;
-  virtual std::vector<Status> MultiGet(
-      const ReadOptions& options,
-      const std::vector<ColumnFamilyHandle*>& column_family,
-      const std::vector<Slice>& keys,
-      std::vector<std::string>* values) override {
-    return db_->MultiGet(options, column_family, keys, values);
-  }
-
-  using DB::KeyMayExist;
-  virtual bool KeyMayExist(const ReadOptions& options,
-                           ColumnFamilyHandle* column_family, const Slice& key,
-                           std::string* value,
-                           bool* value_found = nullptr) override {
-    return db_->KeyMayExist(options, column_family, key, value, value_found);
-  }
-
-  using DB::Delete;
-  virtual Status Delete(const WriteOptions& wopts,
-                        ColumnFamilyHandle* column_family,
-                        const Slice& key) override {
-    return db_->Delete(wopts, column_family, key);
-  }
-
-  using DB::Merge;
-  virtual Status Merge(const WriteOptions& options,
-                       ColumnFamilyHandle* column_family, const Slice& key,
-                       const Slice& value) override {
-    return db_->Merge(options, column_family, key, value);
-  }
-
-
-  virtual Status Write(const WriteOptions& opts, WriteBatch* updates)
-    override {
-      return db_->Write(opts, updates);
-  }
-
-  using DB::NewIterator;
-  virtual Iterator* NewIterator(const ReadOptions& opts,
-                                ColumnFamilyHandle* column_family) override {
-    return db_->NewIterator(opts, column_family);
-  }
-
-  virtual Status NewIterators(
-      const ReadOptions& options,
-      const std::vector<ColumnFamilyHandle*>& column_families,
-      std::vector<Iterator*>* iterators) {
-    return db_->NewIterators(options, column_families, iterators);
-  }
-
-
-  virtual const Snapshot* GetSnapshot() override {
-    return db_->GetSnapshot();
-  }
-
-  virtual void ReleaseSnapshot(const Snapshot* snapshot) override {
-    return db_->ReleaseSnapshot(snapshot);
-  }
-
-  using DB::GetProperty;
-  virtual bool GetProperty(ColumnFamilyHandle* column_family,
-                           const Slice& property, std::string* value) override {
-      return db_->GetProperty(column_family, property, value);
-  }
-
-  using DB::GetApproximateSizes;
-  virtual void GetApproximateSizes(ColumnFamilyHandle* column_family,
-                                   const Range* r, int n,
-                                   uint64_t* sizes) override {
-      return db_->GetApproximateSizes(column_family, r, n, sizes);
-  }
-
-  using DB::CompactRange;
-  virtual Status CompactRange(ColumnFamilyHandle* column_family,
-                              const Slice* begin, const Slice* end,
-                              bool reduce_level = false,
-                              int target_level = -1) override {
-    return db_->CompactRange(column_family, begin, end, reduce_level,
-                             target_level);
-  }
-
-  using DB::NumberLevels;
-  virtual int NumberLevels(ColumnFamilyHandle* column_family) override {
-    return db_->NumberLevels(column_family);
-  }
-
-  using DB::MaxMemCompactionLevel;
-  virtual int MaxMemCompactionLevel(ColumnFamilyHandle* column_family)
-      override {
-    return db_->MaxMemCompactionLevel(column_family);
-  }
-
-  using DB::Level0StopWriteTrigger;
-  virtual int Level0StopWriteTrigger(ColumnFamilyHandle* column_family)
-      override {
-    return db_->Level0StopWriteTrigger(column_family);
-  }
-
-  virtual const std::string& GetName() const override {
-    return db_->GetName();
-  }
-
-  virtual Env* GetEnv() const override {
-    return db_->GetEnv();
-  }
-
-  using DB::GetOptions;
-  virtual const Options& GetOptions(ColumnFamilyHandle* column_family) const
-      override {
-    return db_->GetOptions(column_family);
-  }
-
-  using DB::Flush;
-  virtual Status Flush(const FlushOptions& fopts,
-                       ColumnFamilyHandle* column_family) override {
-    return db_->Flush(fopts, column_family);
-  }
-
-  virtual Status DisableFileDeletions() override {
-    return db_->DisableFileDeletions();
-  }
-
-  virtual Status EnableFileDeletions(bool force) override {
-    return db_->EnableFileDeletions(force);
-  }
-
-  virtual Status GetLiveFiles(std::vector<std::string>& vec, uint64_t* mfs,
-                              bool flush_memtable = true) override {
-      return db_->GetLiveFiles(vec, mfs, flush_memtable);
-  }
-
-  virtual SequenceNumber GetLatestSequenceNumber() const override {
-    return db_->GetLatestSequenceNumber();
-  }
-
-  virtual Status GetSortedWalFiles(VectorLogPtr& files) override {
-    return db_->GetSortedWalFiles(files);
-  }
-
-  virtual Status DeleteFile(std::string name) override {
-    return db_->DeleteFile(name);
-  }
-
-  virtual Status GetDbIdentity(std::string& identity) {
-    return db_->GetDbIdentity(identity);
-  }
-
-  using DB::GetPropertiesOfAllTables;
-  virtual Status GetPropertiesOfAllTables(ColumnFamilyHandle* column_family,
-                                          TablePropertiesCollection* props) {
-    return db_->GetPropertiesOfAllTables(column_family, props);
-  }
-
-  virtual Status GetUpdatesSince(
-      SequenceNumber seq_number, unique_ptr<TransactionLogIterator>* iter,
-      const TransactionLogIterator::ReadOptions& read_options) override {
-    return db_->GetUpdatesSince(seq_number, iter, read_options);
-  }
-
-  virtual ColumnFamilyHandle* DefaultColumnFamily() const override {
-    return db_->DefaultColumnFamily();
-  }
-
- protected:
-  DB* db_;
-};
-
-} //  namespace rocksdb
+#warning This file was moved to rocksdb/utilities/stackable_db.h
+#include "rocksdb/utilities/stackable_db.h"
@@ -3,28 +3,5 @@
 // found in the LICENSE file. See the AUTHORS file for names of contributors.

 #pragma once
-#ifndef ROCKSDB_LITE
-#include <vector>
-#include <string>
-
-#include "utilities/stackable_db.h"
-#include "utilities/db_ttl.h"
-#include "rocksdb/db.h"
-
-namespace rocksdb {
-
-// Please don't use this class. It's deprecated
-class UtilityDB {
- public:
-  // This function is here only for backwards compatibility. Please use the
-  // functions defined in DBWithTTl (utilities/db_ttl.h)
-  // (deprecated)
-  __attribute__((deprecated)) static Status OpenTtlDB(const Options& options,
-                                                      const std::string& name,
-                                                      StackableDB** dbptr,
-                                                      int32_t ttl = 0,
-                                                      bool read_only = false);
-};
-
-} //  namespace rocksdb
-#endif  // ROCKSDB_LITE
+#warning This file was moved to rocksdb/utilities/utility_db.h
+#include "rocksdb/utilities/utility_db.h"
@@ -0,0 +1,54 @@
+# RocksJava Change Log
+
+## By 06/15/2014
+### New Features
+* Added basic Java binding for rocksdb::Env such that multiple RocksDB can share the same thread pool and environment.
+* Added RestoreBackupableDB
+
+## By 05/30/2014
+### Internal Framework Improvement
+* Added disOwnNativeHandle to RocksObject, which allows a RocksObject to give-up the ownership of its native handle.  This method is useful when sharing and transferring the ownership of RocksDB C++ resources.
+
+## By 05/15/2014
+### New Features
+* Added RocksObject --- the base class of all RocksDB classes which holds some RocksDB resources in the C++ side.
+* Use environmental variable JAVA_HOME in Makefile for RocksJava
+### Public API changes
+* Renamed org.rocksdb.Iterator to org.rocksdb.RocksIterator to avoid potential confliction with Java built-in Iterator.
+
+## By 04/30/2014
+### New Features
+* Added Java binding for MultiGet.
+* Added static method RocksDB.loadLibrary(), which loads necessary library files.
+* Added Java bindings for 60+ rocksdb::Options.
+* Added Java binding for BloomFilter.
+* Added Java binding for ReadOptions.
+* Added Java binding for memtables.
+* Added Java binding for sst formats.
+* Added Java binding for RocksDB Iterator which enables sequential scan operation.
+* Added Java binding for Statistics
+* Added Java binding for BackupableDB.
+
+### DB Benchmark
+* Added filluniquerandom, readseq benchmark.
+* 70+ command-line options.
+* Enabled BloomFilter configuration.
+
+## By 04/15/2014
+### New Features
+* Added Java binding for WriteOptions.
+* Added Java binding for WriteBatch, which enables batch-write.
+* Added Java binding for rocksdb::Options.
+* Added Java binding for block cache.
+* Added Java version DB Benchmark.
+
+### DB Benchmark
+* Added readwhilewriting benchmark.
+
+### Internal Framework Improvement
+* Avoid a potential byte-array-copy between c++ and Java in RocksDB.get.
+* Added SizeUnit in org.rocksdb.util to store consts like KB and GB.
+
+### 03/28/2014
+* RocksJava project started.
+* Added Java binding for RocksDB, which supports Open, Close, Get and Put.
@@ -10,7 +10,8 @@ clean:

 java:
 	javac org/rocksdb/util/*.java org/rocksdb/*.java
-	jar -cf $(ROCKSDB_JAR) org/rocksdb/*.class org/rocksdb/util/*.class
+	@cp ../HISTORY.md ./HISTORY-CPP.md
+	@rm -f ./HISTORY-CPP.md
 	javah -d $(NATIVE_INCLUDE) -jni $(NATIVE_JAVA_CLASSES)

 sample: java
@@ -27,6 +28,7 @@ test: java
 	java -ea -Djava.library.path=.:../ -cp "$(ROCKSDB_JAR):.:./*" org.rocksdb.test.BackupableDBTest
 	java -ea -Djava.library.path=.:../ -cp "$(ROCKSDB_JAR):.:./*" org.rocksdb.test.OptionsTest
 	java -ea -Djava.library.path=.:../ -cp "$(ROCKSDB_JAR):.:./*" org.rocksdb.test.ReadOptionsTest
+	java -ea -Djava.library.path=.:../ -cp "$(ROCKSDB_JAR):.:./*" org.rocksdb.test.StatisticsCollectorTest

 db_bench: java
 	javac org/rocksdb/benchmark/*.java
@@ -43,7 +43,10 @@ public class RocksDBSample {
        .setDisableSeekCompaction(true)
        .setBlockSize(64 * SizeUnit.KB)
        .setMaxBackgroundCompactions(10)
-        .setFilter(filter);
+        .setFilter(filter)
+        .setCacheNumShardBits(6)
+        .setCompressionType(CompressionType.SNAPPY_COMPRESSION)
+        .setCompactionStyle(CompactionStyle.UNIVERSAL);
    Statistics stats = options.statisticsPtr();

    assert(options.createIfMissing() == true);
@@ -52,6 +55,9 @@ public class RocksDBSample {
    assert(options.disableSeekCompaction() == true);
    assert(options.blockSize() == 64 * SizeUnit.KB);
    assert(options.maxBackgroundCompactions() == 10);
+    assert(options.cacheNumShardBits() == 6);
+    assert(options.compressionType() == CompressionType.SNAPPY_COMPRESSION);
+    assert(options.compactionStyle() == CompactionStyle.UNIVERSAL);

    assert(options.memTableFactoryName().equals("SkipListFactory"));
    options.setMemTableConfig(
@@ -38,10 +38,10 @@ public class BackupableDBOptions extends RocksObject {
      boolean destroyOldData, boolean backupLogFiles, long backupRateLimit,
      long restoreRateLimit) {
    super();
-    
+
    backupRateLimit = (backupRateLimit <= 0) ? 0 : backupRateLimit;
    restoreRateLimit = (restoreRateLimit <= 0) ? 0 : restoreRateLimit;
-    
+
    newBackupableDBOptions(path, shareTableFiles, sync, destroyOldData,
        backupLogFiles, backupRateLimit, restoreRateLimit);
  }
@@ -0,0 +1,22 @@
+// Copyright (c) 2014, Facebook, Inc.  All rights reserved.
+// This source code is licensed under the BSD-style license found in the
+// LICENSE file in the root directory of this source tree. An additional grant
+// of patent rights can be found in the PATENTS file in the same directory.
+
+package org.rocksdb;
+
+public enum CompactionStyle {
+  LEVEL((byte) 0),
+  UNIVERSAL((byte) 1),
+  FIFO((byte) 2);
+  
+  private final byte value_;
+
+  private CompactionStyle(byte value) {
+    value_ = value;
+  }
+
+  public byte getValue() {
+    return value_;
+  }
+}
@@ -0,0 +1,25 @@
+// Copyright (c) 2014, Facebook, Inc.  All rights reserved.
+// This source code is licensed under the BSD-style license found in the
+// LICENSE file in the root directory of this source tree. An additional grant
+// of patent rights can be found in the PATENTS file in the same directory.
+
+package org.rocksdb;
+
+public enum CompressionType {
+  NO_COMPRESSION((byte) 0),
+  SNAPPY_COMPRESSION((byte) 1),
+  ZLIB_COMPRESSION((byte) 2),
+  BZLIB2_COMPRESSION((byte) 3),
+  LZ4_COMPRESSION((byte) 4),
+  LZ4HC_COMPRESSION((byte) 5);
+  
+  private final byte value_;
+
+  private CompressionType(byte value) {
+    value_ = value;
+  }
+
+  public byte getValue() {
+    return value_;
+  }
+}
--- a/Show More
+++ b/Show More