jabberwocky238/rocksdb
1
0
Fork 0
mirror of https://github.com/facebook/rocksdb.git synced 2026-07-07 14:47:40 +08:00
Code Issues Packages Projects Releases Wiki Activity

Start development 11.4 (#14747)

Browse Source 
Summary:
* Release notes from 11.3 branch
* Update version.h
* Add [11.3.fb](https://github.com/facebook/rocksdb/tree/11.3.fb) (to check_format_compatible.sh)
* Update folly commit hash to: https://github.com/facebook/folly/releases/tag/v2026.05.11.00 (see [comment](https://github.com/facebook/rocksdb/pull/14747#issuecomment-4480217173) for more)
* Copyright headers refresh
* Decouple 11.1.fb from 11.2.fb in `db_forward_with_options_refs`
* Add validation step in `check_format_compatible.sh` to prevent gluey release strings

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14747

Reviewed By: xingbowang

Differential Revision: D105443941

Pulled By: mszeszko-meta

fbshipit-source-id: 8499b18cc2ff118d805b3865463ad2a999868de4
This commit is contained in:
Maciej Szeszko
2026-05-18 13:35:11 -07:00
committed by meta-codesync[bot]
parent c48b020e92
commit 459236341c
31 changed files with 110 additions and 36 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CMakeLists.txt
+38
View File
@@ -654,6 +654,8 @@ if(USE_FOLLY)
FMT_INST_PATH)
exec_program(ls ARGS -d ${FOLLY_INST_PATH}/../gflags* OUTPUT_VARIABLE
GFLAGS_INST_PATH)
exec_program(ls ARGS -d ${FOLLY_INST_PATH}/../glog* OUTPUT_VARIABLE
GLOG_INST_PATH)
set(Boost_DIR ${BOOST_INST_PATH}/lib/cmake/Boost-1.83.0)
if(EXISTS ${FMT_INST_PATH}/lib64)
set(fmt_DIR ${FMT_INST_PATH}/lib64/cmake/fmt)
@@ -661,6 +663,42 @@ if(USE_FOLLY)
set(fmt_DIR ${FMT_INST_PATH}/lib/cmake/fmt)
endif()
set(gflags_DIR ${GFLAGS_INST_PATH}/lib/cmake/gflags)
if(EXISTS ${GLOG_INST_PATH}/lib64/cmake/glog)
set(GLOG_CMAKE_DIR ${GLOG_INST_PATH}/lib64/cmake/glog)
elseif(EXISTS ${GLOG_INST_PATH}/lib/cmake/glog)
set(GLOG_CMAKE_DIR ${GLOG_INST_PATH}/lib/cmake/glog)
endif()
if(NOT GLOG_CMAKE_DIR)
find_library(GETDEPS_GLOG_LIBRARY NAMES glogd glog
PATHS ${GLOG_INST_PATH}/lib64 ${GLOG_INST_PATH}/lib
NO_DEFAULT_PATH)
if(NOT GETDEPS_GLOG_LIBRARY OR
NOT EXISTS ${GLOG_INST_PATH}/include/glog/logging.h)
message(FATAL_ERROR "Could not find getdeps glog under "
"${GLOG_INST_PATH}")
endif()
set(GLOG_CMAKE_DIR ${CMAKE_CURRENT_BINARY_DIR}/getdeps-glog-cmake)
file(MAKE_DIRECTORY ${GLOG_CMAKE_DIR})
file(WRITE ${GLOG_CMAKE_DIR}/glog-config.cmake
"if(NOT TARGET glog::glog)\n"
" add_library(glog::glog UNKNOWN IMPORTED)\n"
" set_target_properties(glog::glog PROPERTIES\n"
" IMPORTED_LOCATION \"${GETDEPS_GLOG_LIBRARY}\"\n"
" INTERFACE_INCLUDE_DIRECTORIES \"${GLOG_INST_PATH}/include\")\n"
"endif()\n"
"set(glog_FOUND TRUE)\n"
"set(Glog_FOUND TRUE)\n"
"set(GLOG_FOUND TRUE)\n"
"set(GLOG_LIBRARY \"${GETDEPS_GLOG_LIBRARY}\")\n"
"set(GLOG_LIBRARIES \"${GETDEPS_GLOG_LIBRARY}\")\n"
"set(GLOG_INCLUDE_DIR \"${GLOG_INST_PATH}/include\")\n")
endif()
set(glog_DIR ${GLOG_CMAKE_DIR})
set(Glog_DIR ${GLOG_CMAKE_DIR})
# Unlike gflags, glog may also be installed system-wide. Pre-resolve the
# getdeps copy so folly-config.cmake cannot fall back to another version.
find_package(glog CONFIG REQUIRED PATHS ${GLOG_CMAKE_DIR}
NO_DEFAULT_PATH)
exec_program(sed ARGS -i 's/gflags_shared//g'
${FOLLY_INST_PATH}/lib/cmake/folly/folly-targets.cmake)
HISTORY.md
+22
View File
@@ -1,6 +1,28 @@
# Rocksdb Change Log
> NOTE: Entries for next release do not go here. Follow instructions in `unreleased_history/README.txt`
## 11.3.0 (05/15/2026)
### New Features
* Add experimental DB option `async_wal_precreate` to precreate the next WAL file in a background thread and reduce foreground WAL rotation latency. The option is sanitized to false when WAL recycling is enabled.
* Added a new `EventListener::OnCompactionPreCommit` callback that fires after a compaction job finishes but before its input files are released (i.e. while `FileMetaData::being_compacted` is still true). Listeners that maintain bookkeeping of which files are currently being compacted can clean up such state in this new callback to avoid races with concurrent compaction picking, where another thread might pick up the same files for a new compaction immediately after `being_compacted` is flipped back to false but before `OnCompactionCompleted` fires. The default implementation is a no-op so this is not a breaking change.
* Add mutable DBOption `optimize_manifest_for_recovery` (default false). When enabled, RocksDB can reduce recovery work after a clean shutdown, which may lower DB::Open latency on warm reopens.
* Added public utility APIs `ParseCompressionNameForDisplay()` to convert `TableProperties::compression_name` into a human-readable compression name for both legacy and format_version 7+ SST metadata, including custom `CompressionManager`-provided display names for custom compression types.
* Add `reuse_manifest_on_open` DBOption (default false). When enabled, DB::Open reuses the existing MANIFEST file for append instead of creating a fresh one, avoiding the cost of serializing the entire database state into a new MANIFEST on the first post-open write. To prevent this feature from interfering with manifest file size auto-tuning, an extra forward-compatible field is now always added to the MANIFEST (to track the last "compacted" size).
### Behavior Changes
* Read-only open with `error_if_wal_file_exists=true` now tolerates empty WAL files so empty precreated WALs do not prevent inspection.
* WriteCommitted TransactionDB now matches WritePrepared and WriteUnprepared compaction filtering in both single- and two-write-queue modes: a compaction filter's FilterMergeOperand will not be invoked on merge operands at or below the latest published sequence number.
### Bug Fixes
* Fixed blob-backed wide-column merge reads to preserve correct status
propagation and resolution across memtable, read-only, and secondary DB
paths.
* Fixed a bug where `DB::GetCreationTimeOfOldestFile()` could return inaccurate results instead of the real creation time when called shortly after opening a legacy DB (one whose manifest lacks `file_creation_time`) with `open_files_async = true`. The API now waits for background SST file loading to complete only when needed; modern DBs are unaffected.
* Fixed merge reads against wide-column/blob-backed base values to preserve precise failure statuses, including `GetMergeOperands()` and direct-write memtable reads.
* Fix bug in range tombstone synthesis that covers live keys added during an IngestExternalFile
* Reject the empty string as a column family name in `DB::CreateColumnFamily` / `DB::CreateColumnFamilies`. Previously such calls returned OK and a usable handle, but the column family was not persisted in the manifest, so any data written to it was silently lost on DB reopen.
* Fixed a bug where a WriteCommitted TransactionDB using commit-bypass WBWI ingestion could drop an entry that is still visible at the published sequence boundary.
## 11.2.0 (04/18/2026)
### New Features
* Added experimental `DBOptions::fast_sst_open` option. When enabled, RocksDB retrieves opaque file system metadata for SST files after flush, compaction, and external file ingestion, persists it in the MANIFEST, and passes it back to the file system on subsequent file opens to accelerate DB open time.
build_tools/getdeps_fallback_mirror.py
+3
View File
@@ -1,4 +1,7 @@
#!/usr/bin/env python3
# Copyright (c) Meta Platforms, Inc. and affiliates.
# This source code is licensed under both the GPLv2 (found in the COPYING file in the root directory)
# and the Apache 2.0 License (found in the LICENSE.Apache file in the root directory).
"""
Pre-download packages with unreliable mirrors using fallback mirrors.
Reads package info from folly's getdeps manifest files.
db/db_etc3_test.cc
+1 -1
View File
@@ -1,4 +1,4 @@
// Copyright (c) 2011-present, Facebook, Inc. All rights reserved.
// Copyright (c) Meta Platforms, Inc. and affiliates.
// This source code is licensed under both the GPLv2 (found in the
// COPYING file in the root directory) and Apache 2.0 License
// (found in the LICENSE.Apache file in the root directory).
db_stress_tool/db_stress_compaction_service.cc
+1 -1
View File
@@ -1,4 +1,4 @@
// Copyright (c) 2011-present, Facebook, Inc. All rights reserved.
// Copyright (c) Meta Platforms, Inc. and affiliates.
// This source code is licensed under both the GPLv2 (found in the
// COPYING file in the root directory) and Apache 2.0 License
// (found in the LICENSE.Apache file in the root directory).
folly.mk
+1 -1
View File
@@ -98,7 +98,7 @@ endif # FMT_SOURCE_PATH
PLATFORM_LDFLAGS += -lglog
endif
FOLLY_COMMIT_HASH = 3b0eed0a128c484a2b077546c3e08c824a311357
FOLLY_COMMIT_HASH = 548b16da0b3cc887d69cdb6ae06496ad8a2a9276
# For public CI runs, checkout folly in a way that can build with RocksDB.
# This is mostly intended as a test-only simulation of Meta-internal folly
include/rocksdb/blob_file_partition_strategy.h
+2 -1
View File
@@ -1,7 +1,8 @@
// Copyright (c) 2011-present, Facebook, Inc. All rights reserved.
// Copyright (c) Meta Platforms, Inc. and affiliates.
// This source code is licensed under both the GPLv2 (found in the
// COPYING file in the root directory) and Apache 2.0 License
// (found in the LICENSE.Apache file in the root directory).
// Copyright (c) 2011 The LevelDB Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file. See the AUTHORS file for names of contributors.
include/rocksdb/perf_context.h
+1 -1
View File
@@ -106,7 +106,7 @@ struct PerfContextBase {
uint64_t compressed_sec_cache_compressed_bytes;
uint64_t block_checksum_time; // total nanos spent on block checksum
uint64_t block_decompress_time; // total nanos spent on block decompression
uint64_t block_decompress_time; // total nanos spent on block decompression
uint64_t block_decompress_count; // total number of block decompressions
uint64_t get_read_bytes; // bytes for vals returned by Get
include/rocksdb/utilities/sorted_run_builder.h
+1 -1
View File
@@ -1,4 +1,4 @@
// Copyright (c) 2011-present, Facebook, Inc. All rights reserved.
// Copyright (c) Meta Platforms, Inc. and affiliates.
// This source code is licensed under both the GPLv2 (found in the
// COPYING file in the root directory) and Apache 2.0 License
// (found in the LICENSE.Apache file in the root directory).
include/rocksdb/version.h
+1 -1
View File
@@ -12,7 +12,7 @@
// NOTE: in 'main' development branch, this should be the *next*
// minor or major version number planned for release.
#define ROCKSDB_MAJOR 11
#define ROCKSDB_MINOR 3
#define ROCKSDB_MINOR 4
#define ROCKSDB_PATCH 0
// Make it easy to do conditional compilation based on version checks, i.e.
table/block_based/block_util.h
+1 -1
View File
@@ -1,4 +1,4 @@
// Copyright (c) 2011-present, Facebook, Inc. All rights reserved.
// Copyright (c) Meta Platforms, Inc. and affiliates.
// This source code is licensed under both the GPLv2 (found in the
// COPYING file in the root directory) and Apache 2.0 License
// (found in the LICENSE.Apache file in the root directory).
tools/check_format_compatible.sh
+29 -7
View File
@@ -148,7 +148,7 @@ EOF
# To check for DB forward compatibility with loading options (old version
# reading data from new), as well as backward compatibility
declare -a db_forward_with_options_refs=("10.4.fb" "10.5.fb" "10.6.fb" "10.7.fb" "10.8.fb" "10.9.fb" "10.10.fb" "10.11.fb" "11.0.fb" "11.1.fb 11.2.fb")
declare -a db_forward_with_options_refs=("10.4.fb" "10.5.fb" "10.6.fb" "10.7.fb" "10.8.fb" "10.9.fb" "10.10.fb" "10.11.fb" "11.0.fb" "11.1.fb" "11.2.fb" "11.3.fb")
# To check for DB forward compatibility without loading options (in addition
# to the "with loading options" set), as well as backward compatibility
declare -a db_forward_no_options_refs=() # N/A at the moment
@@ -174,6 +174,28 @@ declare -a bak_forward_refs=("${db_forward_no_options_refs[@]}" "${db_forward_wi
# (when format_version=2 became the default)
declare -a db_backward_only_refs=("4.6.fb" "4.7.fb" "4.8.fb" "4.9.fb" "4.10.fb" "${bak_backward_only_refs[@]}")
validate_release_refs()
{
local list_name="$1"
local checkout_ref
shift
for checkout_ref in "$@"
do
if [[ ! "$checkout_ref" =~ ^[0-9]+\.[0-9]+\.fb$ ]]; then
echo "Invalid ref in ${list_name}: '${checkout_ref}'" >&2
exit 1
fi
done
}
validate_release_refs "db_backward_only_refs" "${db_backward_only_refs[@]}"
validate_release_refs "db_forward_no_options_refs" "${db_forward_no_options_refs[@]}"
validate_release_refs "db_forward_with_options_refs" "${db_forward_with_options_refs[@]}"
validate_release_refs "ext_backward_only_refs" "${ext_backward_only_refs[@]}"
validate_release_refs "ext_forward_refs" "${ext_forward_refs[@]}"
validate_release_refs "bak_backward_only_refs" "${bak_backward_only_refs[@]}"
validate_release_refs "bak_forward_refs" "${bak_forward_refs[@]}"
if [ "$SHORT_TEST" ]; then
# Use only the first (if exists) of each list
db_backward_only_refs=(${db_backward_only_refs[0]})
@@ -190,20 +212,20 @@ declare -a checkout_refs=()
# Always include first release on each list
for checkout_ref in "${db_backward_only_refs[0]}" "${db_forward_no_options_refs[0]}" "${db_forward_with_options_refs[0]}" "${ext_backward_only_refs[0]}" "${ext_forward_refs[0]}" "${bak_backward_only_refs[0]}" "${bak_forward_refs[0]}"
do
if [ ! -e $db_test_dir/$checkout_ref ]; then
mkdir -p $db_test_dir/$checkout_ref
checkout_refs+=($checkout_ref)
if [ ! -e "$db_test_dir/$checkout_ref" ]; then
mkdir -p "$db_test_dir/$checkout_ref"
checkout_refs+=("$checkout_ref")
fi
done
# Maybe include rest if not short test
if [ "$SHORT_TEST" == "" ]; then
for checkout_ref in "${db_backward_only_refs[@]}" "${db_forward_no_options_refs[@]}" "${db_forward_with_options_refs[@]}" "${ext_backward_only_refs[@]}" "${ext_forward_refs[@]}" "${bak_backward_only_refs[@]}" "${bak_forward_refs[@]}"
do
if [ ! -e $db_test_dir/$checkout_ref ]; then
mkdir -p $db_test_dir/$checkout_ref
if [ ! -e "$db_test_dir/$checkout_ref" ]; then
mkdir -p "$db_test_dir/$checkout_ref"
# Randomly sample remaining releases, unless long test
if [ "$LONG_TEST" ] || [ "$(($RANDOM % 3))" == "0" ]; then
checkout_refs+=($checkout_ref)
checkout_refs+=("$checkout_ref")
fi
fi
done
tools/run_clang_tidy.py
+3
View File
@@ -1,4 +1,7 @@
#!/usr/bin/env python3
# Copyright (c) Meta Platforms, Inc. and affiliates.
# This source code is licensed under both the GPLv2 (found in the COPYING file in the root directory)
# and the Apache 2.0 License (found in the LICENSE.Apache file in the root directory).
"""
Run clang-tidy on locally changed code and filter results to changed lines.
unreleased_history/behavior_changes/error_if_wal_file_exists_empty_wals.md
-1
View File
@@ -1 +0,0 @@
Read-only open with `error_if_wal_file_exists=true` now tolerates empty WAL files so empty precreated WALs do not prevent inspection.
unreleased_history/behavior_changes/write_committed_merge_operand_filtering
-1
View File
@@ -1 +0,0 @@
WriteCommitted TransactionDB now matches WritePrepared and WriteUnprepared compaction filtering in both single- and two-write-queue modes: a compaction filter's FilterMergeOperand will not be invoked on merge operands at or below the latest published sequence number.
unreleased_history/bug_fixes/blob_backed_wide_column_merge_reads.md
-3
View File
@@ -1,3 +0,0 @@
Fixed blob-backed wide-column merge reads to preserve correct status
propagation and resolution across memtable, read-only, and secondary DB
paths.
unreleased_history/bug_fixes/get_creation_time_of_oldest_file_async_open
-1
View File
@@ -1 +0,0 @@
Fixed a bug where `DB::GetCreationTimeOfOldestFile()` could return inaccurate results instead of the real creation time when called shortly after opening a legacy DB (one whose manifest lacks `file_creation_time`) with `open_files_async = true`. The API now waits for background SST file loading to complete only when needed; modern DBs are unaffected.
unreleased_history/bug_fixes/merge_reads_preserve_precise_statuses.md
-1
View File
@@ -1 +0,0 @@
Fixed merge reads against wide-column/blob-backed base values to preserve precise failure statuses, including `GetMergeOperands()` and direct-write memtable reads.
unreleased_history/bug_fixes/range_tombstone_synthesis
-1
View File
@@ -1 +0,0 @@
Fix bug in range tombstone synthesis that covers live keys added during an IngestExternalFile
unreleased_history/bug_fixes/reject_empty_cf_name.md
-1
View File
@@ -1 +0,0 @@
Reject the empty string as a column family name in `DB::CreateColumnFamily` / `DB::CreateColumnFamilies`. Previously such calls returned OK and a usable handle, but the column family was not persisted in the manifest, so any data written to it was silently lost on DB reopen.
unreleased_history/bug_fixes/write_committed_wbwi_ingestion_missing_entry
-1
View File
@@ -1 +0,0 @@
Fixed a bug where a WriteCommitted TransactionDB using commit-bypass WBWI ingestion could drop an entry that is still visible at the published sequence boundary.
unreleased_history/new_features/async_wal_precreate.md
-1
View File
@@ -1 +0,0 @@
Add experimental DB option `async_wal_precreate` to precreate the next WAL file in a background thread and reduce foreground WAL rotation latency. The option is sanitized to false when WAL recycling is enabled.
unreleased_history/new_features/oncompaction_pre_commit_listener.md
-1
View File
@@ -1 +0,0 @@
Added a new `EventListener::OnCompactionPreCommit` callback that fires after a compaction job finishes but before its input files are released (i.e. while `FileMetaData::being_compacted` is still true). Listeners that maintain bookkeeping of which files are currently being compacted can clean up such state in this new callback to avoid races with concurrent compaction picking, where another thread might pick up the same files for a new compaction immediately after `being_compacted` is flipped back to false but before `OnCompactionCompleted` fires. The default implementation is a no-op so this is not a breaking change.
unreleased_history/new_features/optimize_manifest_for_recovery.md
-1
View File
@@ -1 +0,0 @@
Add mutable DBOption `optimize_manifest_for_recovery` (default false). When enabled, RocksDB can reduce recovery work after a clean shutdown, which may lower DB::Open latency on warm reopens.
unreleased_history/new_features/parse_compression_name_for_display.md
-1
View File
@@ -1 +0,0 @@
Added public utility APIs `ParseCompressionNameForDisplay()` to convert `TableProperties::compression_name` into a human-readable compression name for both legacy and format_version 7+ SST metadata, including custom `CompressionManager`-provided display names for custom compression types.
unreleased_history/new_features/reuse_manifest_on_open.md
-1
View File
@@ -1 +0,0 @@
Add `reuse_manifest_on_open` DBOption (default false). When enabled, DB::Open reuses the existing MANIFEST file for append instead of creating a fresh one, avoiding the cost of serializing the entire database state into a new MANIFEST on the first post-open write. To prevent this feature from interfering with manifest file size auto-tuning, an extra forward-compatible field is now always added to the MANIFEST (to track the last "compacted" size).
util/interval_test.cc
+1 -1
View File
@@ -1,4 +1,4 @@
// Copyright (c) 2011-present, Facebook, Inc. All rights reserved.
// Copyright (c) Meta Platforms, Inc. and affiliates.
// This source code is licensed under both the GPLv2 (found in the
// COPYING file in the root directory) and Apache 2.0 License
// (found in the LICENSE.Apache file in the root directory).
utilities/fault_injection_fs_test.cc
+1 -1
View File
@@ -1,4 +1,4 @@
// Copyright (c) 2011-present, Facebook, Inc. All rights reserved.
// Copyright (c) Meta Platforms, Inc. and affiliates.
// This source code is licensed under both the GPLv2 (found in the
// COPYING file in the root directory) and Apache 2.0 License
// (found in the LICENSE.Apache file in the root directory).
utilities/sorted_run_builder/sorted_run_builder.cc
+1 -1
View File
@@ -1,4 +1,4 @@
// Copyright (c) 2011-present, Facebook, Inc. All rights reserved.
// Copyright (c) Meta Platforms, Inc. and affiliates.
// This source code is licensed under both the GPLv2 (found in the
// COPYING file in the root directory) and Apache 2.0 License
// (found in the LICENSE.Apache file in the root directory).
utilities/sorted_run_builder/sorted_run_builder_test.cc
+1 -1
View File
@@ -1,4 +1,4 @@
// Copyright (c) 2011-present, Facebook, Inc. All rights reserved.
// Copyright (c) Meta Platforms, Inc. and affiliates.
// This source code is licensed under both the GPLv2 (found in the
// COPYING file in the root directory) and Apache 2.0 License
// (found in the LICENSE.Apache file in the root directory).
utilities/transactions/write_prepared_transaction_seqno_test.cc
+1 -1
View File
@@ -1,4 +1,4 @@
// Copyright (c) 2011-present, Facebook, Inc. All rights reserved.
// Copyright (c) Meta Platforms, Inc. and affiliates.
// This source code is licensed under both the GPLv2 (found in the
// COPYING file in the root directory) and Apache 2.0 License
// (found in the LICENSE.Apache file in the root directory).