mirror of
https://github.com/facebook/rocksdb.git
synced 2026-07-07 14:47:40 +08:00
zaidoon
1cec28d82d
Summary: This continues and finishes **https://github.com/facebook/rocksdb/issues/14572** ("Add semi-automated code generation for RocksDB C API bindings") by xingbowang. The original author is unavailable to finish it, so I've taken it over. **All 13 of the original commits are preserved** (this branch was created from the PR head and builds on top of it — `git log` shows the original `Xingbo Wang` authorship intact); my follow-up work is in the commits prefixed `C API codegen:`. The underlying design is unchanged and is the original author's: hand-written source templates (`tools/c_api_gen/c_base.h` / `c_base.cc`) plus two generators (auto-discovery from the C++ headers + a spec-driven generator) are inlined into a single, self-contained, `generated` `include/rocksdb/c.h` and `db/c.cc`. This grows the public C API by **668 functions** while keeping `c.h` a single includable header with no `-I` requirement (so `bindgen` and other FFI tools keep working unchanged). This branch reconciles the PR with ~4 months of `main` and addresses the outstanding review feedback (clang-tidy bot, the automated code review, the `c.h` self-containedness discussion, and pdillinger's points about `include/rocksdb` hygiene and `generated` marking). ## What changed on top of the original PR ### Reconciled with current `main` - Merged current `main` (conflicts were confined to the generated/test files) and regenerated. Reconciled the 14 C API functions `main` added since the merge-base (e.g. `rocksdb_set_db_options`, the backup-engine rate limiters, `memtable_batch_lookup_optimization`, `optimize_multiget_for_io`, …) and restored 5 enum constants that upstream had added by hand (`rocksdb_txndb_write_policy_*`, `..._index_block_search_type_auto`, `rocksdb_blob_cache_read_byte`). ### Maintainer feedback (pdillinger) - **No non-user-includable files in `include/rocksdb`.** Moved the hand-written templates out of `include/rocksdb/` and `db/` to `tools/c_api_gen/c_base.{h,cc}`. They were `#include`-ing generated fragments, which broke `make check-headers` and was shipped by `make install`. `include/rocksdb/` now contains only the user-facing, self-contained, `generated` `c.h`. - `c.h` / `c.cc` carry the `// generated` marker. ### Backward compatibility (zero ABI break) - The generator derived each wrapper's C type purely from the C++ field, which had silently changed **5 already-shipped signatures** (e.g. `rocksdb_writeoptions_disable_WAL` `int` → `unsigned char`). Added an ABI type-pinning layer (`tools/c_api_gen/abi_type_overrides.json`) so already-shipped functions keep their historical C signature (the body still casts to the real field type). A repo-wide diff against the merge-base now reports **0 ABI drift**. - New `check_api_compatibility.py` gate (wired into CI + `make`) fails on any removed/changed public function **or** removed enum/typedef symbol, vs a reference revision. Intentional changes go in an allowlist with a reason. ### Correctness (from the automated review) - Restored 5 option setters that were declared in `c.h` but **defined nowhere** (link failure for downstream bindings such as `rust-rocksdb`). Added `check_api_completeness.py` (dependency-free; runs in CI + `make`) asserting every declared function has exactly one definition — this is the gate that would have caught it. - `CopyStringVector` now null-checks `malloc`; the WAL filter `std::move`s the `WriteBatch`; the backup exclude-files callback captures by value instead of the wrapper pointer. ### Build / CI robustness - Removed the dead `C_API_CODEGEN_STAMP` Makefile prerequisite (it was a silent no-op). - The `make check` staleness check is now opt-out-able (behind `SKIP_FORMAT_BUCK_CHECKS`) and skips gracefully when `clang++` is unavailable, so `make check` works without the codegen toolchain. CI remains the authoritative gate. - Pinned `clang-format` consistently through `regen_all.py` / `verify_generated_up_to_date.py` (CI uses clang-format-21) so regeneration is byte-reproducible across environments. - Cleared all 20 `clang-tidy` warnings the bot reported on `db/c.cc` changed lines (fixed in the `c_base.cc` template, not the generated output). - Updated the internal Buck `c_test_bin` wrapper to expose generated `c_api_gen/*.inc` fragments as headers, so sandboxed Buck builds can compile `db/c_test.c` after the generated round-trip tests are included. ### Test coverage - Added `gen_roundtrip_tests.py`, which derives **462 set→get→assert round-trip checks across 25 option objects** from the same generated fragments and wires them into `db/c_test.c`. Coverage now tracks the generated surface automatically. ### Docs - Added the `unreleased_history/public_api_changes` note and fixed `claude_md/add_public_api.md`, which still told contributors to hand-edit the now-`generated` `c.h`/`c.cc`. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14868 Test Plan: - `make c_test && ./c_test` — **passes**, including the 462 generated round-trip assertions (a successful link also confirms the API is complete). - `python3 tools/c_api_gen/check_api_completeness.py` — all 1737 declared functions defined exactly once. - `python3 tools/c_api_gen/check_api_compatibility.py --ref <release>` — 1070 reference functions + 229 enum/typedef symbols preserved, 0 removed/changed. - `python3 tools/c_api_gen/verify_generated_up_to_date.py` — generated output is stable. - `include/rocksdb/c.h` confirmed self-contained (only `<stdbool.h>`, `<stddef.h>`, `<stdint.h>`). cc xingbowang - `buck2 build --flagfile fbcode//mode/dev fbcode//internal_repo_rocksdb/repo:c_test_bin` — passes. - `buck2 build --flagfile fbcode//mode/dev --config fbcode.arch=aarch64 fbcode//internal_repo_rocksdb/repo:c_test_bin` — passes. Reviewed By: pdillinger Differential Revision: D109149150 Pulled By: xingbowang fbshipit-source-id: 3417375345f360a4c78bdfe27e9850b89d0a226a
372 lines
16 KiB
C++
372 lines
16 KiB
C++
// @generated
|
|
// -----------------------------------------------------------------------------
|
|
// Auto-generated by tools/c_api_gen/generate_c_api.py.
|
|
// DO NOT EDIT THIS FILE DIRECTLY.
|
|
// Source: tools/c_api_gen/spec.json
|
|
// -----------------------------------------------------------------------------
|
|
|
|
// To regenerate this file:
|
|
// python3 tools/c_api_gen/generate_c_api.py
|
|
// --spec tools/c_api_gen/spec.json --section 'DB data operations'
|
|
// --section 'WriteBatch' --section 'Transaction'
|
|
// --section 'TransactionDB simple'
|
|
// --header-out
|
|
// c_api_gen/c_generated_subset.h.inc
|
|
// --source-out
|
|
// c_api_gen/c_generated_subset.cc.inc
|
|
|
|
/* DB data operations */
|
|
|
|
void rocksdb_put(rocksdb_t* db, const rocksdb_writeoptions_t* options,
|
|
const char* key, size_t keylen, const char* val, size_t vallen,
|
|
char** errptr) {
|
|
SaveError(errptr,
|
|
db->rep->Put(options->rep, Slice(key, keylen), Slice(val, vallen)));
|
|
}
|
|
|
|
void rocksdb_put_cf(rocksdb_t* db, const rocksdb_writeoptions_t* options,
|
|
rocksdb_column_family_handle_t* column_family,
|
|
const char* key, size_t keylen, const char* val,
|
|
size_t vallen, char** errptr) {
|
|
SaveError(errptr, db->rep->Put(options->rep, column_family->rep,
|
|
Slice(key, keylen), Slice(val, vallen)));
|
|
}
|
|
|
|
void rocksdb_delete(rocksdb_t* db, const rocksdb_writeoptions_t* options,
|
|
const char* key, size_t keylen, char** errptr) {
|
|
SaveError(errptr, db->rep->Delete(options->rep, Slice(key, keylen)));
|
|
}
|
|
|
|
void rocksdb_delete_cf(rocksdb_t* db, const rocksdb_writeoptions_t* options,
|
|
rocksdb_column_family_handle_t* column_family,
|
|
const char* key, size_t keylen, char** errptr) {
|
|
SaveError(errptr, db->rep->Delete(options->rep, column_family->rep,
|
|
Slice(key, keylen)));
|
|
}
|
|
|
|
void rocksdb_merge(rocksdb_t* db, const rocksdb_writeoptions_t* options,
|
|
const char* key, size_t keylen, const char* val,
|
|
size_t vallen, char** errptr) {
|
|
SaveError(errptr, db->rep->Merge(options->rep, Slice(key, keylen),
|
|
Slice(val, vallen)));
|
|
}
|
|
|
|
void rocksdb_merge_cf(rocksdb_t* db, const rocksdb_writeoptions_t* options,
|
|
rocksdb_column_family_handle_t* column_family,
|
|
const char* key, size_t keylen, const char* val,
|
|
size_t vallen, char** errptr) {
|
|
SaveError(errptr, db->rep->Merge(options->rep, column_family->rep,
|
|
Slice(key, keylen), Slice(val, vallen)));
|
|
}
|
|
|
|
void rocksdb_write(rocksdb_t* db, const rocksdb_writeoptions_t* options,
|
|
rocksdb_writebatch_t* batch, char** errptr) {
|
|
SaveError(errptr, db->rep->Write(options->rep, &batch->rep));
|
|
}
|
|
|
|
void rocksdb_put_with_ts(rocksdb_t* db, const rocksdb_writeoptions_t* options,
|
|
const char* key, size_t keylen, const char* ts,
|
|
size_t tslen, const char* val, size_t vallen,
|
|
char** errptr) {
|
|
SaveError(errptr, db->rep->Put(options->rep, Slice(key, keylen),
|
|
Slice(ts, tslen), Slice(val, vallen)));
|
|
}
|
|
|
|
void rocksdb_put_cf_with_ts(rocksdb_t* db,
|
|
const rocksdb_writeoptions_t* options,
|
|
rocksdb_column_family_handle_t* column_family,
|
|
const char* key, size_t keylen, const char* ts,
|
|
size_t tslen, const char* val, size_t vallen,
|
|
char** errptr) {
|
|
SaveError(errptr,
|
|
db->rep->Put(options->rep, column_family->rep, Slice(key, keylen),
|
|
Slice(ts, tslen), Slice(val, vallen)));
|
|
}
|
|
|
|
void rocksdb_delete_with_ts(rocksdb_t* db,
|
|
const rocksdb_writeoptions_t* options,
|
|
const char* key, size_t keylen, const char* ts,
|
|
size_t tslen, char** errptr) {
|
|
SaveError(errptr, db->rep->Delete(options->rep, Slice(key, keylen),
|
|
Slice(ts, tslen)));
|
|
}
|
|
|
|
void rocksdb_delete_cf_with_ts(rocksdb_t* db,
|
|
const rocksdb_writeoptions_t* options,
|
|
rocksdb_column_family_handle_t* column_family,
|
|
const char* key, size_t keylen, const char* ts,
|
|
size_t tslen, char** errptr) {
|
|
SaveError(errptr, db->rep->Delete(options->rep, column_family->rep,
|
|
Slice(key, keylen), Slice(ts, tslen)));
|
|
}
|
|
|
|
void rocksdb_singledelete(rocksdb_t* db, const rocksdb_writeoptions_t* options,
|
|
const char* key, size_t keylen, char** errptr) {
|
|
SaveError(errptr, db->rep->SingleDelete(options->rep, Slice(key, keylen)));
|
|
}
|
|
|
|
void rocksdb_singledelete_cf(rocksdb_t* db,
|
|
const rocksdb_writeoptions_t* options,
|
|
rocksdb_column_family_handle_t* column_family,
|
|
const char* key, size_t keylen, char** errptr) {
|
|
SaveError(errptr, db->rep->SingleDelete(options->rep, column_family->rep,
|
|
Slice(key, keylen)));
|
|
}
|
|
|
|
void rocksdb_singledelete_with_ts(rocksdb_t* db,
|
|
const rocksdb_writeoptions_t* options,
|
|
const char* key, size_t keylen,
|
|
const char* ts, size_t tslen, char** errptr) {
|
|
SaveError(errptr, db->rep->SingleDelete(options->rep, Slice(key, keylen),
|
|
Slice(ts, tslen)));
|
|
}
|
|
|
|
void rocksdb_singledelete_cf_with_ts(
|
|
rocksdb_t* db, const rocksdb_writeoptions_t* options,
|
|
rocksdb_column_family_handle_t* column_family, const char* key,
|
|
size_t keylen, const char* ts, size_t tslen, char** errptr) {
|
|
SaveError(errptr,
|
|
db->rep->SingleDelete(options->rep, column_family->rep,
|
|
Slice(key, keylen), Slice(ts, tslen)));
|
|
}
|
|
|
|
void rocksdb_delete_range_cf(rocksdb_t* db,
|
|
const rocksdb_writeoptions_t* options,
|
|
rocksdb_column_family_handle_t* column_family,
|
|
const char* start_key, size_t start_key_len,
|
|
const char* end_key, size_t end_key_len,
|
|
char** errptr) {
|
|
SaveError(errptr, db->rep->DeleteRange(options->rep, column_family->rep,
|
|
Slice(start_key, start_key_len),
|
|
Slice(end_key, end_key_len)));
|
|
}
|
|
|
|
void rocksdb_flush(rocksdb_t* db, const rocksdb_flushoptions_t* options,
|
|
char** errptr) {
|
|
SaveError(errptr, db->rep->Flush(options->rep));
|
|
}
|
|
|
|
void rocksdb_flush_cf(rocksdb_t* db, const rocksdb_flushoptions_t* options,
|
|
rocksdb_column_family_handle_t* column_family,
|
|
char** errptr) {
|
|
SaveError(errptr, db->rep->Flush(options->rep, column_family->rep));
|
|
}
|
|
|
|
void rocksdb_flush_wal(rocksdb_t* db, unsigned char sync, char** errptr) {
|
|
SaveError(errptr, db->rep->FlushWAL(sync));
|
|
}
|
|
|
|
void rocksdb_pause_background_work(rocksdb_t* db, char** errptr) {
|
|
SaveError(errptr, db->rep->PauseBackgroundWork());
|
|
}
|
|
|
|
void rocksdb_continue_background_work(rocksdb_t* db, char** errptr) {
|
|
SaveError(errptr, db->rep->ContinueBackgroundWork());
|
|
}
|
|
|
|
void rocksdb_disable_file_deletions(rocksdb_t* db, char** errptr) {
|
|
SaveError(errptr, db->rep->DisableFileDeletions());
|
|
}
|
|
|
|
void rocksdb_enable_file_deletions(rocksdb_t* db, char** errptr) {
|
|
SaveError(errptr, db->rep->EnableFileDeletions());
|
|
}
|
|
|
|
void rocksdb_verify_checksum(rocksdb_t* db, char** errptr) {
|
|
SaveError(errptr, db->rep->VerifyChecksum());
|
|
}
|
|
|
|
void rocksdb_verify_file_checksums(rocksdb_t* db, char** errptr) {
|
|
SaveError(errptr, db->rep->VerifyFileChecksums(ReadOptions()));
|
|
}
|
|
|
|
void rocksdb_destroy_db(const rocksdb_options_t* options, const char* name,
|
|
char** errptr) {
|
|
SaveError(errptr, DestroyDB(name, options->rep));
|
|
}
|
|
|
|
void rocksdb_repair_db(const rocksdb_options_t* options, const char* name,
|
|
char** errptr) {
|
|
SaveError(errptr, RepairDB(name, options->rep));
|
|
}
|
|
|
|
/* WriteBatch */
|
|
|
|
void rocksdb_writebatch_clear(rocksdb_writebatch_t* b) { b->rep.Clear(); }
|
|
|
|
void rocksdb_writebatch_put(rocksdb_writebatch_t* b, const char* key,
|
|
size_t klen, const char* val, size_t vlen) {
|
|
b->rep.Put(Slice(key, klen), Slice(val, vlen));
|
|
}
|
|
|
|
void rocksdb_writebatch_put_cf(rocksdb_writebatch_t* b,
|
|
rocksdb_column_family_handle_t* column_family,
|
|
const char* key, size_t klen, const char* val,
|
|
size_t vlen) {
|
|
b->rep.Put(column_family->rep, Slice(key, klen), Slice(val, vlen));
|
|
}
|
|
|
|
void rocksdb_writebatch_delete(rocksdb_writebatch_t* b, const char* key,
|
|
size_t klen) {
|
|
b->rep.Delete(Slice(key, klen));
|
|
}
|
|
|
|
void rocksdb_writebatch_put_log_data(rocksdb_writebatch_t* b, const char* blob,
|
|
size_t len) {
|
|
b->rep.PutLogData(Slice(blob, len));
|
|
}
|
|
|
|
void rocksdb_writebatch_set_save_point(rocksdb_writebatch_t* b) {
|
|
b->rep.SetSavePoint();
|
|
}
|
|
|
|
void rocksdb_writebatch_rollback_to_save_point(rocksdb_writebatch_t* b,
|
|
char** errptr) {
|
|
SaveError(errptr, b->rep.RollbackToSavePoint());
|
|
}
|
|
|
|
void rocksdb_writebatch_pop_save_point(rocksdb_writebatch_t* b, char** errptr) {
|
|
SaveError(errptr, b->rep.PopSavePoint());
|
|
}
|
|
|
|
void rocksdb_writebatch_verify_checksum(rocksdb_writebatch_t* b,
|
|
char** errptr) {
|
|
SaveError(errptr, b->rep.VerifyChecksum());
|
|
}
|
|
|
|
/* Transaction */
|
|
|
|
void rocksdb_transaction_put(rocksdb_transaction_t* txn, const char* key,
|
|
size_t klen, const char* val, size_t vlen,
|
|
char** errptr) {
|
|
SaveError(errptr, txn->rep->Put(Slice(key, klen), Slice(val, vlen)));
|
|
}
|
|
|
|
void rocksdb_transaction_put_cf(rocksdb_transaction_t* txn,
|
|
rocksdb_column_family_handle_t* column_family,
|
|
const char* key, size_t klen, const char* val,
|
|
size_t vlen, char** errptr) {
|
|
SaveError(errptr, txn->rep->Put(column_family->rep, Slice(key, klen),
|
|
Slice(val, vlen)));
|
|
}
|
|
|
|
void rocksdb_transaction_merge(rocksdb_transaction_t* txn, const char* key,
|
|
size_t klen, const char* val, size_t vlen,
|
|
char** errptr) {
|
|
SaveError(errptr, txn->rep->Merge(Slice(key, klen), Slice(val, vlen)));
|
|
}
|
|
|
|
void rocksdb_transaction_merge_cf(rocksdb_transaction_t* txn,
|
|
rocksdb_column_family_handle_t* column_family,
|
|
const char* key, size_t klen, const char* val,
|
|
size_t vlen, char** errptr) {
|
|
SaveError(errptr, txn->rep->Merge(column_family->rep, Slice(key, klen),
|
|
Slice(val, vlen)));
|
|
}
|
|
|
|
void rocksdb_transaction_delete(rocksdb_transaction_t* txn, const char* key,
|
|
size_t klen, char** errptr) {
|
|
SaveError(errptr, txn->rep->Delete(Slice(key, klen)));
|
|
}
|
|
|
|
void rocksdb_transaction_delete_cf(
|
|
rocksdb_transaction_t* txn, rocksdb_column_family_handle_t* column_family,
|
|
const char* key, size_t klen, char** errptr) {
|
|
SaveError(errptr, txn->rep->Delete(column_family->rep, Slice(key, klen)));
|
|
}
|
|
|
|
void rocksdb_transaction_commit(rocksdb_transaction_t* txn, char** errptr) {
|
|
SaveError(errptr, txn->rep->Commit());
|
|
}
|
|
|
|
void rocksdb_transaction_rollback(rocksdb_transaction_t* txn, char** errptr) {
|
|
SaveError(errptr, txn->rep->Rollback());
|
|
}
|
|
|
|
void rocksdb_transaction_put_log_data(rocksdb_transaction_t* txn,
|
|
const char* blob, size_t len) {
|
|
txn->rep->PutLogData(Slice(blob, len));
|
|
}
|
|
|
|
void rocksdb_transaction_set_savepoint(rocksdb_transaction_t* txn) {
|
|
txn->rep->SetSavePoint();
|
|
}
|
|
|
|
void rocksdb_transaction_rollback_to_savepoint(rocksdb_transaction_t* txn,
|
|
char** errptr) {
|
|
SaveError(errptr, txn->rep->RollbackToSavePoint());
|
|
}
|
|
|
|
/* TransactionDB simple */
|
|
|
|
void rocksdb_transactiondb_put(rocksdb_transactiondb_t* txn_db,
|
|
const rocksdb_writeoptions_t* options,
|
|
const char* key, size_t klen, const char* val,
|
|
size_t vlen, char** errptr) {
|
|
SaveError(errptr,
|
|
txn_db->rep->Put(options->rep, Slice(key, klen), Slice(val, vlen)));
|
|
}
|
|
|
|
void rocksdb_transactiondb_put_cf(rocksdb_transactiondb_t* txn_db,
|
|
const rocksdb_writeoptions_t* options,
|
|
rocksdb_column_family_handle_t* column_family,
|
|
const char* key, size_t keylen,
|
|
const char* val, size_t vallen,
|
|
char** errptr) {
|
|
SaveError(errptr, txn_db->rep->Put(options->rep, column_family->rep,
|
|
Slice(key, keylen), Slice(val, vallen)));
|
|
}
|
|
|
|
void rocksdb_transactiondb_write(rocksdb_transactiondb_t* db,
|
|
const rocksdb_writeoptions_t* options,
|
|
rocksdb_writebatch_t* batch, char** errptr) {
|
|
SaveError(errptr, db->rep->Write(options->rep, &batch->rep));
|
|
}
|
|
|
|
void rocksdb_transactiondb_merge(rocksdb_transactiondb_t* txn_db,
|
|
const rocksdb_writeoptions_t* options,
|
|
const char* key, size_t klen, const char* val,
|
|
size_t vlen, char** errptr) {
|
|
SaveError(errptr, txn_db->rep->Merge(options->rep, Slice(key, klen),
|
|
Slice(val, vlen)));
|
|
}
|
|
|
|
void rocksdb_transactiondb_merge_cf(
|
|
rocksdb_transactiondb_t* txn_db, const rocksdb_writeoptions_t* options,
|
|
rocksdb_column_family_handle_t* column_family, const char* key, size_t klen,
|
|
const char* val, size_t vlen, char** errptr) {
|
|
SaveError(errptr, txn_db->rep->Merge(options->rep, column_family->rep,
|
|
Slice(key, klen), Slice(val, vlen)));
|
|
}
|
|
|
|
void rocksdb_transactiondb_delete(rocksdb_transactiondb_t* txn_db,
|
|
const rocksdb_writeoptions_t* options,
|
|
const char* key, size_t klen, char** errptr) {
|
|
SaveError(errptr, txn_db->rep->Delete(options->rep, Slice(key, klen)));
|
|
}
|
|
|
|
void rocksdb_transactiondb_delete_cf(
|
|
rocksdb_transactiondb_t* txn_db, const rocksdb_writeoptions_t* options,
|
|
rocksdb_column_family_handle_t* column_family, const char* key,
|
|
size_t keylen, char** errptr) {
|
|
SaveError(errptr, txn_db->rep->Delete(options->rep, column_family->rep,
|
|
Slice(key, keylen)));
|
|
}
|
|
|
|
void rocksdb_transactiondb_flush_wal(rocksdb_transactiondb_t* txn_db,
|
|
unsigned char sync, char** errptr) {
|
|
SaveError(errptr, txn_db->rep->FlushWAL(sync));
|
|
}
|
|
|
|
void rocksdb_transactiondb_flush(rocksdb_transactiondb_t* txn_db,
|
|
const rocksdb_flushoptions_t* options,
|
|
char** errptr) {
|
|
SaveError(errptr, txn_db->rep->Flush(options->rep));
|
|
}
|
|
|
|
void rocksdb_transactiondb_flush_cf(
|
|
rocksdb_transactiondb_t* txn_db, const rocksdb_flushoptions_t* options,
|
|
rocksdb_column_family_handle_t* column_family, char** errptr) {
|
|
SaveError(errptr, txn_db->rep->Flush(options->rep, column_family->rep));
|
|
}
|