Summary:
- Use the liburing TSAN suppressions in `tools/tsan_suppressions.txt` instead of defining the process-wide `__tsan_default_suppressions()` hook, avoiding conflicts with downstream applications.
- Wire RocksDB TSAN make and crash-test flows to use that suppressions file by default without overriding caller-provided `TSAN_OPTIONS`.
- Cover direct `db_crashtest.py` launches by passing the default suppressions to `db_stress` subprocesses.
- Fix the GCC 16 unity-build warning in `CacheItemHelper` by directly initializing the no-secondary-cache helper fields instead of delegating with `this`.
Imported from D101303486.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14710
Test Plan:
- `COMPILE_WITH_TSAN=1 make -j128 env_test`
- `timeout 60s ./env_test --gtest_filter=EnvPosixTest.IOUringAddressReuseNoTsanFalsePositive`
- `python3 tools/db_crashtest_test.py`
- `python3 -m py_compile tools/db_crashtest.py tools/db_crashtest_test.py`
- CI: `build-linux-unity-and-headers` passed after the `CacheItemHelper` fix
Reviewed By: hx235
Differential Revision: D104103800
Pulled By: xingbowang
fbshipit-source-id: 6066d9abe02a3c44d75f9ce449889468c927ce56
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14286
Add the db_c leanup target which can be used by CI test scripts to delete the db on failure. The db_crashtest.py doesn't automatically delete on error.
Reviewed By: jaykorean
Differential Revision: D91912877
fbshipit-source-id: d36ec0896fba64faaafe055d8673e437e85d0c3a
Summary:
Clearing DB dir for crash test is currently a hodgepodge of
1. Caller of db_crashtest.py maybe tries to clear the dir
2. db_crashtest.py tries to clear the dir in get_dbname() (but ignoring failure)
3. db_crashtest.py passes --destroy_db_initially to some db_stress calls as needed
4. db_crashtest.py tries to clear the dir between some db_stress calls
5. db_crashtest.py tries to clear the dir after everything is done and successful (no artifacts to investigate or save) (but ignoring failure)
6. Try to add more uniqueness to the directory from https://github.com/facebook/rocksdb/issues/14249
This change reverts or replaces 2, 4, 5, and 6 by doubling-down on (expanding) 3 and a small variant of it:
* crash_test.mk passes --destroy_db_initially=1 so that the first run of db_stress clears the db dir.
* After each db_stress invocation, db_crashtest.py resets destroy_db_initially=0 so that the next invocation reuses the same DB, except in cases where there is an incompatibility that requires a fresh DB (from cases 3 and 4 above).
* On success, uses new `db_stress --destroy_db_and_exit` option to clean up the DB dir without needing a custom cleanup_cmd (now ignored)
Note that although case 1 is likely obsolete, it is out of control of an open source PR.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14254
Test Plan: some manual runs
Reviewed By: xingbowang
Differential Revision: D91164731
Pulled By: pdillinger
fbshipit-source-id: 0a66c8c0e130c9eeacc55af411a18a09bc9debdf
Summary:
We are seeing some occasional failures with WRITE_(UN)PREPARED crash test runs, and it's alarming when these are grouped in with WRITE_COMMITTED, which AFAIK is the only one considered mature and mission-critical at this point.
* Mark WRITE_(UN)PREPARED as EXPERIMENTAL in the public APIs
* Separate out the `_with_txn` crash test jobs by write policy, now `_with_wc_txn`, `_with_wp_txn` and `_with_wup_txn` so that the major functional and maturity differences are better grouped.
* Add `_with_multiops_wup_txn` which was apparently missing
* Clean up db_crashtest.py for better consistency
* Get rid of awkard "write_policy" parameter that could conflict with authoritative "txn_write_policy" parameter.
* Similarly, move some multiops logic from different parameter sets to finalize_and_sanitize logic.
Immediate internal follow-up:
* Migrate from `_with_txn` which are now deprecated aliases of `_with_wc_txn` to more jobs with the new variants. And likely also add new multiops job.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/13499
Test Plan: manual runs of modified jobs, at least long enough to spot check things like txn_write_policy
Reviewed By: hx235
Differential Revision: D72015307
Pulled By: pdillinger
fbshipit-source-id: 06b99b2d1f15ac76fe7b8e22c93a51aaa2a42ecf
Summary:
Context:
OptimisticTransactionDB has not been covered by db_stress (including crash test) like TransactionDB.
1. Adding the following gflag options to to test OptimisticTransactionDB
- `use_optimistic_txn`: When true, open OptimisticTransactionDB to test
- `occ_validation_policy`: `OccValidationPolicy::kValidateParallel = 1` by default.
- `share_occ_lock_buckets`: Use shared occ locks
- `occ_lock_bucket_count`: 500 by default. Number of buckets to use for shared occ lock.
2. Opening OptimisticTransactionDB and NewTxn/Commit added per `use_optimistic_txn` flag in `db_stress_test_base.cc`
3. OptimisticTransactionDB blackbox/whitebox test added in crash_test.mk
Please note that the existing flag `use_txn` is being used here. When `use_txn == true` and `use_optimistic_txn == false`, we use `TransactionDB` (a.k.a. pessimistic transaction db). When both `use_txn` and `use_optimistic_txn` are true, we use `OptimisticTransactionDB`. If `use_txn == false` but `use_optimistic_txn == true` throw error with message _"You cannot set use_optimistic_txn true while use_txn is false. Please set use_txn true if you want to use OptimisticTransactionDB"_.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/11513
Test Plan:
**Crash Test**
Serial Validation
```
export CRASH_TEST_EXT_ARGS="--use_optimistic_txn=1 --use_txn=1 --use_put_entity_one_in=0 --occ_validation_policy=0"
make crash_test -j
```
Parallel Validation (no share bucket)
```
export CRASH_TEST_EXT_ARGS="--use_optimistic_txn=1 --use_txn=1 --use_put_entity_one_in=0 --occ_validation_policy=1 --share_occ_lock_buckets=0"
make crash_test -j
```
Parallel Validation (share bucket)
```
export CRASH_TEST_EXT_ARGS="--use_optimistic_txn=1 --use_txn=1 --use_put_entity_one_in=0 --occ_validation_policy=1 --share_occ_lock_buckets=1 --occ_lock_bucket_count=500"
make crash_test -j
```
**Stress Test**
```
./db_stress -use_optimistic_txn -threads=32
```
Reviewed By: pdillinger
Differential Revision: D46547387
Pulled By: jaykorean
fbshipit-source-id: ca19819ca6e0281694966998014b40d95d4e5960
Summary:
This option allows a custom cleanup command line for a non-Posix file system to be used by db_crashtest.py to cleanup between runs.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/10883
Test Plan: Run the whitebox crash test
Reviewed By: pdillinger
Differential Revision: D40726424
Pulled By: anand1976
fbshipit-source-id: b827f6b583ff78f9ca75ced2d96f7e58f5200432
Summary:
Expanded `all_params` to include all parameters crash test may set. Previously, `atomic_flush` was not included in `all_params` and thus was not visible to `finalize_and_sanitize()`. The consequence was manual crash test runs could provide unsafe combinations of parameters to `db_stress`. For example, running `db_crashtest.py` with `-atomic_flush=0` could cause `db_stress` to run with `-atomic_flush=0 -disable_wal=1`, which is known to produce inconsistencies across column families.
While expanding `all_params`, I found we cannot have an entry in it for both `db_stress` and `db_crashtest.py`. So I renamed `enable_tiered_storage` to `test_tiered_storage` for `db_crashtest.py`, which appears more conventional anyways.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/10654
Reviewed By: hx235
Differential Revision: D39369349
Pulled By: ajkr
fbshipit-source-id: 31d9010c760c868b20d5e9bd78ba75c8ff3ce348
Summary:
Having all of TMPD, TMPDIR and TEST_TMPDIR as configuration
parameters is confusing. This change simplifies a number of things by
standardizing on TEST_TMPDIR, while still recognizing the old names
also. In detail:
* crash_test.mk also needs to use TEST_TMPDIR for crash test, so put in
shared common.mk (an upgrade of python.mk)
* Always exporting TEST_TMPDIR eliminates the need to propagate TMPD or
export TEST_TMPDIR in selective places.
* Use --tmpdir option to gnu_parallel so that it doesn't need TMPDIR
environment variable
* Remove obsolete parloop and parallel_check Makefile targets
* Remove undefined, unused function ResetTmpDirForDirectIO()
Pull Request resolved: https://github.com/facebook/rocksdb/pull/9961
Test Plan: manual + CI
Reviewed By: riversand963
Differential Revision: D36212178
Pulled By: pdillinger
fbshipit-source-id: b76c1876c4f4d38b37789c2779eaa7c3026824dd
Summary:
some Makefile refactoring to support Meta-internal workflows,
and add a basic crash_test flow to CircleCI
Pull Request resolved: https://github.com/facebook/rocksdb/pull/9702
Test Plan: CI
Reviewed By: riversand963
Differential Revision: D34934315
Pulled By: pdillinger
fbshipit-source-id: 67f17280096d8968d8e44459293f72fb6fe339f3
xingbowang