Summary:
I am wanting to use std::counting_semaphore for something and the timing seems good to require C++20 support. The internets suggest:
* GCC >= 10 is adequate, >= 11 preferred
* Clang >= 10 is needed
* Visual Studio >= 2019 is adquate
And popular linux distributions look like this:
* CentOS Stream 9 -> GCC 11.2 (CentOS 8 is EOL)
* Ubuntu 22.04 LTS -> GCC 11.x (Ubuntu 20 just ended standard support)
* Debian 12 (oldstable) -> GCC 12.2
* (Debian 11 has ended security updates, uses GCC 10.2)
This required generating a new docker image based on Ubuntu 22 for CI using gcc. The existing Ubuntu 20 image works for covering appropriate clang versions (though we should maybe add a much later version as well, in the next increment of our Ubuntu 22 image; however the minimum available clang build from apt.llvm.org for Ubuntu 22 is clang 13).
Update to SetDumpFilter is to quiet a mysterious gcc-13 warning-as-error.
Removed --compile-no-warning-as-error from a cmake command line because cmake in the new docker image is too old for this option.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/13904
Test Plan: CI, one minor unit test added to verify std::counting_semaphor works
Reviewed By: xingbowang
Differential Revision: D81266435
Pulled By: pdillinger
fbshipit-source-id: 26040eeccca7004416e29a6ff4f6ea93f2052684
Summary:
Building db_bench with clang and DEBUG_LEVEL=0 was failing with unused variable. This was not caught by CI so I have added this to the build-linux-clang-13-no_test_run job.
Also, while I was touching CI:
* Fold build-linux-release-rtti into build-linux-release by reducing the number of combinations tested between static/dynamic lib and rtti/not. I don't expect these to interact meaningfully with an extremely mature compiler.
* Combine build-linux-clang10-asan and build-linux-clang10-ubsan because clang is extremely reliable running both together
Pull Request resolved: https://github.com/facebook/rocksdb/pull/13813
Test Plan: manual builds, CI
Reviewed By: krhancoc
Differential Revision: D79112643
Pulled By: pdillinger
fbshipit-source-id: 4ffc672718c05fa4597d637aacbc5a179ad8a0cf
Summary:
So that we can use --command=recompress with a custom CompressionManager. (It's not required for reading files using a custom CompressionManager because those can already use ObjectLibrary for dependency injection.)
Suggested follow-up:
* These tests should not be using C arrays, snprintf, manual delete, etc. except for thin compatibility with argc/argv.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/13783
Test Plan: unit test added, some manual testing
Reviewed By: archang19
Differential Revision: D78574434
Pulled By: pdillinger
fbshipit-source-id: 609e6c6439090e6b7e9b63fbd4c2d3f04b104fcf
Summary:
See `Makefile` for actual changes:
* ZLIB remains the same
* BZIP2 remains the same
* SNAPPY is a minor update
* LZ4 is a significant update with multithreaded/multicore compression https://github.com/lz4/lz4/releases/tag/v1.10.0
* ZSTD is a significant update RocksDB is called out as benefiting in particular from the performance improvements herein https://github.com/facebook/zstd/releases/tag/v1.5.7
Pull Request resolved: https://github.com/facebook/rocksdb/pull/13609
Reviewed By: archang19
Differential Revision: D77877295
Pulled By: mszeszko-meta
fbshipit-source-id: bf9a257e8f68dec3d02743b339aa2df65df4ab2c
Summary:
Usual release steps
* Release notes from 10.3 branch
* Update version.h
* Add 10.3.fb to check_format_compatible.sh
* Update folly commit hash. Added a few hacks to fix build errors.
Bonus:
* Add a check_format_compatible.sh sanity check to the per-PR GitHub actions jobs. It should be quick enough and catch typos in release diffs as we've seen in the past.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/13622
Test Plan: CI
Reviewed By: jowlyzhang
Differential Revision: D74943843
Pulled By: pdillinger
fbshipit-source-id: 4ff1db9a635e111f8830cadff2d3ee51cf2de512
Summary:
Originally I was trying to update `build-linux-clang10-mini-tsan` to actually use `clang10` (as the name implied). https://github.com/facebook/rocksdb/issues/13220 was supposed to also update this configuration, but I did not see that we had a definition for `build-linux-clang10-mini-tsan` in both `config.yml` and `pr-jobs.yml`. I was wondering why I could not see my changes reflected in the CI checks after merging. After I updated `pr-jobs.yml` for this PR, I found that the CI check started failing https://github.com/facebook/rocksdb/actions/runs/12417441052/job/34668411263?pr=13232. I don't think it makes sense for me to tackle looking into all the TSAN warnings being reported in `clang10` (at least in this PR), so for now I have updated the name of the PR job to accurately reflect the command that is being run.
This PR also gets rid of the entire `.circleci` folder, which I think is the more significant change.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/13232
Test Plan: Existing CI check is unchanged
Reviewed By: pdillinger
Differential Revision: D67462454
Pulled By: archang19
fbshipit-source-id: f1aabfe4c8793616d6cbaae36fdf007319bf7ab2
Summary:
... which is the default for CentOS 9 and Ubuntu 24, the latter of which is now available in GitHub Actions. Relevant CI job updated.
Re-formatted all cc|c|h files except in third-party/, using
```
clang-format -i `git ls-files | grep -E '[.](cc|c|h)$' | grep -v third-party/`
```
Pull Request resolved: https://github.com/facebook/rocksdb/pull/13233
Test Plan: CI
Reviewed By: jaykorean, archang19
Differential Revision: D67461638
Pulled By: pdillinger
fbshipit-source-id: 0c9ac21a3f5eea6f5ade68bb6af7b6ba16c8b301
Summary:
Followed instruction per https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions#defining-access-for-the-github_token-scopes
It turns out that we did not need any of these except `Metadata: read`.
Before
```
GITHUB_TOKEN Permissions
Actions: write
Attestations: write
Checks: write
Contents: write
Deployments: write
Discussions: write
Issues: write
Metadata: read
Packages: write
Pages: write
PullRequests: write
RepositoryProjects: write
SecurityEvents: write
Statuses: write
```
After
```
GITHUB_TOKEN Permissions
Metadata: read
```
Pull Request resolved: https://github.com/facebook/rocksdb/pull/12973
Test Plan: GitHub Actions triggered by this PR
Reviewed By: cbi42
Differential Revision: D61812651
Pulled By: jaykorean
fbshipit-source-id: 4413756c93f503e8b2fb77eb8b684ef9e6a6c13d
Summary:
- Updated pinned folly version to the latest
- gcc/g++ 10 is required since https://github.com/facebook/folly/commit/2c1c617e9e so we had to modify the tests using gcc/g++ 7
- libsodium 1.0.17 is no longer downloadable from GitHub so I found it elsewhere. I will submit a PR for that upstream to folly
- USE_FOLLY_LITE changes
- added boost header dependency instead of commenting out the `#include`s since that approach stopped working
- added "folly/lang/Exception.cpp" to the compilation
Pull Request resolved: https://github.com/facebook/rocksdb/pull/12795
Reviewed By: hx235
Differential Revision: D58916693
Pulled By: ajkr
fbshipit-source-id: b5f9bca2d929825846ac898b785972b071db62b1
Summary:
and also fix comment/label on some MacOS CI jobs. Motivated by a crash test failure missing a definitive indicator of the genesis of the status:
```
file ingestion error: Operation failed. Try again.:
```
Pull Request resolved: https://github.com/facebook/rocksdb/pull/12307
Test Plan: just cosmetic changes. These statuses should not arise frequently enough to be a performance issue (copying messages).
Reviewed By: jaykorean
Differential Revision: D53199529
Pulled By: pdillinger
fbshipit-source-id: ad83daaa5d80f75c9f81158e90fb6d9ecca33fe3
Summary:
* Largely based on https://github.com/facebook/rocksdb/issues/12085 but grouped into one large workflow because of bad GHA UI design (see comments).
* Windows job details consolidated into an action file so that those jobs can easily move between per-pr-push and nightly.
* Simplify some handling of "CIRCLECI" environment and add "GITHUB_ACTIONS" in the same places
* For jobs that we want to go in pr-jobs or nightly there are disabled "candidate" workflows with draft versions of those jobs.
* ARM jobs are disabled waiting on full GHA support.
* build-linux-java-static needed some special attention to work, due to GLIBC compatibility issues (see comments).
Pull Request resolved: https://github.com/facebook/rocksdb/pull/12163
Test Plan:
Nightly jobs can be seen passing between these two links:
https://github.com/facebook/rocksdb/actions/runs/7266835435/job/19799390061?pr=12163https://github.com/facebook/rocksdb/actions/runs/7269697823/job/19807724471?pr=12163
And per-PR jobs of course passing on this PR.
Reviewed By: hx235
Differential Revision: D52335810
Pulled By: pdillinger
fbshipit-source-id: bbb95196f33eabad8cddf3c6b52f4413c80e034d
Peter Dillinger