jabberwocky238/rocksdb
1
0
Fork 0
mirror of https://github.com/facebook/rocksdb.git synced 2026-07-07 14:47:40 +08:00
Code Issues Packages Projects Releases Wiki Activity

Upgrade CI Docker images: clang-18→clang-21, ubuntu24 24.0→24.1 (#14576)

Browse Source 
Summary:
Upgrade the Ubuntu 24 CI Docker image to include clang-21 (from clang-18), and bump all workflow references to the new image tags. Also adds ccache to both images and renames all clang-18 job references to clang-21.

This is mainly for upgrading the clang version later used for generating C API automatically. See PR https://github.com/facebook/rocksdb/issues/14572

### Changes

**`build_tools/ubuntu24_image/Dockerfile`**
- Add clang-21 installation from LLVM snapshot repo (`apt.llvm.org/noble/llvm-toolchain-noble-21`)
- Add ccache
- Add comment pointing Meta employees to internal devvm build guide

**`build_tools/ubuntu22_image/Dockerfile`**
- Add ccache
- Add comment pointing Meta employees to internal devvm build guide

**`.github/workflows/pr-jobs.yml`**
- Rename `build-linux-clang-18-no_test_run` → `build-linux-clang-21-no_test_run`
- Rename `build-linux-clang18-asan-ubsan` → `build-linux-clang21-asan-ubsan`
- Rename `build-linux-clang18-mini-tsan` → `build-linux-clang21-mini-tsan`
- Update all `clang-18`/`clang++-18` references to `clang-21`/`clang++-21`
- Update ccache key prefixes: `clang18-asan-ubsan` → `clang21-asan-ubsan`, `clang18-tsan` → `clang21-tsan`
- Bump `rocksdb_ubuntu:24.0` → `rocksdb_ubuntu:24.1`

**`.github/workflows/nightly.yml`**
- Rename `build-linux-clang-18-asan-ubsan-with-folly` → `build-linux-clang-21-asan-ubsan-with-folly`
- Update clang-18 → clang-21 compiler references
- Bump `rocksdb_ubuntu:24.0` → `rocksdb_ubuntu:24.1`

**`.github/workflows/clang-tidy.yml`**
- Update clang-18 → clang-21
- Bump `rocksdb_ubuntu:24.0` → `rocksdb_ubuntu:24.1`

### New Docker Images

Both images have been built and tested locally. They will be pushed to `ghcr.io/facebook/rocksdb_ubuntu` before this PR is merged.

- `ghcr.io/facebook/rocksdb_ubuntu:22.2` — adds ccache, adds devvm build note
- `ghcr.io/facebook/rocksdb_ubuntu:24.1` — adds clang-21 from LLVM snapshot repo, adds ccache

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14576

Test Plan: Built both images locally and verified CI job names/compiler flags are consistent.

Reviewed By: joshkang97

Differential Revision: D99694293

Pulled By: xingbowang

fbshipit-source-id: c23c27f5cf870fb2c8b4e3d1cba281d0ce63f9d6
This commit is contained in:
Xingbo Wang
2026-04-06 15:00:53 -07:00
committed by meta-codesync[bot]
parent 1b817cff45
commit 3dd6d060e5
5 changed files with 42 additions and 34 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/clang-tidy.yml
+4 -4
View File
@@ -11,7 +11,7 @@ jobs:
runs-on:
labels: 4-core-ubuntu
container:
image: ghcr.io/facebook/rocksdb_ubuntu:24.0
image: ghcr.io/facebook/rocksdb_ubuntu:24.1
steps:
- uses: actions/checkout@v4.1.0
with:
@@ -36,14 +36,14 @@ jobs:
echo "skip=false" >> "$GITHUB_OUTPUT"
- name: Install clang-tidy
if: steps.diff-base.outputs.skip != 'true'
run: apt-get update && apt-get install -y clang-tidy
run: apt-get update && apt-get install -y clang-tidy-21 && ln -sf /usr/bin/clang-tidy-21 /usr/local/bin/clang-tidy
- name: Generate compile_commands.json
if: steps.diff-base.outputs.skip != 'true'
run: |
mkdir build && cd build
cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=ON \
-DCMAKE_C_COMPILER=clang-18 \
-DCMAKE_CXX_COMPILER=clang++-18 ..
-DCMAKE_C_COMPILER=clang-21 \
-DCMAKE_CXX_COMPILER=clang++-21 ..
cd ..
ln -sf build/compile_commands.json compile_commands.json
- name: Run clang-tidy on changed files
.github/workflows/nightly.yml
+6 -6
View File
@@ -41,16 +41,16 @@ jobs:
- uses: "./.github/actions/pre-steps"
- run: make V=1 -j32 check
- uses: "./.github/actions/post-steps"
build-linux-clang-18-asan-ubsan-with-folly:
build-linux-clang-21-asan-ubsan-with-folly:
if: ${{ github.repository_owner == 'facebook' }}
runs-on:
labels: 16-core-ubuntu
container:
image: ghcr.io/facebook/rocksdb_ubuntu:24.0
image: ghcr.io/facebook/rocksdb_ubuntu:24.1
options: --shm-size=16gb
env:
CC: clang-18
CXX: clang++-18
CC: clang-21
CXX: clang++-21
steps:
- uses: actions/checkout@v4.1.0
- uses: "./.github/actions/pre-steps"
@@ -145,13 +145,13 @@ jobs:
runs-on:
labels: 4-core-ubuntu
container:
image: ghcr.io/facebook/rocksdb_ubuntu:24.0
image: ghcr.io/facebook/rocksdb_ubuntu:24.1
options: --shm-size=16gb
steps:
- uses: actions/checkout@v4.1.0
- uses: "./.github/actions/pre-steps"
- name: Build rocksdb lib
run: CC=clang-18 CXX=clang++-18 USE_CLANG=1 make -j4 static_lib
run: CC=clang-21 CXX=clang++-21 USE_CLANG=1 make -j4 static_lib
- name: Build fuzzers
run: cd fuzz && make sst_file_writer_fuzzer db_fuzzer db_map_fuzzer
- uses: "./.github/actions/post-steps"
.github/workflows/pr-jobs.yml
+18 -18
View File
@@ -105,7 +105,7 @@ jobs:
runs-on:
labels: 4-core-ubuntu
container:
image: ghcr.io/facebook/rocksdb_ubuntu:24.0
image: ghcr.io/facebook/rocksdb_ubuntu:24.1
options: --shm-size=16gb
steps:
- uses: actions/checkout@v4.1.0
@@ -295,23 +295,23 @@ jobs:
- uses: "./.github/actions/teardown-ccache"
if: always()
- uses: "./.github/actions/post-steps"
build-linux-clang-18-no_test_run:
if: needs.config.outputs.only_job == 'build-linux-clang-18-no_test_run' || (needs.config.outputs.only_job == '' && github.repository_owner == 'facebook')
build-linux-clang-21-no_test_run:
if: needs.config.outputs.only_job == 'build-linux-clang-21-no_test_run' || (needs.config.outputs.only_job == '' && github.repository_owner == 'facebook')
needs: config
runs-on:
labels: 16-core-ubuntu
container:
image: ghcr.io/facebook/rocksdb_ubuntu:24.0
image: ghcr.io/facebook/rocksdb_ubuntu:24.1
options: --shm-size=16gb
steps:
- uses: actions/checkout@v4.1.0
- uses: "./.github/actions/pre-steps"
- uses: "./.github/actions/setup-ccache"
with:
cache-key-prefix: clang-18
- run: CC=clang-18 CXX=clang++-18 USE_CLANG=1 make -j32 all microbench
cache-key-prefix: clang-21
- run: CC=clang-21 CXX=clang++-21 USE_CLANG=1 make -j32 all microbench
- run: make clean
- run: CC=clang-18 CXX=clang++-18 USE_CLANG=1 DEBUG_LEVEL=0 make -j32 release
- run: CC=clang-21 CXX=clang++-21 USE_CLANG=1 DEBUG_LEVEL=0 make -j32 release
- uses: "./.github/actions/teardown-ccache"
if: always()
- uses: "./.github/actions/post-steps"
@@ -321,7 +321,7 @@ jobs:
runs-on:
labels: 16-core-ubuntu
container:
image: ghcr.io/facebook/rocksdb_ubuntu:24.0
image: ghcr.io/facebook/rocksdb_ubuntu:24.1
options: --shm-size=16gb
steps:
- uses: actions/checkout@v4.1.0
@@ -385,8 +385,8 @@ jobs:
with:
artifact-prefix: "${{ github.job }}-${{ matrix.crash_test_target }}"
# ======================= Linux with Sanitizers ===================== #
build-linux-clang18-asan-ubsan:
if: needs.config.outputs.only_job == 'build-linux-clang18-asan-ubsan' || (needs.config.outputs.only_job == '' && github.repository_owner == 'facebook')
build-linux-clang21-asan-ubsan:
if: needs.config.outputs.only_job == 'build-linux-clang21-asan-ubsan' || (needs.config.outputs.only_job == '' && github.repository_owner == 'facebook')
needs: config
runs-on:
labels: 32-core-ubuntu
@@ -395,15 +395,15 @@ jobs:
matrix:
shard: [0, 1, 2]
container:
image: ghcr.io/facebook/rocksdb_ubuntu:24.0
image: ghcr.io/facebook/rocksdb_ubuntu:24.1
options: --shm-size=16gb
steps:
- uses: actions/checkout@v4.1.0
- uses: "./.github/actions/pre-steps"
- uses: "./.github/actions/setup-ccache"
with:
cache-key-prefix: clang18-asan-ubsan
- run: COMPILE_WITH_ASAN=1 COMPILE_WITH_UBSAN=1 CC=clang-18 CXX=clang++-18 ROCKSDB_DISABLE_ALIGNED_NEW=1 USE_CLANG=1 make V=1 -j40 check
cache-key-prefix: clang21-asan-ubsan
- run: COMPILE_WITH_ASAN=1 COMPILE_WITH_UBSAN=1 CC=clang-21 CXX=clang++-21 ROCKSDB_DISABLE_ALIGNED_NEW=1 USE_CLANG=1 make V=1 -j40 check
env:
CI_SHARD_INDEX: ${{ matrix.shard }}
CI_TOTAL_SHARDS: 3
@@ -412,8 +412,8 @@ jobs:
- uses: "./.github/actions/post-steps"
with:
artifact-prefix: "${{ github.job }}-shard${{ matrix.shard }}"
build-linux-clang18-mini-tsan:
if: needs.config.outputs.only_job == 'build-linux-clang18-mini-tsan' || (needs.config.outputs.only_job == '' && github.repository_owner == 'facebook')
build-linux-clang21-mini-tsan:
if: needs.config.outputs.only_job == 'build-linux-clang21-mini-tsan' || (needs.config.outputs.only_job == '' && github.repository_owner == 'facebook')
needs: config
runs-on:
labels: 32-core-ubuntu
@@ -422,15 +422,15 @@ jobs:
matrix:
shard: [0, 1, 2]
container:
image: ghcr.io/facebook/rocksdb_ubuntu:24.0
image: ghcr.io/facebook/rocksdb_ubuntu:24.1
options: --shm-size=16gb
steps:
- uses: actions/checkout@v4.1.0
- uses: "./.github/actions/pre-steps"
- uses: "./.github/actions/setup-ccache"
with:
cache-key-prefix: clang18-tsan
- run: COMPILE_WITH_TSAN=1 CC=clang-18 CXX=clang++-18 ROCKSDB_DISABLE_ALIGNED_NEW=1 USE_CLANG=1 make V=1 -j32 check
cache-key-prefix: clang21-tsan
- run: COMPILE_WITH_TSAN=1 CC=clang-21 CXX=clang++-21 ROCKSDB_DISABLE_ALIGNED_NEW=1 USE_CLANG=1 make V=1 -j32 check
env:
CI_SHARD_INDEX: ${{ matrix.shard }}
CI_TOTAL_SHARDS: 3
build_tools/ubuntu22_image/Dockerfile
+5 -3
View File
@@ -5,6 +5,8 @@
# the docker and docker-registry groups, and logging out and back in to pick
# those up.)
#
# Meta employees: see https://fburl.com/rocksdb-docker to build this on a devvm.
#
# Follow https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry#authenticating-with-a-personal-access-token-classic
# to login with your GitHub credentials, as in
#
@@ -15,8 +17,8 @@
# Then in the build_tools/ubuntu22_image directory, (bump minor version for
# random docker file updates, major version tracks Ubuntu release)
#
# $ docker build -t ghcr.io/facebook/rocksdb_ubuntu:22.0
# $ docker push ghcr.io/facebook/rocksdb_ubuntu:22.0
# $ docker build -t ghcr.io/facebook/rocksdb_ubuntu:22.2
# $ docker push ghcr.io/facebook/rocksdb_ubuntu:22.2
#
# Might need to change visibility to public through
# https://github.com/orgs/facebook/packages/container/rocksdb_ubuntu/settings
@@ -28,7 +30,7 @@ FROM ubuntu:22.04
RUN apt-get update
RUN apt-get upgrade -y
# install basic tools
RUN apt-get install -y vim wget curl
RUN apt-get install -y vim wget curl ccache
# install tzdata noninteractive
RUN DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC apt-get -y install tzdata
# install git and default compilers
build_tools/ubuntu24_image/Dockerfile
+9 -3
View File
@@ -5,6 +5,8 @@
# the docker and docker-registry groups, and logging out and back in to pick
# those up.)
#
# Meta employees: see https://fburl.com/rocksdb-docker to build this on a devvm.
#
# Follow https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry#authenticating-with-a-personal-access-token-classic
# to login with your GitHub credentials, as in
#
@@ -15,8 +17,8 @@
# Then in the build_tools/ubuntu24_image directory, (bump minor version for
# random docker file updates, major version tracks Ubuntu release)
#
# $ docker build -t ghcr.io/facebook/rocksdb_ubuntu:24.0
# $ docker push ghcr.io/facebook/rocksdb_ubuntu:24.0
# $ docker build -t ghcr.io/facebook/rocksdb_ubuntu:24.1
# $ docker push ghcr.io/facebook/rocksdb_ubuntu:24.1
#
# Might need to change visibility to public through
# https://github.com/orgs/facebook/packages/container/rocksdb_ubuntu/settings
@@ -28,11 +30,15 @@ FROM ubuntu:24.04
RUN apt-get update
RUN apt-get upgrade -y
# install basic tools
RUN apt-get install -y vim wget curl
RUN apt-get install -y vim wget curl ccache
# install tzdata noninteractive
RUN DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC apt-get -y install tzdata
# install git and default compilers
RUN apt-get install -y git gcc g++ clang clang-tools
# install clang-21 from LLVM snapshot repo
RUN curl -fsSL https://apt.llvm.org/llvm-snapshot.gpg.key | gpg --dearmor -o /etc/apt/trusted.gpg.d/llvm.gpg && \
echo "deb https://apt.llvm.org/noble/ llvm-toolchain-noble-21 main" > /etc/apt/sources.list.d/llvm-21.list && \
apt-get update && apt-get install -y clang-21
# install basic package
RUN apt-get install -y lsb-release software-properties-common gnupg
# install gflags, tbb