mirror of
https://github.com/facebook/rocksdb.git
synced 2026-07-07 14:47:40 +08:00
Xingbo Wang
3dd6d060e5
Summary: Upgrade the Ubuntu 24 CI Docker image to include clang-21 (from clang-18), and bump all workflow references to the new image tags. Also adds ccache to both images and renames all clang-18 job references to clang-21. This is mainly for upgrading the clang version later used for generating C API automatically. See PR https://github.com/facebook/rocksdb/issues/14572 ### Changes **`build_tools/ubuntu24_image/Dockerfile`** - Add clang-21 installation from LLVM snapshot repo (`apt.llvm.org/noble/llvm-toolchain-noble-21`) - Add ccache - Add comment pointing Meta employees to internal devvm build guide **`build_tools/ubuntu22_image/Dockerfile`** - Add ccache - Add comment pointing Meta employees to internal devvm build guide **`.github/workflows/pr-jobs.yml`** - Rename `build-linux-clang-18-no_test_run` → `build-linux-clang-21-no_test_run` - Rename `build-linux-clang18-asan-ubsan` → `build-linux-clang21-asan-ubsan` - Rename `build-linux-clang18-mini-tsan` → `build-linux-clang21-mini-tsan` - Update all `clang-18`/`clang++-18` references to `clang-21`/`clang++-21` - Update ccache key prefixes: `clang18-asan-ubsan` → `clang21-asan-ubsan`, `clang18-tsan` → `clang21-tsan` - Bump `rocksdb_ubuntu:24.0` → `rocksdb_ubuntu:24.1` **`.github/workflows/nightly.yml`** - Rename `build-linux-clang-18-asan-ubsan-with-folly` → `build-linux-clang-21-asan-ubsan-with-folly` - Update clang-18 → clang-21 compiler references - Bump `rocksdb_ubuntu:24.0` → `rocksdb_ubuntu:24.1` **`.github/workflows/clang-tidy.yml`** - Update clang-18 → clang-21 - Bump `rocksdb_ubuntu:24.0` → `rocksdb_ubuntu:24.1` ### New Docker Images Both images have been built and tested locally. They will be pushed to `ghcr.io/facebook/rocksdb_ubuntu` before this PR is merged. - `ghcr.io/facebook/rocksdb_ubuntu:22.2` — adds ccache, adds devvm build note - `ghcr.io/facebook/rocksdb_ubuntu:24.1` — adds clang-21 from LLVM snapshot repo, adds ccache Pull Request resolved: https://github.com/facebook/rocksdb/pull/14576 Test Plan: Built both images locally and verified CI job names/compiler flags are consistent. Reviewed By: joshkang97 Differential Revision: D99694293 Pulled By: xingbowang fbshipit-source-id: c23c27f5cf870fb2c8b4e3d1cba281d0ce63f9d6
91 lines
4.4 KiB
Docker
91 lines
4.4 KiB
Docker
# INSTRUCTIONS:
|
|
# I was not able to build docker images on an isolated devserver because of
|
|
# issues with proxy internet access. Use a public cloud or other Linux system.
|
|
# (I used a Debian system after installing docker features, adding my user to
|
|
# the docker and docker-registry groups, and logging out and back in to pick
|
|
# those up.)
|
|
#
|
|
# Meta employees: see https://fburl.com/rocksdb-docker to build this on a devvm.
|
|
#
|
|
# Follow https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry#authenticating-with-a-personal-access-token-classic
|
|
# to login with your GitHub credentials, as in
|
|
#
|
|
# $ docker login ghcr.io -u pdillinger
|
|
#
|
|
# and paste the limited-purpose GitHub token into the terminal.
|
|
#
|
|
# Then in the build_tools/ubuntu22_image directory, (bump minor version for
|
|
# random docker file updates, major version tracks Ubuntu release)
|
|
#
|
|
# $ docker build -t ghcr.io/facebook/rocksdb_ubuntu:22.2
|
|
# $ docker push ghcr.io/facebook/rocksdb_ubuntu:22.2
|
|
#
|
|
# Might need to change visibility to public through
|
|
# https://github.com/orgs/facebook/packages/container/rocksdb_ubuntu/settings
|
|
# or similar.
|
|
|
|
# from official ubuntu 22.04
|
|
FROM ubuntu:22.04
|
|
# update system
|
|
RUN apt-get update
|
|
RUN apt-get upgrade -y
|
|
# install basic tools
|
|
RUN apt-get install -y vim wget curl ccache
|
|
# install tzdata noninteractive
|
|
RUN DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC apt-get -y install tzdata
|
|
# install git and default compilers
|
|
RUN apt-get install -y git gcc g++ clang clang-tools
|
|
# install basic package
|
|
RUN apt-get install -y lsb-release software-properties-common gnupg
|
|
# install gflags, tbb
|
|
RUN apt-get install -y libgflags-dev libtbb-dev
|
|
# install compression libs
|
|
RUN apt-get install -y libsnappy-dev zlib1g-dev libbz2-dev liblz4-dev libzstd-dev
|
|
# install cmake
|
|
RUN apt-get install -y cmake
|
|
RUN apt-get install -y libssl-dev
|
|
# install clang-13
|
|
WORKDIR /root
|
|
RUN wget https://apt.llvm.org/llvm.sh
|
|
RUN chmod +x llvm.sh
|
|
RUN ./llvm.sh 13 all
|
|
# There are incompatibilities between clang with -std=c++20 and libstdc++
|
|
# provided by gcc, so we have to compile with clang-13 using -stdlib=libc++
|
|
# and only one version of libc++ can be installed on the system at one time.
|
|
# So to avoid confusion we remove unusable clang-14 also.
|
|
RUN apt-get install libc++-13-dev libc++abi-13-dev
|
|
RUN apt-get purge -y clang-14 && apt-get autoremove -y
|
|
|
|
# install gcc-10 and more, default is 11
|
|
RUN apt-get install -y gcc-10 g++-10
|
|
RUN add-apt-repository -y ppa:ubuntu-toolchain-r/test
|
|
RUN apt-get install -y gcc-13 g++-13
|
|
# install apt-get install -y valgrind
|
|
RUN apt-get install -y valgrind
|
|
# install folly depencencies
|
|
# Missing compatible libunwind: RUN apt-get install -y libgoogle-glog-dev
|
|
# So instead install from source. This currently requires compiling with
|
|
# -DGLOG_USE_GLOG_EXPORT
|
|
RUN wget https://github.com/google/glog/archive/refs/tags/v0.7.1.tar.gz && tar xzf v0.7.1.tar.gz && cd glog-0.7.1/ && cmake -S . -B build -G "Unix Makefiles" && cmake --build build && cmake --build build --target install && cd .. && rm -rf v0.7.1.tar.gz glog-0.7.1
|
|
# install openjdk 8
|
|
RUN apt-get install -y openjdk-8-jdk
|
|
ENV JAVA_HOME /usr/lib/jvm/java-1.8.0-openjdk-amd64
|
|
# install mingw
|
|
RUN apt-get install -y mingw-w64
|
|
|
|
# install gtest-parallel package
|
|
RUN git clone --single-branch --branch master --depth 1 https://github.com/google/gtest-parallel.git ~/gtest-parallel
|
|
ENV PATH $PATH:/root/gtest-parallel
|
|
|
|
# install libprotobuf for fuzzers test
|
|
RUN apt-get install -y ninja-build binutils liblzma-dev libz-dev pkg-config autoconf libtool
|
|
RUN git clone --branch v1.0 https://github.com/google/libprotobuf-mutator.git ~/libprotobuf-mutator && cd ~/libprotobuf-mutator && git checkout ffd86a32874e5c08a143019aad1aaf0907294c9f && mkdir build && cd build && cmake .. -GNinja -DCMAKE_C_COMPILER=clang-13 -DCMAKE_CXX_COMPILER=clang++-13 -DCMAKE_BUILD_TYPE=Release -DLIB_PROTO_MUTATOR_DOWNLOAD_PROTOBUF=ON && ninja && ninja install
|
|
ENV PKG_CONFIG_PATH /usr/local/OFF/:/root/libprotobuf-mutator/build/external.protobuf/lib/pkgconfig/
|
|
ENV PROTOC_BIN /root/libprotobuf-mutator/build/external.protobuf/bin/protoc
|
|
|
|
# install the latest google benchmark
|
|
RUN git clone --depth 1 --branch v1.7.0 https://github.com/google/benchmark.git ~/benchmark && cd ~/benchmark && mkdir build && cd build && cmake .. -GNinja -DCMAKE_BUILD_TYPE=Release -DBENCHMARK_ENABLE_GTEST_TESTS=0 && ninja && ninja install && cd ~ && rm -rf /root/benchmark
|
|
|
|
# clean up
|
|
RUN rm -rf /var/lib/apt/lists/*
|