Skip to content

Commit

Permalink
Merge bitcoin/bitcoin#28083: ci: Use DOCKER_BUILDKIT for lint image
Browse files Browse the repository at this point in the history
fa2f18a ci: Use DOCKER_BUILDKIT for lint image (MarcoFalke)

Pull request description:

  Currently the lint docker/podman image has many issues:

  * It relies on an EOL debian version.
  * It relies on a debian version different from the one used in the CI lint task.
  * It relies on the legacy docker build command, which requires the user to make `cd ./ci/lint/` before the build step.
  * It doesn't use the `.python-version` file, but a hardcoded version.

  Fix all issues by using the recommended `DOCKER_BUILDKIT=1` to generate the image.

  Also:
  * Rename `/tmp/python` to `/python_build`.
  * Compress all `pip install` commands into one.
  * Bump `.python-version`.

ACKs for top commit:
  jamesob:
    ACK bitcoin/bitcoin@fa2f18a

Tree-SHA512: 804b384904ad753845667998841cc7825f4229933ca2c42af021384713486ec3cca80ba58612d37557fba7ee1921439dacca5e1236aac0557dd75bd9a2f1875d
  • Loading branch information
fanquake committed Jul 18, 2023
2 parents 673acab + fa2f18a commit c6a338b
Show file tree
Hide file tree
Showing 7 changed files with 47 additions and 58 deletions.
2 changes: 1 addition & 1 deletion .cirrus.yml
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ task:
# For faster CI feedback, immediately schedule the linters
<< : *CREDITS_TEMPLATE
python_cache:
folder: "/tmp/python"
folder: "/python_build"
fingerprint_script: cat .python-version /etc/os-release
unshallow_script:
- git fetch --unshallow --no-tags
Expand Down
2 changes: 1 addition & 1 deletion .python-version
Original file line number Diff line number Diff line change
@@ -1 +1 @@
3.8.16
3.8.17
47 changes: 23 additions & 24 deletions ci/lint/04_install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -13,32 +13,31 @@ ${CI_RETRY_EXE} apt-get update
# - gpg (used by verify-commits)
${CI_RETRY_EXE} apt-get install -y curl xz-utils git gpg

if [ -z "${SKIP_PYTHON_INSTALL}" ]; then
PYTHON_PATH=/tmp/python
if [ ! -d "${PYTHON_PATH}/bin" ]; then
(
git clone https://github.com/pyenv/pyenv.git
cd pyenv/plugins/python-build || exit 1
./install.sh
)
# For dependencies see https://github.com/pyenv/pyenv/wiki#suggested-build-environment
${CI_RETRY_EXE} apt-get install -y build-essential libssl-dev zlib1g-dev \
libbz2-dev libreadline-dev libsqlite3-dev curl llvm \
libncursesw5-dev xz-utils tk-dev libxml2-dev libxmlsec1-dev libffi-dev liblzma-dev \
clang
env CC=clang python-build "$(cat "${BASE_ROOT_DIR}/.python-version")" "${PYTHON_PATH}"
fi
export PATH="${PYTHON_PATH}/bin:${PATH}"
command -v python3
python3 --version
PYTHON_PATH="/python_build"
if [ ! -d "${PYTHON_PATH}/bin" ]; then
(
git clone https://github.com/pyenv/pyenv.git
cd pyenv/plugins/python-build || exit 1
./install.sh
)
# For dependencies see https://github.com/pyenv/pyenv/wiki#suggested-build-environment
${CI_RETRY_EXE} apt-get install -y build-essential libssl-dev zlib1g-dev \
libbz2-dev libreadline-dev libsqlite3-dev curl llvm \
libncursesw5-dev xz-utils tk-dev libxml2-dev libxmlsec1-dev libffi-dev liblzma-dev \
clang
env CC=clang python-build "$(cat "./.python-version")" "${PYTHON_PATH}"
fi
export PATH="${PYTHON_PATH}/bin:${PATH}"
command -v python3
python3 --version

${CI_RETRY_EXE} pip3 install codespell==2.2.5
${CI_RETRY_EXE} pip3 install flake8==6.0.0
${CI_RETRY_EXE} pip3 install lief==0.13.2
${CI_RETRY_EXE} pip3 install mypy==1.4.1
${CI_RETRY_EXE} pip3 install pyzmq==25.1.0
${CI_RETRY_EXE} pip3 install vulture==2.6
${CI_RETRY_EXE} pip3 install \
codespell==2.2.5 \
flake8==6.0.0 \
lief==0.13.2 \
mypy==1.4.1 \
pyzmq==25.1.0 \
vulture==2.6

SHELLCHECK_VERSION=v0.8.0
curl -sL "https://github.com/koalaman/shellcheck/releases/download/${SHELLCHECK_VERSION}/shellcheck-${SHELLCHECK_VERSION}.linux.x86_64.tar.xz" | \
Expand Down
29 changes: 0 additions & 29 deletions ci/lint/Dockerfile

This file was deleted.

2 changes: 2 additions & 0 deletions ci/lint/docker-entrypoint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ export LC_ALL=C
# of the mounted bitcoin src dir.
git config --global --add safe.directory /bitcoin

export PATH="/python_build/bin:${PATH}"

if [ -z "$1" ]; then
LOCAL_BRANCH=1 bash -ic "./ci/lint/06_script.sh"
else
Expand Down
19 changes: 19 additions & 0 deletions ci/lint_imagefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# See test/lint/README.md for usage.

FROM debian:bookworm

ENV DEBIAN_FRONTEND=noninteractive
ENV LC_ALL=C.UTF-8

COPY ./.python-version /.python-version
COPY ./ci/lint/docker-entrypoint.sh /entrypoint.sh
COPY ./ci/lint/04_install.sh /install.sh

RUN /install.sh && \
echo 'alias lint="./ci/lint/06_script.sh"' >> ~/.bashrc && \
chmod 755 /entrypoint.sh && \
rm -rf /var/lib/apt/lists/*


WORKDIR /bitcoin
ENTRYPOINT ["/entrypoint.sh"]
4 changes: 1 addition & 3 deletions test/lint/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,8 @@ To run linters locally with the same versions as the CI environment, use the inc
Dockerfile:

```sh
cd ./ci/lint
docker build -t bitcoin-linter .
DOCKER_BUILDKIT=1 docker build -t bitcoin-linter --file "./ci/lint_imagefile" ./

cd /root/of/bitcoin/repo
docker run --rm -v $(pwd):/bitcoin -it bitcoin-linter
```

Expand Down

0 comments on commit c6a338b

Please sign in to comment.