fix(dockerfiles): fix the eof problem of centos for ci cd builder images

[wuhuizuo]

User description

Signed-off-by: wuhuizuo wuhuizuo@126.com

---

PR Type

Bug fix, Enhancement

---

Description

• Updated CentOS 7 repository URLs across multiple Dockerfiles due to CentOS 7 reaching EOL.
• Removed centos-release-scl from package installations where applicable.
• Updated base image versions from v1.8.0 to v1.9.0 across multiple Dockerfiles.
• Added unzip package installation in dockerfiles/cd/builders/tikv/Dockerfile.
• Updated Go version to 1.21.12, ORAS tool to 1.2.0, and Git tool to 2.45.2 in dockerfiles/ci/base/Dockerfile.

---

Changes walkthrough 📝

	Relevant files
Enhancement	3 files skaffold.yaml Update Skaffold template tag version                                          dockerfiles/bases/skaffold.yaml Updated template tag from v1.8.0-fips to v1.9.0-fips. +1/-1      fips.Dockerfile Update TiKV base image version                                                      dockerfiles/bases/tikv-base/fips.Dockerfile Updated base image version from v1.8.0 to v1.9.0. +1/-1      Dockerfile Update Go, ORAS, and Git versions                                                dockerfiles/ci/base/Dockerfile Updated Go version from 1.21.10 to 1.21.12. Updated ORAS tool to use version 1.2.0 from Bitnami. Updated Git tool to use version 2.45.2 from Bitnami. +5/-34
Bug fix	8 files Dockerfile Fix CentOS 7 EOL issues and update base image                        dockerfiles/cd/builders/ng-monitoring/Dockerfile Added commands to update CentOS 7 repository URLs due to EOL. Removed centos-release-scl from package installation. Updated base image version from v1.8.0 to v1.9.0. +6/-2      Dockerfile Fix CentOS 7 EOL issues and update base image                        dockerfiles/cd/builders/pd/Dockerfile Added commands to update CentOS 7 repository URLs due to EOL. Removed centos-release-scl from package installation. Updated base image version from v1.8.0 to v1.9.0. +6/-2      Dockerfile Fix CentOS 7 EOL issues and update base image                        dockerfiles/cd/builders/tidb-dashboard/Dockerfile Added commands to update CentOS 7 repository URLs due to EOL. Updated base image version from v1.8.0 to v1.9.0. +5/-1      Dockerfile Fix CentOS 7 EOL issues and update base image                        dockerfiles/cd/builders/tidb/Dockerfile Added commands to update CentOS 7 repository URLs due to EOL. Removed centos-release-scl from package installation. Updated base image version from v1.8.0 to v1.9.0. +6/-2      Dockerfile Fix CentOS 7 EOL issues and update base image                        dockerfiles/cd/builders/tiflash/Dockerfile Added commands to update CentOS 7 repository URLs due to EOL. Updated base image version from v1.8.0 to v1.9.0. +5/-1      Dockerfile Fix CentOS 7 EOL issues and update base image                        dockerfiles/cd/builders/tiflow/Dockerfile Added commands to update CentOS 7 repository URLs due to EOL. Removed centos-release-scl from package installation. Updated base image version from v1.8.0 to v1.9.0. +7/-3      Dockerfile Fix CentOS 7 EOL issues, add unzip, and update base image dockerfiles/cd/builders/tikv/Dockerfile Added commands to update CentOS 7 repository URLs due to EOL. Removed centos-release-scl from package installation. Added unzip package installation. Updated base image version from v1.8.0 to v1.9.0. +8/-3      Dockerfile Fix CentOS 7 EOL issues                                                                    dockerfiles/ci/release-build-base/Dockerfile Added commands to update CentOS 7 repository URLs due to EOL. +7/-1

---

💡 PR-Agent usage:
Comment /help on the PR to get a list of all available PR-Agent tools and their descriptions

[ti-chi-bot]

I have already done a preliminary review for you, and I hope to help you do a better job.

This pull request aims to fix the end-of-file (EOF) problem of CentOS for CI/CD builder images, by replacing mirror.centos.org with vault.centos.org and modifying the yum repo configuration. The changes were made to several Dockerfiles in the dockerfiles/cd/builders and dockerfiles/ci directories.

The pull request seems to address a specific issue and the changes look reasonable. However, there are a few potential problems that need to be addressed:

• The pull request description is empty. It would be helpful to provide more context about why this change was made and how it may impact the overall CI/CD pipeline.
• The ARG DEVTOOLSET_VER=8 in the tikv Dockerfile is unused and should be removed.
• The --nogpgcheck option used in some yum commands may pose a security risk. It is recommended to use GPG signature verification to ensure package integrity.
• The sed commands used to modify the yum repo configuration in the Dockerfiles may not work as expected on certain systems, especially if the repo files have different names or locations. It may be better to create a separate script to perform the modifications and run it during the build process.

To address these issues, I suggest the following:

• Add more details to the pull request description to explain the problem, the solution, and any potential impacts.
• Remove the ARG DEVTOOLSET_VER=8 line from the tikv Dockerfile.
• Use the --nogpgcheck option with caution and consider enabling GPG signature verification instead.
• Create a separate script to modify the yum repo configuration and run it during the build process, rather than embedding the sed commands in the Dockerfiles.

[ti-chi-bot]

I have already done a preliminary review for you, and I hope to help you do a better job.

This pull request aims to fix the end-of-file (EOF) problem for CentOS in the CI/CD builder images. The changes include updating the base images to use CentOS 7.9.2009 and replacing the mirror source with a vault source. Also, the epel-release, deltarpm packages are updated, and the CentOS-release-scl package is removed.

The changes are straightforward and do not seem to introduce any problems. However, there are a few things to consider:

• The centos-release-scl package is removed, which may impact some applications that depend on it. The impact should be minimal as the package is not commonly used.
• The Git tool is upgraded to version 2.45.2 from 2.40.3, which may introduce compatibility issues with some projects. It is recommended to test thoroughly before upgrading Git.
• The Oras tool is updated to version 1.2.0 from 1.1.0, which may introduce compatibility issues with some projects. It is recommended to test thoroughly before upgrading Oras.

It would be best to test these changes thoroughly in a staging environment before merging them into the main branch.

[ti-chi-bot]

I have already done a preliminary review for you, and I hope to help you do a better job.

The pull request primarily contains the following changes:

1. The base image versions are updated from v1.8.0 to v1.9.0 in several Dockerfiles. This includes changes in skaffold.yaml, fips.Dockerfile, ng-monitoring/Dockerfile, pd/Dockerfile, tidb-dashboard/Dockerfile, tidb/Dockerfile, tiflash/Dockerfile, tiflow/Dockerfile, tikv/Dockerfile, base/Dockerfile, and release-build-base/Dockerfile.

2. In multiple Dockerfiles, the CentOS mirror is changed from mirror.centos.org to vault.centos.org because CentOS 7 has reached its end of life (EOL).

3. The Git tool is upgraded in base/Dockerfile.

4. The Golang version is updated in base/Dockerfile.

5. Removed centos-release-scl package installation from the Dockerfiles.

6. The oras binary is now being copied from the bitnami/oras:1.2.0 image, instead of downloading and installing it manually.

Potential problems and fixing suggestions:

1. Changing base images version could introduce compatibility issues if the new version is not fully compatible with the old one. It is suggested to thoroughly test the new images before merging the PR.

2. Updating CentOS mirror to vault.centos.org should not introduce any issues, but it's good to check if all required packages are still available in the vault repository.

3. The removal of the centos-release-scl package could cause issues if there are dependencies that rely on packages from the SCL. The removal should be validated.

4. Upgrading Git and Golang versions could introduce compatibility issues if the code relies on older versions of these tools. It's suggested to verify the compatibility of the updated versions.

5. The PR does not provide any information about testing. It's highly recommended to test these changes thoroughly given the significant updates to the base images and key development tools.

6. The PR could benefit from a more descriptive summary of the changes. The title and description do not provide enough information about the extensive updates to the Dockerfiles.

[ti-chi-bot]

I have already done a preliminary review for you, and I hope to help you do a better job.

The key changes in this pull request can be summarized as follows:

1. The version of the base image in multiple Dockerfiles has been updated from v1.8.0 to v1.9.0.
2. The CentOS mirror URLs have been updated from mirror.centos.org to vault.centos.org in multiple Dockerfiles, as CentOS 7 has reached EOL (end of life).
3. The installed packages 'centos-release-scl' has been removed from multiple Dockerfiles.
4. The Golang version in the base Dockerfile for CI has been updated from 1.21.10 to 1.21.12.
5. The method of installation for the oras and git tools in the base Dockerfile for CI has been changed to use binaries from Bitnami's Docker images.
6. The Git version in the release-build-base Dockerfile has been updated from 2.40.2 to 2.40.3.

Potential issues:

1. The removal of the 'centos-release-scl' package could potentially cause issues if any processes depend on the Software Collection (SCL) packages.
2. The change in the method of installation for the 'oras' and 'git' tools might cause issues if the Bitnami images don't match the expected configuration or if they are not compatible with the current setup.
3. It's not clear if the version upgrade of the base images and tools have been thoroughly tested. If not, this could lead to unexpected behavior or compatibility issues.

Fixing suggestions:

1. If the 'centos-release-scl' package is needed, consider adding it back or find alternative packages that provide the same functionality.
2. Test the 'oras' and 'git' tools after the installation change to ensure they work as expected.
3. Before merging this pull request, make sure that the new versions of the base images and tools have been thoroughly tested to ensure they are compatible with the existing system and don't introduce any breaking changes.

[codiumai-pr-agent-pro]

CI Failure Feedback 🧐

Action: build images with skaffold (builder-tikv, devtoolset10,local-docker, linux/amd64)

Failed stage: Build images [❌]

Failure summary: The action failed because the Docker build process encountered an error: The command /bin/sh -c FILE=$([ "$(arch)" = "aarch64" ] && echo "protoc-${PROTOBUF_VER#?}-linux-aarch_64.zip" || echo "protoc-${PROTOBUF_VER#?}-linux-$(arch).zip"); curl -LO "https://github.com/protocolbuffers/protobuf/releases/download/${PROTOBUF_VER}/${FILE}" && unzip "$FILE" -d /usr/local/ && rm -f "$FILE" failed. The specific error was /bin/sh: unzip: command not found, indicating that the unzip command is not available in the Docker image.

Relevant error logs: 1: ##[group]Operating System 2: Ubuntu ... 848: #7 12.93 Updating : libssh2-1.8.0-4.el7_9.1.x86_64 32/102 849: #7 12.97 Updating : libcurl-7.29.0-59.el7_9.2.x86_64 33/102 850: #7 13.01 Updating : curl-7.29.0-59.el7_9.2.x86_64 34/102 851: #7 13.04 Updating : rpm-libs-4.11.3-48.el7_9.x86_64 35/102 852: #7 13.13 Updating : rpm-4.11.3-48.el7_9.x86_64 36/102 853: #7 13.16 Updating : openldap-2.4.44-25.el7_9.x86_64 37/102 854: #7 13.22 Updating : rpm-build-libs-4.11.3-48.el7_9.x86_64 38/102 855: #7 13.85 Updating : systemd-219-78.el7_9.9.x86_64 39/102 856: #7 14.03 Failed to get D-Bus connection: Operation not permitted ... 1170: #7 28.07 --> Processing Dependency: rsync for package: git-1.8.3.1-25.el7_9.x86_64 1171: #7 28.07 --> Processing Dependency: perl(Term::ReadKey) for package: git-1.8.3.1-25.el7_9.x86_64 1172: #7 28.07 --> Processing Dependency: perl(Git) for package: git-1.8.3.1-25.el7_9.x86_64 1173: #7 28.07 --> Processing Dependency: perl(Getopt::Long) for package: git-1.8.3.1-25.el7_9.x86_64 1174: #7 28.07 --> Processing Dependency: perl(File::Temp) for package: git-1.8.3.1-25.el7_9.x86_64 1175: #7 28.08 --> Processing Dependency: perl(File::Spec) for package: git-1.8.3.1-25.el7_9.x86_64 1176: #7 28.08 --> Processing Dependency: perl(File::Path) for package: git-1.8.3.1-25.el7_9.x86_64 1177: #7 28.08 --> Processing Dependency: perl(Exporter) for package: git-1.8.3.1-25.el7_9.x86_64 1178: #7 28.08 --> Processing Dependency: perl(Error) for package: git-1.8.3.1-25.el7_9.x86_64 ... 1206: #7 28.11 ---> Package libuv.x86_64 1:1.44.2-1.el7 will be installed 1207: #7 28.11 ---> Package libzstd.x86_64 0:1.5.5-1.el7 will be installed 1208: #7 28.11 ---> Package openssh-clients.x86_64 0:7.4p1-23.el7_9 will be installed 1209: #7 28.11 --> Processing Dependency: openssh = 7.4p1-23.el7_9 for package: openssh-clients-7.4p1-23.el7_9.x86_64 1210: #7 28.12 --> Processing Dependency: fipscheck-lib(x86-64) >= 1.3.0 for package: openssh-clients-7.4p1-23.el7_9.x86_64 1211: #7 28.12 --> Processing Dependency: libfipscheck.so.1()(64bit) for package: openssh-clients-7.4p1-23.el7_9.x86_64 1212: #7 28.12 --> Processing Dependency: libedit.so.0()(64bit) for package: openssh-clients-7.4p1-23.el7_9.x86_64 1213: #7 28.12 ---> Package perl-Carp.noarch 0:1.26-244.el7 will be installed 1214: #7 28.12 ---> Package perl-Error.noarch 1:0.17020-2.el7 will be installed ... 1290: #7 28.22 libedit x86_64 3.0-12.20121213cvs.el7 base 92 k 1291: #7 28.22 libtirpc x86_64 0.2.4-0.16.el7 base 89 k 1292: #7 28.22 libuv x86_64 1:1.44.2-1.el7 epel 145 k 1293: #7 28.22 libzstd x86_64 1.5.5-1.el7 epel 292 k 1294: #7 28.22 openssh x86_64 7.4p1-23.el7_9 updates 510 k 1295: #7 28.22 openssh-clients x86_64 7.4p1-23.el7_9 updates 655 k 1296: #7 28.22 perl-Carp noarch 1.26-244.el7 base 19 k 1297: #7 28.22 perl-Encode x86_64 2.51-7.el7 base 1.5 M 1298: #7 28.22 perl-Error noarch 1:0.17020-2.el7 base 32 k ... 1385: #7 30.42 Installing : perl-File-Temp-0.23.01-3.el7.noarch 24/50 1386: #7 30.43 Installing : perl-File-Path-2.09-2.el7.noarch 25/50 1387: #7 30.43 Installing : perl-threads-shared-1.43-6.el7.x86_64 26/50 1388: #7 30.44 Installing : perl-threads-1.87-4.el7.x86_64 27/50 1389: #7 30.46 Installing : perl-Filter-1.49-3.el7.x86_64 28/50 1390: #7 30.51 Installing : 4:perl-libs-5.16.3-299.el7_9.x86_64 29/50 1391: #7 30.55 Installing : perl-Getopt-Long-2.40-3.el7.noarch 30/50 1392: #7 31.31 Installing : 4:perl-5.16.3-299.el7_9.x86_64 31/50 1393: #7 31.33 Installing : 1:perl-Error-0.17020-2.el7.noarch 32/50 ... 1434: #7 34.99 Verifying : perl-Time-Local-1.2300-2.el7.noarch 23/50 1435: #7 35.00 Verifying : cmake3-data-3.17.5-1.el7.noarch 24/50 1436: #7 35.01 Verifying : 1:perl-Pod-Escapes-1.04-299.el7_9.noarch 25/50 1437: #7 35.01 Verifying : python3-libs-3.6.8-21.el7_9.x86_64 26/50 1438: #7 35.02 Verifying : fipscheck-1.4.1-6.el7.x86_64 27/50 1439: #7 35.02 Verifying : perl-Carp-1.26-244.el7.noarch 28/50 1440: #7 35.03 Verifying : 1:libuv-1.44.2-1.el7.x86_64 29/50 1441: #7 35.03 Verifying : less-458-10.el7_9.x86_64 30/50 1442: #7 35.04 Verifying : 1:perl-Error-0.17020-2.el7.noarch 31/50 ... 1474: #7 35.15 libedit.x86_64 0:3.0-12.20121213cvs.el7 1475: #7 35.15 libtirpc.x86_64 0:0.2.4-0.16.el7 1476: #7 35.15 libuv.x86_64 1:1.44.2-1.el7 1477: #7 35.15 libzstd.x86_64 0:1.5.5-1.el7 1478: #7 35.15 openssh.x86_64 0:7.4p1-23.el7_9 1479: #7 35.15 openssh-clients.x86_64 0:7.4p1-23.el7_9 1480: #7 35.15 perl-Carp.noarch 0:1.26-244.el7 1481: #7 35.15 perl-Encode.x86_64 0:2.51-7.el7 1482: #7 35.15 perl-Error.noarch 1:0.17020-2.el7 ... 1523: #8 0.161 Dload Upload Total Spent Left Speed 1524: #8 0.161 1525: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 1526: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 1527: #8 0.814 1528: 0 1600k 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 1529: 100 1600k 100 1600k 0 0 2411k 0 --:--:-- --:--:-- --:--:-- 156M 1530: #8 0.827 /bin/sh: unzip: command not found 1531: #8 ERROR: process "/bin/sh -c FILE=$([ \"$(arch)\" = \"aarch64\" ] && echo \"protoc-${PROTOBUF_VER#?}-linux-aarch_64.zip\" || echo \"protoc-${PROTOBUF_VER#?}-linux-$(arch).zip\"); curl -LO \"https://github.com/protocolbuffers/protobuf/releases/download/${PROTOBUF_VER}/${FILE}\" && unzip \"$FILE\" -d /usr/local/ && rm -f \"$FILE\"" did not complete successfully: exit code: 127 ... 1540: ------ 1541: Dockerfile:28 1542: -------------------- 1543: 27 | ARG PROTOBUF_VER=v3.15.8 1544: 28 | >>> RUN FILE=$([ "$(arch)" = "aarch64" ] && echo "protoc-${PROTOBUF_VER#?}-linux-aarch_64.zip" || echo "protoc-${PROTOBUF_VER#?}-linux-$(arch).zip"); \ 1545: 29 | >>> curl -LO "https://github.com/protocolbuffers/protobuf/releases/download/${PROTOBUF_VER}/${FILE}" && unzip "$FILE" -d /usr/local/ && rm -f "$FILE" 1546: 30 | 1547: -------------------- 1548: ERROR: failed to solve: process "/bin/sh -c FILE=$([ \"$(arch)\" = \"aarch64\" ] && echo \"protoc-${PROTOBUF_VER#?}-linux-aarch_64.zip\" || echo \"protoc-${PROTOBUF_VER#?}-linux-$(arch).zip\"); curl -LO \"https://github.com/protocolbuffers/protobuf/releases/download/${PROTOBUF_VER}/${FILE}\" && unzip \"$FILE\" -d /usr/local/ && rm -f \"$FILE\"" did not complete successfully: exit code: 127 1549: exit status 1. Docker build ran into internal error. Please retry. 1550: If this keeps happening, please open an issue.. 1551: ##[error]Process completed with exit code 1.

---

✨ CI feedback usage guide:

---

The CI feedback tool (/checks) automatically triggers when a PR has a failed check.
The tool analyzes the failed checks and provides several feedbacks:

• Failed stage
• Failed test name
• Failure summary
• Relevant error logs

In addition to being automatically triggered, the tool can also be invoked manually by commenting on a PR:

/checks "https://github.com/{repo_name}/actions/runs/{run_number}/job/{job_number}"

where {repo_name} is the name of the repository, {run_number} is the run number of the failed check, and {job_number} is the job number of the failed check.

Configuration options

• enable_auto_checks_feedback - if set to true, the tool will automatically provide feedback when a check is failed. Default is true.
• excluded_checks_list - a list of checks to exclude from the feedback, for example: ["check1", "check2"]. Default is an empty list.
• enable_help_text - if set to true, the tool will provide a help message with the feedback. Default is true.
• persistent_comment - if set to true, the tool will overwrite a previous checks comment with the new feedback. Default is true.
• final_update_message - if persistent_comment is true and updating a previous checks message, the tool will also create a new message: "Persistent checks updated to latest commit". Default is true.

See more information about the checks tool in the docs.

[ti-chi-bot]

I have already done a preliminary review for you, and I hope to help you do a better job.

This pull request (PR) appears to be fixing the end of file (EOF) problem in the Dockerfiles used in CI/CD for CentOS. Here is a summary of the key changes:

1. The CentOS 7 mirror URLs have been replaced with the CentOS 7 vault URLs because CentOS 7 has reached the end of life (EOL).
2. The versions of the base images have been updated from v1.8.0 to v1.9.0.
3. The Golang version in the CI base Dockerfile has been updated from 1.21.10 to 1.21.12.
4. The oras tool in the CI base Dockerfile is now copied from the bitnami/oras:1.2.0 image.
5. The Git tool in the CI base Dockerfile is now copied from the bitnami/git:2.45.2 image.
6. The GIT_VERSION argument in the release-build-base Dockerfile has been updated to 2.40.3.

The potential problems in this PR are:

1. If the vault URLs for CentOS 7 are not maintained, it may affect the building of the Docker images.
2. If the updated base images are not backward compatible, it may cause problems in the CI/CD pipeline.
3. If the updated Golang, oras, and Git versions are not backward compatible, it may cause problems in the CI/CD pipeline.

Fixing suggestions:

1. Monitor the vault URLs for CentOS 7 regularly to ensure their availability.
2. Test the updated base images to ensure they are backward compatible.
3. Test the updated Golang, oras, and Git versions to ensure they are backward compatible.

Overall, this PR seems to be a routine update and should not cause major issues if thoroughly tested.

[wuhuizuo]

/describe

[codiumai-pr-agent-pro]

PR Description updated to latest commit (1f1fde6)

• Copy walkthrough table to "Files Changed" Tab

[wuhuizuo]

/approve

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: wuhuizuo

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [wuhuizuo]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment