ci: fix docker cache hash collision #58549

pietroalbini · 2019-02-17T21:38:00Z

#58416 uncovered a bug in our caching for docker images: if the image foo pulls files from the image bar and a file in bar changed, the hash of foo will be the same even though it should be different. In that PR's case, dist-i686-linux pulls scripts from dist-x86_64-linux, and the PR only changed those scripts, causing an hash collision for dist-i686-linux.

We have to fix this, since the image will be rebuilt every time bors switches from testing master to testing beta/stable (and when it switches back), making CI way more painful than it currently is.

The approach used by this PR is to just include all the files in src/ci/docker in the hash. It's a bit heavy-handed and it will cause a rebuild of all the images every time a single image changes, but it's the best I can think of.

r? @Mark-Simulacrum
cc @alexcrichton @kennytm

Mark-Simulacrum · 2019-02-17T21:40:34Z

@bors r+ p=50

bors · 2019-02-17T21:40:35Z

📌 Commit 9c2dc109fe192ef880b17ebc3b14c862707e9358 has been approved by Mark-Simulacrum

pietroalbini · 2019-02-17T21:47:08Z

Removed the backport nominations, stable and beta works fine with the old hashing code and we're not going to backport CI changes to beta 10 days before the released, or even more to stable. Avoiding the backport will save us another cycle of timeout->retry.

bors · 2019-02-17T23:12:39Z

⌛ Testing commit 9c2dc109fe192ef880b17ebc3b14c862707e9358 with merge 4744156ef241eecc5e41435b7e774718f6361b3f...

bors · 2019-02-18T02:13:19Z

💔 Test failed - checks-travis

rust-highfive · 2019-02-18T02:13:21Z

The job dist-x86_64-linux of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

[00:01:13] 
[00:01:13] Total download size: 4.9 M
[00:01:13] Downloading Packages:
[00:01:16] --------------------------------------------------------------------------------
[00:01:16] Total                                           1.7 MB/s | 4.9 MB     00:02     
[00:01:16] warning: rpmts_HdrFromFdno: Header V3 DSA signature: NOKEY, key ID e8562897
[00:01:16] Importing GPG key 0xE8562897 "CentOS-5 Key (CentOS 5 Official Signing Key) <centos-5-key@centos.org>" from /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5
[00:01:16] Running Transaction Test
[00:01:16] Finished Transaction Test
[00:01:16] Transaction Test Succeeded
[00:01:16] Running Transaction
---
[00:04:03] + hide_output make install
[00:04:03] + set +x
[00:04:25] shared.sh: line 1:   352 Terminated              bash -c "while true; do sleep 30; echo \$(date) - building ...; done"
[00:04:25] + cd ..
[00:04:25] + rm -rf openssl-1.0.2k
[00:04:25] ./build-openssl.sh: line 16:  4115 Terminated              bash -c "while true; do sleep 30; echo \$(date) - building ...; done"  (wd: /tmp/openssl-1.0.2k)
[00:04:25] + ln -nsf /etc/pki/tls/cert.pem /rustroot/ssl/
[00:04:26]  ---> 34209d50db24
[00:04:26] Step 14/41 : COPY dist-x86_64-linux/build-curl.sh /tmp/
[00:04:26]  ---> 58d68df0fb9d
[00:04:26] Step 15/41 : RUN ./build-curl.sh
[00:04:26] Step 15/41 : RUN ./build-curl.sh
[00:04:26]  ---> Running in 8b1c82bb1528
[00:04:27] + source shared.sh
[00:04:27] + VERSION=7.51.0
[00:04:27] + curl http://cool.haxx.se/download/curl-7.51.0.tar.bz2
[00:04:28]   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
[00:04:28]                                  Dload  Upload   Total   Spent    Left  Speed
[00:04:30] 
  0 2509k    0 14215    0     0   9267      0  0:04:37  0:00:01  0:04:36  9267
  0 2509k    0 14215    0     0   9267      0  0:04:37  0:00:01  0:04:36  9267
 28 2509k   28  704k    0     0   313k      0  0:00:08  0:00:02  0:00:06  966k
100 2509k  100 2509k    0     0   858k      0  0:00:02  0:00:02 --:--:-- 1796k
[00:04:30] + mkdir curl-build
[00:04:30] + cd curl-build
[00:04:30] + hide_output ../curl-7.51.0/configure --prefix=/rustroot --with-ssl=/rustroot --disable-sspi --disable-gopher --disable-smtp --disable-smb --disable-imap --disable-pop3 --disable-tftp --disable-telnet --disable-manual --disable-dict --disable-rtsp --disable-ldaps --disable-ldap
[00:04:55] + hide_output make -j10
[00:04:55] + set +x
[00:05:09] shared.sh: line 1:    15 Terminated              bash -c "while true; do sleep 30; echo \$(date) - building ...; done"
[00:05:09] + hide_output make install
---
 78 67.8M   78 53.4M    0     0  6857k      0  0:00:10  0:00:07  0:00:03 7353k
 90 67.8M   90 61.4M    0     0  7008k      0  0:00:09  0:00:08  0:00:01 7408k
100 67.8M  100 67.8M    0     0  7055k      0  0:00:09  0:00:09 --:--:-- 7709k
[00:09:08] + cd gcc-5.5.0
[00:09:08] + sed -i 's|ftp://gcc\.gnu\.org/|http://gcc.gnu.org/|g' ./contrib/download_prerequisites
[00:09:08] --2019-02-17 23:23:08--  http://gcc.gnu.org/pub/gcc/infrastructure/mpfr-2.4.2.tar.bz2
[00:09:08] Resolving gcc.gnu.org... 209.132.180.131
[00:09:08] Connecting to gcc.gnu.org|209.132.180.131|:80... connected.
[00:09:08] HTTP request sent, awaiting response... 200 OK
---
[01:18:33]  ---> 2b93fbe595b0
[01:18:33] Step 25/41 : RUN ./build-clang.sh
[01:18:33]  ---> Running in e667474dc3dc
[01:18:34] + source shared.sh
[01:18:34] + LLVM=llvmorg-8.0.0-rc2
[01:18:34] + mkdir llvm-project
[01:18:34] + cd llvm-project
[01:18:34] + curl -L https://github.com/llvm/llvm-project/archive/llvmorg-8.0.0-rc2.tar.gz
[01:18:34] + tar xzf - --strip-components=1
[01:18:34]                                  Dload  Upload   Total   Spent    Left  Speed
[01:18:34] 
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
---
[01:18:52] + cd clang-build
[01:18:52] + INC=/rustroot/include
[01:18:52] + INC=/rustroot/include:/rustroot/lib/gcc/x86_64-unknown-linux-gnu/5.5.0/include-fixed
[01:18:52] + INC=/rustroot/include:/rustroot/lib/gcc/x86_64-unknown-linux-gnu/5.5.0/include-fixed:/usr/include
[01:18:52] + hide_output cmake ../llvm -DCMAKE_C_COMPILER=/rustroot/bin/gcc -DCMAKE_CXX_COMPILER=/rustroot/bin/g++ -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/rustroot -DLLVM_TARGETS_TO_BUILD=X86 '-DLLVM_ENABLE_PROJECTS=clang;lld' -DC_INCLUDE_DIRS=/rustroot/include:/rustroot/lib/gcc/x86_64-unknown-linux-gnu/5.5.0/include-fixed:/usr/include
[01:19:22] Mon Feb 18 00:33:23 UTC 2019 - building ...
[01:19:33] + hide_output make -j10
[01:19:33] + set +x
[01:20:03] Mon Feb 18 00:34:03 UTC 2019 - building ...
---
[02:47:40]  ---> 0b8dc51d10c1
[02:47:40] Step 32/41 : RUN ./build-perl.sh
[02:47:40]  ---> Running in 3fa8f5600bd4
[02:47:41] + source shared.sh
[02:47:41] + curl https://www.cpan.org/src/5.0/perl-5.28.0.tar.gz
[02:47:41]   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
[02:47:41]                                  Dload  Upload   Total   Spent    Left  Speed
[02:47:42] 
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
 98 17.0M   98 16.8M    0     0  13.9M      0  0:00:01  0:00:01 --:--:-- 15.1M
100 17.0M  100 17.0M    0     0  14.0M      0  0:00:01  0:00:01 --:--:-- 15.2M
[02:47:42] + cd perl-5.28.0
[02:47:42] + CC=gcc
[02:47:42] + CFLAGS='-I /rustroot/include -fgnu89-inline'
[02:47:42] + hide_output ./configure.gnu
[02:47:42] + set +x
[02:48:12] Mon Feb 18 02:02:12 UTC 2019 - building ...
[02:48:26] + hide_output make -j10

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

kennytm · 2019-02-18T02:59:38Z

@bors retry

bors · 2019-02-18T05:36:16Z

⌛ Testing commit 9c2dc109fe192ef880b17ebc3b14c862707e9358 with merge 79cb5f6e34b9ff36279632af05e8a5343d0fcacb...

bors · 2019-02-18T08:37:13Z

💔 Test failed - checks-travis

bors · 2019-02-18T14:20:27Z

💔 Test failed - checks-travis

rust-highfive · 2019-02-18T14:20:28Z

The job dist-x86_64-linux of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

[00:01:36] 
[00:01:36] Total download size: 4.9 M
[00:01:36] Downloading Packages:
[00:01:39] --------------------------------------------------------------------------------
[00:01:39] Total                                           1.6 MB/s | 4.9 MB     00:02     
[00:01:39] warning: rpmts_HdrFromFdno: Header V3 DSA signature: NOKEY, key ID e8562897
[00:01:39] Importing GPG key 0xE8562897 "CentOS-5 Key (CentOS 5 Official Signing Key) <centos-5-key@centos.org>" from /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5
[00:01:39] Running Transaction Test
[00:01:39] Finished Transaction Test
[00:01:39] Transaction Test Succeeded
[00:01:39] Running Transaction
---
[00:05:09] + hide_output make install
[00:05:09] + set +x
[00:05:32] shared.sh: line 1:   352 Terminated              bash -c "while true; do sleep 30; echo \$(date) - building ...; done"
[00:05:32] + cd ..
[00:05:32] + rm -rf openssl-1.0.2k
[00:05:32] ./build-openssl.sh: line 16:  4115 Terminated              bash -c "while true; do sleep 30; echo \$(date) - building ...; done"  (wd: /tmp/openssl-1.0.2k)
[00:05:32] + ln -nsf /etc/pki/tls/cert.pem /rustroot/ssl/
[00:05:33]  ---> 73f471e3c451
[00:05:33] Step 14/41 : COPY dist-x86_64-linux/build-curl.sh /tmp/
[00:05:33]  ---> ae3ce200fe3e
[00:05:33] Step 15/41 : RUN ./build-curl.sh
[00:05:33] Step 15/41 : RUN ./build-curl.sh
[00:05:33]  ---> Running in 0d5092356bab
[00:05:33] + source shared.sh
[00:05:33] + VERSION=7.51.0
[00:05:33] + tar xjf -
[00:05:33] + curl http://cool.haxx.se/download/curl-7.51.0.tar.bz2
[00:05:35]                                  Dload  Upload   Total   Spent    Left  Speed
[00:05:36] 
  0 2509k    0 14215    0     0   9242      0  0:04:38  0:00:01  0:04:37  9242
 34 2509k   34  875k    0     0   376k      0  0:00:06  0:00:02  0:00:04 1096k
 34 2509k   34  875k    0     0   376k      0  0:00:06  0:00:02  0:00:04 1096k
100 2509k  100 2509k    0     0   884k      0  0:00:02  0:00:02 --:--:-- 1922k
[00:05:36] + mkdir curl-build
[00:05:36] + cd curl-build
[00:05:36] + hide_output ../curl-7.51.0/configure --prefix=/rustroot --with-ssl=/rustroot --disable-sspi --disable-gopher --disable-smtp --disable-smb --disable-imap --disable-pop3 --disable-tftp --disable-telnet --disable-manual --disable-dict --disable-rtsp --disable-ldaps --disable-ldap
[00:06:03] + hide_output make -j10
[00:06:03] + set +x
[00:06:16] shared.sh: line 1:    13 Terminated              bash -c "while true; do sleep 30; echo \$(date) - building ...; done"
[00:06:16] + hide_output make install
---
 89 67.8M   89 60.6M    0     0  6378k      0  0:00:10  0:00:09  0:00:01 6275k
 98 67.8M   98 66.9M    0     0  6380k      0  0:00:10  0:00:10 --:--:-- 6796k
100 67.8M  100 67.8M    0     0  6363k      0  0:00:10  0:00:10 --:--:-- 6789k
[00:10:59] + cd gcc-5.5.0
[00:10:59] + sed -i 's|ftp://gcc\.gnu\.org/|http://gcc.gnu.org/|g' ./contrib/download_prerequisites
[00:10:59] --2019-02-18 11:32:25--  http://gcc.gnu.org/pub/gcc/infrastructure/mpfr-2.4.2.tar.bz2
[00:10:59] Resolving gcc.gnu.org... 209.132.180.131
[00:10:59] Connecting to gcc.gnu.org|209.132.180.131|:80... connected.
[00:10:59] HTTP request sent, awaiting response... 200 OK
---
[01:16:30]  ---> 8f0b1cb2430a
[01:16:30] Step 25/41 : RUN ./build-clang.sh
[01:16:30]  ---> Running in ad0067fbc26d
[01:16:31] + source shared.sh
[01:16:31] + LLVM=llvmorg-8.0.0-rc2
[01:16:31] + mkdir llvm-project
[01:16:31] + cd llvm-project
[01:16:31] + curl -L https://github.com/llvm/llvm-project/archive/llvmorg-8.0.0-rc2.tar.gz
[01:16:31] + tar xzf - --strip-components=1
[01:16:31]                                  Dload  Upload   Total   Spent    Left  Speed
[01:16:31] 
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100   136    0   136    0     0    555      0 --:--:-- --:--:-- --:--:--   623
---
[01:16:49] + cd clang-build
[01:16:49] + INC=/rustroot/include
[01:16:49] + INC=/rustroot/include:/rustroot/lib/gcc/x86_64-unknown-linux-gnu/5.5.0/include-fixed
[01:16:49] + INC=/rustroot/include:/rustroot/lib/gcc/x86_64-unknown-linux-gnu/5.5.0/include-fixed:/usr/include
[01:16:49] + hide_output cmake ../llvm -DCMAKE_C_COMPILER=/rustroot/bin/gcc -DCMAKE_CXX_COMPILER=/rustroot/bin/g++ -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/rustroot -DLLVM_TARGETS_TO_BUILD=X86 '-DLLVM_ENABLE_PROJECTS=clang;lld' -DC_INCLUDE_DIRS=/rustroot/include:/rustroot/lib/gcc/x86_64-unknown-linux-gnu/5.5.0/include-fixed:/usr/include
[01:17:19] Mon Feb 18 12:38:45 UTC 2019 - building ...
[01:17:31] + hide_output make -j10
[01:17:31] + set +x
[01:18:01] Mon Feb 18 12:39:27 UTC 2019 - building ...
---
[02:44:38]  ---> 63df0ad639c8
[02:44:38] Step 32/41 : RUN ./build-perl.sh
[02:44:38]  ---> Running in c89156415ae9
[02:44:39] + source shared.sh
[02:44:39] + curl https://www.cpan.org/src/5.0/perl-5.28.0.tar.gz
[02:44:39]   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
[02:44:39]                                  Dload  Upload   Total   Spent    Left  Speed
[02:44:40] 
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
 57 17.0M   57  9.8M    0     0  10.3M      0  0:00:01 --:--:--  0:00:01 11.4M
100 17.0M  100 17.0M    0     0  12.8M      0  0:00:01  0:00:01 --:--:-- 13.8M
[02:44:40] + cd perl-5.28.0
[02:44:40] + CC=gcc
[02:44:40] + CFLAGS='-I /rustroot/include -fgnu89-inline'
[02:44:40] + hide_output ./configure.gnu
[02:44:40] + set +x
[02:45:10] Mon Feb 18 14:06:36 UTC 2019 - building ...
[02:45:28] + hide_output make -j10
---
[02:55:57] [RUSTC-TIMING] rustc_errors test:false 9.363
[02:56:53] [RUSTC-TIMING] syntax test:false 55.758
[02:56:53]    Compiling syntax_ext v0.0.0 (/checkout/src/libsyntax_ext)
[02:57:21] [RUSTC-TIMING] syntax_ext test:false 28.099
The job exceeded the maximum time limit for jobs, and has been terminated.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

Before this commit the hash used to cache docker images was calculated from the image's files and the files in the scripts/ directory. This worked fine when all the files used by an image were in those directories, but some images pull files from other images, causing hash collisions in some cases. This commit changes the hash to include the files of all the docker images, causing a rebuild of all the images when a single one changes. That's a bit heavy-handed, but we have no way to track which files an image pulls in and hash collisions are really painful to deal with.

pietroalbini · 2019-02-18T14:29:09Z

@bors r=Mark-Simulacrum

bors · 2019-02-18T14:29:12Z

📌 Commit 2b2045d has been approved by Mark-Simulacrum

bors · 2019-02-18T14:29:23Z

⌛ Testing commit 2b2045d with merge 97886206790a30050e82679c9779a802fc12db8f...

bors · 2019-02-18T17:30:08Z

💔 Test failed - checks-travis

rust-highfive · 2019-02-18T17:30:09Z

The job dist-x86_64-linux of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

[00:01:01] 
[00:01:01] Total download size: 4.9 M
[00:01:01] Downloading Packages:
[00:01:04] --------------------------------------------------------------------------------
[00:01:04] Total                                           1.8 MB/s | 4.9 MB     00:02     
[00:01:04] warning: rpmts_HdrFromFdno: Header V3 DSA signature: NOKEY, key ID e8562897
[00:01:04] Importing GPG key 0xE8562897 "CentOS-5 Key (CentOS 5 Official Signing Key) <centos-5-key@centos.org>" from /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5
[00:01:04] Running Transaction Test
[00:01:04] Finished Transaction Test
[00:01:04] Transaction Test Succeeded
[00:01:04] Running Transaction
---
[00:03:47] + hide_output make install
[00:03:47] + set +x
[00:04:09] shared.sh: line 1:   352 Terminated              bash -c "while true; do sleep 30; echo \$(date) - building ...; done"
[00:04:09] + cd ..
[00:04:09] + rm -rf openssl-1.0.2k
[00:04:09] ./build-openssl.sh: line 16:  4115 Terminated              bash -c "while true; do sleep 30; echo \$(date) - building ...; done"  (wd: /tmp/openssl-1.0.2k)
[00:04:09] + ln -nsf /etc/pki/tls/cert.pem /rustroot/ssl/
[00:04:10]  ---> afc0c6410fad
[00:04:10] Step 14/41 : COPY dist-x86_64-linux/build-curl.sh /tmp/
[00:04:10]  ---> 2b56fbf230d7
[00:04:10] Step 15/41 : RUN ./build-curl.sh
[00:04:10] Step 15/41 : RUN ./build-curl.sh
[00:04:10]  ---> Running in fa260e419b25
[00:04:10] + source shared.sh
[00:04:10] + VERSION=7.51.0
[00:04:10] + curl http://cool.haxx.se/download/curl-7.51.0.tar.bz2
[00:04:12]   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
[00:04:12]                                  Dload  Upload   Total   Spent    Left  Speed
[00:04:13] 
  0 2509k    0  6975    0     0   4490      0  0:09:32  0:00:01  0:09:31  4490
  0 2509k    0  6975    0     0   4490      0  0:09:32  0:00:01  0:09:31  4490
 18 2509k   18  454k    0     0   208k      0  0:00:12  0:00:02  0:00:10  719k
100 2509k  100 2509k    0     0   877k      0  0:00:02  0:00:02 --:--:-- 1915k
[00:04:13] + mkdir curl-build
[00:04:13] + cd curl-build
[00:04:13] + hide_output ../curl-7.51.0/configure --prefix=/rustroot --with-ssl=/rustroot --disable-sspi --disable-gopher --disable-smtp --disable-smb --disable-imap --disable-pop3 --disable-tftp --disable-telnet --disable-manual --disable-dict --disable-rtsp --disable-ldaps --disable-ldap
[00:04:37] + hide_output make -j10
[00:04:37] + set +x
[00:04:51] shared.sh: line 1:    13 Terminated              bash -c "while true; do sleep 30; echo \$(date) - building ...; done"
[00:04:51] + hide_output make install
---
 95 67.8M   95 64.5M    0     0  2540k      0  0:00:27  0:00:26  0:00:01 1790k
 97 67.8M   97 66.4M    0     0  2515k      0  0:00:27  0:00:27 --:--:-- 1786k
100 67.8M  100 67.8M    0     0  2485k      0  0:00:27  0:00:27 --:--:-- 1763k
[00:09:11] + cd gcc-5.5.0
[00:09:11] + sed -i 's|ftp://gcc\.gnu\.org/|http://gcc.gnu.org/|g' ./contrib/download_prerequisites
[00:09:11] --2019-02-18 14:40:21--  http://gcc.gnu.org/pub/gcc/infrastructure/mpfr-2.4.2.tar.bz2
[00:09:11] Resolving gcc.gnu.org... 209.132.180.131
[00:09:11] Connecting to gcc.gnu.org|209.132.180.131|:80... connected.
[00:09:11] HTTP request sent, awaiting response... 200 OK
---
[01:15:26]  ---> 1db9b0f2fa0a
[01:15:26] Step 25/41 : RUN ./build-clang.sh
[01:15:26]  ---> Running in bb756da93eea
[01:15:26] + source shared.sh
[01:15:26] + LLVM=llvmorg-8.0.0-rc2
[01:15:26] + mkdir llvm-project
[01:15:26] + cd llvm-project
[01:15:26] + curl -L https://github.com/llvm/llvm-project/archive/llvmorg-8.0.0-rc2.tar.gz
[01:15:26] + tar xzf - --strip-components=1
[01:15:26]                                  Dload  Upload   Total   Spent    Left  Speed
[01:15:27] 
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100   136    0   136    0     0    410      0 --:--:-- --:--:-- --:--:--   451
---
[01:15:57] + cd clang-build
[01:15:57] + INC=/rustroot/include
[01:15:57] + INC=/rustroot/include:/rustroot/lib/gcc/x86_64-unknown-linux-gnu/5.5.0/include-fixed
[01:15:57] + INC=/rustroot/include:/rustroot/lib/gcc/x86_64-unknown-linux-gnu/5.5.0/include-fixed:/usr/include
[01:15:57] + hide_output cmake ../llvm -DCMAKE_C_COMPILER=/rustroot/bin/gcc -DCMAKE_CXX_COMPILER=/rustroot/bin/g++ -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/rustroot -DLLVM_TARGETS_TO_BUILD=X86 '-DLLVM_ENABLE_PROJECTS=clang;lld' -DC_INCLUDE_DIRS=/rustroot/include:/rustroot/lib/gcc/x86_64-unknown-linux-gnu/5.5.0/include-fixed:/usr/include
[01:16:27] Mon Feb 18 15:47:37 UTC 2019 - building ...
[01:16:37] + hide_output make -j10
[01:16:37] + set +x
[01:17:07] Mon Feb 18 15:48:17 UTC 2019 - building ...
---
[02:45:56]  ---> 2016e3e17e05
[02:45:56] Step 32/41 : RUN ./build-perl.sh
[02:45:56]  ---> Running in b75b9e0b467a
[02:45:56] + source shared.sh
[02:45:56] + curl https://www.cpan.org/src/5.0/perl-5.28.0.tar.gz
[02:45:56]   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
[02:45:56]                                  Dload  Upload   Total   Spent    Left  Speed
[02:45:57] 
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
 30 17.0M   30 5293k    0     0  9038k      0  0:00:01 --:--:--  0:00:01 9470k
100 17.0M  100 17.0M    0     0  15.3M      0  0:00:01  0:00:01 --:--:-- 15.7M
[02:45:57] + cd perl-5.28.0
[02:45:57] + CC=gcc
[02:45:57] + CFLAGS='-I /rustroot/include -fgnu89-inline'
[02:45:57] + hide_output ./configure.gnu
[02:45:57] + set +x
[02:46:27] Mon Feb 18 17:17:37 UTC 2019 - building ...
[02:46:39] + hide_output make -j10
---
[02:57:14] [RUSTC-TIMING] rustc_errors test:false 13.461
[02:58:11] [RUSTC-TIMING] syntax test:false 57.067
[02:58:11]    Compiling syntax_ext v0.0.0 (/checkout/src/libsyntax_ext)
[02:58:40] [RUSTC-TIMING] syntax_ext test:false 28.681
The job exceeded the maximum time limit for jobs, and has been terminated.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

pietroalbini · 2019-02-18T17:34:11Z

@bors retry

bors · 2019-02-18T17:34:25Z

⌛ Testing commit 2b2045d with merge 146aa60...

@Mark-Simulacrum

…crum ci: fix docker cache hash collision #58416 uncovered a bug in our caching for docker images: if the image `foo` pulls files from the image `bar` and a file in `bar` changed, the hash of `foo` will be the same even though it should be different. In that PR's case, `dist-i686-linux` pulls scripts from `dist-x86_64-linux`, and the PR only changed those scripts, causing an hash collision for `dist-i686-linux`. We have to fix this, since the image will be rebuilt every time bors switches from testing master to testing beta/stable (and when it switches back), making CI way more painful than it currently is. The approach used by this PR is to just include all the files in `src/ci/docker` in the hash. It's a bit heavy-handed and it will cause a rebuild of all the images every time a single image changes, but it's the best I can think of. r? @Mark-Simulacrum cc @alexcrichton @kennytm

bors · 2019-02-18T20:18:24Z

☀️ Test successful - checks-travis, status-appveyor
Approved by: Mark-Simulacrum
Pushing 146aa60 to master...

…r=pietroalbini Calculate Docker cache hash precisely from Dockerfile's dependencies rust-lang#58549 changed the Docker cache calculation to include every file under `src/ci/docker`, so that when files under `dist-x86_64-linux` is changed, its dependent image `dist-i686-linux` will also be rebuilt. However, this ultraconservative solution caused the `dist-i686-linux` to be rebuilt every time an irrelevant Dockerfile (e.g. the PowerPC ones) is changed, which increases the building time beyond 3 hours and forcing a spurious but expected failure. This commit instead parses the Dockerfile itself and look for the actual dependencies. The scripts needs to be copied into the Docker image, which must be done with the COPY command, so we just need to find all lines with a COPY command and add the source file into the hash calculator. Note: this script only handles single-lined COPY command in the form `COPY src1 src2 src3 dst`, since these are the only variant used inside this repository.

rust-highfive assigned Mark-Simulacrum Feb 17, 2019

pietroalbini added the T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. label Feb 17, 2019

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Feb 17, 2019

This was referenced Feb 17, 2019

[beta] Rollup backports #58537

Merged

[BETA] Update cargo #58522

Merged

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 17, 2019

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Feb 18, 2019

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 18, 2019

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Feb 18, 2019

pietroalbini force-pushed the fix-docker-ci-hash branch from 9c2dc10 to 2b2045d Compare February 18, 2019 14:28

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 18, 2019

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Feb 18, 2019

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 18, 2019

bors added the merged-by-bors This PR was explicitly merged by bors. label Feb 18, 2019

bors merged commit 2b2045d into rust-lang:master Feb 18, 2019

pietroalbini deleted the fix-docker-ci-hash branch February 18, 2019 20:31

kennytm mentioned this pull request Mar 17, 2019

Calculate Docker cache hash precisely from Dockerfile's dependencies #59253

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ci: fix docker cache hash collision #58549

ci: fix docker cache hash collision #58549

pietroalbini commented Feb 17, 2019 •

edited by pnkfelix

Loading

Mark-Simulacrum commented Feb 17, 2019

bors commented Feb 17, 2019

pietroalbini commented Feb 17, 2019

bors commented Feb 17, 2019

bors commented Feb 18, 2019

rust-highfive commented Feb 18, 2019

kennytm commented Feb 18, 2019

bors commented Feb 18, 2019

bors commented Feb 18, 2019

bors commented Feb 18, 2019

rust-highfive commented Feb 18, 2019

pietroalbini commented Feb 18, 2019

bors commented Feb 18, 2019

bors commented Feb 18, 2019

bors commented Feb 18, 2019

rust-highfive commented Feb 18, 2019

pietroalbini commented Feb 18, 2019

bors commented Feb 18, 2019

bors commented Feb 18, 2019

ci: fix docker cache hash collision #58549

ci: fix docker cache hash collision #58549

Conversation

pietroalbini commented Feb 17, 2019 • edited by pnkfelix Loading

Mark-Simulacrum commented Feb 17, 2019

bors commented Feb 17, 2019

pietroalbini commented Feb 17, 2019

bors commented Feb 17, 2019

bors commented Feb 18, 2019

rust-highfive commented Feb 18, 2019

kennytm commented Feb 18, 2019

bors commented Feb 18, 2019

bors commented Feb 18, 2019

bors commented Feb 18, 2019

rust-highfive commented Feb 18, 2019

pietroalbini commented Feb 18, 2019

bors commented Feb 18, 2019

bors commented Feb 18, 2019

bors commented Feb 18, 2019

rust-highfive commented Feb 18, 2019

pietroalbini commented Feb 18, 2019

bors commented Feb 18, 2019

bors commented Feb 18, 2019

pietroalbini commented Feb 17, 2019 •

edited by pnkfelix

Loading