Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: util/path: CheckSystemDriveAndRemoveDriveLetter to preserve / #5317

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

profnandaa
Copy link
Collaborator

@profnandaa profnandaa commented Sep 10, 2024

The call to CheckSystemDriveAndRemoveDriveLetter() does not preserve the trailing / or \\. This happens because filepath.Clean() strips away any trailing slashes. For example /sample/ will be \\sample on Windows. This function was mainly written for Windows scenarios, which have System Drive Letters like C:/, etc.

This was causing cases like COPY testfile /testdir/ to be intepreted as COPY testfile /testdir, and if testdir is not explictly created before the call, it ends up being treated as a destination file other than a directory.

Fix this by checking that if we have a trailing / or \\, we preserve it after the call to filepath.Clean().

Fixes #5249

PS.

Also fixed for cross-building Windows from Linux, that would fail silently:

Repro dockerfile without RUN:

FROM mcr.microsoft.com/windows/nanoserver:ltsc2022
COPY test1.txt /sample/
#RUN type \sample\test1.txt

Build log:

$ docker buildx build --platform windows/amd64 `
    --builder buildkitd-dev --no-cache --tag=windows-test . `
    --progress plain `
    --output type=local,dest=./output

#6 [2/2] COPY test1.txt /sample/
#6 DONE 0.1s

#7 exporting to client directory
#7 copying files 31B
#7 copying files 230.88MB 0.8s done
#7 DONE 0.8s

Checking results, sample is a file instead of a directory:

# before
$ ls -l output/sample
-rw-r--r-- 1 root root 6 Sep 24 08:57 output/sample

# after
$ ls -l output/sample
total 4
-rw-r--r-- 1 root root 6 Sep 24 08:57 test1.txt

NOTE: also covered cases like, where platform-specific filepath.Clean() won't strip out the \\ on Linux:

COPY test1.txt \\sample\\

@profnandaa profnandaa changed the title fix: util/path: CheckSystemDriveAndRemoveDriveLetter to preserve / fix: util/path: CheckSystemDriveAndRemoveDriveLetter to preserve / Sep 10, 2024
@profnandaa
Copy link
Collaborator Author

Looking into the CI failures...

@profnandaa
Copy link
Collaborator Author

Found the regression caused by this, thankfully coz of the checked in integration tests!

Dockerfile:

FROM mcr.microsoft.com/windows/nanoserver:ltsc2022
COPY test1.txt /sample/
RUN type \sample\test1.txt

COPY test1.txt /
COPY test1.txt /test2.txt

RUN type test1.txt
RUN type test2.txt

Build log:

Dockerfile:5
--------------------
   3 |     RUN type \sample\test1.txt
   4 |     
   5 | >>> COPY test1.txt /
   6 |     COPY test1.txt /test2.txt
   7 |     
--------------------
error: failed to solve: removing drive letter: UNC paths are not supported
Process 5812 has exited with status 1

Fixing.

@profnandaa profnandaa force-pushed the fix-5249-copy-dir-path branch 4 times, most recently from ac8552f to af77ee9 Compare September 11, 2024 08:00
util/system/path.go Outdated Show resolved Hide resolved
util/system/path.go Outdated Show resolved Hide resolved
util/system/path.go Outdated Show resolved Hide resolved
util/system/path.go Outdated Show resolved Hide resolved
@profnandaa profnandaa force-pushed the fix-5249-copy-dir-path branch 3 times, most recently from 36e398e to 1a5289c Compare September 13, 2024 08:21
@profnandaa profnandaa marked this pull request as draft September 13, 2024 08:51
@profnandaa profnandaa marked this pull request as ready for review September 17, 2024 17:25
@profnandaa profnandaa force-pushed the fix-5249-copy-dir-path branch 2 times, most recently from aca0ccb to e8ae74c Compare September 24, 2024 08:27
@profnandaa profnandaa force-pushed the fix-5249-copy-dir-path branch 2 times, most recently from 54b0fbd to 4056da0 Compare September 25, 2024 04:02
@profnandaa
Copy link
Collaborator Author

CI failure seems unrelated?

=== FAIL: client TestClientGatewayIntegration/TestClientGatewayContainerHostNetworkingValidation/worker=containerd/netmode=host (0.97s)
    build_test.go:2133: 
        	Error Trace:	/src/client/build_test.go:2133
        	            				/src/client/build_test.go:2031
        	            				/src/util/testutil/integration/run.go:97
        	            				/src/util/testutil/integration/run.go:211
        	Error:      	Received unexpected error:
        	            	expecting started message, got *moby_buildkit_v1_frontend.ExecMessage_Exit

@profnandaa
Copy link
Collaborator Author

PTAL again, addressed the comments. @tonistiigi @gabriel-samfira

@profnandaa profnandaa force-pushed the fix-5249-copy-dir-path branch 5 times, most recently from 5dd50bb to 2d75bcd Compare October 9, 2024 04:31
util/system/path.go Outdated Show resolved Hide resolved
The call to CheckSystemDriveAndRemoveDriveLetter() does not preserve
the trailing `/` or `\\`. This happens because `filepath.Clean()`
strips away any trailing slashes. For example `/sample/` will be
`\\sample` on Windows and `/sample` on Linux.
This function was mainly written for Windows scenarios, which
have System Drive Letters like C:/, etc.

This was causing cases like `COPY testfile /testdir/` to
be intepreted as `COPY testfile /testdir`, and if `testdir` is
not explictly created before the call, it ends up being treated
as a destination file other than a directory.

Fix this by checking that if we have a trailing `/` or `\\`, we
preserve it after the call to `filepath.Clean()`.

Fixes moby#5249

PS. Also fixed for cross-building from Linux scenario, taking care
for paths like `\\sample\\` that are not changed when run
through `filepath.Clean()`.

Signed-off-by: Anthony Nandaa <profnandaa@gmail.com>
@gabriel-samfira
Copy link
Collaborator

LGTM. Waiting for a second review before merging.

@profnandaa
Copy link
Collaborator Author

profnandaa commented Oct 9, 2024 via email

@profnandaa
Copy link
Collaborator Author

@crazy-max @tonistiigi -- can take a look at this one?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

WCOW: trailing / ignored during COPY if destination dir not present
3 participants