This repository has been archived by the owner on Apr 20, 2023. It is now read-only.
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* Remove duplicate "not found" from some error messages
I noticed this when building a Dockerfile that failed because a file didn't
exist, so went through error messages that looked like they had a duplicate
"not found" in the output;
[+] Building 0.9s (6/9)
=> [internal] load build definition from Dockerfile 0.2s
=> => transferring dockerfile: 306B 0.0s
=> [internal] load .dockerignore 0.1s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/alpine:latest 0.0s
=> CACHED [1/5] FROM docker.io/library/alpine 0.0s
=> [internal] load build context 0.6s
=> => transferring context: 701B 0.5s
=> ERROR [2/5] ADD no-such-file.txt / 0.0s
------
> [2/5] ADD no-such-file.txt /:
------
failed to compute cache key: "/no-such-file.txt" not found: not found
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
* git: support subdir component
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* Update SSH agent socket handling to support Windows OpenSSH using named pipes.
Signed-off-by: Siebe Schaap <siebe@digibites.nl>
* Update Windows named pipe handling for SSH forwarding to use x/sys/windows rather than syscall.
Signed-off-by: Siebe Schaap <siebe@digibites.nl>
* Rename SSH agent forwarding helper parsePlatformSocketPath() to getWindowsPipeDialer().
Signed-off-by: Siebe Schaap <siebe@digibites.nl>
* Fix regular expression to test for Windows named pipe in SSH agent path; allowed backslashes in host.
Signed-off-by: Siebe Schaap <siebe@digibites.nl>
* update to go 1.16
This updates all occurrences of Go 1.13 to Go 1.16; also updated
the code that's used to redact credentials in URLs to use the Go
implementation.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
* go.mod: update docker and docker cli to v20.10.7
- pkg/signal: ignore SIGURG on all platforms
- pkg/archive: use v2 capabilities in layer archives
- update ishidawataru/sctp to fix possible socket leak when bind fails
docker/cli:
- config: print deprecation warning when falling back to ~/.dockercfg
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
* hack: update proto generators for arm64
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* test: fix mirror cache handling
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* cache: fix possible nil dereferences
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* move RegistryConfig to resolver package
This allows using the resolver package without having to import
the buildkit daemon configuration.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
* vendor: fix broken gosum
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* add hack/shell helper for dev shell environment
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* session: make sure all token request keep correct context
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* filesync: ensure sendclose is always called
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* Enable stargz snapshotter to use session for snapshot auth
Signed-off-by: ktock <ktokunaga.mail@gmail.com>
* Implement low-level parser primitives for heredocs
This provides the basic functionality for the parser to recognize and
parse provided heredocs in supported commands.
Signed-off-by: Justin Chadwell <me@jedevc.com>
* Add support for parsing heredocs in ADD/COPY and RUN
Signed-off-by: Justin Chadwell <me@jedevc.com>
* Integrate heredoc support into ADD/COPY and RUN
This modifies the command structures to support inline files, as well as
provides the logic to compile them down into appropriate LLB
definitions.
Signed-off-by: Justin Chadwell <me@jedevc.com>
* Add integration tests for dockerfile heredocs
Signed-off-by: Justin Chadwell <me@jedevc.com>
* Update docker/docker to master / v21.xx-dev (post libnetwork integration)
full diff: https://github.com/docker/docker/compare/v20.10.7..0ad2293d0e5b
This applies the same / similar local changes to buildkit as were made in
the branch that moby is currently vendoring:
moby/buildkit@244e8cd...cpuguy83:update_libnetwork_import
Unfortunately, this (again) requires a "replace" rule (probably until we tagged
a release with this change). Go mod refuses to pick a version from master, and
(without the replace rule), does weird, reverting docker to a very old version,
predating the sirupsen rename:
> [vendored 4/4] RUN --mount=target=/src,rw --mount=target=/go/pkg/mod,type=cache go mod tidy && go mod vendor && mkdir /out && cp -r go.mod go.sum vendor /out:
#10 0.500 go: finding github.com/docker/docker 8dbd90ec00daa26dc45d7da2431c965dec99e8b4
#10 0.599 warning: ignoring symlink /src/examples/kube-consistent-hash
#10 1.027 go: github.com/moby/buildkit/util/testutil/integration imports
#10 1.027 github.com/docker/docker/testutil/daemon imports
#10 1.027 github.com/docker/docker/opts imports
#10 1.027 github.com/docker/libnetwork/ipamutils imports
#10 1.027 github.com/docker/libnetwork/osl imports
#10 1.027 github.com/Sirupsen/logrus: github.com/Sirupsen/logrus@v1.8.1: parsing go.mod:
#10 1.027 module declares its path as: github.com/sirupsen/logrus
#10 1.027 but was required as: github.com/Sirupsen/logrus
------
error: failed to solve: rpc error: code = Unknown desc = executor failed running [/bin/sh -c go mod tidy && go mod vendor && mkdir /out && cp -r go.mod go.sum vendor /out]: exit code: 1
make: *** [vendor] Error 1
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
* Fix doc about Stargz Snapshotter and registry authentication
Signed-off-by: ktock <ktokunaga.mail@gmail.com>
* dockerfile: add documentation for here-docs
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* add differ support for local source
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* llb: don't allow content diff for now
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* Fix file modes with remote ADD commands
Signed-off-by: Justin Chadwell <me@jedevc.com>
* cache: avoid concurrent maps write on prune
remove() needs to be called while holding the manager lock
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* metadata: hold lock on storageitem update
The locks usage is mixed up because two locks separate locks
are actually needed. With a specific lock, calls to SetValue
can be protected.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* Dockerfile: install fuse-overlayfs from apk
fuse-overlayfs is available as an apk since Alpine 3.14.
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
* vendor: update vt100 to fork and remove replace rule
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* add support for opentelemetry
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* replace opentracing with opentelemetry tracers
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* hack: avoid updating generated proto version
This can be done as a separate change when needed.
Also should analyze if this would affect the gogo
incompatibility issues with newer proto.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* session: avoid tracing health checkpoint
Upstream fixes needed for cleaner solution
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* disable otel error logging
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* correctly validate span from context
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* dockerfile: ensure config resolve errors keep source location
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* remove go fork for building windows/arm64
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* Dockerfile: remove nsswitch as default in go1.16
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* retryhandler: use net.ErrClosed for error check
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* replace uses of deprecated containerd/sys.RunningInUserNS()
This utility was moved to a separate package, which has no dependencies.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
* integration: add common context base to all integration tests
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* vendor: update opentelemetry to 1.0.0-rc
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* add current tracing context detection and exec propagation
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit bc9a83144c83e9fd78007b7bfe92e8082c59d40e)
* add transform package to convert from otlp
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* tracing: add delegated exporter
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* otlgrpc: provide a way to get otlp client from grpc conn
Hopefully this can be removed with a future upstream change
that could make this configurable. The package also needs
internal dependency that is copied in.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* enable collecting traces via control api
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* support collecting traces from llb.Exec
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* client: pass delegated exporter as parameter
Avoid client package having dependency on global detect package.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* update runc binary to v1.0.0 GA
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
* handle unconfigured spans without errors
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* llb: add constraints to vertex and validate
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* llb: add constraints to async llb
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* llb: ensure meta resolver uses platform form constraints
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* flightcontrol: reduce contention between goroutines
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* Avoid nil pointer dereference when copying from image with no layers
Fix this panic when copying from an image with no layers:
```
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x50 pc=0xdd8c17]
goroutine 326 [running]:
github.com/moby/buildkit/cache/contenthash.(*cacheManager).Checksum(0xc0005ec030, 0x1682c00, 0xc000842140, 0x0, 0x0, 0xc0005d4023, 0x1, 0x0, 0x0, 0x0, ...)
/src/cache/contenthash/checksum.go:95 +0x37
github.com/moby/buildkit/cache/contenthash.Checksum(0x1682c00, 0xc000842140, 0x0, 0x0, 0xc0005d4023, 0x1, 0x0, 0x0, 0x0, 0x0, ...)
/src/cache/contenthash/checksum.go:59 +0xd5
github.com/moby/buildkit/solver/llbsolver.NewContentHashFunc.func1.1(0x0, 0x4425d6)
/src/solver/llbsolver/result.go:59 +0x20a
golang.org/x/sync/errgroup.(*Group).Go.func1(0xc00056a360, 0xc000594510)
/src/vendor/golang.org/x/sync/errgroup/errgroup.go:57 +0x59
created by golang.org/x/sync/errgroup.(*Group).Go
/src/vendor/golang.org/x/sync/errgroup/errgroup.go:54 +0x66
```
When the path is "/", we allow it because it's a noop.
Based on moby#2185
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
* Add test for copying from scratch
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
* Check that scratch is mounted as empty dir
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
* Make error message consistent when layer is empty
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
* Test with tonistiigi/test:nolayers as well
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
* ensure containerd io is complete and closed before returning
Signed-off-by: Cory Bennett <cbennett@netflix.com>
* [moby#2112] progress.Controller should own the progress.Writer to prevent leaks
Signed-off-by: Cory Bennett <cbennett@netflix.com>
* [moby#2112] progress.FromContext returns a writer factory
this allows progress.Controller to manage the writer lifecycle
Signed-off-by: Cory Bennett <cbennett@netflix.com>
* contenthash: use SeekLowerBound to seek radix tree
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* dockerfile: fix git version detection
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* Add support for heredocs with ONBUILD
Signed-off-by: Justin Chadwell <me@jedevc.com>
* dockerfile: use none differ for dockerfile/dockerignore
This avoids wrong metadata matches on small files
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* progressui: print logs for failed step as summary in plain mode
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* grpcerrors: avoid rpc error wrapping in error messages
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* exec: improve error message on exec errors
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* Improve heredoc parsing to allow more generic words
Previously, heredoc names were restricted to simple alphanumeric
strings. However, heredocs should support much more complex use-cases,
including quoting anywhere, as well as allowing special symbols like `.`
for easily expressing file extensions.
This patch adds support for these more complex cases, by using the shell
lexer to parse each heredoc name. Additionally, we include improvements
to the lexer to optionally preserve escape tokens to avoid problems when
lexing words that have already been lexed before.
Signed-off-by: Justin Chadwell <me@jedevc.com>
* Improve progress and history messages for heredoc-related commands
Signed-off-by: Justin Chadwell <me@jedevc.com>
* Remove unneeded Finalize method from ImmutableRef.
Finalize was only used outside the cache package in one place, which
called it with the commit arg set to false. The code path followed
when commit==false turned out to essentially be a no-op because
it set "retain cache" to true if it was already set to true.
It was thus safe to remove the only external call to it and remove it
from the interface. This should be helpful for future efforts to
simplify the equal{Mutable,Immutable} fields in cacheRecord, which exist
due to the "lazy commit" feature that Finalize is tied into.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
* Fix ref leak if fileop ref fails to mount.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
* add error suggest pkg
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* dockerfile: suggest mistyped flag names
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* dockerfile: provide suggestions for mount options
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* dockerfile: add tests for error suggestions
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* dockerfile: remove unnecessary error wrappings
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* enable riscv64 build
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* Update QEMU emulators
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
* dockerfile: move run network to stable channel
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* Automatically detect default git branch
Instead of just assuming that the default branch is master, use ls-remote to find out. Also removed tests that didn't specifiy a branch but required authentication, because those will fail now that the repo is actually checked.
Signed-off-by: Levi Harrison <levisamuelharrison@gmail.com>
* Moved getDefaultBranch function to gitsource
It is my suspecion that the tests were failing on previous commits because of the lack of authentication and other stuff like that available in gitidentifier as compared to gitsource
Signed-off-by: Levi Harrison <levisamuelharrison@gmail.com>
* Fix tests
Unfortunately, further test cases will have to be removed because gitindentifier will now leave the branch blank instead of filling it in
Signed-off-by: Levi Harrison <levisamuelharrison@gmail.com>
* git: fix default branch detection
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* Enable to forcefully specify compression type
Signed-off-by: ktock <ktokunaga.mail@gmail.com>
* Add full timestamp to logs
Signed-off-by: Yamazaki Masashi <masi19bw@gmail.com>
* Remove meaningless encode
Signed-off-by: Yamazaki Masashi <masi19bw@gmail.com>
* Ignore missing providers for blobs w/ same chainid.
GetByBlob checks to see if there are any other blobs with the same
(uncompressed) ChainID and, if so, reuses their unpacked snapshot if it
exists.
The problem is if this code finds a match, it was trying to get the
matching record, but couldn't do so when the match is lazy because the
caller doesn't necessarily have descriptor handlers setup for it.
This commit changes the behavior to just ignore any match with the same
ChainID that's also lazy as they just aren't usable for the
snapshot-reuse optimization.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
* authprovider: handle eaccess on storing token seeds
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* log with traceID and spanID
Signed-off-by: Morlay <morlay.null@gmail.com>
* github: update CI buildkit to v0.9.0-rc1
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* initial version of github cache
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* vendor: add goactionscache
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* caps: add cap for gha cache backend
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* remove tracetransform package
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* resolver: increase default idle conns reuse
The current default were even lower than stdlib defaults.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* refactor to use util/bklog instead of using logurs directly
Signed-off-by: Morlay <morlay.null@gmail.com>
* GitHub Actions cache docs
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
* Skips getting UID/GUID if passwd/group file is not found
When running a WORKDIR instruction, buildkit will create that folder
and chown it to the currently set user. For this, it will try to read
the /etc/passwd file to get the proper UID, and if that user is not
found in the file, the root user will be considered as the owner.
However, Windows image do not have that file, which will result in
an error while building the image. We can consider not finding
the /etc/passwd file as the same as not finding the user in the file,
which would solve this issue.
Signed-off-by: Claudiu Belu <cbelu@cloudbasesolutions.com>
* add per domain semaphore to limit concurrent connections
This is a safer alternative until we figure out why
http.Transport based limiting fails.
Some connections like cache export/import do not have a
domain key atm and these connections use global pool.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* update to github.com/containerd/containerd v1.5.3
Signed-off-by: coryb <cbennett@netflix.com>
* vendor: update go-actions-cache with custom client support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* tracing: update to otelhttp roundtripper
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* Enhance test matrix
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
* fix dropped pull progress output due to canceled context
fixes moby#2248
Signed-off-by: coryb <cbennett@netflix.com>
* Add span for layer export
This can be a significant amount of time that isn't currently accounted
for in traces.
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
* new implementation for limiting tcp connections
The previous implementation had many issues. Eg. on fetch, even if
the data already existed and no remote connections were needed
the request would still be waiting in the queue. Or if two fetches
of same blob happened together they would take up two places in queue
although there was only one remote request.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* limited: allow extra high-priority connection for json requests
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* ensure wrappers support seeking to continue partial downloads
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* contentutil: change offset to int64 to simplify
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* Exporter config digest typo
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
* daemonless: wait for daemon to finish before exit
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* github: update CI buildkit to v0.9.0
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* add docs for new config options
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* add ktock and crazy-max to maintainers
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* Update Dockerfile references to use 1.3
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* docs: update images-readme to v0.9
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* Bump to codecov/codecov-action v2
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
* wip
Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Co-authored-by: Tonis Tiigi <tonistiigi@gmail.com>
Co-authored-by: Siebe Schaap <siebe@digibites.nl>
Co-authored-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
Co-authored-by: Sebastiaan van Stijn <thaJeztah@users.noreply.github.com>
Co-authored-by: ktock <ktokunaga.mail@gmail.com>
Co-authored-by: Justin Chadwell <me@jedevc.com>
Co-authored-by: Akihiro Suda <suda.kyoto@gmail.com>
Co-authored-by: Aaron Lehmann <alehmann@netflix.com>
Co-authored-by: Cory Bennett <cbennett@netflix.com>
Co-authored-by: Erik Sipsma <erik@sipsma.dev>
Co-authored-by: CrazyMax <crazy-max@users.noreply.github.com>
Co-authored-by: Levi Harrison <levisamuelharrison@gmail.com>
Co-authored-by: masibw <masi19bw@gmail.com>
Co-authored-by: Morlay <morlay.null@gmail.com>
Co-authored-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
Co-authored-by: Claudiu Belu <cbelu@cloudbasesolutions.com>- Loading branch information
18 people
authored
Jul 28, 2021
1 parent
467c8f2
commit 199ad6a