Untar does not support symbolic links

[rrileyca]

Summary

When creating an OCIRepository to monitor the following docker repo: https://artifacthub.io/packages/helm/kiwigrid/spring-cloud-config-server, I get an error saying failed to extract layer contents from artifact: tar file entry bin/bzcmp contained unsupported file type Lrwxrwxrwx.

The errors at loglevel=debug are:

{"level":"error","ts":"2023-04-19T18:53:43.631Z","msg":"failed to extract layer contents from artifact: tar file entry bin/bzcmp contained unsupported file type Lrwxrwxrwx","controller":"ocirepository","controllerGroup":"source.toolkit.fluxcd.io","controllerKind":"OCIRepository","OCIRepository":{"name":"sck-config-server","namespace":"sck-config-server"},"namespace":"sck-config-server","name":"sck-config-server","reconcileID":"2c263b98-2fc8-4522-9ec1-bf3eae575868","error":"failed to extract layer contents from artifact: tar file entry bin/bzcmp contained unsupported file type Lrwxrwxrwx","stacktrace":"github.com/fluxcd/source-controller/internal/reconcile/summarize.logError\n\tgh.neting.cc/fluxcd/source-controller/internal/reconcile/summarize/processor.go:99\ngh.neting.cc/fluxcd/source-controller/internal/reconcile/summarize.ErrorActionHandler\n\tgh.neting.cc/fluxcd/source-controller/internal/reconcile/summarize/processor.go:77\ngh.neting.cc/fluxcd/source-controller/internal/reconcile/summarize.(*Helper).SummarizeAndPatch\n\tgh.neting.cc/fluxcd/source-controller/internal/reconcile/summarize/summary.go:193\ngh.neting.cc/fluxcd/source-controller/controllers.(*OCIRepositoryReconciler).Reconcile.func1\n\tgh.neting.cc/fluxcd/source-controller/controllers/ocirepository_controller.go:210\ngh.neting.cc/fluxcd/source-controller/controllers.(*OCIRepositoryReconciler).Reconcile\n\tgh.neting.cc/fluxcd/source-controller/controllers/ocirepository_controller.go:244\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\tsigs.k8s.io/controller-runtime@v0.14.5/pkg/internal/controller/controller.go:122\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tsigs.k8s.io/controller-runtime@v0.14.5/pkg/internal/controller/controller.go:323\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tsigs.k8s.io/controller-runtime@v0.14.5/pkg/internal/controller/controller.go:274\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\tsigs.k8s.io/controller-runtime@v0.14.5/pkg/internal/controller/controller.go:235"}
{"level":"debug","ts":"2023-04-19T18:53:43.631Z","logger":"events","msg":"failed to extract layer contents from artifact: tar file entry bin/bzcmp contained unsupported file type Lrwxrwxrwx","type":"Warning","object":{"kind":"OCIRepository","namespace":"sck-config-server","name":"sck-config-server","uid":"64986821-33a6-4f1b-b300-45f1b65bdac0","apiVersion":"source.toolkit.fluxcd.io/v1beta2","resourceVersion":"7228322"},"reason":"OCIArtifactLayerOperationFailed"}
{"level":"error","ts":"2023-04-19T18:53:43.652Z","msg":"Reconciler error","controller":"ocirepository","controllerGroup":"source.toolkit.fluxcd.io","controllerKind":"OCIRepository","OCIRepository":{"name":"sck-config-server","namespace":"sck-config-server"},"namespace":"sck-config-server","name":"sck-config-server","reconcileID":"2c263b98-2fc8-4522-9ec1-bf3eae575868","error":"failed to extract layer contents from artifact: tar file entry bin/bzcmp contained unsupported file type Lrwxrwxrwx","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tsigs.k8s.io/controller-runtime@v0.14.5/pkg/internal/controller/controller.go:329\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tsigs.k8s.io/controller-runtime@v0.14.5/pkg/internal/controller/controller.go:274\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\tsigs.k8s.io/controller-runtime@v0.14.5/pkg/internal/controller/controller.go:235"}
{"level":"error","ts":"2023-04-19T18:53:46.223Z","msg":"failed to extract layer contents from artifact: tar file entry bin/bzcmp contained unsupported file type Lrwxrwxrwx","controller":"ocirepository","controllerGroup":"source.toolkit.fluxcd.io","controllerKind":"OCIRepository","OCIRepository":{"name":"sck-config-server","namespace":"sck-config-server"},"namespace":"sck-config-server","name":"sck-config-server","reconcileID":"72bbd7aa-d46b-47ec-8b06-3ad679897b56","error":"failed to extract layer contents from artifact: tar file entry bin/bzcmp contained unsupported file type Lrwxrwxrwx","stacktrace":"github.com/fluxcd/source-controller/internal/reconcile/summarize.logError\n\tgh.neting.cc/fluxcd/source-controller/internal/reconcile/summarize/processor.go:99\ngh.neting.cc/fluxcd/source-controller/internal/reconcile/summarize.ErrorActionHandler\n\tgh.neting.cc/fluxcd/source-controller/internal/reconcile/summarize/processor.go:77\ngh.neting.cc/fluxcd/source-controller/internal/reconcile/summarize.(*Helper).SummarizeAndPatch\n\tgh.neting.cc/fluxcd/source-controller/internal/reconcile/summarize/summary.go:193\ngh.neting.cc/fluxcd/source-controller/controllers.(*OCIRepositoryReconciler).Reconcile.func1\n\tgh.neting.cc/fluxcd/source-controller/controllers/ocirepository_controller.go:210\ngh.neting.cc/fluxcd/source-controller/controllers.(*OCIRepositoryReconciler).Reconcile\n\tgh.neting.cc/fluxcd/source-controller/controllers/ocirepository_controller.go:244\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\tsigs.k8s.io/controller-runtime@v0.14.5/pkg/internal/controller/controller.go:122\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tsigs.k8s.io/controller-runtime@v0.14.5/pkg/internal/controller/controller.go:323\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tsigs.k8s.io/controller-runtime@v0.14.5/pkg/internal/controller/controller.go:274\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\tsigs.k8s.io/controller-runtime@v0.14.5/pkg/internal/controller/controller.go:235"}
{"level":"debug","ts":"2023-04-19T18:53:46.223Z","logger":"events","msg":"failed to extract layer contents from artifact: tar file entry bin/bzcmp contained unsupported file type Lrwxrwxrwx","type":"Warning","object":{"kind":"OCIRepository","namespace":"sck-config-server","name":"sck-config-server","uid":"64986821-33a6-4f1b-b300-45f1b65bdac0","apiVersion":"source.toolkit.fluxcd.io/v1beta2","resourceVersion":"7228334"},"reason":"OCIArtifactLayerOperationFailed"}
{"level":"error","ts":"2023-04-19T18:53:46.247Z","msg":"Reconciler error","controller":"ocirepository","controllerGroup":"source.toolkit.fluxcd.io","controllerKind":"OCIRepository","OCIRepository":{"name":"sck-config-server","namespace":"sck-config-server"},"namespace":"sck-config-server","name":"sck-config-server","reconcileID":"72bbd7aa-d46b-47ec-8b06-3ad679897b56","error":"failed to extract layer contents from artifact: tar file entry bin/bzcmp contained unsupported file type Lrwxrwxrwx","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tsigs.k8s.io/controller-runtime@v0.14.5/pkg/internal/controller/controller.go:329\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tsigs.k8s.io/controller-runtime@v0.14.5/pkg/internal/controller/controller.go:274\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\tsigs.k8s.io/controller-runtime@v0.14.5/pkg/internal/controller/controller.go:235"}

Potential cause

This case statement seems to be the culprit because it does not support FileMode.ModeSymlink.

pkg/tar/tar.go

Lines 112 to 174 in a39c961

	case mode.IsRegular():
	// Make the directory. This is redundant because it should
	// already be made by a directory entry in the tar
	// beforehand. Thus, don't check for errors; the next
	// write will fail with the same error.
	dir := filepath.Dir(abs)
	if !madeDir[dir] {
	if err := os.MkdirAll(filepath.Dir(abs), 0o755); err != nil {
	return err
	}
	madeDir[dir] = true
	}
	if runtime.GOOS == "darwin" && mode&0111 != 0 {
	// The darwin kernel caches binary signatures
	// and SIGKILLs binaries with mismatched
	// signatures. Overwriting a binary with
	// O_TRUNC does not clear the cache, rendering
	// the new copy unusable. Removing the original
	// file first does clear the cache. See #54132.
	err := os.Remove(abs)
	if err != nil && !errors.Is(err, fs.ErrNotExist) {
	return err
	}
	}
	wf, err := os.OpenFile(abs, os.O_RDWR|os.O_CREATE|os.O_TRUNC, mode.Perm())
	if err != nil {
	return err
	}
	n, err := copyBuffer(wf, tr, buf)
	if err != nil && err != io.EOF {
	return fmt.Errorf("error copying buffer: %w", err)
	}
	if closeErr := wf.Close(); closeErr != nil && err == nil {
	err = closeErr
	}
	if err != nil {
	return fmt.Errorf("error writing to %s: %w", abs, err)
	}
	if n != f.Size {
	return fmt.Errorf("only wrote %d bytes to %s; expected %d", n, abs, f.Size)
	}
	modTime := f.ModTime
	if modTime.After(t0) {
	// Ensures that that files extracted are not newer then the
	// current system time.
	modTime = t0
	}
	if !modTime.IsZero() {
	if err = os.Chtimes(abs, modTime, modTime); err != nil {
	return fmt.Errorf("error changing file time %s: %w", abs, err)
	}
	}
	case mode.IsDir():
	if err := os.MkdirAll(abs, 0o755); err != nil {
	return err
	}
	madeDir[abs] = true
	default:
	return fmt.Errorf("tar file entry %s contained unsupported file type %v", f.Name, mode)
	}
	}

.

Is this intentional?

Steps to Reproduce:

flux create source oci sck-config-server \
  --namespace sck-config-server \
  --url oci://docker.io/springcloud/spring-cloud-kubernetes-configserver \
  --tag 3.0.3-SNAPSHOT

Versions

flux CLI version

• 0.41.2

Container versions:

• image: docker.io/fluxcd/helm-controller:v0.31.2
• image: docker.io/fluxcd/kustomize-controller:v0.35.1
• image: docker.io/fluxcd/notification-controller:v0.33.0
• image: docker.io/fluxcd/source-controller:v0.36.1

[darkowlzz]

Hi, thanks for reporting the issue. I did some inspection of the artifact you've used above and my conclusion is that it's not the type of OCI artifact that flux expects. The image contains full linux root filesystem of debian buster/sid.
Please see https://fluxcd.io/flux/cheatsheets/oci-artifacts/#how-does-flux-oci-work for details about how flux works with OCI.

Regarding handling of symlinks, after digging into the history of the untar package, it looks like it was copied from upstream golang project and that was copied from another internal package which was for their specific use case, which didn't account for symlinks. In case of flux, I don't think there would be common use cases for symlinking manifests or other files in an artifact, but because our OCI support is more like a universal artifact format, I think it'd be good to add symlink handling as well.

[rrileyca]

That's fair, thanks for the response.

Yes it might be a good idea to actually use the upstream Golang library instead of using this copy+paste.

[darkowlzz]

it might be a good idea to actually use the upstream Golang library instead of using this copy+paste.

Sounds like some misunderstanding. The upstream code isn't public, refer

pkg/tar/tar.go

Line 6 in 2e870a1

// Adapted from: golang.org/x/build/internal/untar

.
And I meant that in the upstream's context, they may not need to care about symlinks but in our context, we may. So, we can diverge from the upstream and add symlink support.

[rashedkvm]

I am seeing the same issue while extracting image layer with symlink

    - lastTransitionTime: "2023-08-11T14:53:44Z"
      message: 'failed to extract layer contents from artifact: tar file entry ./etc/os-release
        contained unsupported file type Lrwxrwxrwx'
      observedGeneration: 1
      reason: OCIArtifactLayerOperationFailed

[djschny]

I'm seeing the same error as well. It appears that even though Docker images are OCI compliant there are problems.

https://www.docker.com/blog/demystifying-open-container-initiative-oci-specifications/

[stefanprodan]

I'm seeing the same error as well. It appears that even though Docker images are OCI compliant there are problems.

We never said we support Docker images, we made a conscious decision to reject OCI artifacts with symbolic links and other non-standard file types.

@djschny I see you work at VMware, so please sync up with your peer @rashedkvm, we've discuss this topic in detail on Flux dev meetings several times.

[rashedkvm]

@djschny as @stefanprodan mentioned we discussed in the last Flux community meeting and decided to reject the sym-link when untaring image layer(s). If you'd like to talk more hit me up on Slack.

[djschny]

It's OK, I can replace flux with a 4 line shell script in CronJob and be way more flexible.