-
Notifications
You must be signed in to change notification settings - Fork 472
Support the removal of images #197
Comments
Soft deletes support has been implemented in the registry client with this commit e2de2ec. However, we still need to investigate a bit further how it works under the hood before implementing it on the UI. |
For reference, the starting point for the "Distribution supporting deletes" discussion seems to be here: https://github.com/docker/distribution/blob/master/ROADMAP.md#deletes |
Not sure if this is the correct place to ask, is there any update on this feature? In our company, we are evaluating Portus and it seems that this missing feature is really important for maintaining the registry. |
@hhcordero sadly, this is not implemented in docker distribution (the registry) yet. That being said, I'd say that there is work being done in this regard (e.g. pruning). So, hopefully in the near future we will be able to somehow tackle this. |
It looks like Portus could probably take advantage of the soft-deletes implemented in distribution/distribution#461 now, and let the registry deal with the cleanup under the hood (whenever they get around to it). But note that the |
@JonathonReinhart note that soft deletes are already implemented in Portus itself (take a look at the
That's the deal for me. We could implement deletes, but that would be lying until there is no pruning or GC support on registry's side. Thus, I prefer to wait for that. |
Valid point. The one benefit would be removal of the tags, minimizing visual "clutter". |
@JonathonReinhart Yeah, but then you'll run into the problem of "I've deleted all of these tags! Why has my sever run out of disk space?". IMO, the principle of least surprise requires us to not implement a feature that actively lies to users, at least until distribution get around to auto-GCing orphaned tag references. |
These points all sound perfectly valid to me. Maybe all of these issue
cross-references will get someone's attention over at Docker :-)
|
One can only hope. I could prod some of the maintainers. 😸 |
I'm in complete agreement w.r.t. "principle of least surprise" - if the backing store is a cluttered mess, it should look like the cluttered mess it is. One needs to know where the disk space has gone. FYI the reason I found Portus was because I was searching for a (stable) Docker Registry that could handle deletions (v1 registry isn't stable, v2 isn't complete). Lack of support for deletion is a killer issue for me - contrary to popular belief, disk space is not infinite. My CI builds that create docker images can churn through terabytes a week. @cyphar If you can prod docker folks to raise their awareness of this issue, please do so... |
@pjdarton I have the exact same issue. Regarding this issue : +1 for waiting the feature to be fully working on docker side. |
,
Not to speak on their behalf, but the Docker maintainers are very, very aware of this issue, and I'm sure they are getting tired of the continuous stream of +1's being thrown at every issue that talks discusses image deletion. See this comment as an example: docker-archive/docker-registry#45 (comment) I would definitely suggest having a look at the deletes section of the roadmap before attempting to engage with them about this issue, otherwise you'll quickly be dismissed. That being said, I find it quite frustrating that a project as large and corporate-sponsored as Docker hasn't figured out a way to delete things from the registry yet. I mean, would the EXT4 developers ever release a version of their filesystem that didn't let you delete files from it?! So by all means, if SUSE personnel would be able to express that Portus remains incomplete because of Docker shortcomings, please do so. |
I'm aware that it's a non-trivial issue, and have some sympathy for the developers who are at the sharp end of this criticism, but I share your frustration that it got released in this incomplete state. If Portus is just bundling the official Docker v2 registry implementation then Portus cannot claim to be a Docker v2 registry until the deletion part of the API is implemented (either by Portus or within the official Docker v2 registry code). |
@pjdarton We do not claim to be a Docker v2 registry. Portus is an authorization service for a Docker Registry implementing v2 (a.k.a. docker distribution). The description of Portus is:
More specifically, in the first image of this document, Portus fills the "Authorization Service" box, nothing more, nothing less. Therefore, Portus can only support deletion when docker distribution does it ;)
I agree with @JonathonReinhart on this. |
GC support has just been merged in docker distribution and it will be there for 2.4 (see PR). I will investigate on it, and we will hopefully get rid of this issue. |
As a temporary work around this worked for us. |
After doing some work on top of distribution 2.4, I've decided that this is in fact no longer blocked. I've already sent a couple of PRs in preparation of the "big" feature :) |
Since it's such a big feature (as you pointed out by yourself ;-) wouldn't it be worth to issue a interim 2.0.4 release containing especially this feature? Waiting until autumn is quite a long time... |
@holgerreif I'd say no for two reasons:
So, I can only see two options: have it on |
Then I'll stick with master. @JonathonReinhart @rennhak @pjdarton @Seb-Solon does anybody of you have opinions? |
@holgerreif You've contradicted your self here, by suggesting a "big feature" to into a "patch" release. Have a look at Semantic Versioning, which describes what types of things go into what type of release. Personally, I'm on 2.0, but plan to move to |
@holgerreif I've been doing some work around this feature this week and the previous one. I expect to get the feature finished for either this week or the next one. So hopefully you'll be able to test this feature soon enough :) |
Soft delete support has been supported in Docker Distribution since at least 2.1. This was not enough to implement the removal of images/tags in Portus because there was no support to GC these soft deleted blobs. Since 2.4, it's possible to not just delete the manifest, but also to GC blobs no longer referenced by any image manifest. This means that after being able to track digests, we can now safely provide image/tag removal support. For safety concerns, tags with an empty digest will not be allowed to be removed (this is more likely to be the case of Portus versions that have been running for some time). In previous PRs this has already been addressed, so admins can update their DB filling in the empty gaps (e.g. see PRs SUSE#825 or SUSE#830). The main downside of this change is that there is no way for a client to detect whether a remote registry supports GC. Because of this, a configuration option has been provided, which is disabled by default. The rationale is that administrators that are really sure about the availability of GC on their private registry will have to enable it explicitly. Fixes SUSE#197 Signed-off-by: Miquel Sabaté Solà <msabate@suse.com>
Soft delete support has been supported in Docker Distribution since at least 2.1. This was not enough to implement the removal of images/tags in Portus because there was no support to GC these soft deleted blobs. Since 2.4, it's possible to not just delete the manifest, but also to GC blobs no longer referenced by any image manifest. This means that after being able to track digests, we can now safely provide image/tag removal support. For safety concerns, tags with an empty digest will not be allowed to be removed (this is more likely to be the case of Portus versions that have been running for some time). In previous PRs this has already been addressed, so admins can update their DB filling in the empty gaps (e.g. see PRs SUSE#825 or SUSE#830). The main downside of this change is that there is no way for a client to detect whether a remote registry supports GC. Because of this, a configuration option has been provided, which is disabled by default. The rationale is that administrators that are really sure about the availability of GC on their private registry will have to enable it explicitly. Fixes SUSE#197 Signed-off-by: Miquel Sabaté Solà <msabate@suse.com>
Soft delete support has been supported in Docker Distribution since at least 2.1. This was not enough to implement the removal of images/tags in Portus because there was no support to GC these soft deleted blobs. Since 2.4, it's possible to not just delete the manifest, but also to GC blobs no longer referenced by any image manifest. This means that after being able to track digests, we can now safely provide image/tag removal support. For safety concerns, tags with an empty digest will not be allowed to be removed (this is more likely to be the case of Portus versions that have been running for some time). In previous PRs this has already been addressed, so admins can update their DB filling in the empty gaps (e.g. see PRs SUSE#825 or SUSE#830). The main downside of this change is that there is no way for a client to detect whether a remote registry supports GC. Because of this, a configuration option has been provided, which is disabled by default. The rationale is that administrators that are really sure about the availability of GC on their private registry will have to enable it explicitly. Fixes SUSE#197 Signed-off-by: Miquel Sabaté Solà <msabate@suse.com>
It's now possible to remove images & tags from the Portus UI. In the following weeks I'll add support for removing namespaces, teams and users, and I'll try to polish any corner case that might exist. Documentation for this is still to be written, but a summary would be:
|
@mssola could you at least tell me what configuration to add so i can use this feature? |
@EugenMayer certainly. For clarification for other users as well, in my previous comment, when I say:
I mean that we introduced the |
@mssola if found it already, thank you. My issue was, that i was using https://github.com/sshipway/Portus which is rather a fork then just a build for https://hub.docker.com/r/sshipway/portus/ .. and it was very outdated and did not include this feature at all i went forward and made a clean build repo, which uses SUSE/portus without modifications .. publishing them to https://hub.docker.com/r/eugenmayer/portus/ while building is done using https://github.com/EugenMayer/portus-build If that is out of any interest for you - rip it off :) I already tried that feature and it works great. If you wonder what i am doing, i am packing portus for rancher.io so it is installable as a "app" ( catalog ) https://github.com/EugenMayer/kontextwork-catalog/blob/master/templates/registry-slim/1/docker-compose.yml |
One of the efforts that we are doing for the 2.1 release is to come up with an official production-ready Docker image. For this, we've prepared this image, which has three tags:
We are still working on the documentation of this image, but an overview of it is already available here. I'd recommend giving this image a try. Edit: proper URL for the documentation |
great! i could just create a wrapper-image to offer a different entry-point ( due to the configuration over ENV ), but in general, i would be very glad to use your official image! |
Just had a look at the image and created some feedback #912 for now, that image would be not be practical to build on for this purpose. Maybe we can work on this |
Work is being done upstream here. Therefore, this feature, although needed, is blocked by it. We will work on it once that issue is fixed. It's supposed to get out with distribution 2.1
Update: distribution 2.1 has been released and soft deletes have made it in the release. Here's a checklist in order to fix this issue:
The text was updated successfully, but these errors were encountered: