[Docker Hub] Official REST API

[pkennedyr]

Tell us about your request
Official REST API for Docker Hub

Which service(s) is this request for?
Docker Hub

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?
Hub customers have cited a desire to have an officially supported REST API for programmatically performing common Hub operations relating to repositories, organizations, teams, users, search, etc.

Are you currently working around this issue?
N/A

Additional context
N/A

Attachments
N/A

[mgreau]

Hi @pkennedyr

Publishing the Docker images to the store via the Publisher Center is only doable manually today, as far as I know.

It would be great to be able to do this via this REST API.
Would it be part of this work?

Thanks

[pkennedyr]

Hi @mgreau,

Indeed, the Official REST API would also encompass core publisher activities. However, would you mind adding a separate Publisher API issue/request to ensure any publisher-specific use cases that you have in mind are addressed?

Thanks,
Ryan

[ingshtrom]

As part of this, it would be absolutely fantastic to also release a Terraform provider to support this. I'm not sure what % of Docker Hub users use Terraform, but I think it's likely that operators of teams using Docker Hub would really enjoy this.

[pkennedyr]

What class of APIs would the Hub community ideally like to see prioritized first (e.g. repositories, organizations, teams, users, search, etc.)?

[seemethere]

For our workflows it'd be incredibly useful to have team management (and to a greater extent organization management) implemented for Docker Hub.

It's currently a pain to add / remove users from our teams and to do any kind of auditing around who should / should not be part of our current docker hub organizations.

[manishtomar]

As part of this, it would be absolutely fantastic to also release a Terraform provider to support this. I'm not sure what % of Docker Hub users use Terraform, but I think it's likely that operators of teams using Docker Hub would really enjoy this.

@ingshtrom Just curious: what would the terraform provider help provision? Hub resources like repositories, teams, permissions? For what use case is reproducibility of these resources useful?

[ingshtrom]

@manishtomar

I would think it could provision whatever is available through the official API.

I think there are a few forms of reproducibility we could talk about. First, reproducing a whole set up from scratch--in this case, it isn't nice to be able to re-spin up your organization and repos from scratch as it should be safe to assume that Docker Hub can handle that data and won't lose it.

The second form of reproducibility is within the same environment across the same resources that change slowly over time. So for example, you have an organization that has 100 users with 200 repositories in it. As time goes on, users come and go, users change teams, applications evolve to include more parts, and older applications are deprecated. These are changes that pile up over time and having it in Terraform, presumably in a version control system, means you have an audit trail that can easily be looked at to see what has happened, how you did something in the past, etc.

It's really the same reason you would use Terraform (or any IAC, really) for anything. Another example I had heard of with using Terraform was with PagerDuty. This team spun up new teams/schedules/api integrations/etc that were configured in a similar way so it was known across the organization that no matter what team dealt with PagerDuty, you knew it was set up similarly.

Hopefully that helps?

[ob1dev]

@manishtomar, regarding Alex's idea. Something like this: https://auth0.com/blog/use-terraform-to-manage-your-auth0-configuration/

Probably it worth to move it into a separate issue.

[nanoz]

Hi !

Thanks for prioritizing this on the roadmap 😊

As part of a new service bootstrap, all of our workflow to get from code to prod is automated, except for new Docker Hub images declaration, where we have to go on the interface and change the permissions set in order for developers to use the repository. So an image permissions setting accessible via an API endpoint could be great, along with users management.

A terraform provider at least to manage users, groups and to manage images lifecycle sure would be nice too !

[christian-korneck]

@pkennedyr

Tell us about the problem you're trying to solve.
What are you trying to do

I would like to update a repo's short and full descriptions (README).
(I maintain and use a small tool for that purpose).

and why is it hard?

currently the only way I'm aware of doing this for Docker Hub is using an undocumented API that doesn't support login via personal access tokens (only username/password, which only work when 2FA auth is disabled). I would wish for an officially supported REST API that allows to update the repo's description that works for all users, no matter if they have 2FA auth enabled or not.

[jfarraraains]

These ideas are great, but REST API is not complete for many without WebHooks. REST is on request responses. WebHooks are on event responses to outside targets. These should be able to be configured via REST also of course.

[yanjost]

As stated in docker/hub-feedback#2044 having access to the vulnerabilities list and details would be of great help. I am OK to give some feedback on this API part if you have a prototype in mind

[gesellix]

In addition to access the vulnarabilities list, we'd like to enable image scanning via API - or at least get a list with an indicator whether image scanning is enabled for a repository.

[zephinzer]

vulnerabilities list would be awesome so that we don't have to pull images from docker hub just to scan it with an internal tool- also, updating a repo's description via api would help with keeping docs in-sync between the code repository and the image repository

[terowz]

Tell us about your request
Official REST API for Docker Hub

Which service(s) is this request for?
Docker Hub

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?
Hub customers have cited a desire to have an officially supported REST API for programmatically performing common Hub operations relating to repositories, organizations, teams, users, search, etc.

Considering we now need to get developers licenses to continue using docker, we will need to add a bunch of users to our dockerhub team. So prioritizing users/team management via the API would help make this process less painful.

[jdziat]

I completely agree with @terowz, this should have been put in place before that licensing announcement was made.

[terowz]

I completely agree with @terowz, this should have been put in place before that licensing announcement was made.

Don’t get your hopes up. I asked our account rep and they said “great news we are working on it now…. But only docker business can use it”. So triple the cost to use an API endpoint… 🤦
I hope they change this when they launch it. Because it just seems wrong.

[cjolif]

Now that using Docker requires subscription for some companies, there must be a way to programmatically un/register users. Otherwise, this is a manual on-boarding, off-boarding nightmare and this is just pushing looking into alternatives "just" for that reason. So is there any news on that?

[sperryptc]

Would really like the API to have ability to export the Audit log. Docker only keeps 3 months of logs and no ability through the dashboard to export.

[tristanthomas]

It would be great if there were Docker HUB APIs to allow organizations the ability to programmatically add/remove users from teams/org, create/delete/list teams in an org, and list users in an org. It would also be great if these APIs support authentication with a PAT under an MFA/SSO enabled account.

[benchi]

+1 for organization management APIs. This is a must for large customers.

[ChefAustin]

What class of APIs would the Hub community ideally like to see prioritized first (e.g. repositories, organizations, teams, users, search, etc.)?

+1 to what @terowz @jdziat @cjolif and @tristanthomas said.

If an organization is paying for the enterprise-level features of DockerHub's Business tier, then it should be a given that there are a means by which to programmatically:

• Add/remove users to/from organization (ideally by way of email address or DockerID)
• Add/remove users to/from a DockerHub organization Team (ideally by way of SCIM or some other user attribute passed in the SAML assertion during JIT provisioning)
• List users in an org

And on a lower-priority note, I think it would be grand if there was a way to enable SSO bypass for a singular user account (which could be used as a breakglass method for disabling SSO in the event of an IdP outage).

[tehautanop]

+1 @ChefAustin

Using hub-tool you can list uses in a org
It shows only Username and Full name, if full name is populated by user in their account settings.

Hub-tool list should include user email address. ( Already requested #310 )

[tehautanop]

Already mentioned in #310

We need a way to have the entire organisation users data exported (csv or similar) either from DockerHub organisation page and/or using hub-tool (REST API)

[tehautanop]

#316

[ChefAustin]

Albeit tangentially related to this particular issue, I think it would be fantastic if there was an official Terraform provider for the codified management of a DockerHub organization. In order for this to be a feasible means by which to manage all aspects of a DockerHub Org, DockerHub's REST API would need to be a tad more feature-rich.

[nathansegers]

Is it interesting to add a GraphQL API as well? With similar features as the Rest API, but more flexibility in queries?

[dduportal]

As part of this, it would be absolutely fantastic to also release a Terraform provider to support this. I'm not sure what % of Docker Hub users use Terraform, but I think it's likely that operators of teams using Docker Hub would really enjoy this.

@ingshtrom Just curious: what would the terraform provider help provision? Hub resources like repositories, teams, permissions? For what use case is reproducibility of these resources useful?

Use case that we have for the Jenkins Infrastructure project, as we have a set of images big enough to justify automation (around 20-25) but not big enough for us to avoid dealing manually.

We would like to define teams for the "technical users" (e.g. accounts with an API token used by the CI system) with different permissions.
But unless the users are "owners", we have to manually add permission for each image. It means that each time we create/delete an image, a human has to connect to the DockerHub to change permissions.

Having an API, or better a IaC tool such as Terraform would allow better UX: permissions would be defined in a repository, requests to access could be given through PRs/commit/usual config as code workflows.

Otherwise having finer permissions, for instance scoped per object ("allow CRUD on all images but do not allow permissions management neither billing access") would also solve the issue.

[josh-higgs]

Ability to export Docker organisation user data - #316

Something like this is absolutely essential for us to easily manage a large organisation.

[dannysauer]

I guess since SCIM was recently implemented for the highest price business plan, there's very little hope for the API to manage users on the other plans. The official answer will almost certainly be "just double or triple your spend to get SSO support and some other features you really don't want or need". :/

[ryanhristovski]

@ChefAustin I’m happy to share that we’ve released an early-stage Terraform provider for Docker, which we hope you’ll find useful for managing Docker Hub resources. It’s still in development, but we’re excited to expand its capabilities. Feel free to check it out and provide feedback—your insights would be valuable as we continue to improve the provider.

Learn more here: https://www.docker.com/blog/docker-terraform-provider/
Repo: https://github.com/docker/terraform-provider-docker

[technicallyjosh]

As far as an "official REST" API, we have some internal things to accomplish before this can happen sadly. It is definitely happening a bit slower than we'd like, but I can assure you, it will come. For now as @ryanhristovski mentioned, we do have the TF provider which can help, but definitely not the end-all here.

[technicallyjosh]

I guess since #309 for the highest price business plan, there's very little hope for the API to manage users on the other plans

@dannysauer I'm sorry it seems this way. While SCIM is "user management" ultimately, it's also very specific for idPs/directories etc. I don't think we expect everyone to upgrade just to manage their users. When you have an org that is massive in membership as we have many, it's an expected feature as a business to invest 😄.

These features are coming, they just have not been the highest priority at the moment.