OIDC macaroon minting for GitHub (#11272)

* warehouse: make `Macaroon <-> User` optional

Macaroons were previously strongly associated with individual users.

This was the correct association for user-minted tokens, but not
generally. In particular, ODIC-minted tokens are associated with
projects themselves and not the user who registered the OIDC provider.

* migrations: rebase

* warehouse: oidc minting boilerplate

* oidc/views: more boilerplate

* dev, tests, warehouse: NullOIDCProvider

* warehouse: end-to-end OIDC minting functionality

* macaroons/services: docs

* warehouse/locale: `make translations`

* oidc/views: remove import

* warehouse: more DB macaroon modeling

* warehouse, tests: add missing relationship

* warehouse/macaroons: lint fixes

* tests, warehouse: update macaroon handling, OIDC minting view

* warehouse/forklift: begin supporting non-user identities

* warehouse: fill in project ACLs for project identities

* warehouse: fix link in project history

* utils/security_policy: handle unknown identities

* oidc/services: NullOIDCProviderService.verify_for_project actually checks the project

* warehouse/locale: `make translations`

* services, test_services: fix options, add some null OIDC tests

* warehouse: make `JournalEvent.submitted_by` optional

* warehouse: create JournalEvent events for nonusers, update template

* oidc/views: tweak error message

* forklift, oidc: remove "ephemeral" language

Also make the error response more structured when minting.

Signed-off-by: William Woodruff <william@trailofbits.com>

* warehouse/migrations: rebase migration

Signed-off-by: William Woodruff <william@trailofbits.com>

* warehouse: refactor OIDCProvider retrieval

Still needs more work.

Signed-off-by: William Woodruff <william@trailofbits.com>

* warehouse/migrations: remove project_id migration

Unneeded

Signed-off-by: William Woodruff <william@trailofbits.com>

* warehouse/macaroons: update security policy to handle multiple projects in macaroon

Signed-off-by: William Woodruff <william@trailofbits.com>

* warehouse: more migration cleanup

Signed-off-by: William Woodruff <william@trailofbits.com>

* oidc/services: add find_provider to NullOIDCProviderService

Signed-off-by: William Woodruff <william@trailofbits.com>

* macaroons/interfaces: update docstring, iface

Signed-off-by: William Woodruff <william@trailofbits.com>

* warehouse/{macaroons,oidc}: bugfixes, fix query

Almost there.

Signed-off-by: William Woodruff <william@trailofbits.com>

* Use OIDCProvider as an identity

Signed-off-by: William Woodruff <william@trailofbits.com>

* manage/history: render OIDC events more reasonably

Signed-off-by: William Woodruff <william@trailofbits.com>

* manage/views: remove the OIDC provider if it's fully orphaned

This, in turn, cascades and ensures that we delete any lingering
Macaroons associated with the provider (whether expired or not).

Signed-off-by: William Woodruff <william@trailofbits.com>

* macaroons/models: fix constraint name

Signed-off-by: William Woodruff <william@trailofbits.com>

* Avoid DB queries, fail DB lookup gracefully

Signed-off-by: William Woodruff <william@trailofbits.com>

* migrations: rebase

Signed-off-by: William Woodruff <william@trailofbits.com>

* migrations: rebase

Signed-off-by: William Woodruff <william@trailofbits.com>

* Re-remove template

Git merge hell.

Signed-off-by: William Woodruff <william@trailofbits.com>

* migrations: rebase

Signed-off-by: William Woodruff <william@trailofbits.com>

* oidc/services: fix the find_provider API

Signed-off-by: William Woodruff <william@trailofbits.com>

* tests/oidc: begin fixing tests

Signed-off-by: William Woodruff <william@trailofbits.com>

* warehouse/migrations: rebase

Signed-off-by: William Woodruff <william@trailofbits.com>

* tests/unit: fix forklift (legacy) tests

This involves setting the security policy correctly, since we now
check `request.identity` in addition to `request.user`.

Signed-off-by: William Woodruff <william@trailofbits.com>

* tests/unit: fix, fill in macaroon and manage/views tests

Signed-off-by: William Woodruff <william@trailofbits.com>

* warehouse: lint fixes

Signed-off-by: William Woodruff <william@trailofbits.com>

* tests: add GitHubProviderFactory, chip away at coverage

Signed-off-by: William Woodruff <william@trailofbits.com>

* tests: fill in OIDCProviderCaveat tests

Signed-off-by: William Woodruff <william@trailofbits.com>

* macaroons/security_policy: reduce control flow

Each macaroon has a relationship of user XOR OIDC provider,
so we can simply fall back here.

Signed-off-by: William Woodruff <william@trailofbits.com>

* tests/unit: fill in Macaroon service coverage, fix tests for API changes

Signed-off-by: William Woodruff <william@trailofbits.com>

* tests/unit: more forklift/legacy coverage

Signed-off-by: William Woodruff <william@trailofbits.com>

* tests/unit: coverage for management views, OIDC services

Signed-off-by: William Woodruff <william@trailofbits.com>

* warehouse, tests: OIDC utils coverage

Signed-off-by: William Woodruff <william@trailofbits.com>

* tests, warehouse: add OIDC view coverage

Signed-off-by: William Woodruff <william@trailofbits.com>

* warehouse, tests: feature flags for OIDC token minting

This makes the API route respect the same `warehouse.oidc.enabled`
and admin flag as the other OIDC functionality.

Signed-off-by: William Woodruff <william@trailofbits.com>

* warehouse: `make translations`

Signed-off-by: William Woodruff <william@trailofbits.com>

* migrations: rebase

Signed-off-by: William Woodruff <william@trailofbits.com>

* oidc: use EventTag, update API route

Signed-off-by: William Woodruff <william@trailofbits.com>

* templates: put provider spec in <code>

Signed-off-by: William Woodruff <william@trailofbits.com>

* warehouse, tests: display token expiration information, if present

Signed-off-by: William Woodruff <william@trailofbits.com>

* warehouse: `make translations`

Signed-off-by: William Woodruff <william@trailofbits.com>

* oidc/views: remove old TODO

Signed-off-by: William Woodruff <william@trailofbits.com>

* macaroons/services: avoid deprecated API use

Signed-off-by: William Woodruff <william@trailofbits.com>

* warehouse, tests: restrict JSON body to dictionaries

Signed-off-by: William Woodruff <william@trailofbits.com>

* tests, warehouse: distinguish between invalid and missing tokens

Signed-off-by: William Woodruff <william@trailofbits.com>

* caveats: clarify comment

Signed-off-by: William Woodruff <william@trailofbits.com>

* tests, warehouse: distinguish short-lived tokens, render them differently

Signed-off-by: William Woodruff <william@trailofbits.com>

* warehouse: `make translations`

Signed-off-by: William Woodruff <william@trailofbits.com>

* tests: fix, bringup coverage

Signed-off-by: William Woodruff <william@trailofbits.com>

* tests, warehouse: avoid rendering the OIDC provider spec

Signed-off-by: William Woodruff <william@trailofbits.com>

* tests: lint

Signed-off-by: William Woodruff <william@trailofbits.com>

* tests, warehouse: expose verified claims with the provider

We can eventually use this to put more information in the token
creation event, for nicer renderings.

Signed-off-by: William Woodruff <william@trailofbits.com>

* publishing: put each publisher's URL in the table

Signed-off-by: William Woodruff <william@trailofbits.com>

* warehouse: `make translations`

Signed-off-by: William Woodruff <william@trailofbits.com>

* oidc: refactor claim handling

Tests not updated, yet.

Signed-off-by: William Woodruff <william@trailofbits.com>

* tests, warehouse: update tests, bring coverage back up

Signed-off-by: William Woodruff <william@trailofbits.com>

* warehouse/migrations: rebase

Signed-off-by: William Woodruff <william@trailofbits.com>

* warehouse, tests: use provider URL in macaroon comment

Signed-off-by: William Woodruff <william@trailofbits.com>

* warehouse/migrations: rebase

Signed-off-by: William Woodruff <william@trailofbits.com>

Signed-off-by: William Woodruff <william@trailofbits.com>

dev/environment

@@ -39,6 +39,8 @@ BREACHED_PASSWORDS=warehouse.accounts.NullPasswordBreachedService
 
 MALWARE_CHECK_BACKEND=warehouse.malware.services.PrinterMalwareCheckService
 
+OIDC_BACKEND=warehouse.oidc.services.NullOIDCProviderService
+
 METRICS_BACKEND=warehouse.metrics.DataDogMetrics host=notdatadog
 
 STATUSPAGE_URL=https://2p66nmmycsj3.statuspage.io

tests/common/db/oidc.py

@@ -0,0 +1,28 @@
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import factory
+
+from warehouse.oidc.models import GitHubProvider
+
+from .base import WarehouseFactory
+
+
+class GitHubProviderFactory(WarehouseFactory):
+    class Meta:
+        model = GitHubProvider
+
+    id = factory.Faker("uuid4", cast_to=None)
+    repository_name = "foo"
+    repository_owner = "bar"
+    repository_owner_id = 123
+    workflow_filename = "example.yml"

tests/functional/test_templates.py

@@ -32,6 +32,7 @@
     "format_package_type": "warehouse.filters:format_package_type",
     "parse_version": "warehouse.filters:parse_version",
     "localize_datetime": "warehouse.filters:localize_datetime",
+    "ctime": "warehouse.filters:ctime",
     "canonicalize_name": "packaging.utils:canonicalize_name",
 }