Feature: determine if publishing is credential-based or OIDC-based

[di]

Is your feature request related to a problem? Please describe.
GitHub Actions supports OIDC identities for workflow runs: https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/about-security-hardening-with-openid-connect

Some ecosystems now support OIDC-based publishing with these identities, e.g. for PyPI: pypi/warehouse#12465 (currently in closed beta), which can be identified based on the usage of the official action without supplying credentials: https://github.com/pypa/gh-action-pypi-publish#publishing-with-openid-connect

This should be considered more secure than using long-lived credentials like API tokens or username/passwords for publishing. It also has the added benefit of linking the source repository to the published artifact in a verifiable way.

Describe the solution you'd like
For ecosystems that support OIDC-based publishing, Scorecard should penalize repositories that use long-lived credentials for publishing, and reward projects that adopt OIDC-based publishing, either via a new check or an existing check.

[woodruffw]

This would be great!

On the detection side: I think this could be detected by checking the combination of effective permissions and explicit inputs for the configured pypa/gh-action-pypi-publish action: if the effective permissions include id-token: write and no password is explicitly set, then we know the publishing is OIDC-based. Otherwise, it's credential-based.

[di]

As an example, here's an instance of the change that would be necessary to move from less-secure long-lived credentials to more secure OIDC-based publishing: sigstore/sigstore-python#566

[pnacht]

Would this be a new metric for the Packaging check or a new check in itself? It seems a good fit for Packaging, but then that raises the issue of how to score projects in different ecosystems (those that support OIDC and those that don't).

As a separate check this becomes easy to handle: if a project belongs to an OIDC-friendly ecosystem, it gets scored. Otherwise, the check is ignored.

The same thing could be done within Packaging, but it might seem odd to the maintainer that a Python project gets an 8/10 for publishing (without OIDC) while an R project gets a 10/10 for the exact same thing.

[di]

I'm not sure! If it should be part of Packaging, I think there's a larger philosophical question about how checks should change in response to new features: is it relative to that ecosystem or all ecosystems? E.g., in your example, should that R project go from 10/10 to 8/10 once OIDC-based publishing is available for that ecosystem?

I would argue yes, because 10/10 score should be the best a project can do at that time, but a project that doesn't respond or enact new/better security features as they become available shouldn't expect to remain at 10/10 indefinitely.

[laurentsimon]

+1 from me. Great to encourage users to use OIDC instead of long-lived tokens.

[varunsh-coder]

JFYI - I have created an issue in https://github.com/step-security/secure-repo to automate transformation of GitHub Actions workflows to use OIDC.

[di]

More examples of this already available other ecosystems: https://dart.dev/tools/pub/automated-publishing

[laurentsimon]

I'm +1 to have this under Packaging check. Folks who have a 10 today will drop to a lower score (say, 8) and it will encourage them to update. IMO, we don't need "backward compatibility" on the score when best practices are updated based on changes in the ecosystems / attacks, etc

[github-actions]

Stale issue message - this issue will be closed in 7 days

[woodruffw]

Not stale 🙂

[github-actions]

This issue is stale because it has been open for 60 days with no activity.

[woodruffw]

Not stale. Is there a way to disable the stale check on this issue? I believe it can't really go "stale" per se, since it's purely a feature request.

[pnacht]

Not stale. Is there a way to disable the stale check on this issue? I believe it can't really go "stale" per se, since it's purely a feature request.

Yes, the Action has "exempt labels" for issues and PRs. However, Scorecard currently only has that for "bug", "good first issue" and something else. Maybe should add a "never-stale" label.

[woodruffw]

Makes sense. I think "good first issue" would also make sense in this specific case 🙂

[laurentsimon]

Here's another one we should detect https://blog.rubygems.org/2023/12/14/trusted-publishing.html