📖 Mention 2FA relevance although not checked by Scorecard

[joycebrum]

What kind of change does this PR introduce?

(Is it a bug fix, feature, docs update, something else?)

docs update

• PR title follows the guidelines defined in our pull request documentation

Which issue(s) this PR fixes

Fixes #2526

Special notes for your reviewer

Not sure about the title, let me know what you think

Does this PR introduce a user-facing change?

For user-facing changes, please add a concise, human-readable release note to
the release-note

(In particular, describe what changes users might need to make in their
application as a result of this pull request.)

NONE

[codecov]

Codecov Report

Merging #2528 (43c91c9) into main (fa0592f) will not change coverage.
The diff coverage is n/a.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #2528   +/-   ##
=======================================
  Coverage   40.77%   40.77%           
=======================================
  Files         115      115           
  Lines        9604     9604           
=======================================
  Hits         3916     3916           
  Misses       5409     5409           
  Partials      279      279           

[github-actions]

Integration tests success for
[1013a3e]
(https://github.com/ossf/scorecard/actions/runs/3649049293)

[olivekl]

Thanks for this useful addition! I left some comments.

[naveensrinivasan]

I'm undecided about this. Do we want to be the source of security recommendations? How important are they? What should we recommend, and what shouldn't we?

[olivekl]

I'm undecided about this. Do we want to be the source of security recommendations? How important are they? What should we recommend, and what shouldn't we?

Good questions. If we're positioning Scorecard as a checklist of actions to take to improve the security of your project, then neglecting any mention of 2FA seems like a big gap. But where to draw the line? I would be fine with making this exception for 2FA only.

Definitely would like to hear other thoughts on this. @azeemshaikh38 @laurentsimon

[laurentsimon]

I'm undecided about this. Do we want to be the source of security recommendations? How important are they? What should we recommend, and what shouldn't we?

I don't have a clear answer either. But I suppose when it's related to important GH settings, we could consider it. Maybe we could also have a paragraph explaining "what scorecard is not". In a "what scorecard is not" section, we could say things like: we don't check packages, we don't check for malicious code, etc. It is not currently obvious from the "What is scorecard" section that we only look at source repository (not packages); we look at best practices in your settings and code, etc. We could make this a little clearer up front, since some people get lost in the myriad of tools for supply-chain: example, we could explain with diagrams where Scorecard stands in the entire supply-chain story.

Good questions. If we're positioning Scorecard as a checklist of actions to take to improve the security of your project, then neglecting any mention of 2FA seems like a big gap. But where to draw the line? I would be fine with making this exception for 2FA only.

Definitely would like to hear other thoughts on this. @azeemshaikh38 @laurentsimon

I'd be fine having a section about "Limitations and complementary tools" with pointers for users. For 2FA, we could point to using AllStar in conjunction with Scorecard - something we're trying to improve.

Beyond that, we may need a dedicated website: where do I start to secure my supply-chain. I think OSSF / LF have been thinking about this, IIUC

Just some ideas...

[joycebrum]

In my opinion, Scorecard already works with Github Best Practices recommendation and 2FA would certainly be a check if possible.

Considering the objective of improving the supply-chain security, every project that decides to adopt Scorecard would see this 2FA part and would consider implement it, which would increase the supply-chain security by extension.

And I agree with @olivekl that we can make some exceptions consider the relevance of 2FA. Besides, this "what scorecard is not" would be great to also deal with #2474.

[laurentsimon]

In my opinion, Scorecard already works with Github Best Practices recommendation and 2FA would certainly be a check if possible.

Considering the objective of improving the supply-chain security, every project that decides to adopt Scorecard would see this 2FA part and would consider implement it, which would increase the supply-chain security by extension.

Good point. We did not consider it before because it requires an admin token from org maintainers. But we already have a check like WebHook which only runs with special tokens too.. so we could add a check for it. It just won't be available to many users.. but maybe with fine-grained tokens on their way it would work...

[github-actions]

Integration tests success for
[43c91c9]
(https://github.com/ossf/scorecard/actions/runs/3651109243)

[naveensrinivasan]

I'm undecided about this. Do we want to be the source of security recommendations? How important are they? What should we recommend, and what shouldn't we?

Good questions. If we're positioning Scorecard as a checklist of actions to take to improve the security of your project, then neglecting any mention of 2FA seems like a big gap. But where to draw the line? I would be fine with making this exception for 2FA only.

Definitely would like to hear other thoughts on this. @azeemshaikh38 @laurentsimon

Sounds good to me! Thanks

[varunsh-coder]

In my opinion, Scorecard already works with Github Best Practices recommendation and 2FA would certainly be a check if possible.
Considering the objective of improving the supply-chain security, every project that decides to adopt Scorecard would see this 2FA part and would consider implement it, which would increase the supply-chain security by extension.

Good point. We did not consider it before because it requires an admin token from org maintainers. But we already have a check like WebHook which only runs with special tokens too.. so we could add a check for it. It just won't be available to many users.. but maybe with fine-grained tokens on their way it would work...

Would be great if Scorecard had this 2FA check. I am not sure what GitHub APIs need to be called to confirm if all maintainers of a project have 2FA enabled. Does anyone know?

[olivekl]

Would be great if Scorecard had this 2FA check. I am not sure what GitHub APIs need to be called to confirm if all maintainers of a project have 2FA enabled. Does anyone know?

Erring on the side of caution, would we want to surface that information (assuming it's possible)? Would visibility be limited only to project owners and not be made available the general public? It seems risky to make 2FA status generally known.

[varunsh-coder]

Would be great if Scorecard had this 2FA check. I am not sure what GitHub APIs need to be called to confirm if all maintainers of a project have 2FA enabled. Does anyone know?

Erring on the side of caution, would we want to surface that information (assuming it's possible)? Would visibility be limited only to project owners and not be made available the general public? It seems risky to make 2FA status generally known.

Thats a good point. One way to think about it is to only surface info to the public when it is available without authentication/ authorization. If GitHub API needs a token to provide the info, e.g. in case of 2FA, then Scorecard should not make the info public. Just make it available as output of the CLI.

[laurentsimon]

Would be great if Scorecard had this 2FA check. I am not sure what GitHub APIs need to be called to confirm if all maintainers of a project have 2FA enabled. Does anyone know?

Erring on the side of caution, would we want to surface that information (assuming it's possible)? Would visibility be limited only to project owners and not be made available the general public? It seems risky to make 2FA status generally known.

Thats a good point. One way to think about it is to only surface info to the public when it is available without authentication/ authorization. If GitHub API needs a token to provide the info, e.g. in case of 2FA, then Scorecard should not make the info public. Just make it available as output of the CLI.

+1, this would only be used in the CLI.

[laurentsimon]

OK. Let's merge this in given that 2FA is important. It'd be good to be able to link to another doc / website / project for 2FA and the difference between SMS vs OTP-based, etc.