RFC: Eclipse Foundation Secure Software Supply Chain Levels

[mbarbero]

This is a request for review and feedback on a new security framework proposal. It is a pragmatic security framework designed to promote actionable security practices and provide a clear progression path for Eclipse Foundation projects to secure their supply chains.

The draft document for the spec can be found here: https://github.com/eclipse-csi/gradually/blob/main/spec.md.

Here is a little bit of context on our approach and the reasoning behind the creation of this framework:

First, the intended audience is "just" Eclipse Foundation projects: it is not intended nor proposed as an alternative to other frameworks like SLSA, NIST SSDF, or CNCF SSCSP. Rather it builds on them to define Eclipse Foundation-specific policies, interpretations, and practices. Based on those premises, our focus/objectives with this framework are:

• Developer Experience: We need to onboard and convince our projects to implement the promoted practices. Therefore, everything we do must be approachable and make sense to developers.
• Sense of Progress: Security is a journey, and as such, there needs to be a clear path toward better security. We also need to cover all parts of the supply chain, not just a subset.
• Practicality: We want a framework where it's easy to say 'yes' or 'no' regarding whether a practice is implemented. This is crucial because we aim to eventually tool and automate the verification of the implementation of some security levels. Automation does not tolerate non-prescriptive descriptions of practices

Once again, we have drawn significant inspiration from existing frameworks. Please see below how we position our framework in comparison to them:

• SLSA: SLSA's threat modeling of the software supply chain (https://slsa.dev/spec/v1.0/threats) is outstanding and serves as a significant source of inspiration for our framework. While initially aiming to address the entire supply chain, the SLSA framework eventually narrowed its focus to the build part of the supply chain for version 1.0. Progress on the other "tracks" (as SLSA calls them) has slowed since then. We believe that we need to accelerate and provide recommendations to Eclipse Foundation projects on all aspects of the supply chain. We acknowledge that this approach may sacrifice higher formalism or broader applicability, which SLSA is likely to achieve, but it is sufficient for now. Naturally, we will re-evaluate SLSA with each major release, and we aim for Eclipse Foundation projects to easily attain a high level of SLSA compliance by following the practices outlined in our framework.
• NIST SSDF: Although it covers all parts of the supply chain, NIST SSDF is very little prescriptive. It lists practices but does not describe how to implement them, focusing instead on the outcomes of the practices. We have used it to define the implementable requirements laid out in our framework. We believe that offering more concrete actions tailored to the Eclipse Foundation community provides a more approachable developer experience.
• CNCF SSCSP: This framework is prescriptive, practical, and geared toward developers, which is excellent, and we have taken much inspiration from their work. However, it lacks the progressiveness that we think is critical to ease developer adoption. SSCSP is a huge list of practices with very little prioritization of tasks. They categorize practices on two axes: assurance category and risk category, but each of these axes offers only two levels (moderate or high), which provides very little progression capabilities.

Finally, while we have not done it yet, we believe our framework could be easily mapped back to SLSA and NIST SSDF, and we plan on doing so once we have a final 1.0 version.

We look forward to your valuable feedback and guidance.

(edit 2024-10-10: removed references to EF3SCL which was a poor name)

[tellison]

@mbarbero thank you for sharing this work. It is great to see this continued focus on security at the Foundation, and this proposal is a welcome advancement.

Firstly, regarding the motivation to create a new framework, I understand and agree with your observations about the existing secure development frameworks. It is helpful to produce a progressive and practical set of levels for development at Eclipse. I think it will be helpful to cross-reference the EF3SCL practices to established descriptions in the other frameworks. This will help to provide additional context and description from an alternate source, and also allow consumers to align the EF3SCL security levels with their obligations under the existing frameworks.

It may be that entities have existing procurement requirements that are expressed in terms of the established frameworks, unless/until the EF3SCL levels becomes a recognised designation, projects may still need to declare their levels in terms of the existing frameworks to satisfy their supplier obligations.

The categories covered by the EF3SCL framework are comprehensive. A few specific comments:

• I've already raised the topic of improving developer identity assurance at Eclipse. I think this is important because a number of the practices are rooted in trusting developers, and granting individuals appropriate permissions. The XZ vulnerability analysis shows that identity fraud and social engineering are real routes for injecting vulnerabilities when the development processes are made robust.
• The framework doesn't include references to capturing appropriate levels of audit trails beyond source code control. The historical use of services and configuration changes provides information for analysis of developing threats by recognising unusual behavior, and also a way to analyse causes of vulnerabilities when they have been detected. An immutable audit log can help build assurances in a number of the practices described.
• At the higher levels I didn't see much of an opportunity to capture verification or attestations (beyond reproducible builds). A powerful characteristic of an open Foundation's projects is the ability decentralise the security posture. Rather than build 'higher and higher walls' around projects to prevent vulnerabilities, it is worth diversifying the security practices to include different ways to increase assurance. For example, having duplication of work in different systems, and having checks on expected output, would make the system more robust to help address a SolarWinds-type attack. I'd like to see a way to capture third-party attestations of a project's critical points.

There are many fundamental services that can be identified within the EF3SCL framework that should be provided by the Foundation. I appreciate that the document is not intended to cover the implementation details for the items described. While we know some have been outsourced to a trusted external service (e.g. GitHub for source code version control, and the Foundation directly manages some data (e.g. secrets) and services (e.g. code signing), there are more in this framework that should reside at the Foundation level rather than being the responsibility of the project. I trust that these will be forthcoming as projects reach the need for them.

Finally, the name needs fixing ;-) "EF3SCL" is unpronounceable, and "Eclipse Foundation Secure Software Supply Chain Levels" doesn't adequately describe the framework - since it goes beyond supply chain to include development practices and more. While not the most important point, I do believe that you will get better adoption and uptake by choosing a name that people can remember.

Again, I don't want to distract from the great work put into this framework and the levels. I'm definitely in favor of bringing this into production and I am already inspired to work towards some of these points in the Adoptium project. Thanks!

[mbarbero]

Thanks for your feedbacks, @tellison. See my answer below.

It may be that entities have existing procurement requirements that are expressed in terms of the established frameworks, unless/until the EF3SCL levels becomes a recognised designation, projects may still need to declare their levels in terms of the existing frameworks to satisfy their supplier obligations.

This is well understood. We will definitely put more effort into mapping the listed practices against the existing framework as we approach v1.0.

I've already raised the topic of improving developer identity assurance at Eclipse. I think this is important because a number of the practices are rooted in trusting developers, and granting individuals appropriate permissions. The XZ vulnerability analysis shows that identity fraud and social engineering are real routes for injecting vulnerabilities when the development processes are made robust.

This is a sensitive topic. I owe you an answer on this ticket and suggest we continue the discussion there for better community awareness. When/if we reach a common understanding of what this means, it should definitely be incorporated into the framework.

The framework doesn't include references to capturing appropriate levels of audit trails beyond source code control. The historical use of services and configuration changes provides information for analysis of developing threats by recognising unusual behavior, and also a way to analyse causes of vulnerabilities when they have been detected. An immutable audit log can help build assurances in a number of the practices described.

Good point, this is definitely something to be added to the higher levels. Would you mind creating a ticket to discuss this further?

At the higher levels I didn't see much of an opportunity to capture verification or attestations (beyond reproducible builds). A powerful characteristic of an open Foundation's projects is the ability decentralise the security posture. Rather than build 'higher and higher walls' around projects to prevent vulnerabilities, it is worth diversifying the security practices to include different ways to increase assurance. For example, having duplication of work in different systems, and having checks on expected output, would make the system more robust to help address a SolarWinds-type attack. I'd like to see a way to capture third-party attestations of a project's critical points.

I'd argue that there are some practices in the framework with these goals:

• L3: Secure the Build: Build Provenance
• L4: Secure the Build: Build provenance is generated by the build service
• L5: Secure the Build: Release process is not under the control of the project

However, it's true that there is little about duplication in different systems and checks between them. But I don't see that as doable without reproducible builds first. Do you agree? We might want to add another practice at a level higher than the one about reproducible builds then (again, to keep the progressive approach). What do you think?

There are many fundamental services that can be identified within the EF3SCL framework that should be provided by the Foundation

Yes, or by a trusted third party. The goal is to alleviate as much as possible the burden of added security on the developers. We will provide as many services as possible in that regard.

Finally, the name needs fixing

:) This is a bad name, for sure. What about just naming it Gradually (like the repository) and renaming the file to spec.md?

[tellison]

<snip>

The framework doesn't include references to capturing appropriate levels of audit trails beyond source code control. The historical use of services and configuration changes provides information for analysis of developing threats by recognising unusual behavior, and also a way to analyse causes of vulnerabilities when they have been detected. An immutable audit log can help build assurances in a number of the practices described.

Good point, this is definitely something to be added to the higher levels. Would you mind creating a ticket to discuss this further?

Done.

At the higher levels I didn't see much of an opportunity to capture verification or attestations (beyond reproducible builds). A powerful characteristic of an open Foundation's projects is the ability decentralise the security posture. Rather than build 'higher and higher walls' around projects to prevent vulnerabilities, it is worth diversifying the security practices to include different ways to increase assurance. For example, having duplication of work in different systems, and having checks on expected output, would make the system more robust to help address a SolarWinds-type attack. I'd like to see a way to capture third-party attestations of a project's critical points.

I'd argue that there are some practices in the framework with these goals:

• L3: Secure the Build: Build Provenance
• L4: Secure the Build: Build provenance is generated by the build service
• L5: Secure the Build: Release process is not under the control of the project

However, it's true that there is little about duplication in different systems and checks between them. But I don't see that as doable without reproducible builds first. Do you agree?

Partial reproducibility (of components) may be possible but full reproducible builds would be an advanced capability, and aspirational for many projects. There are other characteristics that can be tracked and used to provide levels of verification and attestations. These range from simple technical observations (e.g. expected file sizes, build run times, number of artefacts in a release, etc), the use of tools that scan for vulnerabilities at various key points in the process, unit testing and code coverage, etc.

Without being prescriptive about the techniques used, levels can include practices such as 'verifies the integrity of the build artefacts' in the same way that it currently includes '...ensures the integrity and authenticity of the build process...'.

We might want to add another practice at a level higher than the one about reproducible builds then (again, to keep the progressive approach). What do you think?

At higher levels 'external code audits' would fit into that category, and pen' testing at the top level. As you move up the levels you would be increasingly looking to external verification and cross-checking of the work underway directly in the project; the logic being that you can't hack a wide network who are all cross-checking each others work.

There are many fundamental services that can be identified within the EF3SCL framework that should be provided by the Foundation

Yes, or by a trusted third party. The goal is to alleviate as much as possible the burden of added security on the developers. We will provide as many services as possible in that regard.

+1

Finally, the name needs fixing

:) This is a bad name, for sure. What about just naming it Gradually (like the repository) and renaming the file to spec.md?

I'm no marketing person, but 'gradually' doesn't give any clue about what this is. I don't have a good suggestion, sorry.

[bobcallaway]

FYI, the first revision of the SLSA source track just merged today: https://github.com/slsa-framework/slsa/blob/main/docs/spec/v1.1/source-requirements.md

[mbarbero]

Thanks for the heads up, @bobcallaway. This is great to see. We will provide a mapping between the practices we have listed and SLSA soon.