Create HIP for Adoption of CUE

[JuryA]

Proposal Overview

This HIP proposes the integration of CUE (Configuration Unification Engine) as the primary tool for processing and validating Helm values files, replacing the existing JSON Schema support.

Key Advantages of CUE

• Superset Compatibility: As a superset of YAML, JSON, and JSON Schema, CUE extends existing capabilities while maintaining complete backward compatibility.
• Seamless Transition: Ensures that all current Helm charts and configurations remain fully supported without any disruptions.
• Enhanced Validation: Provides more robust and comprehensive validation mechanisms to ensure configuration integrity.
• Improved Defaulting Mechanisms: Facilitates better handling of default values, reducing manual configuration efforts.
• Simplified Schema Management: Makes schema management easier and more efficient, streamlining the development workflow.

Benefits

By adopting CUE, the integration will offer:

1. Enhanced Validation: More accurate and reliable validation of Helm values files.
2. Improved Defaulting: Simplified management of default configurations, leading to fewer errors and inconsistencies.
3. Easier Schema Management: Streamlined processes for updating and maintaining schemas, saving time and resources.

Conclusion

Integrating CUE as the primary tool for processing and validating Helm values files will not only preserve existing functionalities but also introduce significant improvements in validation, defaulting, and schema management. This transition ensures a more robust and efficient configuration management system for Helm charts and related configurations.

References

• CUE Language Documentation
• Helm Documentation
• JSON Schema Documentation
• Helm PR #13236
• Helm Issue #13260

[gjenkins8]

nit: I would use e.g. superseding rather than replacing, and implementation rather than support. To me, "replacing" and "support" reads like JSON schema support will go away in lieu of using CUE. But I think the HIP means to utilizing CUE to implement JSON schema functionality.

Suggested change

This HIP proposes the integration of CUE (Configuration Unification Engine) as the primary tool for processing and validating Helm values files, replacing the existing JSON Schema support. As a superset of YAML, JSON, and JSON Schema, CUE extends existing capabilities while maintaining complete backward compatibility. This ensures that all current Helm charts and configurations remain fully supported, with the added benefits of enhanced validation, improved defaulting mechanisms, and easier schema management.

This HIP proposes the integration of CUE (Configuration Unification Engine) as the primary tool for processing and validating Helm values files, superseding the existing JSON Schema implementation. As a superset of YAML, JSON, and JSON Schema, CUE extends existing capabilities while maintaining complete backward compatibility. This ensures that all current Helm charts and configurations remain fully supported, with the added benefits of enhanced validation, improved defaulting mechanisms, and easier schema management.

[gjenkins8]

Also, I always thought CUE stood for "Configure Unify Execute" (and not: "Configuration Unification Engine"). Where does this alternate come from?

[gjenkins8]

minor note: helm/helm#13283 would mean Helm can support newer json schema versions

(if that PR lands, we would want to remove this point I think)

[gjenkins8]

One significant benefit to utilizing CUE for values processing, is that CUE would allow "pre-processing" of input values (beyond validation).

A frequent pain-point of Helm charts is that values tend to be very "raw". It is difficult for values to not simply be the exact inputs to substitute in templates. As any business logic that expands or modifies the values needs to be written in gotemplate, either as a "named template" (function) or inline with the template logic. And gotemplate is both a terrible language to write configuration "business" logic. And the actual rendering of charts IMHO should contain as little business logic as possible (should be relegated as much as possible to dumb value substitution).

One motivation to me would be to enable mutation of the input values (which CUE excels at). In order to "expand" the user provided values into values suitable for the (gotemplate) rendering phase.

This HIP could just limit CUE to validation only. But it might be worth considering if CUE could be used beyond validation?

[gjenkins8]

I think, you might be thinking about the above in this point? If so, I would recommend expanding that

Enhanced Features: CUE natively supports advanced features such as defaulting, flexible referencing of values, and other configuration management capabilities critical for complex Helm deployments.

[gjenkins8]

On this note, we have considered Pkl (https://pkl-lang.org/). I'm more familiar with CUE than Pkl. But Pkl would I believe provide both validation and rendering functionality.

That said, there is certainly rationale to keep value processing/validation and manifest rendering separate concerns. Which does mean we could choose separate technologies for each. ie. for values processing/validation we could support (JSON schema, CUE, Pkl), and for manifest rendering we could support (Pkl, gotemplate)

Overall, Helm I think needs to make a fairly strategic choice in how to approach these problem spaces. Personally, I would like to see better support in both. JSON schema (to the point of this HIP) is limited. And using unstructured text formatting ala. gotemplate for producing structured documents (Kubernetes object manifests) is a awkward at best solution.

[gjenkins8]

Thanks for the HIP! Apologies it took so long to begin to review. Overall, I would like to (read: Helm should) think hard about the points in #364 (comment) . And how Helm wants to approach this problem space generally.

Very happy to discuss/brainstorm (ping me of slack if wanted). I do think CUE today could provide significant benefits (so thanks again for writing this up!). I don't want to "block" CUE support in lieu of a fanciful idea of supporting Pkl. But also, I wouldn't want to see CUE introduced if it was immediately going to be superseded (or simply immediately conflict with alternatives) itself. So I think worthy to spend some time discussing options/approaches first.

[abuchanan-airbyte]

Does it make sense to extend this proposal to include CUE as a template language?

[loewenstein]

Does it make sense to extend this proposal to include CUE as a template language?

I would really like to see CUE in multiple places in Helm. However, I could see the discussion for validation to be much easier than that for templating.

I might be wrong, but if not it might be better to have two HIPs - although they might refer to oneanother.