Introduce RFC to correlate resource inputs with output

[idoru]

Readable

closes #744

PR Checklist

Note: Please do not remove items. Mark items as done [x] or use strikethrough if you believe they are not relevant

• Linked to a relevant issue. Eg: Fixes #123 or Updates #123
• Removed non-atomic or wip commits
• Filled in the Release Note section above
• Modified the docs to match changes

[netlify]

✅ Deploy Preview for elated-stonebraker-105904 canceled.

Name	Link
🔨 Latest commit	84460de
🔍 Latest deploy log	https://app.netlify.com/sites/elated-stonebraker-105904/deploys/62557cd877b94900090aa5a6

[waciumawanjohi]

Won't healthy/unhealthy be a concern for resources where the observedMatch is to a field in the status (as well as for resources where the observedMatch is to a field in the spec).

[idoru]

The healthy check only exists when the spec is implicated because we're relying on fields in the spec corresponding to the status.

Once the output is in the status, it is true that the resource may reconcile again and be unhealthy, but as long as the exact inputs specified in correlation rules match the output we can reasonably assume it is the same input.

This, as #drawbacks states is the reason why the template authors must take great care. The matching rules must uniquely identify the instance of the input (e.g. as with URL+revision for a source)

[idoru]

Actually, I take that back. I think you're right, otherwise we may not always be able to correlate inputs and outputs.

[waciumawanjohi]

Supply chains often rely on resources that continually report the latest good output (e.g. kpack). When such an object is processing new input, Cartographer is still able to read the output and use it to assert the proper shape of the following object.

WIth this proposal, Cartographer will need to adopt a caching strategy after reading good outputs of objects. Consider the case of a supply chain where resource B relies on resource A. When resource A has a good output, it will at some point get an update to its spec. It will begin process the new spec and Healthy will become unknown. As a result, Cartographer will no longer read resource A's output. That means that Cartographer will now be unable to assure that resource B has the proper spec, so resource B will be unreadable as well.

[idoru]

I see. A cache, that is, or something like lastGoodOtput on the status in your original RFC?

[scothis]

Where is the cache stored? Assume that the controller pod will crash or be rescheduled at an inconvenient time; a new controller instance needs to be able to pickup and continue processing as if nothing happened.

[idoru]

It would be very convenient to put it on the owner status, but this may cause us to exceed maximum size. One thing we've discussed privately within the team is storing outputs as another custom resource. The latter feels like it may be the only choice, barring the addition of another component for durable cache storage.

[scothis]

Where is the cache stored? Assume that the controller pod will crash or be rescheduled at an inconvenient time; a new controller instance needs to be able to pickup and continue processing as if nothing happened.