Strategy for adding our testing implementation to CI

[shaneutt]

Blixt is a project being donated to Kubernetes SIG Network for use as a vendor-neutral reference and testing Layer 4 implementation of Gateway API (if you missed it, see the mailing list thread on the subject). There are a lot of reasons why this project exists (which I wont go over exhaustively here) however of note for this discussion is that we've been stating interest since the beginning in its potential usage in CI.

Blixt is drawing near it's first release and so we're starting to think about what that CI integration might actually look like. One specific piece of feedback we received recently is a concern from a contributor that they don't want to be forced to update Blixt when they make conformance test changes that would affect it. While needing to take the time to do an exercise of proving new conformance changes work before merging them certainly has its appeal, it seems obvious that contributors will repine such a requirement so we just wanted to make it clear that we're NOT going to go with that approach.

As it stands currently the thinking is to include Blixt conformance tests as a CI workflow which would operate on a "conformance profile" (we don't have these yet) for Layer 4 conformance which would be optional for PRs, but used for releases, and potentially manually trigger-able. The idea being that the maintainers want to know if it's passing conformance prior to cutting any release (and be able to investigate why it may not be) but that there be no requirement on any contributor PRs to do so. We would also like to leave the option open for contributors to implement their changes in Blixt if they wish: there have been several instances where would-be contributors come in wanting to contribute because of the notoriety of Gateway API, but with no specific implementation and this might be helpful to them, or just someone who's particularly interested in verifying they didn't break the reference implementation.

The purpose of this post is simply to share that context and intention to gather feedback: if that general approach sounds good to you (the smaller details are not ironed out yet, and are flexible) please give a thumbs up. If not, please leave a comment below!

[howardjohn]

This addresses my concerns. Thanks!