property "type" must be provided when reading from the Broker

[matzew]

What happened?

Creating a logger-pipe that reads from the broker seems to not work:

apiVersion: camel.apache.org/v1
kind: Pipe
metadata:
  name: log-sink-pipe
spec:
  source:
    ref:
      kind: Broker
      apiVersion: eventing.knative.dev/v1
      name: kamelet
  sink:
    ref:
      kind: Kamelet
      apiVersion: camel.apache.org/v1
      name: log-sink

In theory it should create a trigger that has no filter, which is valid

Steps to reproduce

1. create knative broker
2. create a logger like above
3. see the logs

Relevant log output

{"level":"info","ts":"2024-05-21T09:38:00Z","logger":"camel-k.controller.pipe","msg":"Initializing Pipe","request-namespace":"default","request-name":"log-sink-pipe","api-version":"camel.apache.org/v1","kind":"Pipe","ns":"default","name":"log-sink-pipe"}
{"level":"info","ts":"2024-05-21T09:38:00Z","logger":"camel-k.controller.pipe","msg":"State transition","request-namespace":"default","request-name":"log-sink-pipe","api-version":"camel.apache.org/v1","kind":"Pipe","ns":"default","name":"log-sink-pipe","phase-from":"","phase-to":"Error"}
{"level":"info","ts":"2024-05-21T09:38:00Z","logger":"camel-k.controller.pipe","msg":"Integration error","request-namespace":"default","request-name":"log-sink-pipe","api-version":"camel.apache.org/v1","kind":"Pipe","ns":"default","name":"log-sink-pipe","reason":"Couldn't create an Integration custom resource","error-message":"could not determine source URI: property \"type\" must be provided when reading from the Broker"}
{"level":"error","ts":"2024-05-21T09:38:00Z","msg":"Reconciler error","controller":"pipe-controller","object":{"name":"log-sink-pipe","namespace":"default"},"namespace":"default","name":"log-sink-pipe","reconcileID":"76ed504b-a166-4df7-b457-360c3f606dab","error":"could not determine source URI: property \"type\" must be provided when reading from the Broker","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tsigs.k8s.io/controller-runtime@v0.15.2/pkg/internal/controller/controller.go:324\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tsigs.k8s.io/controller-runtime@v0.15.2/pkg/internal/controller/controller.go:265\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\tsigs.k8s.io/controller-runtime@v0.15.2/pkg/internal/controller/controller.go:226"}

### Camel K version

2.3.1

[squakez]

Thanks for reporting. Do you mind providing the dump of your Integration spec. The Pipe should have created an Integration named log-sink-pipe and we can see there the configuration created and the reason why this may be failing. Thanks!

[matzew]

There is no integration created. The message on the status of the Pipe said that it could not create it, see here k describe:

k describe pipes.camel.apache.org     
Name:         log-sink-pipe
Namespace:    default
Labels:       <none>
Annotations:  <none>
API Version:  camel.apache.org/v1
Kind:         Pipe
Metadata:
  Creation Timestamp:  2024-05-21T10:18:24Z
  Generation:          1
  Resource Version:    2028
  UID:                 b21fd27a-c897-4881-9820-33afe4da23eb
Spec:
  Sink:
    Ref:
      API Version:  camel.apache.org/v1
      Kind:         Kamelet
      Name:         log-sink
  Source:
    Ref:
      API Version:  eventing.knative.dev/v1
      Kind:         Broker
      Name:         kamelet
Status:
  Conditions:
    Last Transition Time:  2024-05-21T10:18:24Z
    Last Update Time:      2024-05-21T10:18:24Z
    Message:               could not determine source URI: property "type" must be provided when reading from the Broker
    Reason:                Couldn't create an Integration custom resource
    Status:                False
    Type:                  IntegrationError
  Observed Generation:     1
  Phase:                   Error
Events:
  Type     Reason        Age   From                     Message
  ----     ------        ----  ----                     -------
  Warning  KameletError  42s   camel-k-pipe-controller  Cannot reconcile Pipe log-sink-pipe: could not determine source URI: property "type" must be provided when reading from the Broker

and here see there are no such resources:

k get integrations.camel.apache.org -A
No resources found

[matzew]

@squakez regardless if I set the:

...
    properties:
      type: <my cloud event type>
...

on the sink or the source, it seems to not work

[matzew]

Update:

I restarted over and now this worked:

apiVersion: camel.apache.org/v1
kind: Pipe
metadata:
  name: log-sink-pipe
spec:
  source:
    ref:
      kind: Broker
      apiVersion: eventing.knative.dev/v1
      name: kamelet
    properties:
      type: com.corp.my.beer.source
  sink:
    ref:
      kind: Kamelet
      apiVersion: camel.apache.org/v1
      name: log-sink

Gave me the trigger I wanted.

[matzew]

OK, I found it out:

When I edit this pipe, the CLI says configured, but nothing really changed. Hence I could not before have it work.

I have to delete the Pipe, edit it, and re-create it, than it works.

I guess that's also a diffenrent bug?

[squakez]

I think the root cause is the second bug you've reported. If you create this from scratch with the right property does it work? in such case, feel free to close this issue and we'll work to fix #5531.

[matzew]

I think this is two pieces:

• Why is type required? It is perfectly valid to have a Trigger w/out any attributes on the filter (this bug)
• when I have it not correct, why does it not reconcile after I fix the yaml? delete and re-apply is IMO just a work-around (the other bug)

[squakez]

Okey, we'll keep this open to answer the first question, thanks.

[squakez]

In the past we hit certain problems when using Brokers (see apache/camel-k-runtime#536). However, this should have been solved as we now leverage directly Camel Knative component. I'm proceeding with a fix.

[christophd]

@squakez I think this is not the same as apache/camel-k-runtime/issues/536. Yes, we have had problems with the broker name (e.g. when using a customized broker name other than default) and this one has been fixed already.

This one here is different. The type information mentioned here refers to the cloud event type information that is used to filter the event stream. In a Pipe users currently need to set this event type in order to filter the event stream, but it is totally valid to read all events from a broker without any type filter.

We need to fix the Trigger filter that is created as part of Camel K Knative traits to allow users to get the full event stream without any filter.

[squakez]

Thanks for having a look at that @christophd - I am dropping the PR and you can continue the work as I understood you were about to work on a side request related. Thanks!