During Eval Agent trace ids are 00000000000000000000000000000000

[jlewi]

In #133 we added code to link eval results to the generate trace. The key here was to return the trace id from the Agent prediction.
However, it looks like the trace id ends up being 00000000000000000000000000000000

Here's an example log message

2024-05-30T15:36:38.079-0700    INFO    agent/agent.go:114      Agent.Generate returning response       {"traceId": "00000000000000000000000000000000", "evalMode": true, "response": {"blocks": [{"kind": "MARKUP", "language": "", "contents": "To get the ids of the execution traces for block `01HZ0W9X2XF914XMG6REX1WVWG`, you can use the following command:", "outputs": [], "trace_ids": [], "id": "01HZ5TBS9ZCPKVRTZCYYFT1HJB"}, {"kind": "CODE", "language": "bash", "contents": "curl http://localhost:8080/api/blocklogs/01HZ0W9X2XF914XMG6REX1WVWG | jq .execTraceIds ", "outputs": [], "trace_ids": [], "id": "01HZ5TBS9ZCPKVRTZCZ23PYSPD"}, {"kind": "MARKUP", "language": "", "contents": "Please execute this command to retrieve the ids of the execution traces for the specified block. Let me know if you need any further assistance!", "outputs": [], "trace_ids": [], "id": "01HZ5TBS9ZCPKVRTZCZ4NCMW0V"}], "trace_id": "00000000000000000000000000000000"}}

I suspect the problem is that we aren't setting up the OTEL trace provider during evaluation.
I think we need to call

foyle/app/pkg/application/app.go

Line 76 in 5a63b46

func (a *App) SetupOTEL() error {

[jlewi]

It looks like that isn't the problem. It looks like no span is defined so we get
a noopspan. I suspect this is because we are calling it as a library function and not as an RPC.

[jlewi]

It seems like there are two possible ways we could potentially deal with this

1. Inside Agent.Generate we could start a new span if no span is set

   • We can't test for a noopspan because that's private
   • I think we can test if its recording

2. In the Eval code we could create initialize a trace and add it to the context

I think I like the second idea better. Its cleaner and it also allows us to link together all the processing in the evaluator.