[Elastic Agent] Agent.id of running processes

[ph]

Agent with System and Endpoint integrations. All agent.id's were different for the same host.

Agent: bffe5cf9-2436-476e-b4cb-39c64235b866
Filebeat: 40154036-66e3-4b71-9505-54c5e6ca74aa
Metricbeat: b9e0d066-a255-4e31-9dfe-13b1419da034
Endpoint: 49ecf6c5-6e09-4a36-8fc5-8f8dbaa77649

I think we should have the same ID on everything so we can correlate events generated by the "Elastic Agent"

[elasticmachine]

Pinging @elastic/ingest-management (Team:Ingest Management)

[ph]

@blakerouse @ruflin WDYT?

[ferullo]

cc @scunningham @kevinlog

For my part, I am ambivalent whether Agent, Beats, Endpoint all share the same ID or not. Endpoint reports elastic.agent.id in all documents, which is Agent's ID, so Endpoint documents can be correlated even if Endpoint and Agent don't share the same ID.

[blakerouse]

@ph surprised they are all different, I would expect them to all be the same.

[ruflin]

I assume all elastic.agent.ids are the same? It is just that the agent.id are different as they are generated by each process? I agree this might be confusing as the processes should be an implementation detail.

[ruflin]

Second thought: I wonder if we should even ship agent.id for the processes, as not in all cases the process itself is actually the agent but sometimes the observer.

[ph]

@ruflin @blakerouse So the fix would makes sure that all process send the elastic.agent.id with the same id, the one from the agent.

Concerning Agent.id, Is this coming from the actual input in metricbeat/filebeat?

[ruflin]

Yes, I would already expect today that elastic.agent.id is identical everywhere. Is this also not the case?

For agent.id, I think each input should decide to send it or not. But that will be more a Beats problem to solve. By default, it should not be sent.

[kevinlog]

For my part, I am ambivalent whether Agent, Beats, Endpoint all share the same ID or not. Endpoint reports elastic.agent.id in all documents, which is Agent's ID, so Endpoint documents can be correlated even if Endpoint and Agent don't share the same ID.

What's important to me is that we still have the elastic.agent.id in the Endpoint so that we can use it in the app. It elastic.agent.id and agent.id end up being the same, that's fine too.

cc @ferullo @ph @ruflin

[ph]

@blakerouse I believe that metrics/logs collected by the Elastic-Agent doesn't include that the agent id?

[andrewkroh]

I think this would be good to change the agent.id of the running processes to match what we show to users in the Fleet UI. It can be confusing to have different IDs and I think it leaks a bit of an implementation detail that Agent currently consists for several different processes (which might change in the future).

@ph @blakerouse What would it take to change this for Beats with Agent (not worrying about Endpoint here)? This was my thinking of the options:

• Add a processor that sets (overwrites) agent.id similar to how elastic_agent.id is set today. This is really simple, but the actual UUID the beat has in its meta.json might leak in other places (logs or monitoring data).
• Add an option to the Beat to configure the agent ID.

[elasticmachine]

Pinging @elastic/agent (Team:Agent)

[ph]

Add a processor that sets (overwrites) agent.id similar to how elastic_agent.id is set today. This is really simple, but the actual UUID the beat has in its meta.json might leak in other places (logs or monitoring data).

Not sure if that would really leak, it think this would take precedence? @michalpristas

[blakerouse]

I would think the processor would overwrite the agent.id set on any event. But possibly leak if the overwritten agent.id is used in another field. In that case they would not match, but again that would be a different field and might be acceptable (if it is even used in another field).

[andrewkroh]

Then would you be OK if I went ahead with opening a PR to add in the processor to overwrite the agent.id value?

[ph]

@andrew Kroh ***@***.***> Yes that would be great!

…

On Thu, Jun 17, 2021 at 10:08 AM Andrew Kroh ***@***.***> wrote: Then would you be OK if I went ahead with opening a PR to add in the processor to overwrite the agent.id value? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#21121 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAAFAG5RQXDF2XUOB57MXLTTH6VPANCNFSM4RPGGCGA> .

-- -- ph