New Datadog Event Sink

[atibdialpad]

Datadog has concepts of a) Metrics b) Events and c) Logs. I see that Vector has already implemented sinks for Datadog Metrics (#585) and Datadog Logs (#1469) This is a feature request for a new Datadog_Event sink.

In addition: I was able to send out Datadog metrics to Datadog in a couple of ways:
a) Using the standard datadog_metrics sink
b) Using the statsd sink (since I have the datadog agent running on my machine that runs dogstatsd as part of itself)

It would be great if Events could also be sent out to datadog similarly in both the ways.

[atibdialpad]

@binarylogic @jszwedko would it be possible to get this done in the near term ? Our transition from logstash to Vector depends on this one feature. Given that vector already has Datadog logs and metrics sinks, I am assuming that this should be trivial. I can also look in it if you think it is a simple ask.

[binarylogic]

@atibdialpad yes, we'll prioritize. Thanks for opening.

[atibdialpad]

Hey @ktff @binarylogic Can you please share the datadog event sink configs (couldn't find it in docs). Wanted to give it a go. Also when can we expect this released ?

[jszwedko]

Hey @atibdialpad ,

You can find the unrendered docs here: https://github.com/timberio/vector/blob/master/docs/reference/components/sinks/datadog_events.cue . They will be published when the sink is released (the third week of July), but you can use the latest nightly build if you want to check it out now.

[ktff]

@atibdialpad you can also checkout docs of similar sink https://vector.dev/docs/reference/configuration/sinks/datadog_logs/.
The differences are:

• type = "datadog_events"
• batch, compression, encoding, region options are not supported

[atibdialpad]

Hi @ktff
Do I need to manually delete/prune fields from the original event that datadog event doesn't expect by default ?
The dataddog event api seems to accept these fields : (Ref : https://docs.datadoghq.com/api/latest/events/#post-an-event )

event(<TITLE>, <TEXT>, <TIMESTAMP>, <HOSTNAME>, <AGGREGATION_KEY>, <PRIORITY>, <SOURCE_TYPE_NAME>, <ALERT_TYPE>, <TAGS>)

[atibdialpad]

I was able to get the events out to datadog events server by manually pruning all the non datadog required fields. Either this is a required step (@ktff can confirm) or may be some of the fields from the original event were poorly structured.

Another interesting point that I noticed was, The datadog_event sink's event.tag works exactly like how datadog treats tags (array of strings) however in datadog_metric sink, tags are objects (a dict). Had this discussion sometime last week in discord, will create an issue for better tracking.

[ktff]

@atibdialpad the sink it self should prune excess fields. I'll recheck it.