[8.x](backport #41516) x-pack/metricbeat/module/openai: Add new module

.github/CODEOWNERS

@@ -223,6 +223,7 @@ CHANGELOG*
 /x-pack/metricbeat/module/iis @elastic/obs-infraobs-integrations
 /x-pack/metricbeat/module/istio/ @elastic/obs-cloudnative-monitoring
 /x-pack/metricbeat/module/mssql @elastic/obs-infraobs-integrations
+/x-pack/metricbeat/module/openai @elastic/obs-infraobs-integrations
 /x-pack/metricbeat/module/oracle @elastic/obs-infraobs-integrations
 /x-pack/metricbeat/module/panw @elastic/obs-infraobs-integrations
 /x-pack/metricbeat/module/prometheus/ @elastic/obs-cloudnative-monitoring

CHANGELOG.next.asciidoc

@@ -417,6 +417,7 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]
 - Add support for location label as an optional configuration parameter in GCP metrics metricset. {issue}41550[41550] {pull}41626[41626]
 - Add support for podman metrics in docker module. {pull}41889[41889]
 - Added `tier_preference`, `creation_date` and `version` fields to the `elasticsearch.index` metricset. {pull}41944[41944]
+- Add new OpenAI (`openai`) module for tracking usage data. {pull}41516[41516]
 - Add `use_performance_counters` to collect CPU metrics using performance counters on Windows for `system/cpu` and `system/core` {pull}41965[41965]
 
 *Metricbeat*

metricbeat/docs/fields.asciidoc

@@ -68,6 +68,7 @@ grouped in the following categories:
 * <<exported-fields-mysql>>
 * <<exported-fields-nats>>
 * <<exported-fields-nginx>>
+* <<exported-fields-openai>>
 * <<exported-fields-openmetrics>>
 * <<exported-fields-oracle>>
 * <<exported-fields-panw>>
@@ -56706,6 +56707,372 @@ type: long
 
 --
 
+[[exported-fields-openai]]
+== openai fields
+
+openai module
+
+
+
+[float]
+=== openai
+
+
+
+
+[float]
+=== usage
+
+OpenAI API usage metrics and statistics
+
+
+
+*`openai.usage.organization_id`*::
++
+--
+Organization identifier
+
+type: keyword
+
+--
+
+*`openai.usage.organization_name`*::
++
+--
+Organization name
+
+type: keyword
+
+--
+
+*`openai.usage.api_key_id`*::
++
+--
+API key identifier
+
+type: keyword
+
+--
+
+*`openai.usage.api_key_name`*::
++
+--
+API key name
+
+type: keyword
+
+--
+
+*`openai.usage.api_key_redacted`*::
++
+--
+Redacted API key
+
+type: keyword
+
+--
+
+*`openai.usage.api_key_type`*::
++
+--
+Type of API key
+
+type: keyword
+
+--
+
+*`openai.usage.project_id`*::
++
+--
+Project identifier
+
+type: keyword
+
+--
+
+*`openai.usage.project_name`*::
++
+--
+Project name
+
+type: keyword
+
+--
+
+[float]
+=== data
+
+General usage data metrics
+
+
+
+*`openai.usage.data.requests_total`*::
++
+--
+Number of requests made
+
+type: long
+
+--
+
+*`openai.usage.data.operation`*::
++
+--
+Operation type
+
+type: keyword
+
+--
+
+*`openai.usage.data.snapshot_id`*::
++
+--
+Snapshot identifier
+
+type: keyword
+
+--
+
+*`openai.usage.data.context_tokens_total`*::
++
+--
+Total number of context tokens used
+
+type: long
+
+--
+
+*`openai.usage.data.generated_tokens_total`*::
++
+--
+Total number of generated tokens
+
+type: long
+
+--
+
+*`openai.usage.data.cached_context_tokens_total`*::
++
+--
+Total number of cached context tokens
+
+type: long
+
+--
+
+*`openai.usage.data.email`*::
++
+--
+User email
+
+type: keyword
+
+--
+
+*`openai.usage.data.request_type`*::
++
+--
+Type of request
+
+type: keyword
+
+--
+
+[float]
+=== dalle
+
+DALL-E API usage metrics
+
+
+
+*`openai.usage.dalle.num_images`*::
++
+--
+Number of images generated
+
+type: long
+
+--
+
+*`openai.usage.dalle.requests_total`*::
++
+--
+Number of requests
+
+type: long
+
+--
+
+*`openai.usage.dalle.image_size`*::
++
+--
+Size of generated images
+
+type: keyword
+
+--
+
+*`openai.usage.dalle.operation`*::
++
+--
+Operation type
+
+type: keyword
+
+--
+
+*`openai.usage.dalle.user_id`*::
++
+--
+User identifier
+
+type: keyword
+
+--
+
+*`openai.usage.dalle.model_id`*::
++
+--
+Model identifier
+
+type: keyword
+
+--
+
+[float]
+=== whisper
+
+Whisper API usage metrics
+
+
+
+*`openai.usage.whisper.model_id`*::
++
+--
+Model identifier
+
+type: keyword
+
+--
+
+*`openai.usage.whisper.num_seconds`*::
++
+--
+Number of seconds processed
+
+type: long
+
+--
+
+*`openai.usage.whisper.requests_total`*::
++
+--
+Number of requests
+
+type: long
+
+--
+
+*`openai.usage.whisper.user_id`*::
++
+--
+User identifier
+
+type: keyword
+
+--
+
+[float]
+=== tts
+
+Text-to-Speech API usage metrics
+
+
+
+*`openai.usage.tts.model_id`*::
++
+--
+Model identifier
+
+type: keyword
+
+--
+
+*`openai.usage.tts.num_characters`*::
++
+--
+Number of characters processed
+
+type: long
+
+--
+
+*`openai.usage.tts.requests_total`*::
++
+--
+Number of requests
+
+type: long
+
+--
+
+*`openai.usage.tts.user_id`*::
++
+--
+User identifier
+
+type: keyword
+
+--
+
+[float]
+=== ft_data
+
+Fine-tuning data metrics
+
+
+
+*`openai.usage.ft_data.original`*::
++
+--
+Raw fine-tuning data
+
+type: object
+
+--
+
+[float]
+=== assistant_code_interpreter
+
+Assistant Code Interpreter usage metrics
+
+
+
+*`openai.usage.assistant_code_interpreter.original`*::
++
+--
+Raw assistant code interpreter data
+
+type: object
+
+--
+
+[float]
+=== retrieval_storage
+
+Retrieval storage usage metrics
+
+
+
+*`openai.usage.retrieval_storage.original`*::
++
+--
+Raw retrieval storage data
+
+type: object
+
+--
+
 [[exported-fields-openmetrics]]
 == Openmetrics fields