Skip to content

Commit

Permalink
fix: AI Gateway metrics info (#8285)
Browse files Browse the repository at this point in the history
Remove enterprise label from Logging, add bullet for AI LLM metrics to Prometheus plugin, add metrics section for Advanced Analytics

Signed-off-by: Diana <75819066+cloudjumpercat@users.noreply.github.com>
  • Loading branch information
cloudjumpercat authored Dec 18, 2024
1 parent c21f139 commit 5bdab92
Show file tree
Hide file tree
Showing 2 changed files with 12 additions and 1 deletion.
3 changes: 3 additions & 0 deletions app/_hub/kong-inc/prometheus/overview/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,9 @@ license signature. Those metrics are only exported on {{site.base_gateway}}.
measures the current number of database entities.
- **Number of Nginx timers** : A gauge metric that measures the total number of Nginx
timers, in Running or Pending state.
{% if_version gte:3.8.x %}
- **[AI LLM metrics](#ai-llm-metrics)** <span class="badge enterprise"></span> : AI LLM metrics are available per provider, model, cache, database name (if cached), embeddings provider (if cached), embeddings model (if cached), and workspace.
{% endif_version %}

{% if_version gte:3.0.x %}
### Metrics disabled by default
Expand Down
10 changes: 9 additions & 1 deletion app/_src/gateway/ai-gateway/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -196,7 +196,6 @@ Kong's AI Gateway allows you to configure semantic caching.
Kong's AI Gateway enables comprehensive observability of your AI services through logging and metrics. These features provide insights into AI usage, performance, and costs, helping you optimize and govern AI operations effectively.

#### Logging
{:.badge .enterprise}

Kong's AI Gateway provides standardized logging formats for AI plugins, allowing you to track and analyze AI usage consistently across various providers.

Expand All @@ -214,8 +213,17 @@ AI operations, helping you monitor performance and costs effectively.

For more information, see [AI Metrics](/gateway/{{ page.release }}/production/monitoring/ai-metrics).

#### {{site.konnect_short_name}} Advanced Analytics

Kong's AI Gateway allows you to expose and visualize AI metrics with {{site.konnect_short_name}} Advanced Analytics. These metrics include
the number of AI requests, the cost associated with AI services, and the token usage per provider and model. This setup provides a real-time view of
AI operations, helping you monitor performance and costs effectively.

For more information, see [{{site.konnect_short_name}} Advanced Analytics](/konnect/analytics/).

{% endif_version %}


## Quickstart script

Kong offers an interactive AI quickstart script that launches a demo instance of {{site.base_gateway}} running AI Proxy:
Expand Down

0 comments on commit 5bdab92

Please sign in to comment.