> ## Documentation Index
> Fetch the complete documentation index at: https://ngrok.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Observability Overview

> Monitor and debug AI Gateway requests.

The AI Gateway provides observability at multiple levels:

<CardGroup cols={2}>
  <Card title="Metrics for Model Selection" icon="chart-line" href="/ai-gateway/observability/metrics">
    Real-time latency and error rate metrics available in CEL expressions for intelligent routing decisions.
  </Card>

  <Card title="Traffic Inspector" icon="magnifying-glass" href="/ai-gateway/observability/traffic-inspector">
    View full request and response bodies for debugging in the ngrok dashboard.
  </Card>

  <Card title="Log Exporting" icon="arrow-right-to-bracket" href="/ai-gateway/observability/log-exporting">
    Export HTTP logs to external systems for logging and analysis.
  </Card>
</CardGroup>

<Note>
  The AI Gateway is in Early Access. We're actively working on additional observability features including dedicated dashboards, provider/retry tracking, token usage visualization, and cost analytics.

  **Have feedback?** We'd love to hear what observability features would be most valuable to you. Email [niji@ngrok.ai](mailto:niji@ngrok.ai) with your ideas.
</Note>

## Current capabilities

| Feature             | Status      | Description                                                                  |
| ------------------- | ----------- | ---------------------------------------------------------------------------- |
| CEL Metrics         | ✅ Available | Latency, error rates, token counts available in `model_selection` strategies |
| Traffic Inspector   | ✅ Available | Standard HTTP request/response inspection                                    |
| Log Exporting       | ✅ Available | Standard ngrok HTTP traffic logs                                             |
| AI-specific Events  | 🚧 Roadmap  | Provider selection, retry tracking, token usage events                       |
| Dedicated Dashboard | 🚧 Roadmap  | AI Gateway-specific analytics and visualizations                             |

## Quick example: Metrics-based routing

Use real-time metrics to route to the fastest, most reliable models:

```yaml theme={null}
on_http_request:
  - type: ai-gateway
    config:
      providers:
        - id: openai
          api_keys:
            - value: ${secrets.get('openai', 'key')}
        - id: anthropic
          api_keys:
            - value: ${secrets.get('anthropic', 'key')}
      model_selection:
        strategy:
          # Prefer low-latency models with good reliability
          - "ai.models.filter(m, m.metrics.global.latency.upstream_ms_avg < 1000 && m.metrics.global.error_rate.total < 0.05)"
          # Fallback to any available model
          - "ai.models"
```

Then connect using the [OpenAI SDK](/ai-gateway/sdks/openai) or [an SDK of your choice](/ai-gateway/sdks).

## Understanding metric scope

<Warning>
  Metrics are **only available within `model_selection` strategy expressions**. They are not available in general CEL expressions elsewhere in your Traffic Policy.

  This is because metrics are populated at runtime during AI Gateway request processing, specifically when evaluating model selection strategies.
</Warning>

For full details on available metrics and their scopes, see [Metrics Reference](/ai-gateway/observability/metrics).

## Coming soon

Enhanced observability features are actively being developed:

* **AI Gateway Dashboard** - Dedicated views for provider performance, token usage, and costs
* **Provider Tracking** - See which providers handle each request
* **Retry Visibility** - Track failover decisions and retry attempts
* **Token Analytics** - Visualize token usage trends across models
* **Cost Estimation** - Automatic cost calculation based on provider pricing

**Want to influence the roadmap?** Email [niji@ngrok.ai](mailto:niji@ngrok.ai) with what observability features you need.