Complete Logs Export

​

Overview

• Full prompt and completion content
• Tool definitions and function calls
• All request parameters (temperature, max_tokens, etc.)
• Complete response metadata

​

Why Use This Feature?

• Unified Observability: Consolidate LLM telemetry with your existing monitoring infrastructure
• External Analysis: Send Portkey logs to specialized GenAI observability tools like LangSmith
• Standards-Based: Follows OpenTelemetry GenAI semantic conventions for consistent attribute naming
• Zero Code Changes: Enable via environment variables without modifying application code

​

Scope

​

Included

• Automatic log export to OTLP-compatible HTTP endpoints
• Full GenAI semantic convention attribute mapping
• Request parameters, response metadata, and token usage
• Tool definitions and prompt/completion content
• LangSmith-compatible attribute overrides

​

Out of Scope

• gRPC OTLP transport (HTTP/JSON only)
• Metrics export (use Analytics Export for that)
• Custom attribute mapping configuration

​

Architecture

1. LLM request is processed through Portkey Gateway
2. After receiving the LLM response, a log object is generated
3. The log is transformed into OTLP span format with GenAI semantic attributes
4. The span is pushed to the configured external OTLP endpoint

​

Configuration

​

Environment Variables

​

Example Configuration

EXPERIMENTAL_GEN_AI_OTEL_TRACES_ENABLED: "true"
EXPERIMENTAL_GEN_AI_OTEL_EXPORTER_OTLP_ENDPOINT: https://api.smith.langchain.com/otel
EXPERIMENTAL_GEN_AI_OTEL_EXPORTER_OTLP_HEADERS: x-api-key=<your-langsmith-api-key>

EXPERIMENTAL_GEN_AI_OTEL_TRACES_ENABLED: "true"
EXPERIMENTAL_GEN_AI_OTEL_EXPORTER_OTLP_ENDPOINT: https://http-intake.logs.datadoghq.com/api/v2/otlp
EXPERIMENTAL_GEN_AI_OTEL_EXPORTER_OTLP_HEADERS: DD-API-KEY=<your-datadog-api-key>

EXPERIMENTAL_GEN_AI_OTEL_TRACES_ENABLED: "true"
EXPERIMENTAL_GEN_AI_OTEL_EXPORTER_OTLP_ENDPOINT: http://otel-collector:4318
EXPERIMENTAL_GEN_AI_OTEL_EXPORTER_OTLP_HEADERS: Authorization=Bearer <token>

​

Span Attributes

​

Resource Attributes

​

Common Attributes

​

Inference Attributes

​

Embeddings Attributes

​

Message Attributes

​

Span Structure

{
  "resourceSpans": [{
    "resource": {
      "attributes": [
        { "key": "service.name", "value": { "stringValue": "portkey" } },
        { "key": "otel.semconv.version", "value": { "stringValue": "1.40.0" } }
      ]
    },
    "scopeSpans": [{
      "scope": { "name": "custom.genai.instrumentation" },
      "spans": [{
        "traceId": "<32-char-hex>",
        "spanId": "<16-char-hex>",
        "parentSpanId": "<optional-16-char-hex>",
        "name": "chat gpt-4o",
        "kind": 3,
        "startTimeUnixNano": "<nanoseconds>",
        "endTimeUnixNano": "<nanoseconds>",
        "status": { "code": 1, "message": "200" },
        "attributes": [/* GenAI attributes */]
      }]
    }]
  }]
}

• Span name follows the format {operation_name} {model} (e.g., chat gpt-4o, embeddings text-embedding-3-small)
• Span kind is CLIENT (3), reflecting that the gateway acts as a client to the LLM provider

​

Trace Context

• Trace ID: Uses Portkey’s trace ID if valid (32 hex chars), otherwise generates a new one
• Span ID: Uses Portkey’s span ID if valid (16 hex chars), otherwise generates a new one
• Parent Span ID: Preserved from parent request if available

​

Error Handling

• Export Failures: Logged internally; does not affect request processing
• Invalid Endpoints: Connection errors are caught and logged
• Malformed Responses: Error responses from the OTLP endpoint are logged with status text

​

Performance Considerations

• Async Processing: Log export runs in parallel with other post-request handlers
• No Request Latency Impact: Export happens after the response is returned
• HTTP/JSON Protocol: Uses standard HTTP POST with JSON payload

​

Security Considerations

• Authentication: Use the headers configuration to pass API keys or bearer tokens
• Data Sensitivity: Prompt and completion content is included in exports
• Network Security: Use HTTPS endpoints for production deployments

​

Integration with LangSmith

​

Verification

1. Enable the feature with your endpoint configuration
2. Make an LLM request through Portkey Gateway
3. Check your external observability platform for incoming spans
4. Verify spans contain expected GenAI attributes

​

Known Limitations

• HTTP Only: gRPC OTLP protocol is not supported
• No Batching: Each log is pushed individually (no span batching)
• No Retry Logic: Failed exports are not retried
• Experimental Conventions: GenAI semantic conventions may change