Tracing LLM Calls

An ai_trace is one row per LLM call. It captures the model, provider, token counts, cost, latency, status, and an optional link to the agent run that made the call.

Automatic capture

The adapters wrap a provider client and record every call. This is the shortest path and the one to prefer: token counts come straight off the provider's usage object, and cost is priced server-side.

import os
import anthropic
from keelwave import Keelwave

client = Keelwave(api_key=os.environ["KEELWAVE_API_KEY"])
llm = client.wrap_anthropic(anthropic.Anthropic())

resp = llm.messages.create(
    model="claude-sonnet-4-5",
    max_tokens=1024,
    messages=[{"role": "user", "content": "hello"}],
)

llm = client.wrap_openai(openai.OpenAI(base_url=...), provider="groq")

Adapters never break your program: if keelwave is unreachable, the recording failure is swallowed and your model call returns normally.

Manual ingest

Send a trace directly when no adapter fits — a provider without one, a custom gateway, or a call you construct by hand.

client.ingest_ai(
    model="gpt-4o",
    status="success",
    provider="openai",
    input_tokens=1200,
    output_tokens=350,
    latency_ms=820,
    request_id="req_abc123",
    agent_run_id=run.id,
    metadata={"feature": "summariser"},
)

Only model and status are required. status must be success, error, or timeout; use error_message / errorMessage to record why a call failed.

Two fields are filled in for you by the API. total_tokens is set to input_tokens + output_tokens when you omit it and send both. cost_usd is computed from the model's catalog rate whenever both token counts are present, overriding a client-supplied value; your value is kept only for models with no known rate. See Cost Tracking for the details.

Linking traces to runs

A trace belongs to a run when agent_run_id is set. Both SDKs do this automatically: opening a run stores it in a context-local variable — a ContextVar in Python, AsyncLocalStorage in TypeScript — and the adapters read it on every call.

with client.run("research-agent", input=question) as run:
    # agent_run_id set automatically from the active run
    resp = llm.messages.create(model="claude-sonnet-4-5", max_tokens=1024,
                               messages=[{"role": "user", "content": question}])
    run.step("think", content="got an answer, deciding next move")

Calls made outside a run are still recorded — they just arrive unlinked, with agent_run_id null. That is the right shape for LLM work that is not part of an agent loop.

The @observe decorator has one behaviour worth knowing: inside a run it emits a step, but outside a run it falls back to writing a standalone trace, using the function name as model and observe as provider. That keeps instrumented helpers visible even when nothing wrapped them.

What shows in the dashboard

Traces surface through the runs they belong to rather than as a standalone browser. Once linked, a trace's tokens and cost roll into its run's total_tokens and total_cost_usd, which drive:

• the run list and run detail views,
• GET /v1/projects/{projectID}/agent/summary — avg_cost_usd and avg_tokens, current window versus previous,
• GET /v1/projects/{projectID}/agent/health — the same averages per agent_name.

Because the rollup is maintained by a trigger on insert, traces that arrive after the run has finished still update its totals.