Observability & rate limits

Turn a slow booking into an explainable trace

In the model-free booking fixture, the first result spends one HTTP call and zero model calls. Selecting the slot normally spends neither another lookup nor a model call while the dependency remains valid. Confirmation spends the POST. Metrics belong to each returned operation, so correlate calls using your session and request IDs rather than treating one result's metrics as the lifetime total.

If a real chat is slow, compare httpMs, modelRecallMs, toolSelectionMs, and conversationMs. If recall is common for straightforward requests, improve metadata before increasing model size. If HTTP dominates, tune or fix the backend before changing routing.

Add instrumentation with a question in mind

Start with status, tool ID, error code, duration, modelCalls, toolCalls, and retryCount. Then add lifecycle events to explain a particular failure. Do not log credentials or blindly retain full user messages.

messages budgets count incoming runtime operations, including continuations, not only natural-language messages. messages_per_session is the lifetime operation budget. ai_calls and tool_calls count actual work; a retry consumes another tool unit. The default window limiter is per session and process, so it is not a global tenant quota.

Test with intentionally low budgets. Expect RATE_LIMITED when a window is exhausted and RATE_SESSION_LIMIT for the session lifetime cap. Distinguish these from API 429, which appears as TOOL_RATE_LIMITED.

Runtime metrics

Each result contains metrics such as:

interface RuntimeMetrics {
  modelCalls: number;
  toolCalls: number;
  inputTokens: number;
  outputTokens: number;
  durationMs: number;
  retryCount: number;
  dependencySteps: number;
  routingMs: number;
  lexicalRetrievalMs: number;
  modelRecallMs: number;
  toolSelectionMs: number;
  conversationMs: number;
  modelRecallCalls: number;
  toolSelectionCalls: number;
  conversationCalls: number;
  httpMs: number;
}

This makes it possible to tell whether latency comes from lexical retrieval, model recall, final tool selection, conversation, dependencies or upstream HTTP.

Lifecycle hook

const runtime = await AgentRuntime.create({
  configPath: "./config.yml",
  onEvent(event) {
    logger.info({
      name: event.name,
      requestId: event.requestId,
      sessionId: event.sessionId,
      at: event.at,
      details: event.details,
    });
  },
});

Events cover major routing, dependency, confirmation, execution, retry and error boundaries. Hooks are redacted and instrumentation exceptions are intentionally prevented from turning a successful business side effect into a failed call.

Debug mode

debug: true

adds bounded routing/execution diagnostics such as candidate scores and mapped field names. It is not a request for hidden chain-of-thought and should not contain credentials.

Rate limiting

Default memory limiter exposes independent counters for:

• AI calls;
• HTTP/tool attempts;
• incoming runtime operations;
• lifetime operations per session.

rate_limits:
  ai_calls: 60
  tool_calls: 120
  messages: 120
  messages_per_session: 1000
  window_ms: 60000

A horizontally scaled deployment needs a shared limiter if limits must be globally enforced across workers.

Circuit diagnostics

runtime.diagnostics()

returns per-tool circuit state for the current runtime instance. Circuit breakers are not shared automatically between workers.