Telemetry

server/orders.ts

import { createServer } from 'experimental-a2/server'
import { otel } from 'experimental-a2/otel'
import { orders } from '@/contracts'

export const ordersServer = createServer({
  contract: orders,
  telemetry: otel(),
  // ...store, scheduler, and handlers
})

With any OpenTelemetry setup registered (on Vercel, @vercel/otel), one trace shows the request, a2.append, the inline a2.drain, an a2.event span per handler, and the appends those handlers make. Failures mark their a2.event span, and a2.event.outcome = dead_lettered is the attribute to alert on. Agent servers add a2.ai.* spans.

`experimental-a2/otel`

otel(options?: { tracer?: Tracer; tracerName?: string }): A2Telemetry

OpenTelemetry instrumentation for a server (@opentelemetry/api is a peer dependency). Pass it as telemetry. Spans nest through the active context, so the happy path lands in one trace. A2Telemetry is a one-method interface: anything with span(name, attributes, fn) works, and otel() is the adapter A2 ships.

The span catalogue

This catalogue is a contract: renames and additions are breaking for dashboards and are called out in the package's CHANGELOG.md. All spans carry a2.contract and a2.session_id. Core spans describe the durable runtime; a2.ai.* spans describe the built-in AI handlers.

Layer	Span	Wraps
Core	`a2.append`	validation + the store write (one batch)
Core	`a2.drain`	one drain pass, including concurrent handlers
Core	`a2.event`	one claimed dispatch, including its handler and completion
Core	`a2.claim`	one `claimAvailable` store operation
Core	`a2.commit`	one `completeAttempt` or `failAttempt` store operation
Core	`a2.state`	one `state()` read: snapshot-plus-tail load + reducer fold
Core	`a2.snapshot`	one background batch of checkpoint writes
Core	`a2.redrive`	one `session.redrive()`: the store operation and starting the session's wakeup
AI	`a2.ai.control`	applying an input/work report/session-close decision and preparing its returned events
AI	`a2.ai.generate`	one generation handler, including Core calls and model/SDK work
AI	`a2.ai.context`	constructing model messages, including tool prompt reconstruction when uncached
AI	`a2.ai.compaction`	automatic summarization or the custom compaction callback
AI	`a2.ai.tool`	one durable tool handler, including application execution
AI	`a2.ai.ack`	waiting for a persisted work report's canonical events and control receipt

Durations include nested work: a2.event contains its AI handler and later commit, AI handlers call Core, and a2.state includes reducer code. Acknowledgements overlap control processing, claims can cover several events, and snapshots run in the background, so summing spans, or subtracting summed children, does not yield exclusive Core or AI overhead. Model and application I/O is included; its child spans have the detail.

Attributes arrive in two waves: at start (passed to span()) and mid-span (via handle.setAttribute, for outcomes and counts known only after the work ran). A recorder must merge both; the ones you alert on are all mid-span.

Attribute	Span	When	Values
`a2.append.source`	`a2.append`	start	`external` \| `handler`
`a2.append.mode`	`a2.append`	start	`inline` \| `dispatch`
`a2.append.types`	`a2.append`	start	comma-joined event types
`a2.append.count`	`a2.append`	start	batch size
`a2.append.armed`	`a2.append`	mid	`false` when the scheduler arm failed and this append degraded to append-driven healing
`a2.drain.outcome`	`a2.drain`	mid	`settled` \| `busy` \| `stalled` \| `failed`
`a2.drain.processed`	`a2.drain`	mid	events processed this pass
`a2.claim.outcome`	`a2.claim`	mid	`claimed` \| `busy` \| `settled`
`a2.claim.count`	`a2.claim`	mid	number of claimed events, or zero
`a2.commit.operation`	`a2.commit`	start	`complete` \| `fail`
`a2.commit.index`	`a2.commit`	start	event log position
`a2.commit.attempt`	`a2.commit`	start	durable 1-based attempt ordinal
`a2.commit.outcome`	`a2.commit`	mid	`completed` \| `failed` \| `dead_lettered` \| `superseded`
`a2.event.type`	`a2.event`	start	the event's type
`a2.event.index`	`a2.event`	start	log position
`a2.event.id`	`a2.event`	start	event id
`a2.event.attempt`	`a2.event`	start	same durable 1-based ordinal as `ctx.attempt`
`a2.event.lane`	`a2.event`	start	stored lane value; absent for concurrent unlaned work
`a2.event.handled`	`a2.event`	start	normally `true`; `false` when a custom-adapter row has no handler
`a2.event.outcome`	`a2.event`	mid	`processed` \| `failed` \| `dead_lettered` \| `superseded` \| `surrendered`
`a2.event.aborted`	`a2.event`	mid	`true` when `abortOn` fired during the run
`a2.state.reducer`	`a2.state`	start	the reducer's name
`a2.state.snapshot`	`a2.state`	mid	`hit` \| `miss` \| `rejected` (schema guard discarded it) \| `memory` (served from the server's in-memory state)
`a2.state.folded`	`a2.state`	mid	events folded past the snapshot
`a2.state.index`	`a2.state`	mid	the frontier the returned state reflects
`a2.snapshot.reducer`	`a2.snapshot`	start	the reducer's name
`a2.snapshot.index`	`a2.snapshot`	start	greatest checkpoint index in the write batch
`a2.snapshot.count`	`a2.snapshot`	start	number of checkpoints in the write batch
`a2.redrive.index`	`a2.redrive`	start	log position of the event being redriven
`a2.redrive.action`	`a2.redrive`	start	`retry` \| `skip`
`a2.redrive.redriven`	`a2.redrive`	mid	`true` when the event was dead-lettered and is now resolved
`a2.redrive.armed`	`a2.redrive`	mid	`false` when the scheduler arm failed and the wakeup degraded to append-driven healing
`a2.ai.control.type`	`a2.ai.control`	start	`ai.control.requested` \| `ai.work.reported` \| `ai.session.closed`
`a2.ai.control.id`	`a2.ai.control`	start	triggering event id
`a2.ai.control.attempt`	`a2.ai.control`	start	durable 1-based attempt ordinal
`a2.ai.generate.request_id`	`a2.ai.generate`	start	generation request event id
`a2.ai.generate.attempt`	`a2.ai.generate`	start	durable 1-based attempt ordinal
`a2.ai.context.request_id`	`a2.ai.context`	start	generation request event id
`a2.ai.context.phase`	`a2.ai.context`	start	`generation` \| `refresh` \| `tool`
`a2.ai.compaction.request_id`	`a2.ai.compaction`	start	generation request event id
`a2.ai.compaction.generation_id`	`a2.ai.compaction`	start	generation attempt id
`a2.ai.compaction.policy`	`a2.ai.compaction`	start	`automatic` \| `custom`
`a2.ai.tool.request_id`	`a2.ai.tool`	start	originating generation request event id
`a2.ai.tool.call_id`	`a2.ai.tool`	start	tool call id
`a2.ai.tool.name`	`a2.ai.tool`	start	tool name
`a2.ai.tool.attempt`	`a2.ai.tool`	start	durable 1-based attempt ordinal
`a2.ai.ack.request_id`	`a2.ai.ack`	start	reporting worker event id
`a2.ai.ack.report_id`	`a2.ai.ack`	start	work report event id, matching the control span's id

Drain outcomes: settled means nothing actionable is left; busy means live per-event claims or a failed event's retry hold remain; stalled means a caught failure in this drain needs a later retry. failed appears only on scheduler deliveries: the drain threw after arming its successor watchdog, the span records the error, and that successor retries the session.
A failing handler marks its a2.event span with the exception and error status. Alert on a2.event.outcome = dead_lettered. A dead-lettered event blocks only later events in its lane, until session.redrive() resolves it; other lanes stay eligible.
superseded means a stale completion or failure did not change the log.