What adopting Second Moment looks like

Export traces from your provider console or observability stack (Langfuse, LangSmith, Helicone, OTel). Any CSV/JSONL with a per-unit cost or tokens + a unit id works — the pricing demo accepts the same formats.

# what we need — nothing else
task_id,cost_usd,timestamp
tkt_1041,0.38,2026-08-02T14:11:09Z
tkt_1042,1.94,2026-08-02T14:12:31Z

Point your existing GenAI telemetry at Second Moment, or wrap your client. Traffic is untouched; Second Moment mirrors costs and publishes a live quote against your real workload. Shadow mode also drafts your spec for you: after a week of traffic, Second Moment emits the envelope your agent already operates in — observed steps, model mix, iteration counts, token budgets — as a draft spec. You tighten the bounds and sign off; you never start from a blank file.

# one line: your existing OTel GenAI spans, duplicated to Second Moment
OTEL_EXPORTER_OTLP_ENDPOINT=https://ingest.secondmoment.co  # illustrative

# or, in code — no behavior change:
client = sm.shadow(OpenAI())   # mirrors usage, never intercepts

First, what the spec is not: it is not a program, and nothing executes it. Your workflow stays your code. The spec is a policy the gateway enforces against your traffic — think firewall rules or rate limits, not an orchestrator. Phase names like triage and respond are just labels your calls carry (like span names in tracing), so the meter knows which limit each call counts against. Real agents are loops — a model call, some tool calls, repeat until done — and the spec simply names the phases of the loop you already have and puts a number on each:

# your code today (LangGraph / custom loop)      # what the spec declares
state = triage(ticket)                            → phase "triage": 1 call, budget tier
while not done:                                   → phase "respond":
    reply = llm(ctx, tools=[kb, orders])              each turn = 1 iteration
    done  = resolved or escalated or turns > N        max_iterations caps N
close(ticket, state)                              → terminal: resolved|escalated|abandoned

The minimal valid spec is one phase with an iteration cap and terminal states — that alone makes the worst case computable. Adding phases and tool budgets tightens the envelope, and a tighter envelope is a lower ceiling — which is a better price. Granularity is a price lever, not homework. One spec per workflow family, in your repo:

# A support bot (Intercom-style): triage, then a conversational
# tool-using loop until the ticket closes. Each agent turn in your
# existing loop = one iteration of the `respond` phase.
workflow: support-bot
archetype: ticket-resolution
unit: ticket
steps:
  triage:                        # 1 cheap call: route + severity
    model_tier: budget
    max_iterations: 1
  respond:                       # your agent loop lives here
    model_tier: frontier
    max_iterations: 12           # turns before forced escalation
    tools:
      kb_search:    {tool_budget_tokens: 20000}
      order_lookup: {tool_budget_tokens: 4000}
terminal: [resolved, escalated, abandoned]

# A coding agent (SWE-agent / OpenHands-style): localize the fault,
# then an edit->run-tests loop until green or out of budget.
workflow: issue-fixer
archetype: code-tasks
unit: issue
steps:
  localize:
    model_tier: budget
    max_iterations: 4            # repo search + file reads
    tools:
      code_search: {tool_budget_tokens: 30000}
  fix:
    model_tier: frontier
    max_iterations: 20           # each edit->test cycle = 1 iteration
    tools:
      run_tests: {tool_budget_tokens: 15000}
terminal: [tests_pass, tests_fail, timeout]

# A research agent (GAIA-style): plan once, a bounded
# search-and-read loop, then one synthesis pass.
workflow: research-brief
archetype: deep-research
unit: brief
steps:
  plan:
    model_tier: frontier
    max_iterations: 1
  gather:
    model_tier: budget
    max_iterations: 25           # each search-or-read action = 1 iteration
    tools:
      web_search: {tool_budget_tokens: 8000}
      fetch_page: {tool_budget_tokens: 60000}
  synthesize:
    model_tier: frontier
    max_iterations: 2
terminal: [delivered, insufficient_sources, timeout]

# A cross-app operator (AppWorld-style RPA successor): plan, act
# across SaaS APIs, verify the end state.
workflow: ops-runner
archetype: app-operation
unit: task
steps:
  plan:
    model_tier: frontier
    max_iterations: 2
  act:
    model_tier: budget
    max_iterations: 30           # each API action = 1 iteration
    tools:
      calendar_api: {tool_budget_tokens: 3000}
      email_api:    {tool_budget_tokens: 6000}
      crm_api:      {tool_budget_tokens: 6000}
  verify:
    model_tier: budget
    max_iterations: 2
terminal: [completed, partial, failed]

# A document pipeline: parse, extract to schema, validate with
# bounded retries. Nearly deterministic — the tightest bound and
# the cheapest fixed price in the registry.
workflow: doc-intake
archetype: document-pipelines
unit: document
steps:
  parse:
    tool: ocr_parse
    tool_budget_tokens: 50000
  extract:
    model_tier: budget
    max_iterations: 2
  validate:
    model_tier: budget
    max_iterations: 2            # schema-check retries
terminal: [extracted, quarantined]

Submit it and Second Moment answers with the two numbers that define the contract — the static worst case, and (after a burn-in on your real sample inputs) the quote:

$ sm workflows push support-bot.sm.yaml
  ✓ wf_7f3a · static bound $3.40/unit · status: needs burn-in

$ sm burnin run wf_7f3a samples.jsonl        # 100–200 real inputs, a few $ of tokens
  ✓ 184 units · mean $0.86 · p99 $2.20
  → quote: $0.95/unit fixed   ·   cap $1.20/unit for $0.06

Your agent executes exactly as before. Two headers tie every model call to its workflow and unit; the gateway enforces the spec (tier substitution, iteration caps, the hard ceiling) below anything your agent's logic can influence. When a unit finishes, your code declares the terminal state — that closes the unit and fixes what you're billed.

client = OpenAI(
    base_url="https://gw.secondmoment.co/v1",           # illustrative
    default_headers={
        "x-sm-workflow": "wf_7f3a",
        "x-sm-unit": ticket.id,                    # your id, any string
        "x-sm-step": "resolve"})                   # optional per-step tag

# ... your existing agent loop, unchanged ...

sm.close(unit=ticket.id, state="resolved")         # or POST /v1/units/{id}/close
# unclosed units resolve by declared timeout rules — nothing hangs open

And here is the same unit from the gateway's side — the spec doing its only job, counting and enforcing per label. Note the refusal is not a crash: it's a signal your loop's existing give-up path handles, exactly like any tool error:

# gateway's view of unit tkt_1042 (spec wf_7f3a)
call 1   step=triage    iter 1/1  · tier budget ✓   → forwarded
call 2   step=respond   iter 1/12 · tier frontier ✓ → forwarded  (substituted: sonnet-tier)
  ⋮
call 13  step=respond   iter 12/12                  → forwarded
call 14  step=respond   iter 13/12 — exceeds cap    → REFUSED (429 sm_cap_exceeded)
your code catches it    → sm.close(unit, state="escalated")
receipt issued          · consumed $0.33 of the $3.40 bound

Untagged calls all count against a single default phase — the minimal spec. Labels can't be gamed into free compute: whatever the labeling, total spend per unit is capped by the static bound, which is what your fixed price was written against.

What ran, what it cost at the prices in force at call time, what you were billed, and how much headroom was left under the bound. Receipts are the audit trail behind every invoice and every renewal — including the units Second Moment loses money on.

{
  "unit": "tkt_1042",
  "workflow": "wf_7f3a",
  "terminal_state": "resolved",
  "billed": {"fixed_price": 0.95, "tier": "M"},
  "execution": {
    "calls": 14, "input_tokens": 61240, "output_tokens": 3891,
    "cost_at_call_time": 1.12, "model_tiers_used": ["budget", "frontier"],
    "ceiling": {"static_bound": 3.40, "consumed": 0.33}
  },
  "sm_margin_this_unit": -0.17
}

At the live stage, yes — and only then. The gateway speaks the OpenAI-compatible API, so the change is base_url= plus two headers; stages 1–2 never touch your traffic. Second Moment forwards each call to the real provider, substituting the specific model only within the tier your spec declares.

In live mode, Second Moment's — deliberately. You pay the fixed per-unit price; Second Moment pays the providers; the difference is Second Moment's margin or Second Moment's loss. The key swap is the risk transfer: on your own keys you'd still be carrying the cost variance and Second Moment would just be a meter. In shadow mode nothing changes — your keys, your endpoint, mirrored telemetry only.

Second Moment fails open. If the gateway is unreachable, the client falls back to calling the provider directly on escrowed keys, and usage reconciles after the fact. Enforcement lapses during the outage — but the party exposed to unenforced spending is Second Moment, not you: your price is fixed either way. A cost control can't fail open (you'd eat the overrun); a cost underwriter can, because the tail lands on our book. Your agent never goes down because Second Moment did.

Mapping is just the header: every call stamped x-sm-unit: tkt_1042 belongs to that unit — your own id, first use opens the unit, close (or timeout) ends it. Context stays where it lives today: LLM APIs are stateless, and your code or framework accumulates the messages between turns — Second Moment forwards the request body untouched and never stores or injects context. That's also why tier substitution is safe mid-unit (full context arrives with every call). One subtlety we handle deliberately: routing is sticky within a unit, so re-sent context keeps hitting the provider's prompt cache — hopping models mid-unit would forfeit cache discounts, and that cost would land on Second Moment's own margin, which is exactly the right incentive.

A passthrough hop adds single-digit milliseconds against calls measured in seconds — the same pattern every AI gateway has normalized. Prompts transit Second Moment's infrastructure under the trace contract: retained for pricing and audit (that's what receipts and renewals are built on), no other use, with regional deployment and BYO-cloud on the enterprise path.