AI proxy

Amba.ai.* proxies LLM requests through amba so your provider keys never ship to a client. You write the prompt once in the console, reference it by prompt_slug from any SDK, and the server fills in the system prompt + variables + model + rate-limits + usage tracking.

Providers wired today: Anthropic (messages.create), OpenAI (chat.completions.create), Mistral (mistral.chat.completions.create, OpenAI-compatible), and Google Gemini (gemini.generateContent). All return the upstream response shape verbatim, plus a usage event so you can attribute cost per user. Register a prompt against any of them by setting its provider — including Gemini, whose distinct request shape (system instruction + multi-turn content + variables) is built for you — bring your own key per provider (the key stays server-side).

Beyond text, the managed AI surface also proxies image generation, text-to-speech, and audio transcription with your key kept server-side and per-call usage metered. OpenAI transcription is duration-priced; Gemini 3.5 Transcribe is priced from its provider-reported audio-input and text-output tokens.

Quick start

import { Amba } from '@layers/amba-web';
 
const reply = await Amba.ai.anthropic.messages.create({
  prompt_slug: 'support_assistant',
  variables: { user_query: 'How do I cancel?' },
  max_tokens: 1024,
});
 
console.log(reply.content);

Operations

ai.anthropic.messages.create({ prompt_slug, variables, max_tokens })

Sends a prompt to Anthropic via amba.

The response shape mirrors Anthropic's Message object, plus a cost_usd field amba adds:

{
  content: [...],        // array of content blocks (text / tool_use)
  usage: {
    input_tokens: 123,
    output_tokens: 456,
    cache_creation_input_tokens: 78,
    cache_read_input_tokens: 90,
  },
  stop_reason: 'end_turn',
  model: 'claude-sonnet-4-5',
  cost_usd: 0.00105,       // computed USD cost of this call
}

cost_usd is the dollar cost of the call (input + output + cache token rates), so you can show per-call spend or sum it per user without a separate metrics call. It's null for a model amba doesn't have a price for — the call still succeeds; the cost is just unattributed.

ai.openai.chat.completions.create({ prompt_slug, variables, max_tokens })

Same shape, but routes to OpenAI. Response mirrors OpenAI's ChatCompletion, plus the same cost_usd field.

Gemini-registered prompts

The vendor path on these SDK methods is just a stable wire surface — the server resolves the real provider from the prompt's registered configuration. So a prompt you registered against gemini is invoked through the same SDK call (e.g. ai.anthropic.messages.create({ prompt_slug })); amba builds Gemini's distinct request body for you (system instruction, multi-turn contents, and variables — multimodal image input included) and parses the response back to the common result shape, with the same cost_usd field. You never touch provider-specific shapes. (To pick the path that matches how you registered the prompt, see the client API reference.)

Per-prompt budgets

Cap a prompt's spend with a per-period budget. Once the period's spend reaches the budget, invocations are denied with an ai_budget_exceeded error (HTTP 429) until the period resets — so a prompt you expose to your app can't run away with spend. Budgets are off by default (unlimited).

Set one from the admin API or an agent (MCP amba_ai_prompts_set_budget):

curl -X PUT 'https://api.amba.dev/v1/admin/projects/$PROJECT_ID/ai/prompts/summarize/budget' \
  -H 'Authorization: Bearer $AMBA_PAT' \
  -H 'Content-Type: application/json' \
  -d '{ "budget_usd": 25, "budget_period": "monthly" }'

When a call is denied, the error carries the budget, the spend so far, and when the period resets, so you can decide whether to raise the budget or wait. This is the fine-grained sibling of the project-wide spend ceiling.

Patterns

Prompt slugs

Prompts live in the console — system prompt, variable schema, default model, default temperature, default max_tokens. The client passes the slug; the server fills the rest. This means you can:

• Change the model behind a slug without redeploying.
• A/B test prompt variants by routing a slug through a feature flag.
• Audit which user sent which prompt via the per-call usage event.

Slug naming: lowercase, underscore-separated, up to 64 characters. Group by feature: support_assistant, summarize_review, onboarding_recommend.

Variable substitution

The prompt template uses {{name}} placeholders. The client's variables object fills them:

// Prompt template (in console):
//   You are a helpful support assistant. The user asks: {{user_query}}
//   Reply in under 200 words.

await Amba.ai.anthropic.messages.create({
  prompt_slug: 'support_assistant',
  variables: { user_query: 'How do I cancel?' },
});

Variables that aren't in the template are silently ignored. Missing required variables return a 400 from the server with the offending key listed.

Structured JSON output

To get parseable JSON back instead of free-form text, set the provider's response_format on the prompt (or pass it through the admin invoke extra_body):

• { "type": "json_object" } — the model returns a syntactically valid JSON object.
• { "type": "json_schema", "json_schema": { "name": "…", "schema": { … } } } — the model returns JSON conforming to your schema (OpenAI / Mistral).

The response is still the provider's native shape — the JSON your model produced lives in the first content block's text, which you then JSON.parse:

const reply = await Amba.ai.openai.chat.completions.create({
  prompt_slug: 'extract_invoice', // registered with response_format: { type: 'json_object' }
  variables: { raw_email: email.body },
});
const data = JSON.parse(reply.choices[0].message.content);

Vision (image input)

Send images to a vision-capable prompt by passing images alongside your variables. Each image is either a remote url or base64 data (with an optional mime). Amba attaches them to your message and maps them to the model's native image format for you — you never touch provider-specific shapes:

const reply = await Amba.ai.anthropic.messages.create({
  prompt_slug: 'describe_photo', // registered against a vision model
  messages: [{ role: 'user', content: 'What is in this picture?' }],
  images: [
    { url: 'https://example.com/cat.png' },
    // or inline base64:
    // { data: base64Bytes, mime: 'image/jpeg' },
  ],
});
console.log(reply.content);

The images attach to the last user message (a string message is kept as text and the images sit beside it). You can also build the multimodal message by hand — pass content as an array of { type: 'text', text } and { type: 'image', url | data, mime } blocks instead of using images; both produce the same request.

The prompt's model must be vision-capable (e.g. claude-sonnet-4-5, gpt-4o, gemini-2.5-pro). Sending an image to a text-only model returns ai_model_not_multimodal (422) rather than silently dropping the image — switch the prompt to a vision model and retry. Image tokens are metered into cost_usd like any other input.

Streaming

The AI proxy supports a streamed (Server-Sent Events) response — pass stream: true to receive provider frames as they arrive, with usage still metered server-side. Today this is a wire-level capability: the buffered SDK methods (Amba.ai.anthropic.messages.create(...)) decode a single JSON envelope and reject stream: true. Until a streaming SDK method lands, consume the SSE stream over the client REST endpoint directly (or from a server function). The client API — ai reference documents the stream shape.

Cost attribution

Every ai.*.create call emits a ai_usage event automatically — same events namespace as everything else, but with usage.input_tokens and usage.output_tokens attached. Query the event stream per-user to attribute cost without instrumenting your own counter.

Managed media endpoints

Beyond text, the managed AI surface proxies image generation and editing, text-to-speech, and audio transcription with your provider key kept server-side and per-call usage metered.

These media endpoints are invoked from a deployed function, not from the client SDK. A function is deployed with the AI gateway already wired up — the runtime injects AMBA_AI_GATEWAY_URL and the credential — so your function calls the gateway directly and returns the result to your app:

Every call records a usage event (ai.image / ai.speech / ai.transcription) so spend is attributed per project without you instrumenting a counter.

When a buffered call can be priced, the response also includes an X-Amba-Cost-Usd header. The header is the common cost surface for JSON, binary audio, and schema-constrained responses without changing their bodies. It is omitted when the model is unknown or the required meter is unavailable. Image cost is variant-aware: DALL-E uses its exact model/quality/size price cell, while GPT Image uses the provider's returned text-input, image-input, and image-output token counts. If that GPT usage envelope is missing or incomplete, Amba records a null cost and omits the header rather than returning a flat per-image estimate. For OpenAI transcription, request response_format=verbose_json so the provider returns the duration needed for per-minute pricing; its transcript body is passed through verbatim. Gemini 3.5 Transcribe instead uses the exact audio-input and text-output token counts in Google's Interaction usage, so every successful Gemini response has a numeric X-Amba-Cost-Usd header. Amba rejects a successful provider response whose usage is incomplete rather than recording a guessed or null cost. audio_seconds analytics use the final WordInfo offset for verbose responses, or Google's documented 25-audio-tokens/second estimate otherwise; audio_seconds_source distinguishes the two.

Gemini 3.5 Transcribe

Register a Gemini key first:

amba ai providers add gemini --from-stdin

Then send model=gemini-3.5-transcribe. Supported audio types are WAV, MP3, AIFF, AAC, OGG, FLAC, MPEG, M4A (including the common audio/mp4 MIME type), L16, Opus, A-law, and mu-law. Amba waits for the resumable Files upload to reach Google's ACTIVE state before invoking the model, fails under a bounded processing deadline, and deletes the temporary file on every terminal path. The default language is automatic detection; omit language or set it to auto. When the language is known, use one of Google's documented BCP-47 hints. Tags are case-insensitive. For compatibility with OpenAI-style transcription clients, Amba also accepts language=en and sends en-US; other bare language subtags are accepted when Google's table has exactly one matching locale. Ambiguous bare hints such as es, pt, bn, and pa require an explicit documented locale or script instead of silently choosing one.

custom_vocabulary is a JSON array of up to 1,000 terms (or use repeated custom_vocabulary[] fields). mode is verbatim by default and may be smart; smart mode cannot be combined with diarization or timestamps. Google's live API also rejects custom vocabulary with timestamp annotations, so Amba accepts custom vocabulary on json or text requests and rejects its combination with verbose_json or timestamp_granularities[] before upload.

response_format=verbose_json selects verbatim mode with speaker diarization and word timestamps. diarize=true is the explicit diarization flag (diarization is also accepted as an alias). Existing OpenAI-compatible clients may send both timestamp_granularities[]=segment and timestamp_granularities[]=word; Amba synthesizes segments by grouping adjacent Gemini WordInfo annotations with the same speaker. Its normalized response is:

{
  "text": "Welcome to PodPod.",
  "duration": 1.42,
  "language": null,
  "cost_usd": 0.00512,
  "segments": [{ "start": 0.1, "end": 1.42, "text": "Welcome to PodPod.", "speaker": "Speaker 1" }],
  "words": [{ "word": "Welcome", "start": 0.1, "end": 0.48, "speaker": "Speaker 1" }]
}

Speaker labels are scoped to one request. If you split a long recording into chunks, stitch speaker identities in your application; Amba does not imply that Speaker 1 in two chunks is the same person. Gemini unary transcription is limited by Google to one hour, and diarization/timestamps to 30 minutes. This surface does not implement the live WebSocket transcription model.

Amba returns words and synthesized segments in chronological order, preserving Google's source order when multiple annotations have the same start time. It preserves valid overlaps and provider timestamps rather than clamping them to the approximate 25-audio-tokens/second duration estimate. The gateway rejects ambiguous offset overruns and omits only unmistakably catastrophic provider outliers; it never presents the token estimate as a decoded-media duration.

Calling from a function

The runtime injects AMBA_AI_GATEWAY_URL and the gateway credential into every deployed function. Authenticate with Authorization: Bearer ${env.AMBA_INTERNAL_TOKEN} and send a request-id header (X-Amba-Request-Id, required for tracing; pass through X-Amba-User-Id to attribute usage to an end user):

export default {
  async fetch(req: Request, env: Env): Promise<Response> {
    const audio = await req.arrayBuffer();
    const form = new FormData();
    form.set('file', new Blob([audio], { type: 'audio/mpeg' }), 'recording.mp3');
    form.set('model', 'gemini-3.5-transcribe');
    // Gemini verbose JSON adds normalized speaker segments + word timestamps.
    form.set('response_format', 'verbose_json');
    form.set('language', 'auto');
 
    const res = await fetch(`${env.AMBA_AI_GATEWAY_URL}/audio/transcriptions`, {
      method: 'POST',
      headers: {
        Authorization: `Bearer ${env.AMBA_INTERNAL_TOKEN}`,
        'X-Amba-Request-Id': crypto.randomUUID(),
      },
      body: form, // multipart boundary set automatically
    });
    return new Response(await res.text(), {
      status: res.status,
      headers: {
        'content-type': res.headers.get('content-type') ?? 'application/json',
        // Forward cost visibility without changing the transcript schema.
        ...(res.headers.get('x-amba-cost-usd')
          ? { 'x-amba-cost-usd': res.headers.get('x-amba-cost-usd')! }
          : {}),
      },
    });
  },
};

Image generation and text-to-speech use a JSON body on the same gateway base URL:

// Image generation — returns JSON with one entry per generated image.
const img = await fetch(`${env.AMBA_AI_GATEWAY_URL}/images/generations`, {
  method: 'POST',
  headers: {
    Authorization: `Bearer ${env.AMBA_INTERNAL_TOKEN}`,
    'X-Amba-Request-Id': crypto.randomUUID(),
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({ model: 'gpt-image-2', prompt: 'a calico cat astronaut', n: 1 }),
});
 
// Image editing — the gateway re-encodes multipart uploads once, without
// retaining a second raw image buffer in the Worker isolate.
const editForm = new FormData();
editForm.set('model', 'gpt-image-2');
editForm.set('prompt', 'add a broadcast microphone');
editForm.set('image', new Blob([sourcePng], { type: 'image/png' }), 'cover.png');
const edit = await fetch(`${env.AMBA_AI_GATEWAY_URL}/images/edits`, {
  method: 'POST',
  headers: {
    Authorization: `Bearer ${env.AMBA_INTERNAL_TOKEN}`,
    'X-Amba-Request-Id': crypto.randomUUID(),
  },
  body: editForm,
});
 
// Text-to-speech — returns binary audio; forward the bytes to your app.
const speech = await fetch(`${env.AMBA_AI_GATEWAY_URL}/audio/speech`, {
  method: 'POST',
  headers: {
    Authorization: `Bearer ${env.AMBA_INTERNAL_TOKEN}`,
    'X-Amba-Request-Id': crypto.randomUUID(),
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({ model: 'tts-1', input: 'Hello from Amba', voice: 'alloy' }),
});

Register a provider key for the model you reference (see AI providers); a missing key returns 424 and an unconfigured gateway returns 503. Transcription bills per audio minute — duration is only known when you request a verbose response, so without it the transcript is still returned but the call records a null cost.

Limits

• Prompt slug must exist: the server returns 404 prompt_slug_not_found for unknown slugs. Define the prompt in the console before referencing it.
• max_tokens ceiling: per-project hard cap (default 4096); the prompt's console default applies if you don't pass one.
• Rate limits: per-prompt-slug rate limits configured in the console. Defaults are conservative; raise per-slug as your usage grows.
• No client-side keys: client SDKs cannot pass an api_key. The server's provider key is the only credential in play.
• No tool-use roundtrip from clients: tool calls (Anthropic) and function-calling (OpenAI) are accepted in the response but the client SDK doesn't auto-execute tools. Run tool dispatch in a server function and only return the final text to the client.

Reference

• Client API — ai — endpoint reference.
• CLI: amba ai prompts — manage prompt slugs.
• Auth feature — prerequisite.
• Per-platform quickstarts: Web, Node, iOS, Android, Flutter, Unity.