AI proxy

Amba.ai.* proxies LLM requests through amba so your provider keys never ship to a client. You write the prompt once in the console, reference it by prompt_slug from any SDK, and the server fills in the system prompt + variables + model + rate-limits + usage tracking.

Providers wired today: Anthropic (messages.create), OpenAI (chat.completions.create), Mistral (mistral.chat.completions.create, OpenAI-compatible), and Google Gemini (gemini.generateContent). All return the upstream response shape verbatim, plus a usage event so you can attribute cost per user. Register a prompt against any of them by setting its provider — including Gemini, whose distinct request shape (system instruction + multi-turn content + variables) is built for you — bring your own key per provider (the key stays server-side).

Beyond text, the managed AI surface also proxies image generation, text-to-speech, and audio transcription with your key kept server-side and per-call usage metered (priced per image / per character / per audio minute rather than per token).

Quick start

import { Amba } from '@layers/amba-web';
 
const reply = await Amba.ai.anthropic.messages.create({
  prompt_slug: 'support_assistant',
  variables: { user_query: 'How do I cancel?' },
  max_tokens: 1024,
});
 
console.log(reply.content);

Operations

ai.anthropic.messages.create({ prompt_slug, variables, max_tokens })

Sends a prompt to Anthropic via amba.

The response shape mirrors Anthropic's Message object, plus a cost_usd field amba adds:

{
  content: [...],        // array of content blocks (text / tool_use)
  usage: {
    input_tokens: 123,
    output_tokens: 456,
    cache_creation_input_tokens: 78,
    cache_read_input_tokens: 90,
  },
  stop_reason: 'end_turn',
  model: 'claude-sonnet-4-5',
  cost_usd: 0.00105,       // computed USD cost of this call
}

cost_usd is the dollar cost of the call (input + output + cache token rates), so you can show per-call spend or sum it per user without a separate metrics call. It's null for a model amba doesn't have a price for — the call still succeeds; the cost is just unattributed.

ai.openai.chat.completions.create({ prompt_slug, variables, max_tokens })

Same shape, but routes to OpenAI. Response mirrors OpenAI's ChatCompletion, plus the same cost_usd field.

Gemini-registered prompts

The vendor path on these SDK methods is just a stable wire surface — the server resolves the real provider from the prompt's registered configuration. So a prompt you registered against gemini is invoked through the same SDK call (e.g. ai.anthropic.messages.create({ prompt_slug })); amba builds Gemini's distinct request body for you (system instruction, multi-turn contents, and variables — multimodal image input included) and parses the response back to the common result shape, with the same cost_usd field. You never touch provider-specific shapes. (To pick the path that matches how you registered the prompt, see the client API reference.)

Per-prompt budgets

Cap a prompt's spend with a per-period budget. Once the period's spend reaches the budget, invocations are denied with an ai_budget_exceeded error (HTTP 429) until the period resets — so a prompt you expose to your app can't run away with spend. Budgets are off by default (unlimited).

Set one from the admin API or an agent (MCP amba_ai_prompts_set_budget):

curl -X PUT 'https://api.amba.dev/v1/admin/projects/$PROJECT_ID/ai/prompts/summarize/budget' \
  -H 'Authorization: Bearer $AMBA_PAT' \
  -H 'Content-Type: application/json' \
  -d '{ "budget_usd": 25, "budget_period": "monthly" }'

When a call is denied, the error carries the budget, the spend so far, and when the period resets, so you can decide whether to raise the budget or wait. This is the fine-grained sibling of the project-wide spend ceiling.

Patterns

Prompt slugs

Prompts live in the console — system prompt, variable schema, default model, default temperature, default max_tokens. The client passes the slug; the server fills the rest. This means you can:

• Change the model behind a slug without redeploying.
• A/B test prompt variants by routing a slug through a feature flag.
• Audit which user sent which prompt via the per-call usage event.

Slug naming: lowercase, underscore-separated, up to 64 characters. Group by feature: support_assistant, summarize_review, onboarding_recommend.

Variable substitution

The prompt template uses {{name}} placeholders. The client's variables object fills them:

// Prompt template (in console):
//   You are a helpful support assistant. The user asks: {{user_query}}
//   Reply in under 200 words.

await Amba.ai.anthropic.messages.create({
  prompt_slug: 'support_assistant',
  variables: { user_query: 'How do I cancel?' },
});

Variables that aren't in the template are silently ignored. Missing required variables return a 400 from the server with the offending key listed.

Structured JSON output

To get parseable JSON back instead of free-form text, set the provider's response_format on the prompt (or pass it through the admin invoke extra_body):

• { "type": "json_object" } — the model returns a syntactically valid JSON object.
• { "type": "json_schema", "json_schema": { "name": "…", "schema": { … } } } — the model returns JSON conforming to your schema (OpenAI / Mistral).

The response is still the provider's native shape — the JSON your model produced lives in the first content block's text, which you then JSON.parse:

const reply = await Amba.ai.openai.chat.completions.create({
  prompt_slug: 'extract_invoice', // registered with response_format: { type: 'json_object' }
  variables: { raw_email: email.body },
});
const data = JSON.parse(reply.choices[0].message.content);

Vision (image input)

Send images to a vision-capable prompt by passing images alongside your variables. Each image is either a remote url or base64 data (with an optional mime). Amba attaches them to your message and maps them to the model's native image format for you — you never touch provider-specific shapes:

const reply = await Amba.ai.anthropic.messages.create({
  prompt_slug: 'describe_photo', // registered against a vision model
  messages: [{ role: 'user', content: 'What is in this picture?' }],
  images: [
    { url: 'https://example.com/cat.png' },
    // or inline base64:
    // { data: base64Bytes, mime: 'image/jpeg' },
  ],
});
console.log(reply.content);

The images attach to the last user message (a string message is kept as text and the images sit beside it). You can also build the multimodal message by hand — pass content as an array of { type: 'text', text } and { type: 'image', url | data, mime } blocks instead of using images; both produce the same request.

The prompt's model must be vision-capable (e.g. claude-sonnet-4-5, gpt-4o, gemini-2.5-pro). Sending an image to a text-only model returns ai_model_not_multimodal (422) rather than silently dropping the image — switch the prompt to a vision model and retry. Image tokens are metered into cost_usd like any other input.

Streaming

The AI proxy supports a streamed (Server-Sent Events) response — pass stream: true to receive provider frames as they arrive, with usage still metered server-side. Today this is a wire-level capability: the buffered SDK methods (Amba.ai.anthropic.messages.create(...)) decode a single JSON envelope and reject stream: true. Until a streaming SDK method lands, consume the SSE stream over the client REST endpoint directly (or from a server function). The client API — ai reference documents the stream shape.

Cost attribution

Every ai.*.create call emits a ai_usage event automatically — same events namespace as everything else, but with usage.input_tokens and usage.output_tokens attached. Query the event stream per-user to attribute cost without instrumenting your own counter.

Managed media endpoints

Beyond text, the managed AI surface proxies image generation, text-to-speech, and audio transcription with your provider key kept server-side and per-call usage metered (priced per image / per character / per audio minute rather than per token).

These media endpoints are invoked from a deployed function, not from the client SDK. A function is deployed with the AI gateway already wired up — the runtime injects AMBA_AI_GATEWAY_URL and the credential — so your function calls the gateway directly and returns the result to your app:

Every call records a usage event (ai.image / ai.speech / ai.transcription) so spend is attributed per project without you instrumenting a counter.

Calling from a function

The runtime injects AMBA_AI_GATEWAY_URL and the gateway credential into every deployed function. Authenticate with Authorization: Bearer ${env.AMBA_INTERNAL_TOKEN} and send a request-id header (X-Amba-Request-Id, required for tracing; pass through X-Amba-User-Id to attribute usage to an end user):

export default {
  async fetch(req: Request, env: Env): Promise<Response> {
    const audio = await req.arrayBuffer();
    const form = new FormData();
    form.set('file', new Blob([audio], { type: 'audio/mpeg' }), 'recording.mp3');
    form.set('model', 'whisper-1'); // a transcription model your provider key supports
    // Ask for a verbose response if you want the audio duration back (used for
    // per-minute cost metering).
    form.set('response_format', 'verbose_json');
 
    const res = await fetch(`${env.AMBA_AI_GATEWAY_URL}/audio/transcriptions`, {
      method: 'POST',
      headers: {
        Authorization: `Bearer ${env.AMBA_INTERNAL_TOKEN}`,
        'X-Amba-Request-Id': crypto.randomUUID(),
      },
      body: form, // multipart boundary set automatically
    });
    return new Response(await res.text(), {
      status: res.status,
      headers: { 'content-type': res.headers.get('content-type') ?? 'application/json' },
    });
  },
};

Image generation and text-to-speech use a JSON body on the same gateway base URL:

// Image generation — returns JSON with one entry per generated image.
const img = await fetch(`${env.AMBA_AI_GATEWAY_URL}/images/generations`, {
  method: 'POST',
  headers: {
    Authorization: `Bearer ${env.AMBA_INTERNAL_TOKEN}`,
    'X-Amba-Request-Id': crypto.randomUUID(),
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({ model: 'gpt-image-1', prompt: 'a calico cat astronaut', n: 1 }),
});
 
// Text-to-speech — returns binary audio; forward the bytes to your app.
const speech = await fetch(`${env.AMBA_AI_GATEWAY_URL}/audio/speech`, {
  method: 'POST',
  headers: {
    Authorization: `Bearer ${env.AMBA_INTERNAL_TOKEN}`,
    'X-Amba-Request-Id': crypto.randomUUID(),
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({ model: 'tts-1', input: 'Hello from Amba', voice: 'alloy' }),
});

Register a provider key for the model you reference (see AI providers); a missing key returns 424 and an unconfigured gateway returns 503. Transcription bills per audio minute — duration is only known when you request a verbose response, so without it the transcript is still returned but the call records a null cost.

Limits

• Prompt slug must exist: the server returns 404 prompt_slug_not_found for unknown slugs. Define the prompt in the console before referencing it.
• max_tokens ceiling: per-project hard cap (default 4096); the prompt's console default applies if you don't pass one.
• Rate limits: per-prompt-slug rate limits configured in the console. Defaults are conservative; raise per-slug as your usage grows.
• No client-side keys: client SDKs cannot pass an api_key. The server's provider key is the only credential in play.
• No tool-use roundtrip from clients: tool calls (Anthropic) and function-calling (OpenAI) are accepted in the response but the client SDK doesn't auto-execute tools. Run tool dispatch in a server function and only return the final text to the client.

Reference

• Client API — ai — endpoint reference.
• CLI: amba ai prompts — manage prompt slugs.
• Auth feature — prerequisite.
• Per-platform quickstarts: Web, Node, iOS, Android, Flutter, Unity.