Experiments

Amba runs A/B experiments end to end: define weighted variants once, assign each user a sticky variant, log exposures when a user actually sees the variant, and read back per-variant conversion rates with a built-in two-proportion significance test. No second experimentation vendor, no client-side bucketing to keep in sync across platforms.

How it works

1. Define an experiment with two or more weighted variants. The first variant is the control — every other variant is compared against it in the results.
2. Assign — the first time a user asks for their variant, Amba buckets them with a deterministic hash of user_id + experiment_key and persists the assignment. Every later call returns the same variant, even if you re-weight the experiment afterwards. Re-weighting only changes bucketing for users assigned after the edit.
3. Expose — record an exposure when the user is actually shown the variant (vs. merely assigned). Exposures are the denominator for conversion: they're the population a conversion can be causally attributed to.
4. Measure — read results filtered by a conversion event. Amba counts distinct exposed users who emitted that event at or after their assignment and runs a two-proportion z-test of each variant against the control.

Assigning + exposing (client)

Assignment and exposure are two client calls made with the end-user's session — the same X-Api-Key + session-token pair you already use for auth and collections. Assignment is sticky and idempotent; exposure is fire-and-forget once the variant is on screen.

// Sticky assignment — returns the same variant for this user every time.
const res = await fetch(`${BASE_URL}/client/experiments/checkout_button_v2/assignment`, {
  headers: {
    'X-Api-Key': CLIENT_API_KEY,
    Authorization: `Bearer ${sessionToken}`,
  },
});
const { data } = await res.json();
 
if (data.variant === 'green') {
  renderGreenButton();
} else {
  renderControlButton();
}
 
// Record an exposure the moment the user actually sees the variant.
await fetch(`${BASE_URL}/client/experiments/checkout_button_v2/exposure`, {
  method: 'POST',
  headers: {
    'X-Api-Key': CLIENT_API_KEY,
    Authorization: `Bearer ${sessionToken}`,
    'Content-Type': 'application/json',
  },
  // `variant` is optional — it defaults to the user's current assignment.
  body: JSON.stringify({}),
});

assignment returns 409 EXPERIMENT_INACTIVE for a brand-new user when the experiment isn't active (a paused or ended experiment still serves variants to users who were already bucketed — in-flight users keep their variant). exposure returns 404 ASSIGNMENT_REQUIRED if the user has no assignment yet — always call assignment first. Full status codes are in the client API reference.

Defining + measuring (admin / agent)

Create, re-weight, pause, and read results from the admin API or over MCP. Your coding agent can do all of this for you — just describe the experiment.

# Create a 50/50 experiment. The FIRST variant is the control.
curl -X POST 'https://api.amba.dev/v1/admin/projects/$PROJECT_ID/experiments' \
  -H 'Authorization: Bearer $AMBA_PAT' \
  -H 'Content-Type: application/json' \
  -d '{
    "key": "checkout_button_v2",
    "name": "Checkout button colour",
    "variants": [
      { "key": "control", "weight": 50 },
      { "key": "green",   "weight": 50 }
    ]
  }'
 
# Read results, attributing conversions to a tracked event.
curl 'https://api.amba.dev/v1/admin/projects/$PROJECT_ID/experiments/checkout_button_v2/results?conversion_event=purchase_completed' \
  -H 'Authorization: Bearer $AMBA_PAT'

Results come back per variant with assigned_count, exposed_count, converted_count, conversion_rate, and — for every non-control variant — a z_score and two-sided p_value versus the control:

{
  "data": {
    "experiment_key": "checkout_button_v2",
    "status": "active",
    "conversion_event": "purchase_completed",
    "control_variant": "control",
    "variants": [
      {
        "variant": "control",
        "is_control": true,
        "exposed_count": 980,
        "converted_count": 142,
        "conversion_rate": 0.1449,
        "z_score": null,
        "p_value": null
      },
      {
        "variant": "green",
        "is_control": false,
        "exposed_count": 1010,
        "converted_count": 196,
        "conversion_rate": 0.1941,
        "z_score": -3.07,
        "p_value": 0.0021
      }
    ]
  }
}

A p_value of 0.0021 means the lift is statistically significant at the conventional 0.05 threshold. z_score / p_value are null until both samples have data (the test is undefined for an empty or degenerate sample).

Reference

• Client API — assignment + exposure
• Admin API — create, update, results
• MCP tools — amba_experiments_create, amba_experiments_list, amba_experiments_get, amba_experiments_update, amba_experiments_delete, amba_experiments_results