Experiments

Admin surface for A/B experiments: CRUD plus a results endpoint that joins exposures against your analytics events and runs a two-proportion z-test of each variant against the control. The client-side assignment + exposure calls feed these results. See the Experiments guide for the flow.

All endpoints are project-scoped and require a developer PAT (Authorization: Bearer).

Endpoints

Experiments are addressed by key (the stable, customer-facing identifier), not by their internal id.

POST /admin/projects/:projectId/experiments

Request

Weights are relative — [{ "key": "a", "weight": 1 }, { "key": "b", "weight": 3 }] is a 25/75 split. They don't have to sum to 100.

Response 201

The created experiment row (including id, variants, status, timestamps).

Errors

• 400 INVALID_KEY / INVALID_NAME — missing or empty.
• 400 INVALID_VARIANTS — fewer than 2 variants, a non-object entry, an empty/duplicate key, or a non-positive / out-of-range weight. error.message says which.
• 400 INVALID_STATUS — status not one of active|paused|ended.
• 409 KEY_TAKEN — an experiment with that key already exists.
• 500 CREATE_FAILED.

PATCH /admin/projects/:projectId/experiments/:key

Update any of name, description, variants, status. Re-weighting via variants never re-rolls already-assigned users — assignments are persisted; the new weights only affect users bucketed after the edit. Set status to "paused" to stop new assignments while still serving existing ones, or "ended" to wind the experiment down.

Errors

• 400 INVALID_NAME / INVALID_VARIANTS / INVALID_STATUS — same rules as create.
• 404 NOT_FOUND — no experiment with that key.
• 500 UPDATE_FAILED.

GET /admin/projects/:projectId/experiments/:key/results

Per-variant analytics with a significance test against the control.

Query

For each variant the response carries:

• assigned_count — distinct users assigned to the variant.
• exposed_count — distinct users with ≥1 exposure for the variant.
• converted_count — distinct exposed users who emitted conversion_event at or after their assignment time.
• conversion_rate — converted_count / exposed_count.
• is_control — true for the first variant.
• z_score / p_value — two-proportion z-test of this variant vs. the control (two-sided). null for the control itself, and null when either sample is empty or the pooled variance is degenerate.

Response 200

{
  "data": {
    "experiment_key": "checkout_button_v2",
    "status": "active",
    "conversion_event": "purchase_completed",
    "control_variant": "control",
    "variants": [
      {
        "variant": "control",
        "is_control": true,
        "weight": 50,
        "assigned_count": 1004,
        "exposed_count": 980,
        "converted_count": 142,
        "conversion_rate": 0.1449,
        "z_score": null,
        "p_value": null
      },
      {
        "variant": "green",
        "is_control": false,
        "weight": 50,
        "assigned_count": 1031,
        "exposed_count": 1010,
        "converted_count": 196,
        "conversion_rate": 0.1941,
        "z_score": -3.07,
        "p_value": 0.0021
      }
    ]
  }
}

Errors

• 400 INVALID_CONVERSION_EVENT — conversion_event present but empty.
• 404 NOT_FOUND — no experiment with that key.
• 500 RESULTS_FAILED.

MCP tools