Experiments

The Experiments surface records manual AI configuration variants and their assignment outcomes. A reported outcome does not change future AI behavior.

Open Experiments

In the dashboard, go to Insights → Analytics → Experiments, or open /analytics/experiments directly.

The list reports the complete tenant-wide, all-time experiment collection. It also shows the latest experiment update time and the number of incompatible legacy definitions. This view is not scoped to the selected workspace.

Retained experiment definitions

Existing definitions can contain:

• a required name and optional description
• at least two named variants
• a name and assignment weight for each variant
• weights that total exactly 100%
• an optional model override
• an optional temperature override

The retained definition contract requires stable, unique variant IDs; the historical creation validator rejected duplicates rather than repairing them.

At runtime, a model override is used only when its model name matches the current provider candidate: claude- for Anthropic, or gpt-/o-series for OpenAI. A temperature override is passed only to Anthropic. Prompt, brand-voice, response-style, and other unrecognized variant fields do not alter the ticket agent loop.

The retained contract does not admit response-style, response-length, metric, duration, hypothesis, prompt, or recurrence configuration.

Assignment behavior

Candidate assignment is deterministic: source hashes SHA-256(ticketId:experimentId) and maps the result across the configured weights. The first durable assignment for that tenant, experiment, and ticket is authoritative on retry; a later conflicting candidate returns the already stored variant.

Source currently selects one active tenant experiment by ordering updatedAt descending and then id descending before limit(1). This makes the selected active experiment deterministic when multiple active experiments exist. Do not interpret current assignment behavior as multi-experiment orchestration.

Manual mutation boundary

Existing records can retain draft, active, paused, or completed status. The historical forward-transition model was:

• draft → active
• active → paused or completed
• paused → active or completed

Experiment reads require analytics.view. The current POST, PATCH, and DELETE handlers preserve unauthenticated 401, then return private, non-cacheable 503 with code EXPERIMENT_MUTATION_SUSPENDED before params, body validation, tenant, database, provider, or audit work. Manual creation, update, status changes, and deletion are suspended because no approved experiment-mutation capability or workspace policy exists. Existing records and results remain available through read-only manual dashboard and API surfaces; the dashboard exposes no creation or lifecycle controls. Admitted runtime assignment and pending outcome processing are unchanged by this manual mutation suspension.

Results and coverage

Open an experiment from the list to view its aggregate assignment outcomes. The results route reports tenant-wide, all-time coverage with an assignment count, latest assignment timestamp, and experiment update timestamp.

The current result categories are:

• auto-sent
• approved
• rejected
• escalated
• pending

These are aggregate outcome counts. The detail page labels (autoSent + approved) / total as Resolution %. That display is not a measure of ticket resolution and is not a significance calculation. The product does not derive a confidence score or choose a preferred variant from it.

Incompatible legacy definitions

Some historical rows do not match the current variant-array contract. Keva preserves those records but returns an empty variant list with invalid_legacy_definition metadata rather than inventing a conversion.

For those rows, the list labels the definition unavailable and the detail page suppresses variant results until an approved migration exists.

Autonomous orchestration boundary

A nightly experiment-orchestrator job is defined, but the handler has one stable fail-closed result: skipped with reason tenant_enrollment_and_mutation_custody_not_implemented. It performs no tenant scan, provider work, or experiment mutation.

Deployment, provider synchronization, scheduled invocation, configured database behavior, and product outcomes are not available as a customer-facing automated flow. Treat the manual records and aggregate views above as the current documented surface.