Test
11 endpoints
POST /api/v5/artifacts/test/get
Get Test Artifact
Get benchmark test artifact details with tests/invocations in parallel.
Parameters:
| Name | In | Required | Description |
|---|---|---|---|
X-Api-Key | header | No | — |
authorization | header | No | — |
X-MCP | header | No | — |
Request body:
{
test_id: `string`
}Response:
{
test?: `GetTestResponse` | `null`
invocations?: `GetTestInvocationResponse`[]
status?: `string`
eval_name?: `string` | `null`
eval_description?: `string` | `null`
rubric_name?: `string` | `null`
infinite_mode?: `boolean`
runs?: `TestRunItem`[]
status_summary?: `TestStatusSummary` | `null`
show_controls?: `boolean`
current_invocation_id?: `string` | `null`
has_runs_or_groups?: `boolean`
entries?: `TestEntries` | `null`
resources?: `TestResources` | `null`
}POST /api/v5/artifacts/test/archive
Archive Test Artifacts
Archive or unarchive benchmark tests by IDs.
Parameters:
| Name | In | Required | Description |
|---|---|---|---|
X-Api-Key | header | No | — |
authorization | header | No | — |
X-MCP | header | No | — |
Request body:
{
test_ids: `string`[]
archived?: `boolean`
}Response:
{
updated_count?: `integer`
}POST /api/v5/artifacts/test/refresh
Test Refresh
Refresh test materialized views and invalidate caches.
Parameters:
| Name | In | Required | Description |
|---|---|---|---|
X-Api-Key | header | No | — |
authorization | header | No | — |
X-MCP | header | No | — |
Response:
{
success: `boolean`
refreshed_views: `string`[]
invalidated_tags: `string`[]
}POST /api/v5/artifacts/test/export
Export Test
Export test data as a clean, denormalized ZIP.
Parameters:
| Name | In | Required | Description |
|---|---|---|---|
X-Api-Key | header | No | — |
authorization | header | No | — |
X-MCP | header | No | — |
Request body:
{
test_id: `string`
}Response:
{
upload_id: `string`
file_name: `string`
row_count: `integer`
}POST /api/v5/artifacts/test/docs
Get Test Docs Endpoint
Get composed documentation for the test analytics.
Parameters:
| Name | In | Required | Description |
|---|---|---|---|
X-Api-Key | header | No | — |
authorization | header | No | — |
X-MCP | header | No | — |
Request body:
{
entity_id?: `string` | `null`
}Response:
{
name: `string`
type: `string`
description: `string`
artifact?: `DocsResponse` | `null`
entries: `DocsResponse`[]
resources: `DocsResponse`[]
permissions: `OperationInfo`[]
api_operations: `OperationInfo`[]
page_metadata?: `DocsApiResponse` | `null`
}POST /api/v5/artifacts/test/start
Start Test
Create a new test.
Parameters:
| Name | In | Required | Description |
|---|---|---|---|
X-Api-Key | header | No | — |
authorization | header | No | — |
X-MCP | header | No | — |
Request body:
{
benchmark_id: `string`
infinite_mode?: `boolean`
}Response:
{
test_id: `string`
}POST /api/v5/artifacts/test/next
Next Test
Find next pending run in an existing test.
Parameters:
| Name | In | Required | Description |
|---|---|---|---|
X-Api-Key | header | No | — |
authorization | header | No | — |
X-MCP | header | No | — |
Request body:
{
test_id: `string`
}Response:
{
invocation_id: `string`
run_id: `string`
current_run: `integer`
total_runs: `integer`
}POST /api/v5/artifacts/test/run
Run Test
Run one auto-regressive replay. Returns immediately; progress via socket.
Parameters:
| Name | In | Required | Description |
|---|---|---|---|
X-Api-Key | header | No | — |
authorization | header | No | — |
X-MCP | header | No | — |
Request body:
{
test_id: `string`
test_invocation_id: `string`
run_id: `string`
}Response:
{
test_id: `string`
invocation_id: `string`
run_id: `string`
}POST /api/v5/artifacts/test/end
End Test
End a single invocation within a test.
Browser client: sends grade=True, internal AI generates grade + feedback. Agent: can optionally provide score, passed, feedback to skip AI.
Parameters:
| Name | In | Required | Description |
|---|---|---|---|
X-Api-Key | header | No | — |
authorization | header | No | — |
X-MCP | header | No | — |
Request body:
{
test_id: `string`
test_invocation_id: `string`
run_id: `string`
grade?: `boolean`
score?: `number` | `null`
passed?: `boolean` | `null`
feedback?: `string` | `null`
}Response:
{
invocation_id: `string`
grade_id?: `string` | `null`
score?: `number` | `null`
passed?: `boolean` | `null`
feedback?: `string` | `null`
}POST /api/v5/artifacts/test/stop
Stop Test
Stop current test execution.
Parameters:
| Name | In | Required | Description |
|---|---|---|---|
X-Api-Key | header | No | — |
authorization | header | No | — |
X-MCP | header | No | — |
Request body:
{
invocation_id: `string`
}Response:
{
invocation_id: `string`
success: `boolean`
message?: `string` | `null`
}POST /api/v5/artifacts/test/search
Search Test
Search tests — composable infra architecture.
Parameters:
| Name | In | Required | Description |
|---|---|---|---|
X-Api-Key | header | No | — |
authorization | header | No | — |
X-MCP | header | No | — |
Request body:
{
eval_ids?: `string`[] | `null`
department_ids?: `string`[] | `null`
is_archived?: `boolean` | `null`
start_date?: `string` | `null`
end_date?: `string` | `null`
eval_search?: `string` | `null`
department_search?: `string` | `null`
page_size?: `integer`
page_offset?: `integer`
}Response:
{
actor_name?: `string` | `null`
tests?: `SearchTestItem`[] | `null`
eval_filter?: `ListFilterSection` | `null`
department_filter?: `ListFilterSection` | `null`
total_count?: `integer` | `null`
}