Datasets

Golden test case collections used to drive evaluations.

List all datasets with a computed test case count.

{
  "datasets": [
    {
      "id": "ds_001",
      "name": "Customer Support QA",
      "description": "Golden set of common customer support questions and expected answers.",
      "testCaseIds": ["case_001", "case_002", "..."],
      "createdAt": "2026-06-02T10:00:00.000Z",
      "updatedAt": "2026-07-14T22:05:00.000Z",
      "testCaseCount": 8
    }
  ]
}

Get one dataset with its full test cases resolved (not just ids).

{
  "dataset": { "id": "ds_001", "name": "Customer Support QA", "...": "..." },
  "testCases": [
    {
      "id": "case_001",
      "input": "What is your refund policy?",
      "expectedOutput": "Refunds are available within 30 days of purchase with a valid receipt.",
      "context": ["Our refund policy allows returns within 30 days of purchase.", "..."],
      "tags": ["billing", "policy"],
      "createdAt": "2026-06-08T22:22:00.000Z"
    }
  ]
}

404 if datasetId doesn't match any dataset.

Add a test case to a dataset.

curl -X POST https://preview.devx-ai.xyz/api/datasets/ds_001/cases \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Do you offer a student discount?",
    "expectedOutput": "Yes, students get 20% off with a valid .edu email address.",
    "tags": ["billing"]
  }'

Returns 201 with { testCase }. 400 if input or expectedOutput is missing. 404 if the dataset doesn't exist.

Remove a test case from a dataset (the case's id is unlinked from testCaseIds; it isn't scrubbed from historical evaluation results that already reference it).

curl -X DELETE https://preview.devx-ai.xyz/api/datasets/ds_001/cases/case_009

Returns 200 with { ok: true }.

Data model