Quickstart

Go from zero to a real, scored evaluation run in under five minutes — no sign-up, no API keys, no setup.

1. Open the preview

Go to preview.devx-ai.xyz — it opens straight into the Dashboard. Everything is pre-seeded, so there's already data to look at.

2. Trigger a real evaluation run

1. Click Evaluations in the sidebar.
2. Click New run at the top of the page.
3. Pick a dataset (e.g. "Customer Support QA") and a model.
4. Optionally paste a prompt template using {{question}} and {{context}} to see real Mustache rendering — leave it blank to use the raw test case input.
5. Click Run evaluation.

You'll land on the new run's detail page with real, computed metric scores — not canned numbers. See Metrics for how they're calculated.

3. Or do it from the command line

Prefer curl? The same action, one request:

curl -X POST https://preview.devx-ai.xyz/api/evaluations/run \
  -H "Content-Type: application/json" \
  -d '{
    "datasetId": "ds_001",
    "model": "gpt-4o",
    "promptTemplate": "Context: {{context}}\nQuestion: {{question}}"
  }'

The response includes the full run with per-test-case scores. Grab the id and open https://preview.devx-ai.xyz/evaluations/<id> to see it in the UI.

4. Compare it against a previous run

Open Compare, set your new run as the candidate (B) and any existing run on the same dataset as the baseline (A). You'll get a per-test-case, per-metric diff — this is the core loop the whole app is built around.

5. Keep exploring

1. Datasets — add your own test case and re-run the evaluation to see it scored.
2. Prompts — version a prompt, then open a version directly in the Playground.
3. Traces — open any row for a span-level timeline of a simulated LLM call.
4. Alerts — toggle a rule on/off; it calls the real API immediately.