Support

Contact us

The fastest way to reach us is email. We read everything and aim to reply within one business day.

Frequently asked questions

How do I send eval results to the hub?

Create an ingest key in Settings, then have your eval job POST its run payload to /api/ingest/runs with that key as a bearer token. The payload is validated against the run record contract and the raw body is stored before anything is derived from it, so a result can always be traced back to what was sent.

How is a prompt assigned to an aspect?

Through an explicit tag-to-aspect map rather than anything inferred: a prompt’s tags decide its aspect (A1–A8). The map is an editable table on the settings page, so when a prompt is filed under the wrong aspect the team can move it there — no code change and no re-ingest.

Why does every rate show an interval?

Because a bare percentage hides how much evidence is behind it — 3 of 4 and 750 of 1000 are both 75%, and they mean very different things. Every rate is shown with its Wilson 95% interval and its n. Vacuous and errored trials are excluded from the denominator and counted separately, and a comparison whose interval spans zero is reported as inconclusive, never as a regression.

How do I get the weekly digest?

Subscribe from Settings → Weekly digest; it goes out every Monday at 15:00 UTC, after the 13:00 eval, and you can preview the current one in the browser at /digest/preview (or ?text=1 for plain text). Unsubscribe from the same place or with the one-click link at the foot of every digest — that link works without signing in.

Where is the eval design package?

It lives in this repository at docs/eval-design/00-index.html — the index, the architecture note, and one page per aspect from A1 to A8. That package is the source for the aspect map, the cases and the clauses the hub reports against.

Legal

Looking for the fine print? Read our Terms of Service and Privacy Policy.