AuraOne Open / Browser evaluation

Score LLM outputs against a rubric in your browser.

Load a rubric and a set of responses, run deterministic scoring, and inspect the evidence behind every score. Nothing leaves the browser unless you export it.

Input

  • Rubric JSON with weighted criteria
  • Response JSONL, one model output per line
  • Optional gold labels for agreement checks

Install and run

pip install auraone-evalkit

The example inputs live in the repository. Get them first:

git clone --depth 1 https://github.com/auraoneai/open.git && cd open

Then run the example from the repository root:

evalkit score --rubric packages/evalkit/examples/tutorial/rubric.jsonl --responses packages/evalkit/examples/tutorial/model_outputs.jsonl --labels packages/evalkit/examples/tutorial/labels.jsonl --format csv
Real outputThe same scoring engine, run from the EvalKit CLI on the bundled tutorial fixture. auraone-evalkit 0.3.0, captured 2026-11-20.
output_id,score,passed,applicable_weight,weighted_points,missing_criteria
out-001,0.8375,True,1.0,0.8375,
out-002,0.3,False,1.0,0.3,
out-003,0.8,True,1.0,0.8,

Limitations

  • Scoring is deterministic and criterion-based; it does not call a model judge.
  • Large datasets are bounded by browser memory. Use the EvalKit CLI for batch runs.
  • Results stay in the tab until you export them.

What you can do

Follow the work from input to a usable result.

Import and validate

Rubric and response files are schema-checked before scoring, with line-level errors.

Score with evidence

Each criterion shows the points it awarded and why, so a score can be reviewed.

Graduate to CI

Run the same rubric with the EvalKit CLI or the EvalKit GitHub Action.

EvalKit Playground workflow

See how the product moves from setup through a useful output.

  1. 01

    Install

    Install auraone-evalkit 0.2.0 from source.

    Package version

  2. 02

    Run on the bundled example

    Run the quick start on the synthetic fixture and compare it with the output above.

    Captured output

  3. 03

    Run on your data

    Point the same command at your own files, locally or in CI.

    Local report

Release specifications

What is included in this release

Review the current source, release, runtime, and support information.

Package
auraone-evalkit 0.2.0Discovery snapshot registry readback
Source tag
v0.2.0 (40ae44072efb)Public GitHub release tag
Sample
packages/evalkit/examples/tutorial/, captured 2026-11-20public/open/samples.json
Data handling
Imported evaluation data remains in the browser unless the user explicitly exports or shares it.Project README

Availability

Current release and install details.

Review the current version, source, packages, downloads, supported platforms, and install options.

Verified

EvalKit Playground availability

Version 0.2.0 is available from source tag v0.2.0.

When the work grows

Need expert reviewers to calibrate or score this evaluation?

Talk to AuraOne