Rubric validation
Lint, diff, and convert portable rubrics before they reach a release gate.
AuraOne Open / Evaluation CLI
Validate rubrics, score outputs, check reviewer agreement and leakage, and write portable reports. Runs locally or inside GitHub Actions.
pip install auraone-evalkitThe example inputs live in the repository. Get them first:
git clone --depth 1 https://github.com/auraoneai/open.git && cd openThen run the example from the repository root:
evalkit score --rubric packages/evalkit/examples/tutorial/rubric.jsonl --responses packages/evalkit/examples/tutorial/model_outputs.jsonl --labels packages/evalkit/examples/tutorial/labels.jsonl --format csv- uses: auraoneai/[email protected]
with:
rubric-path: evals/rubric.jsonl
responses-path: evals/outputs.jsonl
labels-path: evals/labels.jsonloutput_id,score,passed,applicable_weight,weighted_points,missing_criteria
out-001,0.8375,True,1.0,0.8375,
out-002,0.3,False,1.0,0.3,
out-003,0.8,True,1.0,0.8,Included packages
python -m pip install rubric-specpython -m pip install eval-adapterpython -m pip install eval-conformance-suitepython -m pip install eval-run-manifestpython -m pip install synthetic-disagreementWhat you can do
Lint, diff, and convert portable rubrics before they reach a release gate.
Weighted criterion scores with pass/fail thresholds and missing-criterion reporting.
EvalKit Action posts one consistent check on every pull request.
See how the product moves from setup through a useful output.
01
Install auraone-evalkit 0.3.0 from the package registry.
Package version
02
Run the quick start on the synthetic fixture and compare it with the output above.
Captured output
03
Point the same command at your own files, locally or in CI.
Local report
Release specifications
Review the current source, release, runtime, and support information.
Availability
Review the current version, source, packages, downloads, supported platforms, and install options.
PyPI 0.3.0 is available with a linked source tag v0.3.0.
When the work grows
Continue exploring
Each one lists its source, current version, and install path.