AuraOne / Foundry

From real work to repeatable tasks and evaluations.

Use supported production methods, domain-specific graders and compatible environments; retain the option to license private evaluation suites.

Two streams on one foundry: climb the benchmark you care about, or license a sealed suite and score against it.

Foundry run · 06Illustrative workspace
Locked benchmark

Support triage

Working target0.74
ARIS by stage0.58 → 0.74
  1. 01Measurebaseline 0.58
  2. 02Diagnosebaseline 0.58
  3. 03Bridgebaseline 0.58
  4. 04Improvebaseline 0.58
  5. 05Re-measuretarget 0.74
  6. 06Certifypending
Baseline
0.58
Target
0.74
Regression
Not run
Locked rubric · 0.1% overlap targetREGRESSION_GATE

Certifications are tuple-bound and revocable upon contamination or staleness.

Custom post-training data Available by signed scope

Your model picks the problem. Foundry builds the proof.

Bring the model, the harness, the benchmark and the failure. We find a source that allows your use. We build tasks against that failure, record a baseline, try the wrong fixes, get the reviews, and deliver training data apart from sealed holdouts.

  1. 01 / Target

    Your model, harness, benchmark and failure.

    Won't pass if The failure can't be stated as something a test can check.

  2. 02 / Source

    A source that allows your use.

    Won't pass if The rights don't cover training or evaluation for you.

  3. 03 / Tasks

    Runnable tasks, reset to a clean start.

    Won't pass if A task can't be reset and run again from scratch.

  4. 04 / Probes

    A baseline, then the wrong fixes.

    Won't pass if A wrong fix or a bypass we tried gets through.

  5. 05 / Review

    A named reviewer and a second one.

    Won't pass if Either reviewer hasn't signed.

  6. 06 / Split

    Training apart from sealed holdouts.

    Won't pass if A task shows up on both sides.

  7. 07 / Acceptance

    Your bar, your call.

    Won't pass if The accepted tasks miss the bar you set.

Stream B

License a sealed suite built for a real decision.

Use a reviewed evaluation suite while its tasks, solutions, and licensing boundary remain controlled.

Explore the suites

Foundry workbench

Inspect the loop before you scope the engagement.

Explore how a benchmark climb and a sealed-suite license organize the same fundamentals: locked criteria, reviewed failure modes, and evidence attached to the decision.

Illustrative program model
Stream A / Active climb

Support triage

Scope preview
Fixture baseline0.58
Working target0.74
Representative failure modes
  • Escalation policy
  • Sparse context
  • Tool sequencing

The climb loop

One locked bar. Scored from the runs you submit.

Your system runs the benchmark and submits the scores. Each of the six stages keeps the bar, the proposed fix, and the evidence legible enough to review.

  1. 01 / Measure

    Measure the system as it runs today.

    Lock the task, the baseline, and the quality bar before anything changes.

  2. 02 / Diagnose

    Name the failures worth fixing.

    Expert review classifies failure modes and traces them to representative cases.

  3. 03 / Bridge

    Build targeted practice material.

    Proprietary Task Foundry Output aimed at the diagnosed failures, kept separate from sealed evaluation sets.

  4. 04 / Improve

    Fix the prompt, workflow, or data.

    Apply the smallest change that addresses the diagnosed failure mode.

  5. 05 / Re-measure

    Score the same locked benchmark again.

    Repeated rollouts under identical criteria show whether the change held.

  6. 06 / Certify

    Issue the regression report.

    A regression report is issued when the re-measured result clears the bar with no regressions; a positive certification additionally requires hosted proof under a signed scope.

Suite inventory Illustrative

Representative benchmarks, visible before the scope.

Each cell shows how baseline models perform across specific failure categories before targeted post-training and rubric calibration.

1,200 tasks
Illustrative example

Support triage

Classify, route, and resolve representative customer support requests.

Baseline resolution rate
61%
ARIS (reliability-adjusted)
0.58
Known-failure coverage
34 modes
2,500 tasks
Illustrative example

Document review

Extract terms, flag risk, and summarize representative business documents.

Baseline extraction accuracy
74%
ARIS (reliability-adjusted)
0.71
Known-failure coverage
41 modes
800 tasks
Illustrative example

Workflow automation

Complete multi-step operational workflows against representative inputs.

Baseline completion rate
52%
ARIS (reliability-adjusted)
0.49
Known-failure coverage
27 modes
1,200 tasks
Illustrative example

Agentic coding & traces

Reconstruct, verify, and challenge multi-step repository tasks with hidden checks and blind adversarial grading.

4-part spec alignment
Intent · Prompt · World · Verifier
Adversary challenge
Wrong-patch & bypass probes
Cold-start determinism
3 identical runs

Example suites, not available inventory. Task counts and baselines are fixtures. Real inventory, if any, is confirmed with its source and rights in scope.

Evidence model

Trust is part of the evaluation design.

Every Foundry program combines sealed evaluation suites, expert failure annotations, and reproducible release gates.

One score, in plain English.

ARIS is a single reliability-adjusted score: how often the system succeeds, weighted by how much each failure matters, with shaky results discounted.

Repeated rollouts, not one lucky run.

In a scoped engagement, each benchmark runs multiple times under locked criteria. ARIS is always shown with the raw pass rate, sample counts, uncertainty and severe failures, and a score counts toward production only after at least five rollouts per task on a held-out set.

A 0.1% overlap target.

Sealed evaluation sets are built to a 0.1% overlap target against seen material, and the method is named in scope. A set above the line is quarantined and cannot be certified. A failed or incomplete check counts as not cleared, and a later revocation makes any certificate that relied on it stale. The check measures n-gram and term-frequency similarity; it does not prove a model's training data was clean.

Expert review before release.

In a scoped engagement, qualified reviewers adjudicate disputed cases and sign off on every suite before it ships.

Engagement shapes

Enter at the depth your decision requires.

Start bounded, establish the evidence path, then expand only where the result supports it.

Stream A / Improvement

Benchmark climb

Custom data

Training tasks built around your model's failures, delivered apart from sealed holdouts. Available by signed scope.

Sample

A bounded first climb on one benchmark to show the loop working on your system.

Setup

A scoped engagement: locked baselines, diagnosed failures, and the first regression-reported improvement.

Retainer

Repeated hill-climbing across releases under a signed retainer, with regression evidence on every cycle.

Stream B / Licensing

Sealed suite

Eval

License a sealed suite to score a system you already run.

Training

License improvement material built from reviewed failure trajectories for a separately scoped training decision.

Bundle

Pair a sealed evaluation with its improvement material under one scope.

Evaluate under license

Choose a sealed suite for your decision.

We will confirm fit, permitted use, delivery shape, and the terms that keep it sealed.

License a private evaluation suite