Climb a benchmark that matters to your system.
Lock the baseline, diagnose the failures, improve the system, and re-measure against the same criteria.
Inspect the climb Or have tasks built for your modelAuraOne / Foundry
Use supported production methods, domain-specific graders and compatible environments; retain the option to license private evaluation suites.
Two streams on one foundry: climb the benchmark you care about, or license a sealed suite and score against it.
Support triage
Certifications are tuple-bound and revocable upon contamination or staleness.
Custom post-training data Available by signed scope
Bring the model, the harness, the benchmark and the failure. We find a source that allows your use. We build tasks against that failure, record a baseline, try the wrong fixes, get the reviews, and deliver training data apart from sealed holdouts.
Won't pass if The failure can't be stated as something a test can check.
Won't pass if The rights don't cover training or evaluation for you.
Won't pass if A task can't be reset and run again from scratch.
Won't pass if A wrong fix or a bypass we tried gets through.
Won't pass if Either reviewer hasn't signed.
Won't pass if A task shows up on both sides.
Won't pass if The accepted tasks miss the bar you set.
Lock the baseline, diagnose the failures, improve the system, and re-measure against the same criteria.
Inspect the climb Or have tasks built for your modelUse a reviewed evaluation suite while its tasks, solutions, and licensing boundary remain controlled.
Explore the suitesFoundry workbench
Explore how a benchmark climb and a sealed-suite license organize the same fundamentals: locked criteria, reviewed failure modes, and evidence attached to the decision.
The climb loop
Your system runs the benchmark and submits the scores. Each of the six stages keeps the bar, the proposed fix, and the evidence legible enough to review.
Lock the task, the baseline, and the quality bar before anything changes.
Expert review classifies failure modes and traces them to representative cases.
Proprietary Task Foundry Output aimed at the diagnosed failures, kept separate from sealed evaluation sets.
Apply the smallest change that addresses the diagnosed failure mode.
Repeated rollouts under identical criteria show whether the change held.
A regression report is issued when the re-measured result clears the bar with no regressions; a positive certification additionally requires hosted proof under a signed scope.
Suite inventory Illustrative
Each cell shows how baseline models perform across specific failure categories before targeted post-training and rubric calibration.
Classify, route, and resolve representative customer support requests.
Extract terms, flag risk, and summarize representative business documents.
Complete multi-step operational workflows against representative inputs.
Reconstruct, verify, and challenge multi-step repository tasks with hidden checks and blind adversarial grading.
Example suites, not available inventory. Task counts and baselines are fixtures. Real inventory, if any, is confirmed with its source and rights in scope.
Evidence model
Every Foundry program combines sealed evaluation suites, expert failure annotations, and reproducible release gates.
ARIS is a single reliability-adjusted score: how often the system succeeds, weighted by how much each failure matters, with shaky results discounted.
In a scoped engagement, each benchmark runs multiple times under locked criteria. ARIS is always shown with the raw pass rate, sample counts, uncertainty and severe failures, and a score counts toward production only after at least five rollouts per task on a held-out set.
Sealed evaluation sets are built to a 0.1% overlap target against seen material, and the method is named in scope. A set above the line is quarantined and cannot be certified. A failed or incomplete check counts as not cleared, and a later revocation makes any certificate that relied on it stale. The check measures n-gram and term-frequency similarity; it does not prove a model's training data was clean.
In a scoped engagement, qualified reviewers adjudicate disputed cases and sign off on every suite before it ships.
Engagement shapes
Start bounded, establish the evidence path, then expand only where the result supports it.
Training tasks built around your model's failures, delivered apart from sealed holdouts. Available by signed scope.
A bounded first climb on one benchmark to show the loop working on your system.
A scoped engagement: locked baselines, diagnosed failures, and the first regression-reported improvement.
Repeated hill-climbing across releases under a signed retainer, with regression evidence on every cycle.
License a sealed suite to score a system you already run.
License improvement material built from reviewed failure trajectories for a separately scoped training decision.
Pair a sealed evaluation with its improvement material under one scope.
We will scope the model, the harness, the task, the review boundary, and the evidence you need back.
We will confirm fit, permitted use, delivery shape, and the terms that keep it sealed.
Connected surfaces