AuraOne / Resources / Glossary
Plain definitions of the terms AI teams use when they buy human data and evaluate models: RLHF, preference data, SFT, evals, red teaming and more. Where an AuraOne page covers a term, the entry links to it.
Each definition is general. Where there is a note under it, the note says what AuraOne's own pages state, and nothing more.
Training data
A way to improve a model with human judgment. People compare or rate model answers, those judgments train a reward model, and the reward model steers the model toward the answers people prefer.
On AuraOne: RLHF and preference data. Experts compare model answers side by side and write the reason. On expert programs, a second reviewer checks the work.
Records of human choices between model answers: the prompt, the answers compared, which one is better, and why. RLHF and DPO learn from it.
On AuraOne: RLHF and preference data. Each score stays tied to the rubric version it used, and accepted work is delivered with its review record.
A training method that tunes a model directly on preference pairs, without training a separate reward model.
On AuraOne: RLHF and preference data. AuraOne prepares the preference data. Training is a separate scope.
Further training of a model on prompts paired with the answers you want it to give. SFT data is that set of prompts and answers, often written or corrected by experts.
On AuraOne: SFT data. Qualified specialists create or assess difficult examples, and the review history stays attached.
A model trained on preference data to score answers. Training uses its score as a stand-in for a human judge.
On AuraOne: Model Improvement. Model Improvement lists graders and reward signals among the signal it creates. Training itself is a separate scope.
Attaching labels, rankings or corrections to training examples so a model can learn from them or be tested on them.
On AuraOne: Annotation. Specialists create, rank, and correct training examples against your task and rubric. Uncertain work goes to review.
Data that is generated rather than recorded. It can fill gaps, but it needs its own validation and should stay labeled apart from observed data.
On AuraOne: Physical AI, Synthetic Research. Physical AI programs keep observed capture, machine enrichment and synthetic coverage distinct. Synthetic Research builds synthetic respondents for market research, with methodology and limitations attached to every result.
Evaluation and safety
A test of how well a model does a task, scored against explicit criteria. Human evaluation uses expert graders. Automated evaluation uses code or another model as the judge.
On AuraOne: Model Improvement. Model Improvement builds task-specific test sets from representative work, has experts grade the results, and scores a candidate against its baseline under the same criteria.
A fixed set of test cases with a scoring method, shared so that models can be compared. A benchmark measures only what its cases cover.
On AuraOne: Model Improvement, Foundry. Model Improvement builds task-specific test sets from representative work. Foundry suites package reviewed evaluation work by domain, with baselines shown and sealed content kept sealed.
Adversarial testing of an AI system. People try to make it fail or misbehave so a team can find the problems before release. Findings can become test cases.
The written criteria a grader applies, with what each score means. A versioned rubric shows which criteria produced which grade.
On AuraOne: Rubric Studio, Rubric Studio Open. Rubric Studio lets a team author versioned criteria and approve each version. Every grade keeps the version it used.
A set of examples with answers that experts have verified. Teams use it to calibrate annotators and to check the quality of annotators or models.
On AuraOne: Expert Reviewer Certification Template. The template keeps gold-set performance, disagreements and sign-off for the exact rubric version.
A measure of how often different annotators give the same label to the same item, usually adjusted for chance. Low agreement points to unclear instructions or an ambiguous task.
On AuraOne: Expert AI Quality. The Expert AI Quality page says every batch ships with inter-annotator agreement metrics, reviewer qualification IDs, and criterion-level rationales.
Settling a disagreement between reviewers. A named adjudicator decides against the written criteria, so the disagreement is not averaged away.
On AuraOne: Expert AI Quality. When two experts disagree, a named adjudicator decides. The competing judgments and the rationale are recorded.
Speech and robotics
ASR turns speech into text. A speech dataset pairs recordings with transcripts, and with the consent, language and recording details a team needs to use them.
On AuraOne: Voice AI. Every voice program delivers audio files, time-aligned transcripts, prosody and emotion markers, and verified speaker consent records in one package.
Recordings of a person remotely driving a robot through a task, used to teach robots by example, a method called imitation learning. Human demonstration data records people doing the task themselves.
On AuraOne: Physical AI. Physical AI programs cover teleoperation, human demonstration and multimodal capture. Reviewers check every episode before it ships.
Operations
A workflow where software does the steady steps and people review the exceptions or the uncertain cases.
On AuraOne: Enterprise Intelligence. Software applies your rules and qualified people review the exceptions. Each workflow runs under a signed scope.
Questions
SFT trains a model on one good answer for each prompt. RLHF uses human comparisons of answers to train a reward model, and then optimizes the model against it. Many teams use SFT first and preference methods afterward.
An evaluation is any test of a model against explicit criteria. A benchmark is a fixed, shared set of cases and a scoring method, so different models can be compared. A benchmark measures only what its cases cover, so teams also build task-specific evaluations.
Red teaming is one kind of evaluation. It looks for failures by attacking the system on purpose. Other evaluations measure ordinary quality on ordinary tasks.
Not by default. Generated data needs its own validation, and it should stay labeled apart from observed data. AuraOne's Physical AI program keeps observed capture, machine enrichment and synthetic coverage distinct.
Each entry first defines the term in general words. Where an AuraOne page covers the term, the note under it names that page and restates what it says. The glossary is not a product list: scope, deliverables and use terms are agreed per program.
Guides
Tell us the word your team uses through the contact form. To see how a program is scoped, start with Human Data or Model Improvement. The Resources hub lists the rest of the guides.
