Adoption · Operating decision guide
AI training that proves independent task performance
Otomat editorial · Revised
A role-based competency rubric using fresh tasks, exception handling, correction effort and maintenance/restoration evidence.
Train for the task and its exceptions
Attendance and logins show participation, not capability. Agree what each role must do independently and what must be escalated. Use realistic authorized tasks for practice, then different held-out tasks for assessment. Include missing sources, conflicting information and requests outside scope.
Record eligible users, intended task frequency, accepted completion, correction time, escalations and sustained use over an agreed period. Make the denominator visible. A model upgrade that saves generation time but doubles review effort may worsen the workflow.
Agree the rubric before rollout
Set task-specific numerical thresholds with the functional owner before assessment; there is no universal pass rate. Critical unauthorized actions fail the gate regardless of an average score. Record accommodations and permitted reference materials so assessment measures the real job.
| Role | Fresh assessment | Evidence / gate |
|---|---|---|
| Executive | Defend a go/no-go decision from baseline, full cost and risk | Can explain uncertainty, alternatives and retained human decisions |
| Staff | Complete normal and exception tasks | Accepted result; correct escalation; review effort within agreed limit |
| Power user | Reproduce a failure and propose a bounded fix | Source/version identified; evidence preserved; no policy bypass |
| Maintainer + backup | Change, evaluate, release and restore | Independent record; held-out tests; rollback succeeds |
Remediate and reassess on fresh work
Classify failures as unclear procedure, missing knowledge, interface problem, model quality or permission problem. Assign the corrective action to the responsible owner. Retest on new cases and track reviewer effort; repeated coaching on the same example does not demonstrate transfer.
Maintain competence after changes
Material workflow or model changes need targeted reassessment. Primary and backup maintainers each perform an approved change and restoration without the builder taking over. Keep open limitations and support coverage visible in the handover record.
Use the working tools
Blank templates are ungated. Adapt them to the workflow; inputs are not submitted.
Sources and limits
These are Otomat’s proposed operating methods and original decision artifacts. External references inform risk management; they do not certify this method or establish client results. Linked sources were selected for this review; their live revision dates may change.
For anonymous documentation and proposal patterns, see delivery evidence and its boundaries.
Related decisions
Bring a workflow to the discussion
Share the business goal and constraints through the mandate process. Do not send sensitive production data in the initial brief.
Scope an Operating Mandate