Why AI projects stall: a diagnostic before another pilot

Classify the failure before selecting a fix

A pilot can produce a plausible answer while failing the business task. Separate wrong answers, missing sources, unauthorized actions, integration failures and slow human review. These have different remedies; another model is not a universal repair.

Do not infer an economy-wide failure rate from a selective study or use it to justify a purchase. For this workflow, count attempts, accepted tasks, exceptions, abandoned tasks and human effort. Include work that never reached the model.

Run a one-week evidence review

Choose a representative sample with permission, label the observed failure and identify the layer responsible. Keep examples outside the tuning set so the proposed fix faces an independent test.

• Task: can the owner state an acceptable result and escalation?
• Data: are required sources current, authorized and available?
• Workflow: does the result enter the real system of record?
• Authority: can the model request actions that the application must refuse?
• Adoption: can staff finish fresh normal and exception tasks without the builder?

Make the next decision reversible

Select one bounded hypothesis, a named owner and a review window. Release only if accepted task performance or full cost improves without critical permission or quality regression. If the limiting factor is unavailable data rights or reviewer capacity, hold the build and fix that dependency first.

Use the working tools

Blank templates are ungated. Adapt them to the workflow; inputs are not submitted.

• Open the operating charter

Sources and limits

These are Otomat’s proposed operating methods and original decision artifacts. External references inform risk management; they do not certify this method or establish client results. Linked sources were selected for this review; their live revision dates may change.

• NIST AI Risk Management Framework (2023)

For anonymous documentation and proposal patterns, see delivery evidence and its boundaries.

Related decisions

• AI training that proves independent task performance
• AI-agent permissions: define the data and action boundary

Bring a workflow to the discussion

Share the business goal and constraints through the mandate process. Do not send sensitive production data in the initial brief.