AI initiatives often start with a model demonstration and then struggle to reach production. The demonstration works on a curated dataset. Production involves messy documents, inconsistent terminology, access boundaries, changing content and users who notice when the answer is wrong.
The model is the smallest part
A language model is a general capability. The value comes from the context it is given, the actions it can take, the controls around it and the evaluation that proves it is useful for the specific task.
Organisations that treat AI as a model procurement exercise tend to build impressive prototypes and fragile products. Organisations that treat it as a data and control problem tend to build less visible but more durable capabilities.
Context needs ownership and permission
Retrieval-augmented generation is often described as a technical pattern. In practice it is a data governance problem: which documents are authoritative, how are they versioned, who is allowed to see them and how do you prevent the system from combining information across permission boundaries?
Without those answers, the AI feature becomes a new path around existing access controls. That is a security and privacy risk, not just a quality risk.
Evaluate the task, not the model
A model benchmark does not tell you whether the system produces a usable answer for a specific workflow. Evaluation should be based on representative tasks, known edge cases, human review and a clear definition of acceptable error.
For high-consequence tasks, the design should include human confirmation, source citation, confidence signals and a path to escalate. The goal is not to remove human judgement; it is to make it better informed.
Start with a bounded, measurable use case
The best first use cases have a clear user, a measurable outcome and a tolerable error rate. Document summarisation, knowledge retrieval and structured extraction are often better starting points than open-ended decision support.
A bounded use case also makes cost visible. Token consumption, retrieval volume, human review and integration effort are easier to measure when the workflow is specific.
What we recommend
- Start with the task and the decision, not the model.
- Map the authoritative content, ownership and permission boundaries before retrieval.
- Define an evaluation set with representative tasks and edge cases.
- Design human review and escalation for high-consequence outputs.
- Measure cost per task and error rate, not just model accuracy.
Where we disagree
We do not believe every organisation needs a custom model or a large AI platform. Many useful capabilities can be delivered with a bounded workflow, a governed retrieval layer and a commercial model. We also do not believe AI should be adopted because competitors are announcing it; the use case has to justify the operating cost and risk.
A practical test
Pick one proposed AI use case and write the evaluation before the prototype. Define the user, the task, the acceptable error, the source of authoritative context, the permission boundary and the human review step. If the organisation cannot describe what a good answer looks like, the problem is the task definition, not the model.