Start with the questions your AI needs to answer.
A practical way to shape evaluation before choosing a model.
A model demo can make a project feel close to finished. The useful question is more specific: how does it behave on the work your users actually need to do?
Collect examples before picking a winner.
Start with representative inputs and the outcome you would consider useful. Include ordinary cases, incomplete information and situations where the system should ask for help. Keep a separate set of examples for checking changes.
Decide what good means.
A response may be accurate but unusable in the next step of a workflow. Look at factual correctness, required structure, source support and whether the user can act on it. Record the failures separately so improvements have a clear target.
Evaluate the whole system.
Prompts, retrieval, tools and interface decisions all affect the result. Compare changes against the same examples. Include latency and cost in the decision, alongside answer quality.
This gives the team a shared way to discuss progress and a more useful foundation for deciding whether adaptation or fine-tuning is the next step.