By Shivacha Engineering
The model is the cheapest part to start with
Access to capable language models is inexpensive to begin with, which is why demos are quick. The cost of a production AI system sits elsewhere: getting data into shape, defining and measuring quality, integrating with real workflows and keeping the system safe and affordable at scale.
Driver 1: data and retrieval
For assistants that answer from your documents, parsing, chunking, indexing and permission filtering are the core work. Messy PDFs, scanned documents and scattered sources add effort; so does respecting who may see what.
Driver 2: evaluation
An evaluation set — representative questions or tasks with expected results — is what turns AI development into engineering. Building it takes time with your subject-matter experts, and it pays for itself on every model or prompt change afterwards.
Driver 3: integrations and actions
Agents that act — create tickets, update records, draft payments — need typed tools, permissions, approval steps and audit logs. Each system integrated adds work, and higher-risk actions need more guardrails.
- Tool and API integrations
- Human approval for consequential actions
- Logging and review workflows
- Fallbacks when the model is unsure
Driver 4: running cost
Per-request model cost, hosting, vector storage and monitoring are ongoing. Caching, choosing smaller models where quality allows and limiting context size keep them predictable. Ask for an estimate of running cost alongside build cost.