When the AI is the product, not a feature of it
A custom AI application is software whose central function depends on a model: a research tool, a document analysis platform, a domain assistant. Because the model is the product rather than a feature, evaluation, cost per use and output quality become product requirements rather than technical details.
Adding an AI feature to existing software is an engineering task. Building an application whose value is the AI is a different proposition, because everything uncertain about model behaviour is now uncertain about your product. Quality that varies by input, costs that scale with usage, and capability that changes when a provider updates a model are product risks rather than implementation details.
These projects go wrong at the boundary between demonstration and product. A prototype works impressively on the examples its builder chose, which is the least representative sample available. Nobody defines what acceptable quality means, so there is no basis for deciding whether it is ready. Cost per use is not modelled, so unit economics break at the volume that was supposed to justify the build. And no evaluation set exists, so nobody can tell whether a change made things better.
When this work makes sense
Questions worth asking any supplier
What does acceptable quality mean, specifically, and how is it measured?
What does one use cost, and does the pricing model cover it at volume?
What is the moat? If a general assistant does this adequately, the product is exposed.
Is there proprietary data or domain knowledge, or only a prompt?
What happens when the underlying model changes? Behaviour shifts with provider updates.
Deliverables
A defined quality bar with an evaluation set measuring against it
The application, with model behaviour treated as a product surface
Retrieval over your proprietary content where that is the differentiator
Cost per operation modelled against your pricing
Output quality controls, validation, constraints, and human review where needed
Provider abstraction, so a model change is not a rewrite
Usage monitoring including cost per customer
Regression testing so provider updates do not silently degrade the product
Built with
Process
The evaluation set is built before the application, from real cases with expert-agreed correct outputs. It defines what the product must achieve and becomes the regression test for every subsequent change, including changes you did not make, model providers update models, and an application with no evaluation discovers that through customer complaints rather than through a failing test.
The standard nine stages, discovery, strategy, UX/UI, development, content and SEO, testing, launch, measurement and continuous improvement, apply to every project. The paragraph above is what differs for this one.
What moves the number
Pricing depends on scope, functionality, integrations and content requirements. These are the factors that change it most:
Wrapper, retrieval application, or fine-tuned model
Prompt wrapper
Fits: Testing an idea quickly with minimal build
Limits: Little defensibility, a general assistant may do the same thing free
Retrieval over proprietary data
Fits: Products whose value is your content or domain knowledge
Limits: Requires that data to exist and be in usable condition
Fine-tuned or specialised model
Fits: Narrow tasks with substantial labelled examples
Limits: Significant data requirement; retrieval solves most cases more cheaply
Case studies
No published AI product case study.
MBGDesk operates from 4 offices in India, United States, United Kingdom, United Arab Emirates. We hold no industry certifications and do not imply otherwise.
Frequently asked questions
What stops a general AI assistant doing this?
Frequently nothing, and that is the question worth answering before building. Defensible products usually rest on proprietary data, a specific workflow, integrations, or domain constraints that a general tool does not have. A prompt is not a product, and it is worth being honest about that early.
Should we fine-tune a model?
Usually not first. Retrieval over your own content solves most cases more cheaply and stays current as your content changes. Fine-tuning suits narrow tasks with substantial labelled examples, and it is a bigger commitment than most teams expect.
How do we price it?
Starting from cost per use, which model calls make variable rather than fixed. Flat-rate pricing on a product with usage-based costs can invert your margins at exactly the customers who use it most. Model that before pricing rather than after.
What happens when the provider updates the model?
Behaviour changes, sometimes noticeably and rarely announced in detail. Applications with an evaluation set detect it in a failing test; those without detect it in customer complaints. This is the main argument for building the evaluation harness first.
How good does it need to be?
That is a product decision, not a technical one, and it must be made explicitly. A research assistant that is right most of the time is useful. A medical or legal tool at the same accuracy is a liability. The bar determines the build.
Can we build this as an MVP?
Yes, and the MVP should include the evaluation set even when it excludes features. Shipping an unmeasured AI product means you cannot tell improvement from regression, which is the worst position to iterate from.
Do we need our own infrastructure?
Only if data sensitivity or volume economics require it. Most products start on provider APIs. Self-hosting is a real option and a real operational commitment, and it is worth deciding on evidence rather than principle.
Services that connect to this
Talk to us about this
Tell us what you are trying to build and we will come back with scope, approach and an estimate.