The same playbook, read from two sides. A founder reads it forward, how to put AI in a product without burning the budget. An investor reads it backward, how to tell a real advantage from a wrapper. Each side gets sharper by understanding the other.
If you are building
You want AI to earn its place: the right job, the cheapest approach that works, a moat the model does not give you for free, and a way to prove it works before you ship.
If you are evaluating
You want to see through the pitch: whether the AI is load-bearing or decoration, whether the spend matches the rung, whether the moat is real, and whether anyone measured it.
Six questions, two answers each
01
Does this even need AI?
Building
Users buy outcomes, not the letters A and I. The cheapest win is one narrow job a model visibly does better, not a pipeline that automates everything before the core workflow is even proven. And some places users actively reject it: in one 2025 survey, 64% said they would turn off an AI assistant they had not asked for.
Evaluating
"Powered by AI" is a claim, not a moat. The test is simple: delete the model and see what breaks. If the product still works, the AI is decoration. If everything breaks but the thing breaking is a thin wrapper around someone else’s API, the defensibility lives elsewhere, or nowhere.
The ladder, cheapest rung first: start with a good prompt (free), add retrieval (RAG) when answers must cite fresh or private documents, and fine-tune only when you need the same behavior every time regardless of the prompt. Training a model from scratch is almost never the move for an application.
Evaluating
When a company says "our own model," find out which rung they are standing on. A system prompt dressed up as a custom model has a different cost, latency, and defensibility profile than a real fine-tune, which is different again from a foundation model. The rung tells you both the spend and the moat.
A popular model gives you no moat: a competitor calls the same API next week. Durable advantage comes from somewhere the model is not, proprietary data, workflow lock-in, switching costs, network effects, distribution. Build the moat around the model, not in it.
Evaluating
This is the entire diligence question. Without proprietary data, real switching costs, or a workflow others cannot copy, an "AI feature" is replicable in weeks, and competitors often ship a clone within them. Score the moat, not the demo.
The demo is cheap; the last mile is not. Reliability, monitoring, evals, and edge cases are where the cost hides, and the jump from 95% to 99% can cost roughly ten times the first 95%. Price the token and context cost into the unit economics before you commit, not after.
Evaluating
Inference is cost of goods sold. As tokens get cheaper, usage rises to fill the room (Jevons), which rewards whoever owns demand and squeezes whoever sells the commodity. Ask where this company sits relative to the price floor, and who captures the savings.
AI fails differently from software: it is wrong confidently. By one widely-reported 2025 study, around 95% of enterprise generative-AI pilots showed no measurable P&L impact, usually because nobody cleared the quality bar before shipping. Write the eval before you write the feature.
Evaluating
"How do you measure that it works?" is a fast filter. A team with a real eval harness and a stated quality bar is derisked; a team showing a cherry-picked demo is not. The absence of evaluation is itself a finding.
A coding agent does not understand your codebase by default. Give it a persistent map (a project memory file), reusable procedures (skills) for recurring tasks, and scoped sub-agents so exploration does not crowd the context, then wire it to your tools. This is, in fact, how this site is built.
Evaluating
How a team ships is a signal. A throwaway vibe-coded demo and a production system that survives contact with users are not the same asset. Velocity that compounds, with evals and review behind it, is what separates the two.
A teaching aid for your own work, not professional engineering, design, legal, or financial advice, and not a recommendation to buy or sell anything. See the disclaimer.