Build vs Buy for AI Features: API, Fine-Tune, or Self-Host? A Decision Model for PMs

A PM gets a request: "Add AI summaries to the support console." Engineering says the fastest path is a hosted frontier-model API. Someone on the platform team says self-hosting an open-weight model will be "ten times cheaper at scale." A data scientist says fine-tuning a small model will beat both on quality. All three are right in some situations, and nobody has a shared way to decide.

The result is usually one of two bad outcomes: the team over-builds (self-hosts a GPU cluster for a feature with 2,000 requests a day), or under-builds (stays on a premium API at 50 million requests a month because migrating "isn't on the roadmap"). This post gives you a model to avoid both: three sourcing options, the variables that actually drive the decision, a break-even calculator, and a rubric you can put in a PRD.

The three options, defined precisely

Option	What you own	What you pay for	Typical time to first release
Hosted API, prompted	Prompts, evals, orchestration	Per-token usage	Days
Hosted or small model, fine-tuned	Training data, eval set, tuned weights	Training run + (usually cheaper) per-token inference	Weeks
Self-hosted open-weight model	Everything above plus serving, scaling, upgrades	GPU hours + engineering time	Weeks to months

Note that these aren't strictly exclusive. A common mature setup is a fine-tuned small model handling 80% of traffic with a frontier API as fallback for hard cases. That's a routing decision on top of the sourcing decision, covered in our model-routing guide.

Why the "obvious" cost comparison is wrong

The tempting comparison is price per million tokens: API at X, self-hosted at Y, done. This misses four things.

Utilization. A GPU you rent by the hour costs the same whether it serves 1 request or 1,000. APIs bill per use. Self-hosting only wins when you keep hardware busy. At low or spiky traffic, you pay for idle capacity.
People cost. Someone has to run the inference server, handle upgrades, monitor latency, and be on call. That cost is real even if it never appears on the cloud bill.
Quality delta. If a frontier model resolves 92% of cases and your self-hosted model resolves 84%, the cost of the 8% gap (escalations, retries, churn) belongs in the comparison.
Price drift. Hosted API prices and capabilities change frequently, and generally the direction has been toward more capability per dollar. A self-hosting business case built on today's API price can get stale within a couple of quarters. Always check current provider pricing pages rather than relying on numbers in a document from last year.

A break-even model you can run today

The core question: at what monthly request volume does the fixed cost of self-hosting (or fine-tuning) beat the variable cost of the API?

from dataclasses import dataclass


@dataclass
class ApiOption:
    input_price_per_m: float   # USD per 1M input tokens (check provider pricing page)
    output_price_per_m: float  # USD per 1M output tokens
    avg_input_tokens: int
    avg_output_tokens: int

    def cost_per_request(self) -> float:
        return (
            self.avg_input_tokens * self.input_price_per_m
            + self.avg_output_tokens * self.output_price_per_m
        ) / 1_000_000


@dataclass
class SelfHostOption:
    gpu_hourly_usd: float      # per GPU, from your cloud quote
    gpus: int                  # minimum for redundancy, usually >= 2
    engineer_monthly_usd: float  # fully loaded cost
    engineer_fte_fraction: float # e.g. 0.3 of one engineer
    requests_per_gpu_per_hour: int  # from YOUR load test, not a benchmark blog

    def fixed_monthly(self) -> float:
        gpu = self.gpu_hourly_usd * self.gpus * 24 * 30
        people = self.engineer_monthly_usd * self.engineer_fte_fraction
        return gpu + people

    def capacity_per_month(self) -> int:
        return self.requests_per_gpu_per_hour * self.gpus * 24 * 30


def break_even_requests(api: ApiOption, host: SelfHostOption) -> dict:
    api_cost = api.cost_per_request()
    fixed = host.fixed_monthly()
    cap = host.capacity_per_month()
    be = fixed / api_cost if api_cost > 0 else float("inf")
    return {
        "api_cost_per_request": round(api_cost, 5),
        "self_host_fixed_monthly": round(fixed, 0),
        "break_even_requests_per_month": int(be),
        "self_host_capacity_per_month": cap,
        "feasible": be <= cap,   # if break-even exceeds capacity, add GPUs and recompute
    }


if __name__ == "__main__":
    api = ApiOption(
        input_price_per_m=3.0, output_price_per_m=15.0,
        avg_input_tokens=1800, avg_output_tokens=250,
    )
    host = SelfHostOption(
        gpu_hourly_usd=2.5, gpus=2,
        engineer_monthly_usd=14000, engineer_fte_fraction=0.3,
        requests_per_gpu_per_hour=900,
    )
    print(break_even_requests(api, host))

The numbers above are placeholders for illustration, not current prices. Replace every input with your own: token counts from production logs, GPU price from your cloud quote, throughput from a load test on your actual prompts.

With these placeholder inputs the API costs under a cent per request (about $0.009), self-hosting has about $7,800 a month in fixed cost (GPUs plus a fraction of an engineer), and break-even lands around 850,000 requests a month. Below that, the API wins on cost alone. Above it, self-hosting starts to pay, provided the quality is good enough and the capacity holds.

The most useful thing this model does is expose which input dominates. In most real cases it's not the GPU price; it's the engineer fraction and the utilization. Run the sensitivity: halve the throughput and see how far break-even moves.

Where fine-tuning fits

Fine-tuning is rarely a cost play on its own and more often a quality or latency play. It makes sense when:

You have a narrow, repetitive task (classification, extraction, routing, a fixed tone) where a small tuned model can match a large general one.
You have at least a few hundred to a few thousand clean labeled examples, and an eval set held out from training.
Prompting is already near its ceiling: you've tried better instructions, few-shot examples, and structured outputs.

It doesn't make sense when the task changes weekly, when your labels are noisy, or when you haven't yet built an eval. Fine-tuning without an eval is guessing with extra steps. Also budget for re-tuning: when the base model is deprecated or your policy changes, you retrain.

A scoring rubric for your PRD

Score each factor 1 (favors API) to 5 (favors owning the model). Weights reflect how often each factor is decisive.

Factor	Weight	Score 1 means...	Score 5 means...
Monthly volume	3	Under 100k requests	Millions, steady
Traffic shape	2	Spiky, unpredictable	Flat, predictable
Task narrowness	2	Open-ended, varied	Narrow, repetitive
Data sensitivity / residency	3	No restrictions	Cannot leave your boundary
Latency requirement	2	Seconds are fine	Tight, sub-second
Team ML/infra capacity	3	No one to run it	Experienced platform team
Quality bar vs. frontier	2	Needs frontier quality	Small model is enough
Speed to market	2	Ship this month	Can wait a quarter

WEIGHTS = {
    "volume": 3, "traffic_shape": 2, "task_narrowness": 2,
    "data_sensitivity": 3, "latency": 2, "team_capacity": 3,
    "quality_bar": 2, "speed_to_market": 2,
}

def recommend(scores: dict[str, int]) -> str:
    total = sum(WEIGHTS[k] * scores[k] for k in WEIGHTS)
    max_total = 5 * sum(WEIGHTS.values())
    ratio = total / max_total
    if ratio < 0.45:
        return "Hosted API, prompted"
    if ratio < 0.65:
        return "Fine-tune a smaller model, or hybrid routing"
    return "Self-host (after validating with a pilot)"

Treat the output as a conversation starter, not a verdict. Two factors deserve veto power regardless of score: data residency (if data can't leave, the API is off the table) and team capacity (if no one can operate it, self-hosting fails no matter how good the math looks).

Common pitfalls

Benchmarking on public leaderboards instead of your task. Build a 100-example eval from real traffic first. Every option gets scored on it.
Ignoring the migration path. Start with an API behind a thin abstraction layer so you can swap providers or models later. Premature self-hosting is far more common than premature API use.
Counting only inference. Include eval runs, retraining, monitoring, and on-call load.
Locking in on a single price point. Re-run the break-even model quarterly. Both API prices and open-weight model quality move fast.
Forgetting the fallback. Self-hosted systems fail differently. Decide what happens when your GPU pool is saturated or down, and whether a hosted API is the backstop.
Treating the decision as permanent. The best teams treat sourcing as a dial. Start at API, collect traces, and graduate specific high-volume, narrow paths to cheaper options once the data justifies it.

A practical sequence

Launch on a hosted API behind an internal interface. Log inputs, outputs, cost, and latency.
Build an eval set from real traffic within the first weeks.
After a month, rank your features by volume and narrowness. The top one or two are candidates for fine-tuning or a smaller model.
Run the break-even model with real numbers. Pilot the cheaper option in shadow mode against the API, and compare on your eval set.
Promote only when quality is within your agreed tolerance and the economics clear break-even with margin for people cost.

Next steps

This week, pull one month of production logs for your highest-volume AI feature, compute average tokens per request, and plug them into the calculator above. If break-even is more than 3x your current volume, stay on the API and invest in evals and caching instead. If you're within reach, design a shadow-mode pilot.

If you're a PM who wants to make these sourcing, evaluation, and pricing calls with confidence, Joinloop's AI Product Management cohort walks through exactly these decisions with hands-on exercises.