You are reading the free preview: the preface and chapters 1 and 2. Get the full book (£5.99) to read all 16 chapters online, plus the PDF and EPUB.

Chapter 2 The Production Mindset

The Problem

A two-person team runs a document-summarisation feature. A user complains that summaries of legal contracts are dropping the termination clauses. An engineer opens the prompt, adds a line — “always preserve clauses about termination and liability” — confirms the contract from the ticket now summarises correctly, and ships it before lunch. The fix works. It also, unnoticed, makes summaries of short marketing emails twice as long and weirdly formal, because the new instruction nudged the model toward exhaustive, legalistic output everywhere. Two weeks later a different ticket arrives about bloated email summaries. A different engineer adds “be concise for short documents,” which fixes the emails and quietly reintroduces the original clause-dropping for a subset of contracts. Six months and forty edits later, the prompt is a 600-word palimpsest of contradictory instructions. No one knows which lines still matter, every change fixes one ticket and risks two more, and nobody can say whether today’s prompt is better or worse than the one from March. There were never any regressions recorded, because there was never anything measuring them.

The team in that story is not incompetent. Each individual decision was reasonable: a user reported a problem, an engineer reproduced it, changed the prompt until it went away, and confirmed the fix on the reported case. That is exactly how we are trained to fix bugs in ordinary software. The trouble is that the method depends on a property LLM systems do not have — that a verified fix on one input tells you something reliable about every other input — and the absence of that property is precisely what makes the loop accumulate damage instead of progress.

This chapter is about the change of mind that has to happen before any of the techniques in the rest of the book make sense. We saw in Chapter 1 why prompts fail in ways ordinary code does not. This chapter is about the working method that replaces the instinct those failures defeat. We will call it the production mindset, and its core is a single substitution: you stop tweaking and hoping, and you start changing and measuring.

2.1 From “tweak and hope” to evaluation-driven development

The summarisation team was running a loop, even if they would not have called it one. It looked like this: a complaint arrives, an engineer forms a hunch about the cause, edits the prompt, checks the one case that prompted the complaint, and ships. Call it the tweak-and-hope loop. Its defining feature is that the only measurement in the entire cycle is a single human eyeballing a single output. The loop has no memory — nothing it learns on Tuesday constrains what it breaks on Thursday — and no breadth, because it never looks at more than the case in front of it.

The production mindset replaces this with a loop that has both memory and breadth. Each change is measured against a fixed, representative set of inputs before it ships, and production behaviour is watched after it ships so that the next change starts from evidence rather than a hunch. The shape is a cycle:

Change. You modify a prompt, swap a model, adjust a parameter, or alter a chain. This is the only step the tweak-and-hope loop has, and the only one it shares with the mature one.
Eval gate. Before the change can ship, it runs against a curated dataset of inputs with known-good expectations, and is scored. If the score on any input you already cared about drops, the gate blocks the change. This is what turns a hunch into a measured decision; Chapter 5 builds the scoring machinery and Chapter 6 builds the gate.
Traced production traffic. Once shipped, every real call is logged with enough context — input, output, prompt version, model version, latency, cost — that you can later reconstruct exactly what happened. Chapter 9 is about this. Without it, production is a black box and your only sensor is the customer-complaint queue.
Scored regression detection. Production traffic is sampled and scored on the same criteria as the eval set, so that drift and slow degradation surface as a falling number on a dashboard rather than as a spike in tickets.
Next change. The failures the previous steps surface — both from the gate and from production — become the inputs to the next change, and frequently become new cases in the eval set so that the same regression can never ship twice.

The single most important property of this loop is that step 5 feeds back into step 2. The eval set is not static; every production failure you find is a candidate to be frozen into the gate, so the system’s ability to catch regressions grows monotonically over time. The tweak-and-hope loop gets worse as it ages, because contradictions accumulate and nothing remembers them. The evaluation-driven loop gets better as it ages, because every incident leaves behind a test that prevents its recurrence. That asymmetry — degradation versus compounding improvement — is the whole argument for the extra machinery.

It is worth being precise about what the eval gate buys you, because it is easy to misread it as bureaucracy. The summarisation team’s real problem was not that they made changes; it was that they could not see the side effects of a change. A prompt is a single dense object, and adjusting one instruction shifts behaviour across every input the prompt handles. The eval gate is simply an instrument that makes those side effects visible before a user finds them. It does not stop you changing things. It stops you changing things blind.

2.2 What “done” means for an LLM feature

In conventional software, “done” is a meaningful state. You implement a function, write tests, the tests pass, you ship, and — barring new requirements or discovered bugs — the function is finished. It will compute the same thing next year. The specification is stable and the implementation is deterministic, so completion is a coherent idea.

For an LLM feature, “done” in that sense never arrives, and pretending otherwise is the source of a great deal of organisational pain. There are three independent reasons completion does not apply.

The first is non-determinism, covered in Chapter 1: the same input does not reliably yield the same output, so “I verified it works” is a statement about a sample, not a guarantee about behaviour. The second is drift: the model behind the API can change under you, and the distribution of real user inputs shifts as your product and its users change, so a feature that was correct in March can be wrong in September with no edit to your code. The third is that the quality bar itself moves — competitors raise expectations, new failure modes are discovered, and what counted as an acceptable summary last year reads as clumsy now.

The honest mental model is not a button on a form, which is built once and then simply exists. It is a fraud-detection model, which is deployed and then maintained — monitored for degrading precision, retrained as patterns shift, and watched indefinitely because the thing it models keeps moving. An LLM feature is a system you operate, not an artefact you complete. This is not a counsel of despair; it is a budgeting fact, and treating it as one is what separates teams that ship LLM features calmly from teams that lurch from incident to incident. We will return to the budgeting consequences in the team section (Sculley et al. 2015).

2.3 Prompts as artifacts, not text

The deepest shift in the production mindset is a shift in what kind of thing you believe a prompt is. In the tweak-and-hope world, a prompt is text: a string you open, edit, and save, the way you would edit a paragraph in a document. Text has no identity over time, no owner, no history you can interrogate, and no contract about how it behaves. That is fine for a paragraph. It is a liability for the component that determines what your product says to users.

In the production mindset, a prompt is an artifact: a versioned, owned, observed object with a defined behavioural contract, exactly like a deployed service or a trained model. The difference is not cosmetic. Four concrete properties separate a production LLM system from a prototype, and each is something a prompt-as-text simply cannot have.

A reproducible evaluation gate. The system can answer, mechanically and repeatably, “is this version better or worse than the last one?” by running both against the same scored dataset. A prototype answers that question with an engineer’s memory and optimism. The gate is what makes change safe; it is the subject of Chapters 5 and 6.

Versioning. Every prompt has an identity and a history. You can say which version is in production, what it said last week, who changed it, and why. When behaviour shifts, you can correlate the shift with a specific change. A prompt living as a scattered string literal can answer none of these; Chapter 4 is about giving prompts a home and a history.

Observability. Production behaviour is visible. You can see what inputs arrive, what outputs go out, what they cost, how long they take, and how their quality scores trend over time. A prototype’s only observability is a user complaint. Chapter 9 builds this out.

Defined ownership. Someone is responsible for the artifact — for its quality, its cost, its incidents, and its maintenance. In the opening story, the prompt was owned by whoever last touched it, which is another way of saying it was owned by no one. Ownership is the human property that makes the other three durable; without it, the eval set rots, the version history fills with unexplained changes, and the dashboards go unwatched. Chapter 12 addresses ownership formally.

These four are not a wish-list of nice-to-haves. They are the minimum structure under which the evaluation-driven loop can actually run. The gate needs the eval set; correlating failures needs versioning; detecting drift needs observability; and keeping all three alive needs an owner. Remove any one and the loop quietly degrades back toward tweak-and-hope.

2.4 We already know how to do this

None of this is unprecedented, and that is the reassuring part. The software profession has twice before taken an artefact that people initially treated as informal and subjected it to engineering discipline, and both times the discipline was resisted as overkill before becoming obviously non-negotiable.

The first was application code itself. There was a time when code was edited directly on servers, with no version control, no review, and no automated tests — and shipping meant copying files and hoping. We now consider source control, code review, and continuous integration to be table stakes, not because a committee mandated them but because the alternative was demonstrably more expensive once systems grew past a certain size. Version control answered “what changed and who changed it”; review answered “is this change sound”; CI answered “did this change break anything we already cared about.” Those three questions map almost exactly onto prompt versioning, prompt review, and the eval gate.

The second was machine-learning models. Early ML in production was notorious for being un-engineered: models trained in notebooks, deployed by hand, and left to silently degrade as the world drifted away from their training data. The response was the set of practices now grouped under MLOps — model versioning, experiment tracking, monitoring for drift, and retraining pipelines. The motivating insight was precisely that a model is non-deterministic relative to its inputs and decays over time, so it must be operated, not merely deployed.

Prompts sit squarely in the same category as ML models on the property that matters: they are probabilistic components whose behaviour depends on inputs you do not fully control and on a model that can change underneath you. It would be strange to grant rigorous lifecycle management to the model weights and then drive their behaviour with an un-versioned, un-tested, un-monitored string. This chapter, and the book, is the unsurprising argument that the discipline we already apply to code and to models should extend to the prompts that steer them. The techniques differ in detail — you score outputs rather than assert equality, you watch quality trends rather than error rates — but the shape is one the profession already knows.

2.5 A maturity model for prompt engineering

It helps to make the journey concrete, because “be more disciplined” is not actionable and most teams are somewhere in the middle rather than at either extreme. The following five levels describe a progression from prompts as scattered strings to a fully governed lifecycle. The point of the model is not to shame anyone at a low level — every system starts at zero — but to let you locate yourself honestly and identify the single next move that buys the most safety.

Level	Name	Prompts live as	Change process	What’s measured	Failures found by	Ownership
0	Ad hoc	Scattered string literals in code	Edit in place, ship	Nothing	Customer complaints	Whoever last touched it
1	Tracked	Named constants / a prompts file, in version control	Edit, commit, ship	Diffs are visible in git	Complaints, but now traceable to a commit	Implicit (the repo)
2	Tested	Versioned, with an eval dataset	Edit, run eval locally, ship if it looks better	Scores on a fixed eval set, run manually	The eval set, when someone remembers to run it	A nominated engineer
3	Gated	Versioned artifacts with a behavioural contract	Edit, automated eval gate in CI blocks regressions, ship	Eval scores in CI plus sampled, scored production traffic	The gate (pre-ship) and production monitoring (post-ship)	A named owner with an on-call path
4	Governed	First-class artifacts in a managed lifecycle	Proposed change, reviewed, gated, staged rollout, monitored, auditable	Full eval + production observability + cost/latency + audit trail	The gate, monitoring, and automated drift detection	Defined ownership with documented accountability and review cadence

A few things are worth drawing out of the table. The jump from Level 1 to Level 2 is the introduction of measurement — the first time anyone scores a change against more than one case — and it is the most important single step a team can take, because it is the precondition for everything above it. The jump from Level 2 to Level 3 is the introduction of automation and production feedback: the eval stops being a thing someone runs when they remember and becomes a gate that runs on every change, and you start scoring real traffic rather than only your curated set. The jump from Level 3 to Level 4 is largely organisational — review, audit, staged rollout, and accountability — and it is the level that regulated environments require, which is why Chapter 12 is devoted to it.

Most teams reading this book are at Level 0 or 1 and feel the pain of it. The worst place to be, though, is a false Level 3: elaborate dashboards and a CI pipeline, but an eval set that no longer represents real traffic and an owner who left the company. Maturity is not the tooling; it is the live, maintained loop the tooling supports. A genuine Level 2 with a small, honest, well-tended eval set beats a decorative Level 3 every time.

2.6 Implementing the smallest honest version of the loop

The gap between Level 1 and Level 2 is conceptually large but mechanically small, and it is worth showing how little code it takes to cross it, because the barrier is almost always mindset rather than effort. You do not need a platform. You need a handful of representative cases, a way to score an output, and the habit of running them before you ship. Everything in Chapters 5 through 9 is an elaboration of this skeleton; none of it replaces the skeleton’s logic.

First, give the prompt an identity and a version instead of leaving it as an inline literal. Even this much moves you from Level 0 to Level 1.

# prompts.py -- prompts as named, versioned artifacts, not scattered strings.
from dataclasses import dataclass


@dataclass(frozen=True)
class Prompt:
    name: str
    version: str
    template: str

    def render(self, **kwargs) -> str:
        return self.template.format(**kwargs)


SUMMARISE = Prompt(
    name="summarise_document",
    version="2025-05-30",  # bump on every change; pairs with git history
    template=(
        "Summarise the document below. Preserve any clauses about "
        "termination and liability. Match the summary length to the "
        "document: brief for short documents, fuller for long ones.\n\n"
        "Document:\n{document}\n\nSummary:"
    ),
)

Next, capture the cases you care about. The crucial move is that the case from the opening story — the contract whose termination clause went missing — does not just get fixed; it becomes a permanent member of the eval set, so the regression can never silently return.

# eval_cases.py -- each production failure becomes a permanent test case.
from dataclasses import dataclass
from typing import Callable


@dataclass
class EvalCase:
    name: str
    document: str
    # A check returns True if the output is acceptable for this case.
    check: Callable[[str], bool]


CASES = [
    EvalCase(
        name="contract_keeps_termination_clause",
        document="...[a contract whose termination clause was dropped]...",
        check=lambda out: "terminat" in out.lower(),
    ),
    EvalCase(
        name="short_email_stays_short",
        document="...[a three-line marketing email]...",
        # The regression the quick fix caused: short inputs got bloated.
        check=lambda out: len(out.split()) <= 60,
    ),
]

Finally, the gate itself. Note what it does and does not do: it runs the current prompt against every case, scores each, and refuses to pass if any case that previously worked now fails. That single rule is the difference between a loop that accumulates regressions and one that cannot ship the same regression twice.

# gate.py -- the minimum viable eval gate. Run before every ship.
from prompts import SUMMARISE
from eval_cases import CASES


def run_model(prompt: str) -> str:
    """Call your provider here. Model-agnostic by design."""
    raise NotImplementedError("wire up your LLM client")


def evaluate() -> bool:
    failures = []
    for case in CASES:
        output = run_model(SUMMARISE.render(document=case.document))
        if not case.check(output):
            failures.append(case.name)

    if failures:
        print(f"GATE FAILED ({SUMMARISE.version}): {', '.join(failures)}")
        return False
    print(f"GATE PASSED ({SUMMARISE.version}): {len(CASES)} cases")
    return True


if __name__ == "__main__":
    import sys
    sys.exit(0 if evaluate() else 1)

The check functions here are deliberately crude — substring and length tests — because the point is the structure of the loop, not the sophistication of the scoring. Real scoring uses graded rubrics, model-graded evaluation, and statistical treatment of non-determinism, all of which Chapter 5 develops properly. But a team that wires up even this crude version, and adds a case for every incident, is already running the production loop. They have crossed from hoping to knowing. Everything after this is refinement.

For Your Team

The most expensive mistake in scoping LLM work is budgeting it like a feature when it behaves like a service. A feature has a build cost and then runs for free; a service has a build cost and a run cost that continues for as long as it is live. Because an LLM feature is never “done” — it drifts, the model changes, the quality bar rises — its evaluation and monitoring are not a one-off line item but an ongoing operating cost, and a plan that omits them is not cheaper, it is just wrong.

Three practical moves make this fundable rather than a fight. First, when you estimate an LLM feature, attach an explicit ongoing percentage — a useful rule of thumb is that maintenance, evaluation, and monitoring run at roughly 20–30% of the initial build effort per year, indefinitely. Naming the number turns an invisible cost into a planned one. Second, make the eval set and the monitoring dashboards deliverables of the initial build, not aspirations for later — “later” never comes, and a feature shipped without them is shipped at Level 0 regardless of how polished the demo was. Third, assign a named owner before launch, not after the first incident; ownership with no name attached is the failure mode that lets the eval set rot and the dashboards go unwatched. The framing that lands with most leadership is the fraud-detection analogy: no one expects to build a fraud model once and walk away, and an LLM feature is the same kind of object. Budget for the operating cost up front and the work stays calm; omit it and you will pay it anyway, in incidents, at a worse time and a higher price.

2.7 Locating yourself: a self-assessment

The maturity model is only useful if you can place yourself on it honestly. The following self-assessment is deliberately blunt — yes/no, no middle option, so that comfortable half-answers are not available. Answer for the system as it actually is today, not as you intend it to become.

Takeaway

Production Mindset Self-Assessment. Answer yes or no, honestly, for one real LLM feature you own.

Can you say, right now, exactly which prompt text is running in production?
Is that prompt under version control, with a history of who changed it and why?
Do you have a fixed set of representative inputs you can run a prompt against?
When you change a prompt, do you score it against that set before shipping?
Does something automatically block a change that regresses a case you already cared about?
Is real production traffic logged with its prompt version, model version, cost, and latency?
Is any of that production traffic sampled and scored for quality on an ongoing basis?
Would you detect a slow quality decline from drift before a customer reported it?
Is there a single named person accountable for this feature’s quality and maintenance?
Does your budget for this feature include ongoing evaluation and monitoring, not just the initial build?

Scoring. Count your yes answers. 0–2: Level 0 — prompts as scattered strings; your highest-leverage next move is question 3, building a small eval set. 3–4: Level 1 — tracked but unmeasured; cross to Level 2 by making question 4 a habit. 5–6: Level 2 — tested but not gated; automate the gate (question 5) and start logging production (question 6). 7–8: Level 3 — gated and observed; the remaining work is organisational. 9–10: Level 4 — a governed lifecycle; your job now is to keep the loop alive, because maturity decays without tending. The lowest-numbered question you answered no to is, in almost every case, exactly where to start.

Chapter summary

The tweak-and-hope loop accumulates untraceable regressions because its only measurement is one human checking one output; it gets worse as it ages.
Evaluation-driven development replaces it with a loop — change, eval gate, traced production traffic, scored regression detection, next change — that gets better as it ages, because every incident becomes a permanent test.
An LLM feature is never “done”: non-determinism, drift, and a rising quality bar mean it must be operated and maintained, like a fraud-detection model rather than a button on a form.
A production LLM system has four properties a prototype lacks: a reproducible evaluation gate, versioning, observability, and defined ownership — the minimum structure under which the loop can run.
This is the same discipline the profession already applied to application code (version control, review, CI) and to ML models (MLOps); extending it to prompts is the unsurprising next step.
The maturity model runs from Level 0 (scattered strings) to Level 4 (governed lifecycle); the most valuable single jump is Level 1 to 2, the introduction of measurement, and the self-assessment locates where to start.

References

Sculley, D., Gary Holt, Daniel Golovin, Eugene Davydov, Todd Phillips, Dietmar Ebner, Vinay Chaudhary, Michael Young, Jean-François Crespo, and Dan Dennison. 2015. ‘Hidden Technical Debt in Machine Learning Systems’. In Advances in Neural Information Processing Systems 28 (NIPS 2015), 2503–11. https://papers.nips.cc/paper/5656-hidden-technical-debt-in-machine-learning-systems.

That is the end of the free preview

The rest of the book turns the mindset into practice: prompt architecture and patterns, versioning, evaluation frameworks, regression testing, cost and latency, reliability, observability, chains and agents, security and prompt injection, and governance, plus a pattern library and ready-to-use checklists.

Buy the full book · £5.99 See all chapters Prefer Kindle? Buy the Kindle edition on Amazon. Read online on any device, or download the PDF and EPUB. One-time payment, no subscription.