Prompt Management: A Registry, Not a Google Doc

A customer-facing summary comes out wrong on a Tuesday. Someone asks which prompt produced it. The engineer points at a file in the repository. The growth lead says that one was replaced weeks ago and the good version is in the shared doc. A third person has a better version in their chat history, and that is the one quietly pasted into the production notebook.

All three are telling the truth. That is the problem.

The working prompt lives in six places

Count them on any team that has shipped AI features for a year. A notebook where the prompt was developed. A document where it was pasted for review. A chat window where it was refined. Two or three codebases holding copies that diverged the first time someone fixed a bug locally. Each copy drifts, and none records the model it was tuned against.

Nobody did anything wrong. Prompts are text, text is easy to copy, and copying is faster than opening a pull request. The consequence is that when quality moves you cannot say what changed, because the thing that changed was never an artefact.

Run this check in ten minutes. Take the prompt behind your most important AI feature and ask three people to send you the current version. Compare them character by character. Most teams find at least two differences, and at least one that changes behaviour.

A registry holds far more than the text

The text is the least interesting part. A usable registry record carries:

The prompt and a version identifier. A content hash or a semantic version, immutable once released.

The model family and tier it was tuned against, with its parameters: temperature, maximum tokens, stop sequences, response format.

The eval set it passes, and the scores at release, so a later regression has a baseline.

The owner and the approver, by name, not by team.

Its scope: which tools it may call, which sources it may read, which systems it may write to.

That last field turns a filing cabinet into a control. A prompt that can trigger an outbound email sits in a different risk class from one that summarises a document, and the registry is where that boundary is written down and enforced rather than assumed. It is the approval gate argument applied one layer down.

Reviewing a prompt change is not reading the diff

A reviewer who only reads the added lines will approve most of the damage. Four things deserve attention.

What was removed. Prompts accrete constraints, and the risky change is the deletion of a rule somebody added after an incident whose reason was never recorded. Require the description to state why each removed constraint is no longer needed.

The eval delta, case by case. An overall pass rate that improves while three previously passing cases regress is not an improvement, it is a trade, and the reviewer has to accept it knowingly rather than read one number.

Rendered outputs, side by side, on five fixed inputs. Described behaviour is not reviewable. Actual text is.

Scope changes: a new variable, a new tool, a new data source. These change what the prompt can do, not just what it says.

The commit message carries more weight here than in code, because intent is invisible in prose. "Tighten summary" tells a future reader nothing. "Removed the two-sentence limit because support answers were truncating mid-procedure" is a record that outlives its author.

A prompt is coupled to a model, and models get deprecated

Prompts are tuned against one model's behaviour: how literally it obeys a negative instruction, how it handles a long list of rules, how reliably it produces valid JSON. Move the same text to another family and output shifts, usually in small ways that survive a glance and fail an eval.

So pin the model and version in the registry entry rather than in a configuration file nobody reads. When a provider announces a deprecation, normally with a few months of notice, the registry is the query that lists which prompts are affected and who owns them. Without it the migration begins with a code search and ends with a surprise in production.

Treat that migration as re-tuning, not find and replace. Run the eval set against the new model with the existing prompt, measure the drop, adjust, re-review, release under a new version. Budget half a day per important prompt and more for anything depending on strict output structure. The gateway that makes the switch operational is the routing layer.

Rollback is a pointer change or it does not exist

The registry should serve prompts to the application by reference. The application asks for "summariser, production" and receives whatever version that pointer currently names. Rollback is then moving the pointer: seconds, no deploy, no release train.

Applications that inline the prompt string in source make rollback a deployment, minutes at best and blocked by whatever else is queued. That is the difference between reverting a bad prompt during the customer call and reverting it tomorrow.

Keep released versions immutable and have every logged output reference the version that produced it. Without that reference an incident review is guesswork about which text was live at 14:20.

Variables are where user text becomes instructions

Templating fills slots at runtime with retrieved passages, CRM fields, inbound email bodies, form input. That is the injection surface: untrusted text lands inside the same string as your instructions, and models have no reliable way to tell the two apart.

The mitigations are partial and worth having anyway. Place untrusted content inside clearly marked delimiters and state that everything within them is data, never instruction. Strip those delimiters from injected values so the content cannot close its own fence. Keep tool scope minimal, so a successful injection has nowhere useful to go. And validate output structure before acting on it: an injection that produces odd prose is noise, one that emits a tool call is an incident.

A registry helps here by holding the variable schema: name, type, source, and whether the source is trusted. A reviewer can then see at a glance when a change has widened the untrusted surface.

What a registry will not tell you

It answers which version, which model, which parameters, who approved it and what it may touch. It says nothing about whether the prompt is any good. That is the job of an eval set, and a registry without one is a well-organised opinion.

It also adds friction to iteration, which is a real cost. Someone tuning a prompt tries thirty variants in an afternoon, and making each a pull request would end the work. Keep an unregistered sandbox for exploration and draw the line at dependency: the moment anything other than the author relies on a prompt, it gets registered.

And registries rot the way wikis do. Fifty entries, twelve live, nobody sure which. Fix that mechanically rather than with a tidying session: have the runtime report which versions it actually served in the last thirty days, mark anything unserved dormant, and delete dormant entries after a quarter.

Frequently asked questions

What is a prompt registry?

A prompt registry is a versioned store that holds each production prompt as an artefact rather than as text in a document. Each entry carries the prompt, an immutable version identifier, the model and parameters it was tuned against, the eval set it passes with its scores, a named owner and approver, and the scope of tools and data it may use. Applications request prompts by reference, which is what makes rollback and auditing possible.

Why version prompts if they are just text?

Because output quality moves when the text moves, and without versions you cannot connect the two. A prompt change is a behaviour change in a production system, so it needs what a code change gets: a diff, a reviewer, a record of intent, a release, and a way back. Logged outputs should reference the prompt version that produced them, otherwise an incident review cannot establish what was live.

How should a prompt change be reviewed?

Look first at what was removed, since deleted constraints are usually the ones added after an earlier failure. Read the eval results case by case rather than as a single score, because an improved average can hide three regressions. Read rendered outputs side by side on fixed inputs. Then check whether the change widened scope: new variables, new tools, new data sources.

What happens to prompts when a provider deprecates a model?

They stop being tuned for anything. Prompts are shaped around one model's habits, so moving to a new family changes output in ways that pass casual inspection and fail evaluation. Pin the model version in the registry entry so a deprecation notice becomes a query listing every affected prompt and owner, then treat each migration as re-tuning against the eval set rather than a configuration change.