Every Model Release Is a Change Ticket

A new model version changes what your system does, whether it arrives on the vendor's schedule or yours. Give it what every other production change gets: a pin, a frozen test built from your own work, an approver, and a record.

Three frontier models shipped in under two weeks. Anthropic released Claude Opus 5.5 on September 22 and Claude Sonnet 5.5 on September 28, and OpenAI published GPT-6.1 Sol, an upgrade to GPT-6 Sol, on September 29. Each launch page leads with a cost claim, and each claim is measured in a different unit. None of them is measured on your work.

My rule for a regulated shop is flat: a model version change is a production change, and it gets a ticket like one. Same input, new model, different output, and no commit lands in your repository for anyone to review. I've argued before that model selection is routing and capacity planning, and that a floating alias has no place on a critical path. This piece is about what comes after the pin: the change itself, what triggers it, how you test it, and what record you keep.

Three launches, three different cost units

Read the three pages side by side and the numbers stop lining up.

• Opus 5.5. Anthropic's page says input and output tokens are $4 and $20 per million, "20% less than Opus 5." The same page says that at default settings the model "will cost 40% less than Opus 5 on typical workloads." Two numbers, two units. The gap between them is tokens the new model doesn't spend.
• Sonnet 5.5. Anthropic prices it the same as Sonnet 5, $2 and $10 per million, "but it typically needs far fewer tokens to do the same work." In Anthropic's testing it "costs up to 30% less per task." Same price per token, cheaper per task, and "up to" is doing real work in that sentence.
• GPT-6.1 Sol. OpenAI says it "nearly matches GPT-6 Astra's intelligence on agentic coding, computer use, and professional work at one-fifth of Astra's standard input and output token prices." That's a per-token price, compared against a different and pricier model, the one OpenAI calls its most intelligent.

None of those is wrong, and none of them is your number. "Typical workloads" are Anthropic's. The tests behind them are the vendor's. The figure I'd put in front of a CFO is cost per accepted task on our own assignments: everything spent on a piece of work, including the runs that failed, divided by the results that met the bar.

Effort makes this sharper. Anthropic's models overview lists the default effort as medium for Opus 5.5 and high for Sonnet 5.5. Its Sonnet page says lower settings answer faster and use fewer tokens, and that on several benchmarks Sonnet 5.5 at Low or Medium effort beats Sonnet 5's best score "for about a tenth of the cost per task." Code that never sets effort inherits whatever default ships with the model. Swap one of those models for the other and you've changed the effort level too, and nobody wrote that down.

Good changes are still changes

The case for treating a model like any other dependency isn't that new models are worse. Often they're better. Anthropic says Opus 5.5 is "much less likely than recent models to take hard-to-reverse actions or act outside the boundaries it's been given." For anyone running agents, that's welcome news. It's also a behavior change in exactly the part of the system your controls were tuned around: approval gates, tool scopes, and the alerts that fire when an agent reaches for something it shouldn't. If the old model regularly tripped an approval gate and the new one never does, you want to know whether the gate still gets exercised, or whether your thresholds were tuned to the old model's habits.

The interface moves too. Anthropic's deprecation page says that setting temperature, top_p, or top_k to a non-default value "Returns a 400 error" on Claude 4.7 and later models. Code that sets temperature to zero for repeatability doesn't get different answers after a move across that line. It gets errors.

So a model version gets what a library upgrade or a database engine upgrade gets: a ticket, a test plan, an approver who isn't the person asking for it, a rollback target, and a record that outlives the people involved. The record is the part that's easiest to skip. It's also the only part an examiner can read.

Three ways the model changes under you

Change arrives through three doors, and only two of them knock.

1. You choose to upgrade. A release ships, someone wants it, and it goes through normal change. Opus 5.5, Sonnet 5.5 and GPT-6.1 Sol all arrive this way. The risk here is speed, because a strong launch page makes the test feel optional.
2. The vendor retires your pin. Anthropic says it gives customers with active deployments at least 60 days' notice before retiring a publicly released model. On September 30 it notified Sonnet 4.5 developers that the model retires from the Claude API on November 30, with claude-sonnet-5-5 as the recommended replacement. OpenAI's deprecation page promises generally available models at least six months' notice unless safety or compliance concerns require less, and on October 1 it announced that gpt-5.1 leaves the API on April 1, 2027, with gpt-6-sol as the recommended replacement. That's a scheduled change with a deadline you didn't set. Partner platforms add a wrinkle: Anthropic's page says Amazon Bedrock and Google Cloud set their own retirement schedules. We built AgentOS with a single governed AWS Bedrock egress, and that design carries a general lesson: when every call leaves through one managed provider, the retirement calendar that binds you is the provider's, not the model maker's. Track the date for the platform you actually call.
3. Something changes behind a name you kept. OpenAI's page for chat-latest says it "points to the latest Instant model currently used in ChatGPT" and that "The underlying model snapshot will be regularly updated." The same page recommends a different model for production API use. On the Claude API, aliases for models before the 4.6 generation, such as claude-sonnet-4-5, point to the most recent dated snapshot for that minor version.

The third door has a quieter version that pinning doesn't close. Anthropic's documentation is unusually direct about it. It says Anthropic "does not update the weights or configuration of an existing model ID," and that updated versions ship under new IDs, which is the right design. It then says the serving infrastructure around the model, including the request router, safety classifiers, and sampling logic, can change over time, and that "Occasionally, infrastructure updates produce minor differences in observable behavior even when the model ID and weights have not changed."

A pin is a guarantee about weights, not about behavior. That's why the test can't run only when a release ships. It runs on a calendar too.

Freeze a test built from your own work

Benchmarks are someone else's assignments. Vendor cost claims are someone else's workload. The test that tells you whether to move is built from work your teams already do every week, frozen so that the only thing that changes between runs is the model.

Pick a set of recurring assignments where a wrong answer would cost you something: a policy summary, a contract clause review, a reconciliation exception write-up, a code change against a known repository. A couple dozen is enough to start. For each one, freeze five things:

• Inputs. The exact files and records, masked or synthetic where the real ones are regulated data.
• Instructions. The prompt and system instructions, verbatim, under version control.
• Tools. The tool definitions and permissions the model gets. A model with a different tool surface is taking a different test.
• Settings. Effort, output limits, and every other parameter, set explicitly rather than inherited from a default that can change with the model.
• Definition of done. Written by the business owner before the first run, in sentences a reviewer can check without asking the author: "The summary cites the policy section it relies on, states the dollar threshold exactly, and routes any exception to a named reviewer." If the owner can't write that sentence, the assignment isn't ready to delegate to any model.

Then hold it still. If you change the test, that's a new baseline, and the model you run today reruns against it too. Otherwise you're comparing two models on two different exams and calling the difference progress.

Anthropic's Sonnet 5.5 page carries a quote from Curtis Allen, a principal engineer at Slack, that shows the practice working: "Without changing any of our prompts, Claude Sonnet 5.5 did better than Sonnet 5 on almost all of our offline Slackbot evals, in fewer steps and with about 14% fewer output tokens." Unchanged prompts, their own evals, a new model. For a risk team, the important word in that sentence is "almost." The scorecard exists to name the remaining cases before a customer does.

The scorecard is the change record

Every run of the frozen set writes one row per assignment, and the rows together are the change record. I'd want these columns at minimum:

• the resolved model ID, the platform, the effort setting, and the date
• pass or fail against the definition of done, and who or what decided
• the rework a person needed to make the output acceptable
• tokens in and out, and cost per accepted task
• the failure type, from a typed register rather than one bucket, because "the model was bad" is not a root cause

Under the rows sits one decision: promote, hold, or promote with limits. The approver is someone other than the person who wants the upgrade. The rollback target is the prior pinned ID, which stays callable until its own retirement date, so keep a fallback wired in and know that date before you need it.

The Federal Reserve's SR 26-2, issued April 17 with the FDIC and with the OCC as Bulletin 2026-13, does not cover the models in this piece. It places generative and agentic AI outside its scope, the agencies say it is most relevant to banking organizations with over $30 billion in total assets, and the OCC's bulletin says it sets no enforceable standards. I borrow its shape anyway, because the logic carries.

It says "The timing, nature, and frequency of validation activities vary based on model purpose, model methodology, frequency and scope of model changes, data limitations, and other practical constraints." Applied to a vendor that ships a new model every few weeks, that logic argues for testing more often, not less. It says validation "can reveal performance deterioration over time and inform judgments about acceptable performance ranges," which is what a frozen set with history gives you. It also covers the case where you can't wait. When an urgent business need means using a model before validation is complete, sound practice involves "greater attention to the model's limitations," informing stakeholders, and controls such as "placing limits on model use or more closely monitoring its performance." Borrowed for a language model, that's the emergency path for a retirement date that crept up on you, written down before you need it.

An auditor asks four things of any change: what changed, when, who signed off, and how you knew it still worked. A pinned ID with a scorecard row answers all four in one artifact. A vendor's launch page answers none of them.

Monday: open the tickets you already owe

Start with an inventory, because some of these changes already have deadlines.

1. List every production call path and the exact model string it sends. Mark each one as a pinned ID, an alias, or a deployment name. On Microsoft Foundry, Anthropic's docs note, the deployment name is what you send, so record what it resolves to.
2. Look up the retirement date for each on the platform you actually call. Anything under 90 days is an open change ticket today. Sonnet 4.5 on the Claude API is one, due November 30.
3. Search the code for temperature, top_p and top_k before any move across model generations, so the migration fails in a test instead of in production.
4. Pick one workflow and sit with its business owner long enough to write five definitions of done. Freeze the inputs, run the model you have pinned today, and record the baseline row by row.
5. Set two triggers on that set: every release you're considering, and a monthly rerun of the model you already run, to catch the changes nobody announces.

The next release will ship before you finish. That's fine. Its ticket starts with the baseline you record this week.