Home › Study guides › CCAR-P › Domain 6 › Lesson 6.5

CCAR-P · Domain 6 · 14% of the exam · Lesson 6.5 · 22 min read

Supporting the lifecycle: phase gates, handoff and owned iteration

How an architect carries a Claude solution from discovery to iteration: exit tests per phase, staged rollout, named owners, eval-gated change and retirement.

Written against objective 6.5 of the official CCAR-P exam guide (Version 1.0, effective July 2026). An independent resource, not affiliated with Anthropic; the practice questions are written from scratch.

6.5.1 Why the last dealer chatbot died of neglect

Brenmark Motors sells its cars through 900 franchised dealerships, whose service advisers, warranty administrators and parts staff send about 2,000 questions a day to Brenmark's dealer-support contact centre. Is a failed water pump on a 26-month-old car still under warranty? Which technical service bulletin (TSB, the manufacturer's approved fix for a known fault) covers a rattling sunroof? Has part BR-44120 been superseded? The answers sit in the warranty manual, about 3,000 bulletins and the parts catalogue, yet a dealer waits hours for a reply while the customer's car waits on the lift.

Brenmark has tried to fix this before. Three years ago a project team launched a dealer FAQ chatbot and moved on, and nobody was named to own it. New bulletins were never loaded, a warranty rule change never reached its answers, and complaints went to a mailbox nobody read. Within a year dealers had stopped trusting it. Nothing crashed. It decayed.

Bronwen, the solution architect for Brenmark's new Claude-based dealer-support assistant, knows the prototype is the easy part; the year after launch is the hard one. Lifecycle support is the architect's work of carrying a solution through five phases: discovery, design, handoff, monitoring and iteration. We will follow her assistant through its first year, up to a model upgrade and a new market.

6.5.2 Five phases, each with an exit test

Here is the belief that trips up architects new to AI systems: the lifecycle is a straight line that ends at go-live. For a conventional system, launch is when change slows down; for a Claude solution it is when change speeds up. Dealers ask new questions, bulletins arrive weekly, models are deprecated and the business enters new markets. So the five phases form a line that ends in a loop, and each closes on an exit test: the evidence that must exist before work moves on.

Memorise the shape: a role that shrinks from lead to consulted, and exit tests made of evidence.

Notice where the running team first appears: in discovery's exit test, months before handoff. Dealer-support IT, the team led by Hamish that will run the assistant, sits in the discovery interviews and design reviews. A relay works the same way: the next runner is already moving when the baton arrives, and a baton handed to someone standing still gets dropped.

Exit tests also answer calendar pressure. Constance, the aftersales director who sponsors the project, wants the assistant live for the dealer conference in May. With an exit test, the choice becomes scope or date: launch in May for warranty and bulletin questions, and add parts questions once they pass their eval. Anthropic's advice to make success criteria specific and measurable is what makes such a test checkable.

Discovery's 400 questions become design's eval suite: real questions with agreed answers, against which every version of the assistant is scored. Passing it at an agreed threshold is the eval gate, and no version reaches dealers without it. The design exit also asks whether this is the simplest design that meets the criteria; Anthropic advises adding complexity only when it demonstrably improves outcomes. Brenmark's design is retrieval over the manual and bulletins, one read-only parts tool and a hand-off to the contact centre. Claim approvals stay with the warranty team.

6.5.3 Pilot to production: stages, gates and signatures

"The pilot dealers love it. Can we switch it on for all 900 next Monday?" Constance asks three weeks into the pilot. The pilot proved something narrower: thirty keen volunteer dealerships in one region asked questions much like those in the eval suite. Nine hundred dealerships bring unfamiliar questions, less patient users, Monday-morning peaks, and a contact centre that absorbs every failure.

The answer is a staged rollout: widen the audience in steps, and open each step with a go/no-go gate, a decision whose criteria were agreed before the stage started and which named people sign.

Stage 0 puts the assistant in front of experts first: contact-centre agents know the right answers, so every draft they correct is a graded sample of live traffic. Here is the gate Bronwen wrote before stage 1 began, for the move to stage 2. Look at three things: every criterion is a number or a yes-or-no fact, two failures stop the rollout and send it back a stage whatever the averages say, and each signature covers a different risk.

Gate 1 to 2: widen from 30 pilot dealerships to 150 in three regions. Decision at the end of pilot week 6.
GO requires all of: warranty and bulletin answers correct in at least 97% of the 400-question eval suite and in at least 97% of 200 live answers sampled and graded by warranty administrators; hand-offs to the contact centre correct in at least 95% of sampled cases; no open severity-1 incident; cost per resolved question under $0.60; dealer-support IT on-call rota live; rollback to "contact centre only" rehearsed once.
STOP and roll back to the previous stage, whatever the averages, on: any answer that approves or denies a warranty claim; any wrong answer about a safety recall.
Signs: Constance (aftersales director) for value and dealer experience; Hamish (dealer-support IT) for operational readiness; the warranty compliance lead for warranty content; information security for the parts-catalogue integration. Bronwen presents the evidence and recommends.
If NO-GO: stay at 30 dealerships, fix the named gap, run the gate again. The pilot never widens on a date alone.

Bronwen recommends on the evidence but signs nothing herself: each risk is signed for by the person who will carry it. A launch to everyone at once suits only changes that are low-risk and quick to reverse, such as an internal tool for one small team.

Notice what the gate does not demand: perfection. Zero tolerance is kept for two named harms; everything else gets a threshold. Success criteria must be achievable as well as measurable, and a bar of "no errors at all" is never met by a probabilistic system, so in practice the calendar overrides it. The 95% hand-off line is the escalation-accuracy target Anthropic's guide to support assistants suggests.

6.5.4 Handoff: every asset gets a named owner

Here is the failure this objective exists to prevent: the project team disbands after launch, and nobody owns quality. It rarely looks like negligence. Projects are funded to launch, and afterwards the assistant belongs to "IT" or "the business", which means nobody. A Claude solution also has more parts that decay than a conventional application: prompts, an eval suite and its thresholds, retrieved content, integrations, a pinned model and a token budget. The pinned model is one fixed model version, and its provider will one day retire it. The old chatbot let three of these parts rot at once.

The remedy is a responsibility matrix, often drawn as a RACI chart (responsible, accountable, consulted, informed): every asset, the one accountable person who has accepted it, and who else is involved. Bronwen drafts Brenmark's during design and finalises it before stage 2.

Three rules make the matrix work. First, the owner is one person in a named role, never a team, a mailbox or "the vendor"; when that person leaves, the role is reassigned, not dropped. Second, content has two owners: warranty operations owns what is TRUE, and IT owns the job that loads it. The old chatbot failed at exactly that seam.

Third, the budget is an asset too. An organisation admin gives the assistant its own production workspace, the Claude Console's unit for grouping API keys, limits and usage. It carries the monthly spend limit and alert thresholds the budget owner chose. Limits cannot be set on the Default Workspace, one more reason each production use case gets its own.

A handoff needs a meeting and a documentation package, but it is neither. It transfers accountability, and only the receiving team doing the work proves that. After stage 2 Bronwen gives Hamish's team six weeks of hypercare, a period of extra support right after go-live. Her being on call is support, not proof. The proof is behavioural: without her help, the team ships one change through the eval gate and runs one incident drill, rolling back to "contact centre only" and restoring. Anything that needed her goes into the documentation and the matrix, and that test runs again.

6.5.5 Monitoring into iteration: one backlog, one gate

After launch, signals arrive from everywhere: graded samples of live answers, dealers' thumbs-down, the contact centre's hand-off reasons, incidents, spend alerts and notices from Anthropic. Without a route from signal to change, they pile up unseen, as with the old chatbot. Or people fix things directly in production: a warranty administrator pastes a rule into the system prompt after one complaint, and breaks answers nobody was looking at.

Brenmark's rule is one backlog and one gate. Every signal becomes an item in a single backlog, triaged weekly by the product owner. Every quality item is first written as an eval case, a real question with its agreed answer, and added to the suite. And every change, whether to the prompt, the content, a tool or the model, must pass the full regression suite before it reaches dealers. That is the whole eval suite rerun on every change, to catch anything that used to work and now fails.

In month 8 the loop gets its first real test. A dealer is told that a hybrid battery is covered for eight years; a newer bulletin extended cover to ten years for one model line, but retrieval ranked the older one first. The operations engineer adds a supersession field to the bulletin index, and the QA lead adds 25 superseded-bulletin questions to the suite. A proposed one-line prompt patch, "hybrid batteries are covered for ten years", fails the suite at once: it is wrong for every other model line.

In month 9 comes a notice. Anthropic deprecates Claude Sonnet 4.5, the model the assistant launched on. A deprecated model still works but is no longer recommended; it has a retirement date, here about two months out, and a named replacement, Claude Sonnet 5.5. Anthropic gives at least 60 days' notice before retiring a publicly released model, and after retirement every request to it fails. The operations engineer who owns the model watches the deprecations page, so the notice is a backlog item that week. The Console's usage export, broken down by API key and model, finds every caller, including a forgotten staging key.

The migration guide lists what changes. Thinking, the reasoning the model does before it answers, billed as output tokens, now runs on requests that never asked for it. The same text makes about 30% more tokens than on Sonnet 4.5. And assistant prefills, where your code writes the first words of Claude's reply, now return an error and must be replaced. The QA lead runs the full suite on both models, the budget owner re-baselines cost and the spend limit, and the new model goes out through the same stages, weeks before retirement.

6.5.6 When the business moves: rediscover or retire

Some signals are not defects. In month 11 Brenmark enters a new country: 120 dealerships, bulletins in a second language, and warranty terms shaped by local consumer law the home manual never mentions. Hamish's team proposes translating the system prompt and loading the new bulletins, which is tempting because everything else works. But the eval suite has no questions in the new language, and nobody local has said what a right answer is. That is a discovery gap, not a prompt change.

So the lifecycle loops back, with Bronwen leading again. She runs a scoped rediscovery: only what changed, but done properly. That means new stakeholders such as the local importer's warranty team and legal counsel, new data and its owners, and success criteria and eval cases for the new market. Then come design changes, new rows in the matrix, and the same staged gates. Skip this and the missing requirements arrive later as incidents.

Retirement deserves the same care. Late in the year Brenmark's new dealer portal starts showing live parts stock, so parts-availability questions leave the assistant. The product owner proposes the change and Constance signs it. Bronwen's checklist: announce a date and redirect dealers; remove the tool and revoke its credential; make those eval cases expect a pointer to the portal; keep logs for their retention period; record the decision. Retiring a whole use case ends by archiving its workspace, which keeps historical data for reporting, deactivates the workspace and archives every API key created for it. Archiving cannot be undone.

Memorise the four triggers and where each one sends you.

6.5.7 The exam traps

Every trap here leaves a transition without evidence or without an owner.

• ✗ Letting the project team disband at launch with no named owners. ✓ Name one accountable owner per asset before launch; every asset without one decays.
• ✗ Calling the handoff done after a document and a meeting, with the architect on call as the safety net. ✓ Prove it: the running team ships a change and runs an incident drill without the architect. Hypercare supports that period; it is not the proof.
• ✗ Widening from pilot to everyone on a date because the pilot "went well". ✓ Stage the rollout through go/no-go criteria agreed in advance and signed by each risk owner.
• ✗ Patching the live prompt after an incident. ✓ Write the failure as an eval case, fix the cause, and release through the full regression suite.
• ✗ Leaving the model alone until retirement, or swapping it because the new one is "better". ✓ A named owner watches deprecations and migrates on evals, staged, well before retirement.
• ✗ Entering a new market by translating the prompt. ✓ Rerun discovery for what changed, then extend the design, evals and matrix.

6.5.8 Put it together: run one change through the lifecycle

You now have every piece: exit tests, signed rollout gates, a proven handoff, one backlog with one gate, and the triggers to rediscover or retire. To make it stick, run one change through the gate yourself and watch an ungated one break things.

Each phase goes deeper elsewhere in the guide. Structured discovery (6.1) takes the first phase from interviews to a requirements canvas, and documentation (6.4) fills the handoff package. Monitoring (4.6) builds the signals your backlog needs, and A/B testing (4.3) proves a gated change on live traffic.

Key takeaways

• ✓ A Claude solution changes fastest after launch, so its lifecycle is a line (discovery, design, handoff) that ends in a loop (monitoring, iteration).
• ✓ Each phase ends on an exit test of evidence, and the architect's role shrinks from lead to consulted while the running team joins in discovery.
• ✓ Pilots widen in stages through go/no-go gates agreed in advance and signed by the sponsor, the running team and each risk owner.
• ✓ Handoff transfers accountability: every asset has one named owner, and the running team proves it by changing and recovering the system alone.
• ✓ Every signal lands in one owned backlog, and every change, model upgrades included, passes the full regression suite before a staged release.
• ✓ A change of scope reopens discovery, and a use case whose value or owner is gone is retired on a plan: announce, redirect, revoke, archive.

Check your understanding

4 questions written for this lesson, then one from the CCAR-P question bank on the same topic. Every answer option is explained, including the ones you did not pick. Nothing is stored.

Sources

• Model deprecations
• Migrating to Claude Sonnet 5.5
• Define success criteria and build evaluations
• Workspaces
• Customer support agent
• Building effective agents
• The CCAR-P exam guide, summarised