Welcome to Certsleader, your ultimate source for top-quality CCAR-P dumps tailored for Anthropic CCAR-P exam. Our comprehensive resources are designed to help you excel in your exam preparations and achieve your certification goals. Whether you are a beginner looking to start a career in Anthropic or an experienced professional seeking to advance your skills, Certsleader has the right tools to support your journey.
Why Certsleader is Your Best Choice:
Expertly Curated Content: Our study materials are meticulously crafted and verified by a panel of IT experts, ensuring they are accurate, relevant, and up-to-date with the latest industry standards.
Real Exam Questions: Our resources include authentic CCAR-P exam questions and detailed answers, allowing you to familiarize yourself with the exam format and question types, and practice effectively.
Comprehensive Study Guides: Each certification guide is designed to provide in-depth knowledge and understanding of the subject matter, helping you to grasp even the most complex concepts.
Convenient Access: Our study materials are available in easy-to-download PDF files, making it convenient for you to study anytime, anywhere, and on any device.
Guaranteed Success
At Certsleader, we are committed to your success. Our practice questions answers are designed to improve your knowledge and help you pass your exams on the first attempt with high scores. In the rare event that you do not succeed, we offer a full refund, taking responsibility for your satisfaction.
Start Your Journey with Certsleader
Join thousands of satisfied learners who have successfully passed their certification exams with Certsleader. Explore our study materials, download your PDF files, and take the first step towards a rewarding IT career today.
Anthropic CCAR-P Sample Questions
Question # 1
A team must reduce the cost of a production Claude workload by 40% without a material quality regression.Select TWO approaches that are most consistent with disciplined optimization.
A. Profile where tokens are actually consumed, then cache stable prefixes and eliminate redundant context that adds no measured value. B. Reduce the evaluation set size so that testing costs less. C. Disable logging and tracing to reduce infrastructure spend. D. Route simple requests to a smaller model and reserve the larger model for requests that evaluation shows genuinely require it. E. Cap all responses at 100 tokens regardless of the request.
Answer: A,D Explanation
Why this is correct. Both are evidence-driven and preserve the ability to detect regression. Profiling first (A) is the discipline that separates optimization from guesswork — you find that a repeated prefix or an unused retrieved field dominates the bill, and you remove or cache it with no information loss. Tiered routing (D)
exploits the fact that request difficulty is not uniform: most traffic is straightforward and does not need the largest model, and evaluation tells you where the boundary sits rather than intuition. Together they typically deliver large savings while leaving quality measurable.
Why B is wrong. Shrinking the evaluation set reduces your ability to detect the quality regression the requirement forbids. It also saves trivial money, since evaluation is a small fraction of production spend. This is cutting the instrument panel to save fuel.
Why C is wrong. Removing observability destroys the diagnostic capability you most need while making cost and latency changes. Logging is also a minor line item next to inference.
Why E is wrong. A blanket 100-token cap is an indiscriminate quality cut that truncates any response legitimately requiring more, directly violating the no-material-regression constraint.
Blueprint objective: Optimize token usage, latency, and cost-performance trade-offs.
Question # 2
A summarization system occasionally introduces facts that do not appear in the source document. Which combination of measures most directly reduces this failure?
A. Increase temperature to encourage more creative summaries. B. Summarize the summary a second time to remove errors. C. Instruct the model to base every statement on the source and to state when information is absent; addan automated faithfulness check that verifies claims and entities against the source; and route lowscoring outputs to review. D. Increase the maximum output length so the model has room to be accurate.
Answer: C
Explanation
Why this is correct. Fabrication is best attacked in layers. At the prompt level, explicitly grounding the task and — critically — giving the model a legitimate way to say information is absent removes the pressure to fill gaps, since an instruction to always produce a complete summary implicitly rewards invention. At the verification
level, an automated faithfulness check that traces claims and named entities back to the source detects what the prompt did not prevent. At the operational level, routing low-scoring outputs to human review contains the residual. Prevention, detection, and containment together, which is the standard pattern for a probabilistic
failure mode.
Why A is wrong. Higher temperature increases sampling diversity, which increases the likelihood of unsupported content. This makes the problem worse, and "creative" is the opposite of what a faithful summary requires.
Why D is wrong. Fabrication is not caused by insufficient output space. More room to write is more room to invent.
Why B is wrong. A second summarization pass has no access to the source at that point, so it cannot distinguish invented content from real content — it will happily preserve and even reinforce the fabrication while shortening the text.
Blueprint objective: Diagnose system issues (prompt failure, hallucinations, model mismatch); implement guardrails.
Question # 3
A model-based grader (LLM-as-judge) is used to score answer quality at scale. Which practice most improves the trustworthiness of its scores?
A. Using the same model and prompt that generated the answers, for consistency. B. Defining an explicit rubric with concrete criteria and score anchors, and periodically measuring the grader's agreement with human raters on a held-out subset. C. Averaging scores across many runs at high temperature to reduce variance. D. Asking the grader for a single overall score from 1 to 10 with no further guidance.
Answer: B Explanation
Why this is correct. A model-based grader is itself a measurement instrument, and instruments require specification and calibration. An explicit rubric with concrete criteria and score anchors — what distinguishes a 4 from a 5, with examples — converts a vague impression into a repeatable judgment and makes disagreements
inspectable. Periodic agreement measurement against human raters is the calibration step: it tells you whether the grader's numbers actually track human judgment, and it catches drift when prompts or data change.
Without it, you have precise-looking scores of unknown validity.
Why A is wrong. Using the generating model and prompt as the judge introduces self-preference bias; models tend to rate their own outputs favourably, and shared prompt framing means shared blind spots. Independence between generator and grader is desirable.
Why D is wrong. An unanchored 1–10 score is the least reliable form of model grading. The scale means something different across runs and across examples, and there is no way to audit a disagreement because no criteria were stated.
Why C is wrong. Averaging high-temperature runs reduces variance around whatever the grader's central tendency is, but it does nothing about bias. A grader that is systematically wrong will be consistently, precisely wrong.
Blueprint objective: Design evaluation datasets and test frameworks using mixed methodologies.
Question # 4
An agent completes its task successfully in 91% of evaluation runs. In the 9% of failures, the team observes no common pattern in the final output. Which diagnostic approach is most likely to identify the root cause?
A. Examine the full execution traces of failing runs — tool calls, arguments, results, and intermediate reasoning — and compare them against traces of successful runs on similar inputs. B. Replace the model with a larger one and re-measure. C. Increase the evaluation set size until a pattern emerges in the outputs. D. Re-run the failing cases with a higher temperature to see whether results improve.
Answer: A Explanation
Why this is correct. When failures share no visible pattern in the final output, the divergence happened earlier in the execution path. Traces expose where: a tool called with a malformed argument, a retrieval that returned nothing and was silently treated as empty rather than as an error, a loop that exhausted its budget, a reasoning
step that made an unwarranted assumption. Comparing failing traces against successful traces on similar inputs is the key technique, because it isolates the step at which the two diverge rather than leaving you to read traces in isolation.
Why D is wrong. Raising temperature increases randomness. If some runs then succeed, you have learned only that the failure is stochastic, not why it occurs — and you have made the system less predictable.
Why C is wrong. More examples of an unexplained failure produce more unexplained failures. Volume does not substitute for visibility into intermediate state.
Why B is wrong. Swapping the model is a shot in the dark that may move the number without explaining it. If the root cause is a malformed tool argument or an unhandled empty retrieval, a larger model may mask it intermittently — leaving a latent defect that resurfaces later.
Question # 5
A system that performed well at launch shows gradually declining user satisfaction over four months, while offline evaluation scores remain unchanged. What is the most probable explanation?
A. The evaluation set is too large to be sensitive to changes. B. Users have become less tolerant of the same quality level. C. The model has degraded over time through repeated use. D. The production input distribution has drifted away from the fixed evaluation set, so the eval no longer represents real traffic.
Answer: D
Explanation
Why this is correct. The contradiction between stable offline scores and falling real-world satisfaction is the defining symptom of distribution drift. A static evaluation set measures performance on the world as it was when the set was built; production traffic moves — new products, changed customer language, seasonal patterns, new use cases users discovered on their own. The system may genuinely still be excellent at what the eval measures and poor at what users now send. The remedy is to refresh the evaluation set from recent production traffic on a schedule, and to monitor input characteristics directly so drift is visible before satisfaction falls.
Why C is wrong. Model weights do not degrade through inference. A deployed model is a fixed artifact; usage does not wear it out.
Why B is wrong. Shifting expectations are a real phenomenon over long horizons, but they are a weak explanation for a steady four-month decline and are unfalsifiable as stated. It also conveniently locates the problem entirely outside the system, which should raise suspicion.
Why A is wrong. Larger evaluation sets are generally *more* sensitive, not less. Size is not the issue; representativeness is.
Blueprint objective: Monitor system performance using logging and observability tools; design evaluation datasets.
Question # 6
A generation step costs more than budgeted. Profiling shows that 78% of input tokens come from a static instruction block and a fixed reference table sent identically on every request, and 22% from variable user content. Output tokens are modest. Which optimization has the greatest cost impact for the least quality risk?
A. Reduce the maximum output tokens. B. Compress the variable user content before sending it. C. Enable prompt caching on the static prefix so the repeated 78% is billed at the reduced cached rate rather than reprocessed on every call. D. Remove the reference table and rely on the model's general knowledge.
Answer: C
Explanation
Why this is correct. Optimization should target the largest cost component that can be reduced without losing information, and the profile hands you the answer: 78% of input tokens are byte-identical across requests.
Caching that prefix means those tokens are written once and subsequently billed at a substantially reduced rate, with no content removed and therefore no quality risk — the model sees exactly the same input it saw before. It also reduces time-to-first-token as a side benefit. The pairing of "largest share" and "identical every
time" is the signal for caching.
Why A is wrong. Output tokens are stated to be modest, so the addressable saving is small. Capping output length also risks truncating legitimate responses, which is a quality regression for a minor gain.
Why D is wrong. Deleting the reference table removes information the system was designed to use and replaces it with the model's general knowledge, which will not contain this organization's specific reference data. This trades correctness for cost — the opposite of "least quality risk."
Why B is wrong. Compressing the variable 22% attacks the smaller share, and lossy compression of user content risks discarding the specifics the request depends on.
Blueprint objective: Optimize token usage, latency, and cost-performance trade-offs.
Question # 7
A team plans to A/B test a new prompt against the current one in production. Which experimental design flaw would most undermine the result?
A. Assigning users to variants at random rather than by geography. B. Deploying the new prompt to variant B while simultaneously upgrading variant B's model and increasing its retrieval depth. C. Running the test for two weeks rather than one. D. Measuring both task success rate and cost per request.
Answer: B
Explanation
Why this is correct. Changing three variables at once destroys attribution. If variant B performs better, you cannot tell whether the prompt, the model upgrade, or the deeper retrieval is responsible — and if the changes interact, the individual effects may point in opposite directions, so you could ship a worse prompt because a
model upgrade masked it. Controlled experiments isolate one variable; if multiple changes must be evaluated, they need separate arms or a factorial design that can separate the effects. This confound is also expensive to unwind later, since the team will have learned nothing transferable.
Why A is wrong. Random assignment is the correct approach. Assigning by geography would introduce confounds — different regions differ in language, query mix, and time-of-day patterns — so A describes good practice, not a flaw.
Why C is wrong. A longer run generally improves statistical power and captures weekly seasonality. Two weeks is more defensible than one, not less.
Why D is wrong. Measuring both quality and cost is sound multi-dimensional evaluation. A prompt that raises success rate while tripling cost is a trade-off the business should see, not a design flaw.
Blueprint objective: Conduct A/B testing and iterative improvements.
Question # 8
A team wants to compare two system prompts for a summarization feature. Summaries have no single correct answer, and the qualities that matter are faithfulness to the source and usefulness to the reader. Which evaluation methodology is most appropriate?
A. A mixed approach: programmatic checks for objective properties such as length limits and absence of fabricated entities, plus model-based grading against an explicit rubric, calibrated against a humanlabelled subset. B. Exact string matching against reference summaries written by the team. C. Measure only average output length, since concise summaries are better. D. Ask the model that produced each summary to rate its own quality.
Answer: A
Explanation
Why this is correct. Open-ended generation requires layered evaluation because different qualities are best measured by different instruments. Deterministic code is perfect for objective, checkable properties — length limits, required sections, whether every named entity in the summary appears in the source — and it is fast and
free. Faithfulness and usefulness are judgment calls, so a model-based grader against an explicit rubric scales the assessment. The calibration clause is what makes the answer complete: you validate the grader against a human-labelled subset, because an uncalibrated automated judge produces confident numbers with unknown
correspondence to human opinion.
Why B is wrong. Exact string matching assumes one correct output. For summarization, a summary can be excellent and share almost no character sequences with the reference, so the metric would penalize good work and reward mimicry.
Why D is wrong. Self-rating by the generating model is systematically biased toward its own output and correlates poorly with quality. If a model could reliably detect its own faithfulness failures, it would avoid making them.
Why C is wrong. Length is a proxy that ignores content entirely. A short summary that omits the key finding is worse than a longer one that captures it, and optimizing for brevity alone will produce exactly that failure.
Blueprint objective: Design evaluation datasets and test frameworks using mixed methodologies.
Question # 9
An architect is defining evaluation metrics for a production Claude system. Select TWO statements that reflect sound evaluation design.
A. Evaluation should be performed once before launch and repeated only if users complain. B. Metrics should span quality, latency, cost, and safety, because optimizing any one alone can degrade the others. C. Safety evaluation is only necessary for consumer-facing systems. D. A single aggregate accuracy figure is sufficient if it is measured on a large enough sample. E. Metrics should be segmented by meaningful slices — such as request type, customer tier, or language — because aggregate figures conceal localized failure.
Answer: B,E Explanation
Why this is correct. Sound evaluation is multi-dimensional and segmented. Multi-dimensional (B) because these systems have coupled properties: a change that raises accuracy may double latency or cost, and a change that reduces cost may weaken safety behaviour. If you measure only one, you will optimize it into a regression
somewhere else and not notice. Segmented (E) because aggregate metrics are averages, and averages hide concentrated harm — a system at 94% overall may be at 62% for one language or one high-value customer segment, which is invisible until that segment escalates.
Why D is wrong. Sample size fixes statistical noise, not dimensional blindness. A large sample still tells you nothing about latency, cost, safety, or per-segment behaviour.
Why C is wrong. Internal systems carry safety and security risk too — data leakage across entitlements, prompt injection via ingested documents, harmful automation of a business action. The audience changes the risk profile; it does not eliminate the need to evaluate.
Why A is wrong. User complaints are the slowest and most expensive detector available, and they arrive after harm. Model behaviour, data, and usage all drift, which is why evaluation is continuous and paired with production monitoring.
A team is building the first evaluation set for a customer-email classification system before launch. Which composition is most appropriate?
A. 1,000 examples generated by a language model to cover the label space quickly. B. The 50 emails the team found most interesting during development. C. A random sample of production traffic with labels assigned by the system itself. D. A set built primarily from real historical emails labelled by domain experts, deliberately including known edge cases, ambiguous items, and the rare-but-costly categories, with class balance recorded rather than artificially equalized.
Answer: D
Explanation
Why this is correct. An evaluation set is only useful insofar as performance on it predicts performance in production, which requires real distributional characteristics and trustworthy labels. Expert-labelled historical emails give you ground truth grounded in actual customer language rather than a model's idea of it.
Deliberately including edge cases, ambiguous items, and rare-but-costly categories is what makes the set diagnostic — average accuracy on easy examples hides exactly the failures that matter. Recording rather than equalizing class balance keeps the metric interpretable: if a category is 2% of real traffic, forcing it to 14% of the
eval set produces numbers that do not correspond to anything you will experience.
Why A is wrong. Synthetic data has a role — filling gaps in rare categories, generating adversarial variants — but as the primary basis it bakes the generating model's blind spots into the yardstick. You end up measuring how well the system handles the kind of email a model imagines, and real customers do not write that way.
Why B is wrong. Fifty developer-selected examples are both too small for statistical confidence and selected by a biased process. "Interesting during development" over-represents cases the team already thought about and under-represents the ordinary traffic that dominates production.
Why C is wrong. Labelling production data with the system under test is circular: it will score near-perfectly by construction, because you have defined its own output as correct. Sampling production traffic is an excellent idea; the labels must come from an independent source.
Blueprint objective: Design evaluation datasets and test frameworks using mixed methodologies.
Question # 11
A RAG-based support assistant begins returning confident but incorrect answers immediately after a scheduled documentation refresh. The model version, prompt, temperature, and p95 latency are all unchanged. What is the most likely first place to investigate?
A. The context window size has been reduced. B. The model weights have been silently updated by the provider. C. The retrieval and indexing step is returning stale, irrelevant, or improperly parsed chunks following the refresh. D. The temperature setting has become too low, making the model overconfident.
Answer: C Explanation
Why this is correct. Diagnosis in a multi-component system starts with what changed. The one variable that moved is the document corpus, and the symptom — fluent, confident answers that are factually wrong — is the classic signature of a model faithfully summarizing bad context. A refresh can break retrieval in several ways: a
re-index that failed partway leaving a mix of old and new content, an embedding regeneration that did not complete so vectors no longer correspond to their text, a parser that mishandled a changed document format, or metadata that no longer matches. Inspecting the actual retrieved chunks for a failing query is the fastest way
to confirm.
Why B is wrong. This is not how versioned model endpoints work, and more importantly it does not explain the timing. An unrelated provider change coinciding exactly with the refresh would be a remarkable coincidence; the correlation points elsewhere.
Why D is wrong. Temperature is a configuration value that does not drift on its own, and it is stated to be unchanged. Low temperature also produces consistency, not fabrication — confidence in tone is not caused by the sampling parameter.
Why A is wrong. A shrunken context window would typically produce truncation errors or dropped content rather than confidently wrong answers, and again nothing in the scenario changed it.
Blueprint objective: Diagnose system issues (prompt failure, hallucinations, model mismatch).
Question # 12
Two autonomous systems built by different departments must collaborate: a procurement agent that can request quotes and a finance agent that can approve budget. Neither team will expose its internal tools to the other, and each must retain its own authorization boundary. Which integration approach is most appropriate?
A. Have a human manually relay messages between the two agents. B. An agent-to-agent interaction in which each system exposes a narrow, contract-defined interface for the specific collaboration, with each retaining its own authorization and audit boundary. C. Have the procurement agent call the finance system's internal APIs directly using shared credentials. D. Merge both agents into one with the union of all tools.
Answer: B Explanation
Why this is correct. The scenario states two constraints that jointly determine the answer: neither side will expose internal tools, and each must keep its own authorization boundary. Agent-to-agent interaction is designed for exactly this — autonomous systems collaborate through a narrow, explicitly contracted interface rather than by sharing internals. Each side continues to enforce its own permissions and produce its own audit trail, and the collaboration surface is small enough to review and version independently. This is the third of the three integration mechanisms in the blueprint, and the tell is peer systems with separate ownership.
Why D is wrong. Merging violates both constraints at once: it dissolves the authorization boundaries and creates a single agent holding both procurement and budget-approval powers, which is a segregation-of-duties failure that finance and audit functions exist to prevent.
Why C is wrong. Shared credentials collapse the two identities, so finance loses the ability to attribute actions and enforce its own rules. It also directly contradicts the requirement not to expose internal interfaces.
Why A is wrong. Manual relay is not an integration design; it inserts a human clerical step into a system built for automation, and it will not scale or produce a reliable audit trail.
Blueprint objective: Evaluate connection protocols and select the appropriate integration mechanism (MCP, API/CLI, agent-to-agent).
Question # 13
During a design review, a team proposes letting the agent construct and execute arbitrary SQL against the production data warehouse so it can answer any analytical question. What is the strongest architectural objection?
A. Arbitrary query execution grants unbounded read scope and resource consumption; the safer pattern is parameterized queries or curated views with enforced row-level security, column masking, and resource limits. B. SQL is an outdated interface and should be replaced with a REST API. C. The data warehouse will be too slow to respond within a conversation. D. SQL generation is a task language models cannot perform at all.
Answer: A Explanation
Why this is correct. The objection is scope, not capability. "Any analytical question" means the agent can read any table, including columns holding personal or financial data the requesting user has no right to see, and can issue a query that consumes enormous warehouse resources. Both risks are structural, not model failures — a
perfectly correct query can still be an unauthorized or ruinously expensive one. The safer pattern narrows what is reachable: expose curated views rather than raw tables, apply row-level security and column masking so the database enforces entitlements regardless of what the agent generates, use parameterized templates where
the question space is known, and impose query timeouts and cost ceilings.
Why D is wrong. Models generate SQL competently, and the scenario does not claim otherwise. Overstating the objection weakens it — the review will simply demonstrate a working query and move on.
Why C is wrong. Warehouse latency is a real design consideration and argues for async patterns or result caching, but it is a performance concern, not the strongest objection when unbounded access to sensitive data is on the table.
Why B is wrong. SQL is the native and appropriate interface to a data warehouse. Replacing it with REST does not address scope or resource control; you would simply need the same controls behind a different protocol.
Blueprint objective: Analyze authentication and authorization requirements to identify security gaps.
Question # 14
An architect must connect a Claude application to a legacy mainframe system that exposes only a batch file interface with a four-hour processing window. Business users expect conversational responses. Which integration design is most realistic?
A. Replace the mainframe before building the Claude application. B. Instruct the model to approximate mainframe data from its general knowledge when a query arrives. C. Have the agent call the mainframe synchronously during the conversation and wait for the batch result. D. Maintain a synchronized read model — periodically extracted mainframe data in a queryable store the agent reads in real time — and handle writes as asynchronous submissions with status tracking and clear user expectations about timing.
Answer: D Explanation
Why this is correct. Good integration design respects the constraints of the systems being integrated rather than wishing them away. A four-hour batch window cannot serve a conversational read, so reads are served from a synchronized store extracted on the mainframe's own schedule, giving real-time query performance at the cost of bounded staleness — a trade-off you make explicit to users. Writes cannot be made instantaneous either, so they are modelled honestly as asynchronous submissions with a tracking identifier and status visibility. Separating the read and write paths and setting accurate expectations is the mature answer.
Why C is wrong. Blocking a conversation for up to four hours is not an integration; it is a timeout. No conversational interface survives this.
Why B is wrong. Fabricating enterprise data from general knowledge is the most damaging option available.
The model has no access to this organization's mainframe records, so every answer would be invented while appearing authoritative.
Why A is wrong. Mainframe replacement is a multi-year programme unrelated to the current deliverable. An architect who can only deliver by first removing the constraint has not solved the problem. Modernization may be a valid parallel recommendation, but it is not the integration design.
Blueprint objective: Evaluate connection protocols and select the appropriate integration mechanism; manage stakeholder expectation alignment.
Question # 15
A RAG pipeline over a legal corpus returns passages that are topically related to the query but frequently miss the single most authoritative passage, which sits lower in the ranking. Retrieval recall at 50 is high; precisionat 5 is poor. Which technique most directly addresses this?
A. Increase the number of passages sent to the model from 5 to 50. B. Lower the similarity threshold so more candidates qualify. C. Add a reranking stage that scores the top 50 candidates against the query with a more precise model and passes the top 5 to generation. D. Reduce the corpus to only the most recent documents.
Answer: C Explanation
Why this is correct. The diagnosis is given precisely: high recall at 50 means the right passage *is* being retrieved, and poor precision at 5 means it is not being ranked highly enough to survive into the generation context. That is the textbook signature of a ranking problem, and reranking is the technique built for it — a twostage design where a fast retriever casts a wide net and a slower, more accurate cross-encoder reorders the candidates. You get the recall of a broad search with the precision of an expensive scorer, paying the expensive scoring cost on only 50 items rather than the whole corpus.
Why A is wrong. Sending all 50 passages floods the context with 45 mostly irrelevant documents, raising cost and latency and diluting the authoritative passage among near-misses. On a legal corpus, surrounding the correct clause with superficially similar incorrect ones is actively dangerous.
Why D is wrong. Recency is not authority, particularly in law where an older statute or precedent may be controlling. This discards potentially essential material based on a proxy that does not track the actual problem.
Why B is wrong. Lowering the threshold admits more weak candidates, which worsens precision. The system's problem is not that it retrieves too little.
Blueprint objective: Apply retrieval strategies matched to data shape and query pattern; design a RAG pipeline.
Question # 16
A fraud-review assistant currently achieves 96% accuracy with a p95 latency of 4.1 seconds by retrieving 20 documents and using an extended reasoning configuration. The business states that reviewers abandon the tool above 2 seconds, and that a 2-point accuracy drop is acceptable if it keeps reviewers in the tool. Which configuration decision is best justified?
A. Reduce retrieval to a single document, which will minimize latency. B. Reduce retrieval depth and reasoning budget to land near 2 seconds, validate that accuracy remains at or above 94% on the evaluation set, and route only low-confidence cases to the slower high-accuracy path. C. Keep the configuration and add a progress indicator so reviewers are willing to wait longer. D. Keep the current configuration, because accuracy is paramount in fraud review.
Answer: B Explanation
Why this is correct. The business has done the hard part: it has stated the trade-off explicitly, giving a latency threshold and an accuracy tolerance. The architect's job is to find the configuration that satisfies both and to verify it rather than assume it — hence tuning retrieval depth and reasoning budget toward the 2-second target and then *validating* against the evaluation set that accuracy stayed within the stated 94% floor. The tiered routing clause is what makes the answer strong: sending only low-confidence cases down the slow, thorough path preserves accuracy where it matters most while keeping the common case fast. A tool that reviewers abandon has an effective accuracy of zero, which is the reasoning behind the business's position.
Why D is wrong. It overrides an explicit business decision with a technical preference. Accuracy that is never consumed because users have left the tool is not paramount; it is unused.
Why A is wrong. Single-document retrieval is an unvalidated overcorrection that will almost certainly breach the 94% accuracy floor. The requirement was to hit 2 seconds, not to minimize latency at any cost.
Why C is wrong. A progress indicator manages perception, not latency. It may buy a little patience, but the stated threshold came from observed abandonment behaviour, and dressing up a 4-second wait does not make it a 2-second one.
Blueprint objective: Evaluate accuracy-latency trade-offs and justify configuration decisions.
Question # 17
A production agentic system serves 50,000 requests per day across multi-step tool-calling sessions. The tea can currently see only the final response and total latency. Select TWO observability capabilities that would most improve their ability to diagnose failures.
A. A daily count of total requests served. B. Aggregate CPU utilization of the application servers. C. Trace-level capture of each step in a session — tool calls, arguments, results, and intermediate model outputs — correlated by a session identifier. D. Per-step latency and token attribution, so cost and time can be traced to specific tools or reasoning steps. E. A dashboard showing average response length in characters.
Answer: C,D Explanation
Why this is correct. In agentic systems the interesting failures happen *inside* the session, not at its edges.
Step-level tracing correlated by session ID (C) is the foundational capability: without it, a wrong final answer is an unexplainable event, whereas with it you can see that the agent called the right tool with a malformed argument, or looped three times on a failing retrieval. Per-step latency and token attribution (D) is the second
half, converting aggregate cost and latency into actionable signal — you learn that one tool accounts for 70% of p95 latency, or that a single reasoning step consumes most of the token budget. Together they turn a black box into something diagnosable.
Why A is wrong. A daily request count is a volume metric. It tells you nothing about why any individual session failed.
Why E is wrong. Average response length is a weak proxy that correlates poorly with quality. Both good and bad answers come in all lengths.
Why B is wrong. CPU utilization measures infrastructure health. In an LLM system, time and cost are dominated by model inference and external tool calls, so application-server CPU is close to irrelevant for diagnosing quality or latency failures.
Blueprint objective: Analyze observability challenges and select monitoring strategies at scale.
Question # 18
A Claude agent is integrated with an internal HR system through a service account that holds broadadministrative permissions, because "the agent needs to serve every employee." End users authenticate to the chat interface, but their identity is not propagated to the HR system. What is the most significant security gap?
A. The agent operates as a confused deputy — every user's request executes with full administrative rights, so a user can retrieve data they are not entitled to see. B. The HR system may rate-limit the service account under load. C. The service account credential may expire and interrupt service. D. Service account activity will appear in logs under a single identity, complicating capacity planning.
Answer: A
Explanation
Why this is correct. This is the classic confused-deputy vulnerability. The agent is a privileged intermediary that acts on behalf of unprivileged callers without carrying their identity, so the HR system's own access controls — which presumably prevent an employee from reading a colleague's salary or performance record — are entirely
bypassed. Authorization has collapsed onto whatever the agent's prompt happens to enforce, which is a probabilistic control protecting sensitive personal data. The correct pattern is to propagate the end user's identity so the downstream system evaluates permissions per request, or at minimum to scope the agent's credential to the least privilege any user requires and enforce per-user filtering in a trusted layer.
Why C is wrong. Credential expiry is an availability and operations concern. It is real but not a security gap, and it does not expose data.
Why B is wrong. Rate limiting is a capacity concern. It may degrade the service; it does not compromise it.
Why D is wrong. Attribution loss in logs is a genuine consequence of shared service accounts — it undermines auditability — but framing it as a capacity planning issue misses the point entirely, and it is a lesser harm than the unauthorized data access in A.
Blueprint objective: Analyze authentication and authorization requirements to identify security gaps.
Question # 19
An agent connects to an MCP server that exposes 200 tools. The team wants the agent to remain effective without loading all 200 tool definitions into context on every request. Which strategy best fits?
A. Split the agent into 200 single-tool agents, one per tool. B. Load a fixed subset of 20 tools chosen by the development team and ignore the rest. C. Load all 200 definitions but shorten each description to one sentence. D. Use progressive discovery — expose a compact index of available capabilities and let the agent load full definitions for the tools it determines are relevant to the current task.
Answer: D
Why this is correct. Progressive discovery is the direct answer to large capability surfaces. Rather than paying the full definition cost of 200 tools on every request, the agent sees a lightweight index sufficient to identify what might be relevant, then loads complete schemas on demand for the few tools it actually needs. The context cost becomes proportional to what the task requires rather than to the size of the catalogue, and selection accuracy improves because the agent chooses among a handful of candidates rather than 200.
Why C is wrong. Truncating descriptions reduces the token bill somewhat while making tool selection harder — descriptions are the primary signal the model uses to choose correctly. You pay in accuracy for a partial cost saving and still load all 200.
Why A is wrong. 200 single-tool agents converts a context problem into an orchestration nightmare, with routing logic that must itself know about all 200 capabilities.
Why B is wrong. Hard-coding a 20-tool subset is progressive disclosure's crude cousin: it caps cost by permanently removing capability, so any task needing tool 21 simply fails. It also requires the development team to predict usage correctly in advance.
Blueprint objective: Evaluate progressive discovery vs. monolithic context strategy.
Question # 20
An enterprise wants Claude-based assistants built by four different teams to access the same set of interna systems — a CRM, a ticketing system, and a data warehouse — without each team writing and maintaining its own integration code. Which integration mechanism is most appropriate?
A. Each team embeds the systems' data as static context in its system prompt, refreshed nightly. B. Each team writes direct REST API clients for each system inside its own application. C. Expose each internal system through a Model Context Protocol server that any compliant client can connect to, so integrations are built once and reused across teams. D. Route all four assistants through a single shared agent that owns every integration.
Answer: C
Why this is correct. MCP exists precisely for the N-clients-by-M-systems problem. Implementing each system once as an MCP server means the integration — its tools, resources, authentication handling, and schemas — is written and maintained in one place, and any compliant client can consume it. The four teams then connect rather than build, upgrades to a server propagate to all consumers, and integration behaviour is consistent across assistants. This is the standard signal for MCP in an exam scenario: multiple consumers, shared systems, a desire to avoid bespoke duplicated glue.
Why B is wrong. This is the status quo the requirement rejects: four independent implementations per system, four sets of auth handling, four places to fix a bug, and inevitable behavioural drift between teams.
Why A is wrong. Static nightly snapshots of CRM and ticketing data are stale by construction and cannot support write actions at all. Embedding operational data in prompts also scales badly and destroys prompt caching.
Why D is wrong. Funnelling four assistants through one shared agent couples unrelated products, creates a single point of failure, and confuses two layers — the need is shared *integration plumbing*, not a shared reasoning agent. It also complicates per-team authorization, since the shared agent would hold the union of all
permissions.
Blueprint objective: Evaluate connection protocols and select the appropriate integration mechanism (MCP, API/CLI, agent-to-agent).
