Topic 3: AI Agents & Orchestration interview questions

An AI agent is a system that uses a model to interpret a goal, decide on actions, use tools or context, and evaluate results. Unlike a simple prompt-response application, it can follow a multi-step loop. A production agent still needs boundaries: explicit tools, authorization checks, time and cost limits, observable state, and a human escalation path for risky or ambiguous actions. Interview signal: Define the agent boundary and give a bounded support-triage example instead of calling every chatbot an agent.

A workflow follows predefined steps and branching rules, so its behavior is predictable and easy to test. An agent chooses among possible actions at runtime based on the task and available context. Many reliable systems combine both: deterministic workflow code controls critical transitions, while an agent handles flexible tasks such as classification, planning, or selecting the most appropriate tool. Interview signal: Contrast a deterministic approval workflow with an agent choosing among tools, then explain why a hybrid is often safer.

A typical agent has a model for reasoning, instructions defining its role and constraints, tools for external actions, memory or retrieved context, a state store, and an orchestration loop. It also needs policy enforcement, validation, retries, tracing, and evaluation. Keeping these concerns separate makes the system easier to change, test, secure, and operate than putting everything into one prompt. Interview signal: Name model, policy, tools, state, memory, orchestration, validation, and telemetry as separate production components.

I would start with a bounded loop: receive the task, load relevant state, ask the model for either a final response or a structured tool call, validate that call, execute it, append the result, and repeat. The loop ends on a valid answer, a maximum-step limit, timeout, budget exhaustion, or policy failure. Every iteration should emit trace data for debugging. Interview signal: Describe the loop as structured decision, validated tool call, execution, observation, and bounded termination.

Planning is the process of decomposing a goal into actions or subgoals before execution. It can be explicit, such as producing a task list, or implicit, with the model deciding one action at a time. I prefer plans that are short, structured, and revisable. The executor should verify each step rather than blindly trusting a plan generated before new information appears. Interview signal: Use a revisable plan for a research task and explain why each step needs evidence-based verification.

Multiple agents are useful when responsibilities are genuinely different, such as research, coding, review, or domain-specific decision support, and when separate context or permissions improve quality. They add latency, cost, coordination complexity, and more failure modes. I would first establish that a single agent or deterministic workflow cannot meet the requirement, then define narrow roles and a clear handoff contract. Interview signal: Justify multiple agents only when context, permissions, or expertise are genuinely separable; otherwise quantify the added cost.

Common patterns include a sequential pipeline, a router that selects a specialist, parallel workers followed by aggregation, a supervisor coordinating specialists, and an evaluator-reviser loop. The right pattern depends on dependency structure and risk. I use sequential execution for dependent steps, parallelism for independent work, and explicit aggregation schemas so one worker cannot silently distort another worker’s output. Interview signal: Choose sequential, router, parallel, supervisor, or evaluator patterns from dependency structure rather than framework preference.

I enforce multiple independent limits: maximum iterations, wall-clock timeout, tool-call budget, token budget, and repeated-action detection. The agent should receive a structured termination condition and the orchestrator should stop it regardless of model output. On termination, the system should return a useful partial result or request human review, while recording the reason and intermediate state. Interview signal: Mention step, time, token, tool, spend, and repeated-action limits; one limit alone will not stop every runaway loop.

Tools should be narrow, typed, deterministic where possible, and documented with clear preconditions and failure behavior. Inputs need schema validation, authorization, and limits such as allowed resources or record counts. Destructive operations should require confirmation or a separate approval tool. Returning concise, structured results helps the model reason accurately and reduces context growth. Interview signal: Give a typed tool example with scope and side-effect metadata, and separate read tools from confirmation-gated writes.

I classify failures as transient, permanent, validation, authorization, or business-state errors. Transient failures can use bounded exponential backoff with idempotency protection. Permanent and policy errors should be explained without repeated attempts. The tool response should expose a safe error code and recovery hint, while logs retain diagnostic details. The orchestrator should decide whether to retry, choose another tool, or escalate. Interview signal: Classify tool errors and show why retrying an authorization failure is both ineffective and potentially dangerous.

Agent memory is persisted information used across steps or conversations. Short-term memory is current task state; long-term memory may contain preferences, facts, or past outcomes. Risks include stale facts, accidental sensitive-data retention, prompt injection through stored content, and unbounded context. I use explicit schemas, provenance, retention rules, access controls, and retrieval filters instead of treating every past message as trusted memory. Interview signal: Separate working memory from durable memory and require provenance, retention, access control, and injection handling.

I keep the working context bounded by summarizing completed work, retaining only relevant tool results, and storing detailed artifacts outside the prompt. Summaries should preserve decisions, unresolved questions, identifiers, and evidence links. Retrieval should be selective and permission-aware. I also monitor context size and model quality because aggressive compression can remove a fact needed for a later decision. Interview signal: Explain summaries as state compression that must preserve decisions, identifiers, unresolved questions, and evidence.

Reliability comes from system design, not from prompting alone. I constrain tools, validate outputs, use deterministic code for critical rules, add time and budget limits, and provide retries with idempotency. I test representative tasks and adversarial cases, instrument every step, define fallback behavior, and review failures. For high-impact actions, the agent proposes while a policy or human approves. Interview signal: Tie reliability to deterministic policy code, bounded execution, validation, fallback, and traces—not a better prompt alone.

I evaluate both final outcomes and intermediate behavior. A test set should include normal, ambiguous, adversarial, and tool-failure tasks. Metrics can cover task success, factuality, tool-selection accuracy, policy violations, latency, cost, and escalation quality. Traces make failures diagnosable. Offline evaluations establish a baseline, while sampled production reviews and regression tests detect drift after prompts, tools, or models change. Interview signal: Evaluate outcome success and intermediate tool behavior, including cost, policy violations, and escalation quality.

Human-in-the-loop orchestration pauses an agent at defined decision points so a person can approve, edit, reject, or clarify an action. It is appropriate for financial, legal, security, or irreversible operations. The approval screen should show the proposed action, inputs, evidence, and consequences. The system must persist the pending state and safely resume or cancel without duplicating the action. Interview signal: Describe approval as a persisted state transition with exact arguments, reviewer identity, expiry, and safe cancellation.

I treat retrieved documents, web pages, emails, and tool outputs as untrusted data, not instructions. System policy remains separate from content, and tools enforce authorization independently of the model. I restrict domains and actions, sanitize inputs where useful, require confirmation for sensitive operations, and test indirect injection scenarios. Logging suspicious instructions and offering a safe refusal path are also important. Interview signal: Treat web pages, emails, retrieved text, and tool output as untrusted data while enforcing authority outside the model.

I set per-request budgets, choose smaller models for routing and simple steps, cache stable results, limit retrieved context, and parallelize independent calls. I avoid unnecessary planning loops and make tool outputs compact. Traces should attribute tokens, model time, tool time, and retries to each task. If a budget is reached, the system should return a bounded result or escalate predictably. Interview signal: Attribute latency and spend to each step, then use smaller models, caching, parallelism, and compact tool results selectively.

Idempotency means repeating the same request produces the same intended result rather than duplicating an effect. Agents need it because models, networks, and orchestrators may retry after uncertain failures. I pass an operation or request key to side-effecting tools, persist execution status, and make the tool safely return the existing result on replay. This is essential for payments, emails, and record updates. Interview signal: Use idempotency keys and durable execution records for retries around email, payment, or record-update tools.

I persist a versioned task record containing the goal, current step, validated tool calls, results, status, timestamps, and retry metadata. Each transition should be atomic or recoverable so a process restart cannot lose the workflow position. Sensitive fields need encryption and access controls. State schemas should support migration because prompts, tools, and orchestration logic evolve over time. Interview signal: Persist versioned state transitions so restarts resume safely and schema migrations remain possible.

An agent should refuse or escalate when the request is outside its authority, evidence is insufficient, a policy blocks the action, the user’s intent is materially ambiguous, or the impact is high and approval is required. The response should explain the limitation briefly and offer a safe next step. Silent guessing is worse than a bounded refusal because it creates untraceable risk. Interview signal: Escalate when authority, evidence, intent, or risk is unacceptable, and provide a recoverable next step rather than guessing.