Databricks Certified Generative AI Engineer Associate

Question 1

A Generative AI Engineer is running Mosaic AI Agent Evaluation on a RAG-based customer support assistant. The evaluation dataset includes a column with expert-written reference answers for each question. The team wants to measure how factually and semantically close the agent's generated responses are to these reference answers, independent of whether the retrieved context supported them. Which built-in Agent Evaluation metric directly addresses this requirement?

Accepted Answer

answer correctness, which compares the agent's response against the provided ground-truth reference answers

Answer

groundedness, which checks whether the response is supported by the retrieved context

Answer

chunk relevance (retrieval precision), which measures how relevant retrieved documents are to the request

Answer

safety, which flags whether the response contains harmful or toxic content

Question 2

A team runs Mosaic AI Agent Evaluation nightly on a golden dataset of 500 questions for their production RAG assistant. After a recent change that swapped the generation model to a larger foundation model, the evaluation report shows the average correctness and groundedness scores each improved by about 3%, but the total_input_token_count and total_output_token_count metrics roughly tripled, and mean response latency rose from 1.2s to 3.8s. The product owner requires sub-2s p90 latency and a fixed monthly serving budget. What is the most appropriate way to use these evaluation results to decide next steps?

Accepted Answer

Weigh the marginal 3% quality gain against the tripled token cost and latency SLA violation, and either revert or test cheaper alternatives (smaller model, tighter prompts, fewer retrieved chunks) that meet the latency and budget constraints.

Answer

Keep the larger model because any improvement in correctness and groundedness always outweighs cost and latency regressions in a production assistant.

Answer

Disable the cost and latency metrics in Agent Evaluation so the report focuses only on correctness and groundedness, which are the metrics that matter for quality.

Answer

Ignore the offline evaluation entirely and rely solely on production inference table monitoring, since offline token counts are not representative of real traffic.

Question 3

A GenAI engineer runs Mosaic AI Agent Evaluation on a customer-support RAG agent. The built-in judges (correctness, groundedness, relevance, safety) all pass, but the compliance team insists that every response must also avoid making promises about refund timelines, a domain-specific policy not captured by any built-in judge. The engineer wants this policy checked automatically on each evaluation run alongside the existing metrics. What is the most appropriate approach?

Accepted Answer

Define a custom LLM-judge metric with a guidelines/prompt describing the refund-timeline policy and pass it to mlflow.evaluate so it runs alongside the built-in judges.

Answer

Lower the safety judge's threshold so it also flags refund-timeline promises during evaluation.

Answer

Manually inspect every evaluation output for refund-timeline promises after each run and record findings in a spreadsheet.

Answer

Add the refund policy text to the system prompt and rely on the correctness judge to catch violations automatically.

Question 4

A Generative AI Engineer wants to run Mosaic AI Agent Evaluation on a deployed RAG chatbot using an offline evaluation dataset. The team has collected 200 representative user questions and expected reference answers, but no retrieval traces. They want to compute both answer correctness (comparing to the expected answers) and groundedness (whether responses are supported by retrieved context). Which approach correctly structures the evaluation so both metric types can be computed?

Accepted Answer

Provide a dataset with 'request' and 'expected_response' columns, and let Agent Evaluation invoke the agent so that retrieved context is captured in the trace for groundedness scoring.

Answer

Provide only a 'request' column with the 200 questions; Agent Evaluation will infer expected answers and retrieved context automatically.

Answer

Provide 'request' and 'expected_response' columns only; groundedness cannot be computed without manually pre-populating a 'retrieved_context' column for every row.

Answer

Split the work into two datasets—one with 'request' and 'expected_response' for correctness, and a separate one with only 'request' for groundedness—since the two metrics cannot share an evaluation run.

Question 5

A Generative AI Engineer is using Mosaic AI Agent Evaluation to assess a deployed RAG chatbot for a healthcare knowledge base. Stakeholders complain that some answers sound confident but contain claims not supported by the retrieved documents. The engineer has a golden evaluation dataset with questions but no ground-truth answers. Which built-in LLM-as-judge metric should the engineer prioritize to catch these unsupported claims?

Accepted Answer

Groundedness — measures whether the response is factually supported by the retrieved context

Answer

Answer correctness — compares the response against a labeled ground-truth answer

Answer

Chunk relevance — measures whether each retrieved chunk is relevant to the question

Answer

Latency — measures the end-to-end response time of the agent

Question 6

A Generative AI Engineer has built a RAG-based customer support agent and wants to systematically compare two candidate prompt templates before promoting either to production. They have assembled a curated dataset of representative questions, each with a reference (ground-truth) answer and the relevant supporting documents. They want to run Mosaic AI Agent Evaluation to measure answer correctness, groundedness, and retrieval relevance for both templates, then choose the better one. Which approach correctly uses Agent Evaluation for this offline comparison?

Accepted Answer

Run mlflow.evaluate() with the Agent Evaluation harness on the curated evaluation dataset for each template, comparing the built-in LLM-judge metrics (correctness, groundedness, retrieval relevance) across runs.

Answer

Deploy both templates to production serving endpoints first, then compare their inference table logs after a week of real traffic to decide which is better.

Answer

Enable Lakehouse Monitoring on the source Delta tables and use its data drift metrics to determine which template produces higher-quality answers.

Answer

Skip the labeled dataset and rely solely on measuring p95 latency and cost-per-request for each template, selecting whichever is cheaper and faster.

Question 7

A Generative AI Engineer runs Mosaic AI Agent Evaluation on a RAG chatbot. The aggregate scores show high groundedness but low answer relevance. The team wants to understand whether the problem originates in retrieval or in generation before making changes. Within the Agent Evaluation results, which artifact should they inspect FIRST to pinpoint the source of the low relevance?

Accepted Answer

The per-request detailed trace, which shows the retrieved chunks alongside the generated response and the individual judge assessments for each evaluated row

Answer

The overall aggregate metric summary table averaged across the entire evaluation dataset

Answer

The model serving endpoint's inference table payload logs from the last 24 hours of production traffic

Answer

The Lakehouse Monitoring drift dashboard comparing the current scoring window to the baseline table

Question 8

A Generative AI Engineer runs Mosaic AI Agent Evaluation on a deployed RAG chatbot. The overall answer-correctness score is low. The evaluation report shows that the retrieval metrics (context precision and context recall) are strong, but the answer-groundedness metric is poor. Which conclusion best explains this pattern, and what should the engineer prioritize fixing?

Accepted Answer

The correct context is being retrieved, but the LLM is generating claims not supported by that context; the engineer should focus on the generation prompt and model, not the retriever.

Answer

The retriever is pulling irrelevant documents; the engineer should tune the vector search index and re-embed the corpus.

Answer

The golden dataset labels are wrong; the engineer should discard the evaluation results and rely only on production inference tables.

Answer

The chunk size is too small; the engineer should increase chunk size to improve both retrieval and groundedness simultaneously.

Question 9

A GenAI engineer has a customer-support RAG agent deployed with inference tables enabled. Product leadership wants a weekly quality report showing, for each request, whether the response was grounded in retrieved context and whether it correctly answered the question. The team has thousands of daily requests and only two subject-matter experts who can review a small sample manually. Which approach best scales the per-request quality assessment while keeping human effort focused where it matters most?

Accepted Answer

Run Mosaic AI Agent Evaluation with LLM-as-judge metrics (e.g., correctness and groundedness) across all logged requests, then route only low-scoring or low-confidence cases to the two SMEs for manual review.

Answer

Have the two SMEs manually label every request each week, since only human judgment can reliably assess groundedness and correctness at the required quality.

Answer

Rely solely on Lakehouse Monitoring drift metrics on the inference table to infer whether response quality is acceptable, avoiding any per-request scoring.

Answer

Compute average response latency and token count per request from the inference table and use those as proxies for answer quality in the weekly report.

Question 10

You are building a multi-step agent that answers customer questions about their orders. The workflow requires the agent to (1) look up a customer's order ID from their email, (2) retrieve the shipment status using that order ID, and (3) generate a natural-language summary. During testing, the agent frequently fails at step 2 because it invokes the shipment-status tool with the customer's email instead of the order ID returned by step 1. What is the most effective way to fix this behavior?

Accepted Answer

Ensure each tool returns clearly structured output (e.g., a JSON field named order_id) and write tool descriptions that explicitly state which input each tool requires, so the agent can carry the intermediate result forward

Answer

Increase the agent's max_iterations so it has more attempts to eventually call the shipment-status tool with the correct argument

Answer

Combine all three steps into a single tool so the agent never has to pass intermediate results between calls

Answer

Switch the underlying foundation model to a larger pay-per-token model with a bigger context window

Question 11

You are building a Mosaic AI agent on Databricks that answers customer questions by calling multiple tools (a product catalog lookup, an inventory checker, and a shipping estimator). During testing, some queries cause the agent to enter long reasoning loops, repeatedly invoking tools without converging on a final answer. This drives up latency and token costs. Which configuration change most directly prevents these runaway loops while still allowing legitimate multi-step reasoning?

Accepted Answer

Set a max_iterations (maximum reasoning steps) limit on the agent executor so it terminates and returns after a bounded number of tool-calling cycles.

Answer

Switch the underlying foundation model from a pay-per-token endpoint to a provisioned throughput endpoint.

Answer

Increase the model's max_tokens output parameter so it can produce a complete answer in a single response.

Answer

Reduce the temperature parameter to 0 so the model produces fully deterministic tool calls.

Question 12

You are building a Mosaic AI agent for a retail customer-support app. The agent has access to a tool `issue_refund(order_id, amount)` that calls an internal API to process a monetary refund. During testing, you notice that when the LLM's reasoning loop occasionally re-invokes the same tool call after a transient response parsing error, customers are sometimes refunded twice for the same order. Which design change most directly prevents duplicate side effects from repeated tool invocations?

Accepted Answer

Add an idempotency key (e.g., a hash of order_id + refund request ID) so the refund API treats repeated calls for the same order as a single operation

Answer

Increase the agent's max_iterations so it has more attempts to complete the reasoning loop successfully

Answer

Lower the model temperature to 0 so the agent produces more deterministic reasoning and never repeats a tool call

Answer

Rewrite the tool description to instruct the LLM to call issue_refund only once per conversation

Question 13

You are building a Mosaic AI agent that has access to three registered Unity Catalog functions: `get_order_status`, `lookup_product_inventory`, and `search_return_policy`. During testing, you notice the LLM frequently calls `search_return_policy` even when users ask about where their package is. The tool bindings and endpoint work correctly, and each function executes without error when called directly. What is the most effective change to improve the agent's tool selection accuracy?

Accepted Answer

Write clear, distinct docstrings and parameter descriptions for each function so the LLM can accurately match user intent to the correct tool

Answer

Reduce the temperature of the foundation model endpoint to 0 to make tool calls deterministic

Answer

Increase the agent's maximum iteration limit so it has more chances to call the right tool

Answer

Switch the endpoint from a pay-per-token model to provisioned throughput for more consistent responses

Question 14

You are building a Mosaic AI agent that calls an external inventory API tool to answer customer stock-availability questions. During testing, the API intermittently returns timeouts and malformed responses, causing the agent to either crash or return the raw error text to the user. You need to make the agent robust while keeping responses helpful. Which approach best addresses this within the agent's tool-execution logic?

Accepted Answer

Wrap the tool call with retry-with-backoff logic and a fallback that returns a structured, user-friendly message the LLM can incorporate when the tool ultimately fails

Answer

Increase the foundation model's max_tokens so the LLM can absorb and rewrite the raw API error text before responding

Answer

Remove the inventory tool from the agent and instead embed all stock data into the system prompt so no external call is needed

Answer

Lower the model temperature to 0 so the agent stops producing inconsistent responses when the API fails

Question 15

You are building a Mosaic AI agent that calls a Unity Catalog function to look up a customer's order by an order ID. During testing, the LLM occasionally passes malformed arguments — sometimes an empty string, sometimes a full sentence like 'the order from yesterday' — causing the tool to fail. You want to reduce these failures by making the tool's expected input explicit to the LLM so it produces correctly formatted arguments more reliably. Which approach most directly addresses this?

Accepted Answer

Define a strict argument schema (e.g., a Pydantic model or typed function signature with descriptions) for the tool so the LLM receives a clear structured specification of the required input format.

Answer

Increase the model temperature so the agent explores a wider variety of argument formats until one succeeds.

Answer

Wrap the tool in a retry loop that re-invokes it with the same arguments until the call returns successfully.

Answer

Raise the agent's maximum iteration limit so it has more attempts to eventually call the tool correctly.

Question 16

You are building a Mosaic AI agent that answers customer queries like 'Compare the shipping cost and estimated delivery date for my order to the standard rate.' The agent has three tools: get_order_details (returns order ID metadata), get_shipping_cost (requires the order's weight and destination from get_order_details), and get_standard_rate (returns a fixed published rate with no inputs). You want to minimize end-to-end latency without producing incorrect tool inputs. How should you structure the agent's tool execution?

Accepted Answer

Run get_order_details first, then run get_shipping_cost and get_standard_rate in parallel once its output is available.

Answer

Run all three tools in parallel at the start to minimize latency, then reconcile any missing inputs afterward.

Answer

Run all three tools strictly sequentially in the order they were registered to guarantee correct inputs.

Answer

Merge get_order_details and get_shipping_cost into a single tool and skip get_standard_rate since rates are published.

Question 17

You are building a customer-facing shopping assistant on Databricks. Customers ask two kinds of questions: (1) explanations of the store's return and warranty policies, which are documented in a static PDF knowledge base, and (2) current stock levels and prices for specific SKUs, which change minute-to-minute and live in a transactional database exposed through a REST API. You want a single Mosaic AI agent that answers both accurately. Which design correctly handles both question types?

Accepted Answer

Use a RAG retriever over the policy documents for policy questions, and register a tool (function) that calls the live inventory REST API for stock and price questions, letting the agent choose based on the query.

Answer

Index both the policy PDFs and a nightly snapshot of the inventory table in Vector Search, and answer every question with pure RAG retrieval.

Answer

Fine-tune a foundation model on both the policy PDFs and current inventory data so it can answer all questions from its weights.

Answer

Store the inventory data as few-shot examples in the system prompt and use RAG only for the policy documents.

Question 18

A customer support team receives tickets in multiple languages. The GenAI application must: (1) detect and translate each ticket into English, (2) classify the ticket into a support category, and (3) generate a concise English summary for the support agent. The team wants a maintainable design where each step can be tested and swapped independently, and where a failure or low-confidence result in one step can be inspected before the next step runs. Which design approach best meets these requirements?

Accepted Answer

Decompose the workflow into a sequential chain of discrete steps (translate → classify → summarize), where the output of each step is passed as input to the next and each step is an independently testable component.

Answer

Build a single large prompt that instructs one LLM call to translate, classify, and summarize the ticket all at once, returning a combined result.

Answer

Fine-tune a single model on translated, classified, and summarized examples so the model performs all three tasks implicitly without separate steps.

Answer

Deploy three completely independent applications that each process the raw ticket in parallel, then merge their unrelated outputs at the end.

Question 19

You are building a multi-turn customer support agent on Databricks that uses a foundation model with an 8,192-token context window. Each turn appends the full conversation history plus a system prompt plus retrieved documents. After roughly 15 exchanges, users start receiving errors that the context length has been exceeded. The business requires that the agent still remember key facts from earlier in the conversation (like the customer's account ID and reported issue). What is the most appropriate way to manage context so the agent keeps working across long conversations?

Accepted Answer

Use a conversation summary memory that progressively condenses older turns into a running summary while keeping the most recent turns verbatim.

Answer

Increase the max_tokens generation parameter so the model has more room to produce output.

Answer

Truncate the conversation by dropping the oldest turns entirely once the context limit is approached.

Answer

Switch to a larger provisioned-throughput endpoint of the same model to raise the throughput limit.

Question 20

A Generative AI Engineer is preparing a knowledge base of long-form product manuals for a RAG chatbot. During evaluation, users report that answers frequently seem 'cut off' or miss information that spans across paragraph boundaries. The team is using fixed-size chunking of 500 tokens with zero overlap. Which adjustment to the chunking strategy is MOST likely to resolve this issue while keeping chunk sizes manageable?

Accepted Answer

Introduce a chunk overlap (e.g., 50-100 tokens) between consecutive chunks so context that spans boundaries is preserved in adjacent chunks.

Answer

Increase the fixed chunk size to 4000 tokens with zero overlap so each chunk contains more complete information.

Answer

Switch to character-based chunking of exactly 500 characters with zero overlap to make chunks more granular.

Answer

Remove chunking entirely and embed each full manual as a single vector to avoid losing any information.

Question 21

A Generative AI Engineer is preparing a corpus of long-form product manuals (avg 40 pages each) for a RAG application. They plan to use an embedding model with a maximum input context of 512 tokens. During testing, they notice that chunks frequently exceed this limit and are silently truncated, causing the tail of each chunk's content to be excluded from the embedding. What is the most appropriate action to ensure complete content is represented in the vector index?

Accepted Answer

Reduce the target chunk size so that each chunk (plus any overlap) stays within the embedding model's 512-token limit before generating embeddings.

Answer

Keep the large chunks but increase the number of retrieved results (top-k) at query time to compensate for truncated content.

Answer

Concatenate multiple manuals into a single document before chunking so the embedding model sees more context.

Answer

Switch to a larger vector dimension in the index so more of each chunk's tokens can be stored.

Question 22

You are building a RAG application over a customer support knowledge base composed of thousands of short FAQ entries. Each entry is a self-contained question paired with a concise answer, and entries are typically 2-4 sentences long. Retrieval accuracy is currently poor because chunks frequently split a question from its answer or merge multiple unrelated FAQs together. Which chunking strategy best fixes this?

Accepted Answer

Treat each FAQ question-and-answer pair as a single chunk, aligning chunk boundaries to the natural document structure.

Answer

Use a fixed-size character chunker of 512 characters with 50-character overlap applied uniformly across the concatenated document.

Answer

Chunk at the sentence level so each individual sentence becomes its own embedding for maximum granularity.

Answer

Merge the entire knowledge base into one large chunk so all context is always available to the retriever.

Question 23

You are preparing a large internal knowledge base for a RAG application. The documents are Markdown files with clear hierarchical headers (H1 for product areas, H2 for features, H3 for individual procedures). Users typically ask specific procedural questions like 'How do I reset the API key for the billing service?'. You want each retrieved chunk to be self-contained and topically coherent so the LLM has focused context. Which chunking approach best fits these documents?

Accepted Answer

Use header-aware (structure-based) splitting that segments at Markdown headers and prepends the parent header path to each chunk

Answer

Split the text into fixed 512-token windows with no overlap, ignoring the Markdown structure

Answer

Store each entire Markdown file as a single chunk to preserve all context

Answer

Split on every sentence boundary to maximize the number of retrievable units

Question 24

You are building a RAG system over a corpus of customer support tickets. The tickets vary widely in length: many are short one-paragraph questions, but some are long multi-issue threads spanning several distinct problems. Users query for solutions to specific individual problems. You want each retrieved chunk to represent a coherent, self-contained unit so that retrieval returns focused answers without mixing unrelated issues. Which chunking approach best fits this requirement?

Accepted Answer

Use semantic (content-aware) chunking that splits threads at topic boundaries so each chunk captures one distinct issue

Answer

Apply a fixed-size token window (e.g., 512 tokens) uniformly across all tickets regardless of structure

Answer

Store each entire ticket as a single chunk regardless of length to preserve full context

Answer

Split every ticket into single-sentence chunks to maximize retrieval granularity

Question 25

You are building a RAG pipeline over quarterly financial reports delivered as PDFs. Each report contains long narrative sections interspersed with large tabular data (balance sheets, income statements). During testing, users ask questions about specific line-item values, but retrieved chunks frequently return partial tables where column headers are separated from their data rows, producing wrong or unanswerable results. Which document preparation approach best addresses this problem?

Accepted Answer

Use a table-aware extraction step that detects and serializes each table as a self-contained unit (headers plus rows) into its own chunk, while chunking narrative text separately with recursive character splitting

Answer

Increase the fixed chunk size to 4000 tokens so entire tables are more likely to fit inside a single chunk

Answer

Lower the embedding dimension to capture tabular structure more compactly and improve retrieval recall

Answer

Strip all tables from the PDFs during ingestion and rely solely on the narrative sections for answering questions

Question 26

You are building a RAG application over a large corpus of software engineering documentation that mixes prose explanations with fenced code blocks and API tables. During testing, users complain that when they ask 'show me the function that validates tokens,' the retrieved chunks return only the surrounding explanation text and cut code samples in half, making the answers useless. You are currently using a fixed-size character chunker with 500-character chunks and no overlap. Which change to your chunking strategy will most directly improve retrieval of complete, coherent code examples?

Accepted Answer

Switch to a structure-aware (recursive/markdown-aware) chunker that respects code-fence and section boundaries, and add chunk overlap to preserve context.

Answer

Reduce the chunk size to 200 characters so more granular chunks are indexed and code is easier to match.

Answer

Increase the number of results (top_k) returned by Vector Search from 3 to 50 so the code block is eventually included.

Answer

Replace the embedding model with a larger general-purpose text embedding model to better understand code semantics.

Question 27

A Generative AI Engineer has run Mosaic AI Agent Evaluation on version 1 of a RAG chatbot and logged the results as an MLflow run. After changing the retriever's chunking strategy and re-running the evaluation on the same evaluation dataset, the engineer wants to determine whether the change improved or degraded answer quality before promoting the new version. What is the most effective way to make this decision using the evaluation results?

Accepted Answer

Compare the aggregate LLM-as-judge and retrieval metrics across the two MLflow evaluation runs to see whether scores like groundedness and retrieval relevance improved or regressed.

Answer

Deploy both versions to production behind a traffic split and rely on user thumbs-up/thumbs-down feedback to decide which version is better.

Answer

Inspect the inference tables from the production endpoint for the new version to measure latency and token cost differences.

Answer

Manually re-read a random sample of 5 generated answers from the new version without referencing the version 1 results.

Question 28

A software company wants to build a generative AI assistant that reviews pull requests. For each PR, the assistant must: (1) summarize the code changes, (2) flag potential security vulnerabilities against the company's internal secure-coding guidelines, and (3) generate a natural-language review comment that references specific findings. The security guidelines are a large, frequently updated internal document. As the GenAI engineer designing this application, what is the most appropriate high-level architecture?

Accepted Answer

A chain that first summarizes the diff, then performs a RAG step retrieving relevant guideline sections from a vector index to identify vulnerabilities, then composes the review comment from the prior outputs

Answer

A single prompt that pastes the full secure-coding guidelines and the entire code diff into one LLM call, asking it to return the summary, vulnerabilities, and review comment together

Answer

Fine-tune a base model on the secure-coding guidelines and use it directly to produce all three outputs from the raw diff in one step

Answer

Three fully independent LLM calls that each receive only the raw diff, with no shared context or retrieval, and concatenate their outputs

Question 29

A financial services company wants to build a customer support assistant that can answer questions about a customer's account. A single request may require the assistant to: (1) look up the customer's current account balance from an internal transactional database, (2) retrieve relevant policy explanations from an internal knowledge base of PDFs, and (3) perform a currency conversion calculation before presenting the final answer. The team wants a design where the LLM dynamically decides which capability to invoke based on the user's question. Which application design best fits these requirements?

Accepted Answer

An agent that reasons over the request and selects among defined tools: a SQL/function-calling tool for the balance lookup, a vector-search retriever tool for policy documents, and a calculator/code tool for currency conversion

Answer

A single large prompt that concatenates the full account database, all policy PDFs, and conversion tables into the context window for every request

Answer

A pure RAG pipeline that embeds the user question, retrieves the top-k policy chunks, and passes them to the LLM to generate all answers including balances and conversions

Answer

A fine-tuned model trained on historical account balances and policy text so it can answer all three types of questions without any external calls

Question 30

A financial services firm wants to build a system that answers analyst questions like 'Summarize the risk factors from the three latest 10-K filings for Company X and compare them to sector peers.' The number of documents to retrieve and the comparison steps vary per query, and the system sometimes needs to call a live market-data API mid-workflow depending on what the question asks. The team must decide how to structure the application. Which design approach best fits these requirements?

Accepted Answer

An agent with access to a retrieval tool and a market-data API tool, allowing the LLM to reason about which tools to invoke and in what order per query

Answer

A fixed sequential chain with predefined steps: retrieve documents, summarize, then compare, using the same tool order for every query

Answer

A single large prompt that stuffs all possible 10-K filings and market data into the context window and asks the model to answer directly

Answer

A batch fine-tuning job that trains the base model on all historical 10-K filings so it can answer from parametric memory

Question 31

A Generative AI Engineer is building a RAG application over a corpus of internal support articles that were exported from three different systems. During data profiling, they discover that roughly 30% of the articles are near-duplicates (same content with minor formatting differences) because articles were migrated multiple times. Early retrieval tests show that the top-k results are frequently filled with several copies of the same article, crowding out other relevant content. What is the most effective step to improve retrieval quality before generating embeddings?

Accepted Answer

Deduplicate the source documents during preprocessing (e.g., using content hashing or similarity-based clustering) so only one canonical version of each article is embedded and indexed

Answer

Increase the number of retrieved documents (k) so that non-duplicate content is more likely to appear in the results

Answer

Switch to a higher-dimensional embedding model so duplicate articles produce more distinct vectors

Answer

Reduce the chunk size so that duplicate articles are split into smaller, less-overlapping segments

Question 32

You are designing a customer-support generative AI application for an e-commerce company. Business requirements state that the assistant must: (1) answer general product policy questions, (2) look up a specific customer's live order status by order ID, and (3) initiate refund requests that require writing to an internal transactional system. The company wants a single conversational entry point that decides which capability to use per turn. Which application design best satisfies these requirements?

Accepted Answer

An agent that uses an LLM to reason about intent and select among tools: a RAG retriever over policy documents, a read-only order-lookup function, and a refund-submission function that writes to the transactional system.

Answer

A single large prompt that concatenates the policy documents, the order database schema, and refund instructions, relying on the LLM to generate answers, order statuses, and refund confirmations directly from context.

Answer

A fixed sequential chain that always retrieves policy documents, then queries order status, then submits a refund on every user turn regardless of the question asked.

Answer

A fine-tuned model trained on historical support transcripts, deployed without external tools, that memorizes policies, order data, and refund outcomes.

Question 33

A retail company wants a generative AI application that lets customer support agents upload a photo of a damaged product and receive a plain-language description of the visible damage, which is then combined with the customer's typed complaint to draft a suggested resolution email. The engineering team must translate this business requirement into an application design. Which design approach best satisfies the requirement?

Accepted Answer

Use a multimodal (vision-capable) model to generate the damage description from the image, then pass that description plus the typed complaint to an LLM step that drafts the resolution email

Answer

Use a single text-only LLM and instruct agents to manually describe the photo in words before submitting the prompt

Answer

Build a RAG pipeline that retrieves similar past product images from a vector index and returns the most visually similar prior resolution email verbatim

Answer

Fine-tune a text embedding model on complaint text so it can classify damage severity directly from the uploaded image bytes

Question 34

A Generative AI Engineer is designing a customer-support RAG assistant for a mid-size retailer. Business requirements state that roughly 80% of incoming questions are simple FAQ-style lookups (store hours, return policy) that must be answered in under 1 second at low cost, while the remaining 20% are complex, multi-part troubleshooting questions where answer quality matters most and slightly higher latency is acceptable. The team has both a small, fast, inexpensive LLM and a large, high-quality, more expensive LLM available as Model Serving endpoints. Which application design best satisfies these requirements?

Accepted Answer

Add a lightweight query-classification/routing step that sends simple FAQ queries to the small fast model and routes complex troubleshooting queries to the large high-quality model.

Answer

Route every query to the large high-quality model to guarantee the best possible answer quality across all question types.

Answer

Route every query to the small fast model and rely on more retrieved context chunks to compensate for its lower reasoning ability on complex questions.

Answer

Fine-tune the small fast model on the complete FAQ and troubleshooting corpus so a single model can handle both query types without any routing.

Question 35

A Generative AI Engineer must embed 4 million product-description records stored in a Delta table before creating a Databricks Vector Search index. They plan to call a Databricks Model Serving endpoint hosting an embedding model. Initial testing shows the job is extremely slow because each row is sent to the endpoint one at a time in a Python loop. Which change will most improve embedding throughput while remaining a supported Databricks pattern?

Accepted Answer

Use ai_query or a pandas_udf to send batched requests of many records per call, leveraging Spark parallelism across partitions.

Answer

Use a Spark UDF that sends one record per invocation, relying on cluster autoscaling to add more single-request executors.

Answer

Switch to a Delta Sync Vector Search index so the embeddings are computed sequentially by the sync process without any batching.

Answer

Reduce the dataset to 100,000 rows so the single-row loop completes in an acceptable amount of time.

Question 36

A Generative AI Engineer built a RAG application on Databricks. They embedded a corpus of 50,000 product manual chunks using the 'bge-large-en' embedding model and stored the vectors in a Databricks Vector Search index. During testing, retrieval quality is poor even for queries that clearly match documents in the corpus. On inspection, the engineer discovers that the query embeddings at search time are being generated by a different model ('all-MiniLM-L6-v2') than the one used to build the index. Which action correctly resolves the root cause of the poor retrieval?

Accepted Answer

Re-embed the user queries at search time with the same 'bge-large-en' model used to build the index, so query and document vectors share the same embedding space.

Answer

Increase the number of results (k) returned from the index so that more candidate chunks are retrieved to compensate.

Answer

Add a reranker after retrieval to reorder the mismatched results by relevance.

Answer

Reduce the chunk size of the documents and rebuild the index to improve match density.

Question 37

A Generative AI Engineer is building a RAG application over a company's internal knowledge base. Stakeholders require that every generated answer include a citation showing the document title, section heading, and a link back to the original source page. The engineer is designing the document preparation pipeline that produces chunks for embedding and indexing in Databricks Vector Search. What is the most effective approach to satisfy the citation requirement?

Accepted Answer

Attach metadata fields (document title, section heading, source URL) to each chunk during preparation so they are stored alongside the embeddings and returned with retrieval results.

Answer

Store only the raw chunk text in the index and instruct the LLM in the prompt to invent plausible citations for each answer.

Answer

Concatenate the document title and URL into the chunk text itself before embedding, and rely on the LLM to extract them from the retrieved passage.

Answer

Increase the chunk size so each chunk contains an entire document, guaranteeing the citation information is always present in the context.

Question 38

A Generative AI Engineer has built a RAG pipeline over a corpus of internal knowledge base articles. During evaluation, users report that retrieved chunks are often topically related to the query but frequently lack the specific detail needed to answer the question, forcing the LLM to say it cannot find the answer. Inspection shows each chunk is 1500 tokens with no overlap, and many chunks combine an FAQ heading with several unrelated sub-answers. What is the most effective first change to improve retrieval relevance?

Accepted Answer

Reduce chunk size and add semantic-aware splitting so each chunk captures a single coherent topic, with a small token overlap between adjacent chunks

Answer

Switch to a higher-dimensional embedding model while keeping the same 1500-token chunks

Answer

Increase the number of retrieved chunks (top-k) from 3 to 20 so more context reaches the LLM

Answer

Re-index the vector store more frequently using Delta Sync to ensure freshness of the embeddings

Question 39

A Generative AI Engineer has built a RAG application over an internal knowledge base and wants to quantitatively evaluate whether the vector search retriever is returning relevant documents before investing in prompt tuning. The team can manually label a small set of representative questions with the specific source chunks that should be retrieved for each. Which evaluation approach best measures retrieval relevance in this situation?

Accepted Answer

Build a golden evaluation dataset of question-to-relevant-chunk pairs and compute retrieval metrics such as recall@k and precision@k against the retriever's returned results

Answer

Measure the average cosine similarity score between each query embedding and its top returned chunk, treating higher scores as proof of correct retrieval

Answer

Send the final generated answers to an LLM judge that scores fluency and grammar, using those scores as a proxy for retrieval quality

Answer

Track the end-to-end latency of the vector search endpoint and use lower latency as the indicator that retrieval is returning the most relevant chunks

Question 40

A Generative AI Engineer at a healthcare startup needs to integrate OpenAI's GPT-4 into a Databricks-based application. Company policy requires that the OpenAI API key never be hard-coded in notebooks or application code, and that all model calls flow through a governed, centrally-managed interface that supports rate limiting and usage tracking on Databricks. What is the recommended approach to configure this integration?

Accepted Answer

Store the OpenAI API key in a Databricks secret scope and create a Mosaic AI Model Serving external model endpoint that references the secret

Answer

Hard-code the OpenAI key in a shared cluster environment variable and call the OpenAI SDK directly from each notebook

Answer

Provision throughput for GPT-4 through the Foundation Model APIs pay-per-token endpoint and pass the key as a query parameter

Answer

Download GPT-4 weights from OpenAI and register the model in Unity Catalog for local serving

Question 41

A Generative AI Engineer is building a RAG system over an internal knowledge base exported from a wiki. During inspection of the chunked output, they notice many chunks contain only navigation breadcrumbs, page footers with copyright notices, empty table cells, or single-word headers with no surrounding content. These chunks are being embedded and indexed alongside meaningful content, and retrieval quality has degraded because irrelevant chunks are frequently returned. What is the most effective step to take before generating embeddings and populating the Vector Search index?

Accepted Answer

Apply content-quality filters that drop chunks below a minimum token/character threshold and remove boilerplate patterns (breadcrumbs, footers, empty cells) prior to embedding

Answer

Increase the chunk size so that low-value fragments are merged into larger chunks that also contain meaningful text

Answer

Switch to a higher-dimensional embedding model so the vector representation better distinguishes low-value chunks at query time

Answer

Add a reranker after retrieval to push the boilerplate chunks lower in the result ordering

Question 42

A Generative AI Engineer is building a RAG application on Databricks. They need to generate embeddings for both the initial batch indexing of 500,000 documents and for real-time query embeddings at inference time. They want to minimize operational overhead by using a Databricks-hosted embedding model rather than deploying a custom model. Which approach correctly uses the Databricks Foundation Model APIs for this task?

Accepted Answer

Call the pay-per-token Foundation Model API embedding endpoint (e.g., databricks-bge-large-en) for both batch indexing and query-time embedding, ensuring the same model is used in both paths.

Answer

Use the pay-per-token embedding endpoint for real-time queries, but use a fine-tuned chat completion endpoint to generate embeddings during batch indexing.

Answer

Register an external model endpoint pointing to a third-party chat completion API and use it to produce embeddings for both indexing and queries.

Answer

Use the Foundation Model API text generation endpoint (e.g., DBRX Instruct) to embed documents, since instruction-tuned models produce higher-quality vectors.

Question 43

A Generative AI Engineer is building an internal prototype chat assistant on Databricks. The traffic is sporadic and unpredictable during the exploration phase, and the team wants to experiment quickly with a state-of-the-art open model (Llama) without committing to reserved capacity or managing infrastructure. They want the lowest operational overhead and to pay only for what they actually use. Which Foundation Model API option should they choose?

Accepted Answer

Use pay-per-token Foundation Model APIs, which expose pre-configured base model endpoints billed per token consumed

Answer

Provision a dedicated provisioned throughput endpoint sized for peak concurrency

Answer

Register the model in Unity Catalog and deploy it to a custom Model Serving endpoint with autoscaling to zero disabled

Answer

Configure an external model endpoint pointing to a self-hosted Llama server outside Databricks

Question 44

A Generative AI Engineer is deploying a customer-facing chatbot on Databricks that must serve a large open-source LLM (Llama 3 70B) with predictable, low latency during business hours when traffic is consistently high. The team has strict SLAs on response time and needs guaranteed capacity rather than best-effort serving. Which Databricks Foundation Model API serving option best meets these requirements?

Accepted Answer

Provisioned Throughput Foundation Model APIs, which reserve dedicated capacity for guaranteed performance

Answer

Pay-per-token Foundation Model APIs, since they automatically scale to any request volume

Answer

External Models routing to a third-party OpenAI endpoint through the AI Gateway

Answer

A CPU-based Model Serving endpoint hosting the model with autoscaling enabled

Question 45

A Generative AI Engineer has built a customer-support summarization chain that calls a Databricks Foundation Model API pay-per-token endpoint using the OpenAI-compatible client. Leadership now wants the ability to switch the underlying LLM (e.g., from Llama to a Claude external model) without rewriting the application code that makes the completion calls. Which Databricks capability lets the engineer change the served model behind a stable interface so the application code stays unchanged?

Accepted Answer

Configure a Model Serving endpoint (with an external-model or foundation-model entity) and keep the application pointed at the stable endpoint name while updating the served entity

Answer

Register each candidate model in Unity Catalog and reference it by its three-level name in every request

Answer

Hardcode conditional logic in the chain that instantiates a different client library for each provider based on an environment variable

Answer

Enable provisioned throughput on the endpoint so that the model can be resized dynamically

Question 46

A Generative AI Engineer is building an invoice-processing application that calls a Databricks Foundation Model API endpoint (a chat model that supports structured outputs). The application must receive strictly valid JSON matching a fixed schema (vendor, invoice_number, total_amount) so it can be inserted directly into a Delta table without post-parsing failures. The team has already tried adding 'respond only in JSON' to the prompt but occasionally gets responses wrapped in markdown code fences or with explanatory prose. Which approach most reliably guarantees schema-conformant output from the endpoint?

Accepted Answer

Pass a response_format parameter specifying a JSON schema in the API request so the model is constrained to emit output matching that schema.

Answer

Increase the temperature to 0 and repeat the same prompt instruction more forcefully with capital letters.

Answer

Add a regular-expression post-processor that strips markdown fences and hopes the remaining text is valid JSON before insertion.

Answer

Switch the endpoint to provisioned throughput mode, which enforces structured JSON responses automatically.

Question 47

A Generative AI Engineer deploys a customer-facing chatbot on a Databricks Model Serving endpoint. Compliance requires that any generated response containing toxic, hateful, or unsafe content must be blocked before it reaches the end user, and the blocking decision must happen on outputs the LLM produces (not just user inputs). The team wants to use Databricks-native tooling with minimal custom code. Which approach best satisfies this requirement?

Accepted Answer

Enable AI Guardrails safety filters on the serving endpoint so that both prompts and model responses are screened, and unsafe outputs are blocked or flagged before returning to the user

Answer

Rely solely on a system prompt instructing the model to never produce harmful content, since well-crafted prompts guarantee safe outputs

Answer

Apply Unity Catalog column masking on the source Delta tables so toxic content is never retrieved during RAG

Answer

Configure inference tables to log all responses and run a nightly batch job that reviews flagged outputs after they have been sent to users

Question 48

A GenAI engineer deployed a RAG chatbot to a Model Serving endpoint two weeks ago. Leadership now wants a Lakehouse Monitoring dashboard that tracks request volume, latency trends, and response quality drift starting from launch day. When the engineer opens the endpoint, they discover no inference table exists and no payload logs were captured. What is the correct conclusion about building the requested historical dashboard?

Accepted Answer

Historical data from launch cannot be recovered; inference tables only capture payloads logged after the feature is enabled on the endpoint, so monitoring can only begin going forward once logging is turned on.

Answer

Lakehouse Monitoring can automatically backfill the inference table by replaying all historical requests stored in the endpoint's internal cache.

Answer

The engineer should enable MLflow autologging retroactively, which will reconstruct the two weeks of missing request and response payloads.

Answer

Because the endpoint already served traffic, the inference table exists by default and simply needs to be attached to a Lakehouse Monitoring profile.

Question 49

A Generative AI Engineer has deployed a RAG-based customer support assistant with inference tables enabled to capture request and response payloads. Over the past month, users have started asking about a newly launched product line that did not exist when the app was first evaluated. The engineer wants to detect that the distribution of incoming questions has shifted away from the topics covered in the original evaluation set, so they can decide when to refresh the knowledge base and re-run evaluation. Which approach best enables ongoing detection of this shift?

Accepted Answer

Attach a Lakehouse Monitoring inference profile to the inference table, define the original evaluation topic distribution as the baseline, and monitor for drift in the incoming request features.

Answer

Configure Lakehouse Monitoring on the inference table using a snapshot profile that only tracks row counts per day.

Answer

Increase the LLM-as-judge frequency to score every production response for correctness in real time.

Answer

Enable route-optimized serving on the endpoint to reduce latency for the new product questions.

Question 50

A GenAI engineer has deployed a customer-support RAG assistant with an inference table enabled and a Lakehouse Monitoring monitor of type 'InferenceLog' configured on top of it. The team wants the monitoring dashboards (drift, quality, and volume metrics) to reflect new production traffic automatically without an engineer manually triggering computation each morning. The monitor currently only updates when someone clicks 'Refresh metrics' in the UI. What is the most appropriate way to keep the monitor's metric tables and dashboard current on an ongoing basis?

Accepted Answer

Configure a schedule on the monitor so that its profile and drift metric tables are recomputed automatically at a defined cron interval.

Answer

Enable Delta Change Data Feed on the inference table so the monitor recomputes metrics on every row insert.

Answer

Increase the model serving endpoint's provisioned throughput so inference logs are written faster to the monitor tables.

Answer

Recreate the monitor as a 'Snapshot' type monitor, which continuously streams metrics as new data arrives.

Question 51

You are building a customer-support agent on Databricks using LangChain. The agent must answer product-availability questions by querying a live inventory system exposed as a Python function, and it must also handle general product FAQs from a knowledge base. You want the LLM to decide autonomously when to call the inventory function versus when to answer directly. Which approach correctly enables the agent to invoke the inventory function only when needed?

Accepted Answer

Wrap the inventory function as a LangChain tool with a clear name and description, bind the tools to a function-calling-capable chat model, and let the agent's reasoning loop decide when to call it.

Answer

Concatenate the full inventory table into the system prompt so the model can read stock levels directly without any tool calls.

Answer

Create a fixed LCEL chain that always calls the inventory function first, then passes the result to the LLM for every user query.

Answer

Fine-tune the foundation model on inventory snapshots so it memorizes current stock levels and can respond without external calls.

Question 52

You are building a customer-support assistant on Databricks using LangChain. The assistant must handle long, multi-turn troubleshooting sessions that can exceed 30 exchanges. Early in the session, users describe their environment (OS version, product edition, error codes), and this detail must remain available for the model throughout the whole conversation. However, you are hitting the context window limit of your foundation model after about 20 turns because the full raw history is being passed each time. Which conversation memory approach best preserves the critical early context while staying within the token budget?

Accepted Answer

Use ConversationSummaryBufferMemory, which keeps recent turns verbatim and progressively summarizes older turns to retain key details within a token limit.

Answer

Use ConversationBufferMemory to always pass the complete raw message history so no detail is ever lost.

Answer

Use ConversationBufferWindowMemory with k=5 to keep only the last five exchanges and discard everything older.

Answer

Disable memory entirely and rely on the vector search retriever to fetch prior turns as needed.

Question 53

You are building a RAG chain with LangChain Expression Language (LCEL) on Databricks. Your chain must call two independent retrievers — one against a product-documentation vector index and another against a support-ticket vector index — then merge their results before passing everything to the prompt. Profiling shows the two retrieval calls currently run one after the other, adding significant latency. Which LCEL construct should you use to invoke both retrievers concurrently and combine their outputs into a single dictionary for the downstream prompt?

Accepted Answer

Wrap the two retrievers in a RunnableParallel (a dict of runnables) so both execute concurrently and their outputs are collected under named keys.

Answer

Chain the two retrievers sequentially with the pipe operator and rely on LCEL to automatically parallelize independent steps.

Answer

Use a RunnablePassthrough to forward the query to both retrievers, which guarantees they run in parallel.

Answer

Place both retrievers inside a RunnableBranch, which routes the query to each retriever concurrently based on content.

Question 54

You are building a LangChain (LCEL) chain that classifies incoming customer emails using a Databricks Foundation Model API endpoint. During a nightly job, you must process roughly 50,000 emails that are already loaded into a Python list. Your current code loops over the list and calls chain.invoke() once per email, but the job is too slow to finish within its window. Without changing the model or endpoint, what is the most appropriate way to improve throughput while keeping the same LCEL chain?

Accepted Answer

Replace the loop with a single call to chain.batch() passing the full list of inputs so the runnable processes them concurrently.

Answer

Wrap the chain in chain.stream() so tokens are returned incrementally and the job overlaps generation with I/O.

Answer

Call chain.invoke() inside a Python for-loop but set temperature=0 to make each request return faster.

Answer

Convert the chain to an agent with a classification tool so it can decide how to parallelize the work automatically.

Question 55

You are building a customer support assistant on Databricks. The application must handle two very different request types: (1) factual questions that should be answered from a Vector Search-backed knowledge base, and (2) requests to calculate refund amounts that require a deterministic Python function. Both request types arrive at the same chat endpoint. You want a single LangChain LCEL pipeline that inspects each incoming query and dispatches it to the appropriate sub-chain. Which LCEL construct is the most appropriate way to implement this dispatch logic?

Accepted Answer

Use RunnableBranch to evaluate conditions on the input and route to the retrieval sub-chain or the calculation sub-chain accordingly

Answer

Use RunnableParallel to execute both the retrieval sub-chain and the calculation sub-chain simultaneously and merge their outputs

Answer

Use RunnablePassthrough to forward the raw query unchanged so the foundation model decides how to respond

Answer

Chain the two sub-chains sequentially with the pipe operator so retrieval always runs before calculation

Question 56

You are building a customer-facing RAG chatbot on Databricks using LangChain LCEL. The chain calls a pay-per-token Foundation Model API endpoint for generation. During load testing, occasional 429 rate-limit errors and transient endpoint failures cause the entire chain to fail and return errors to users. You want the chain to automatically try a secondary provisioned-throughput endpoint when the primary call fails, without rewriting the chain logic. Which LangChain approach best satisfies this requirement?

Accepted Answer

Wrap the primary model runnable with .with_fallbacks([secondary_model]) so LCEL invokes the backup endpoint when the primary raises an exception.

Answer

Increase the retriever's top_k so more context is passed, reducing the chance the model endpoint returns an error.

Answer

Set a higher temperature on the primary model so it produces output even when the endpoint is rate-limited.

Answer

Use RunnableParallel to always call both endpoints simultaneously and return whichever finishes first.

Question 57

You are building a RAG chain with LangChain Expression Language (LCEL) on Databricks. Your chain retrieves context from a Vector Search index and formats a prompt that needs BOTH the retrieved context AND the user's original question. You write a retrieval step that takes the incoming question string, embeds it, and returns formatted document text. However, when you compose the chain, the prompt template's `{question}` variable ends up empty because the retriever consumed the input and only passed forward the document text. What is the most idiomatic LCEL construct to fix this so the prompt receives both the context and the original question?

Accepted Answer

Wrap the retrieval in a `RunnableParallel` (dict) that maps `context` to the retriever and `question` to `RunnablePassthrough()`, so both keys flow into the prompt.

Answer

Add a `RunnableLambda` after the prompt template that re-injects the question from a global variable captured at chain-build time.

Answer

Call the retriever with `.batch()` instead of `.invoke()` so it returns the question alongside the documents automatically.

Answer

Set `return_source_documents=True` on the retriever so the original question is preserved in the chain output metadata.

Question 58

A Generative AI Engineer is building a RAG chain with LangChain on Databricks. Business stakeholders require that every generated answer include a list of the source documents that were retrieved and passed to the LLM, so end users can verify claims. The engineer is composing the chain using LCEL (LangChain Expression Language) with a Databricks Vector Search retriever and a Foundation Model API chat model. What is the most appropriate way to satisfy this requirement while keeping the generated answer and the sources in a single structured output?

Accepted Answer

Use RunnableParallel to run the retriever and pass through the question, then structure the output so that the retrieved documents flow alongside the LLM-generated answer into a final dictionary containing both 'answer' and 'context' keys.

Answer

Increase the retriever's k value so more documents are retrieved, and instruct the LLM in the prompt to copy every retrieved chunk verbatim into its answer text.

Answer

Enable inference tables on the model serving endpoint so the retrieved documents are automatically appended to each response returned to the client.

Answer

Store the retrieved documents in a separate Delta table keyed by request timestamp so users can join them to the answer afterward.

Question 59

You are building a customer-support RAG assistant using LangChain LCEL on Databricks. The chain retrieves context from Mosaic AI Vector Search, injects it into a prompt, and calls a Foundation Model API chat endpoint. Product wants the assistant to display the model's answer progressively as it is generated in the web chat UI, but the retrieval and prompt-assembly steps should complete before any output appears. Which approach correctly implements this behavior with a single LCEL chain?

Accepted Answer

Invoke the full chain with `.stream()`; the retriever and prompt steps run to completion first, and only the final LLM step yields incremental tokens as they are produced.

Answer

Call `.invoke()` on the chain and set `temperature=0` so the endpoint returns tokens one at a time to the UI.

Answer

Wrap the retriever in a `RunnableParallel` so it streams document chunks to the UI while the LLM generates the answer simultaneously.

Answer

Use `.batch()` on the chain with a batch size of 1, which forces the endpoint into streaming mode for the final step.

Question 60

A Generative AI Engineer is building a LangChain LCEL chain for a customer support assistant. For each incoming ticket, the chain must simultaneously (1) classify the ticket priority and (2) extract the product name, then feed both results into a final prompt that drafts a response. Both the classification and extraction steps call the same Foundation Model endpoint and are independent of each other. The engineer wants to minimize end-to-end latency. Which LCEL construct should be used to run the two independent steps?

Accepted Answer

Wrap the two steps in a RunnableParallel (RunnableMap) so they execute concurrently and return a dict of both outputs to the downstream prompt

Answer

Chain the two steps sequentially with the pipe (|) operator so the extraction runs after classification completes

Answer

Use a RunnableBranch to route the ticket to either classification or extraction based on a condition

Answer

Use RunnablePassthrough alone to forward the raw ticket text to the final prompt without transformation

Question 61

A Generative AI Engineer is building a customer-facing chat assistant on Databricks using LangChain Expression Language (LCEL). The frontend team requires that partial responses appear token-by-token in the UI as the foundation model generates them, rather than waiting for the full completion. The chain is composed as: prompt | chat_model | StrOutputParser(). Which approach correctly enables incremental token delivery from this LCEL chain?

Accepted Answer

Invoke the chain with chain.stream(input) and iterate over the yielded chunks, forwarding each to the UI as it arrives

Answer

Call chain.invoke(input) inside a loop and send the accumulated string on each iteration

Answer

Wrap the chain with chain.batch([input]) so multiple partial results are returned as a list

Answer

Set the StrOutputParser to async mode and call chain.ainvoke(input) to receive tokens as they complete

Question 62

A Generative AI Engineer builds a LangChain chain on Databricks that extracts contact details (name, email, phone) from customer inquiry emails. The LLM is instructed to return the details, and the extracted values must be passed to a downstream Python function that expects a validated dictionary with those three keys. Currently the chain returns raw text from the model, and the downstream function frequently fails when the model adds explanatory prose or inconsistent formatting. Which change most directly addresses this problem within the LangChain chain?

Accepted Answer

Append a Pydantic-based structured output parser (e.g., with_structured_output or a PydanticOutputParser) so the model response is coerced and validated into the expected schema

Answer

Add a StrOutputParser to the end of the chain so the raw model string is cleanly extracted before the function call

Answer

Increase the temperature of the foundation model so it produces more varied phrasings of the contact details

Answer

Wrap the LLM call in a RunnableParallel so the contact fields are generated concurrently and merged

Question 63

A Generative AI Engineer is building a RAG chain in LangChain on Databricks. Documents are already indexed in a Databricks Vector Search Delta Sync index backed by a Foundation Model API embedding endpoint. The engineer wants the chain to retrieve relevant chunks at query time and pass them to a chat model, using the least custom code and the officially supported integration. Which approach should the engineer use to wire the vector index into the LangChain chain?

Accepted Answer

Instantiate DatabricksVectorSearch with the index name and call its as_retriever() method, then compose it into a retrieval chain with the chat model.

Answer

Manually query the index with the Vector Search Python SDK inside a custom RunnableLambda, embedding the query with a separate SentenceTransformers model each time.

Answer

Load all chunks from the underlying Delta table into an in-memory FAISS vector store at chain startup and retrieve from FAISS.

Answer

Use the SQL connector to run a similarity query against the Delta table directly and parse the returned rows as documents in a custom retriever.

Question 64

A Generative AI Engineer is building a RAG chain with LangChain over a Databricks Vector Search index of product manuals. During testing, users ask questions whose answers span multiple sections of a manual (e.g., 'What are all the steps to reset and recalibrate the device?'). The current retriever is configured with search_kwargs={'k': 2}, and the LLM frequently produces incomplete answers that miss steps. Retrieval relevance for the top results is high, and the embedding model is appropriate for the domain. What is the most effective adjustment to improve answer completeness?

Accepted Answer

Increase the retriever's k value so more relevant chunks are passed into the prompt context, staying within the model's context window.

Answer

Switch the Foundation Model API endpoint from pay-per-token to provisioned throughput to reduce latency.

Answer

Replace the embedding model with a larger one to increase retrieval relevance.

Answer

Lower the LLM temperature to 0 so the model generates more deterministic answers.

Exam domain	Exam weight	Practice
Design Applications	14%	Practice this topic
Data Preparation	14%	Practice this topic
Application Development	30%	Practice this topic
Assembling and Deploying Applications	22%	Practice this topic
Governance	8%	Practice this topic
Evaluation and Monitoring	12%	Practice this topic

Databricks Certified Generative AI Engineer Associate (Databricks GenAI Engineer Associate) practice exam & study guide

Start studying Databricks GenAI Engineer Associate

All Databricks GenAI Engineer Associate study resources

Databricks GenAI Engineer Associate exam domains

Sample Databricks GenAI Engineer Associate questions

Key Databricks GenAI Engineer Associate terms

Databricks GenAI Engineer Associate frequently asked questions

Design Applications (14%)

Data Preparation (14%)

Application Development (30%)

Assembling and Deploying Applications (22%)

Governance (8%)

Evaluation and Monitoring (12%)