Hard AI-300 practice questions

You maintain a fraud-detection model deployed to a managed online endpoint in Azure Machine Learning. A model monitor already computes prediction drift and feature-attribution drift on a recurring schedule. The business requires that when the monitored metric exceeds a defined threshold, a retraining pipeline is automatically kicked off without a data scientist manually reviewing the results first. Which approach best implements this automated retraining trigger?

You are configuring an automated machine learning classification job in Azure Machine Learning to predict fraudulent transactions. Only about 2% of the transactions in your training dataset are labeled as fraud. Your business goal is to correctly identify as many fraudulent transactions as possible while keeping false positives manageable. Which primary metric should you configure for the AutoML job to best guide model selection?

Your team must provision an Azure Machine Learning workspace using Bicep as part of a GitHub Actions pipeline. Company security policy requires that the workspace and its associated resources (storage account, key vault) not be reachable from the public internet, and that all traffic flow through the corporate virtual network. Which combination should you configure in the Bicep template to meet this requirement?

You are training a large deep learning model on Azure Machine Learning using a compute cluster of 8 GPU nodes. The model no longer fits into the memory of a single GPU, so you need to shard the model parameters, gradients, and optimizer states across all available GPUs to reduce per-device memory consumption. Which approach should you configure for the training job?

Your team's Azure Machine Learning workspace is configured with a private endpoint and public network access disabled. When you submit a job that references a custom environment defined by a Dockerfile, the environment image build fails because the default build process cannot reach the workspace's container registry over the public network. You need to allow custom environment images to build successfully while keeping the workspace isolated from the public internet. What should you do?

You are preparing a synthetic dataset to fine-tune a domain-specific chat model in Azure AI Foundry. The synthetic examples were generated by a larger teacher model using prompts derived from your internal knowledge base. Before training, you split the data into training and evaluation sets. After fine-tuning, the model reports excellent evaluation scores, but real-world performance on held-out user queries is noticeably worse. Which action should you take first to make the evaluation scores trustworthy?

You fine-tuned a base language model on a large domain-specific dataset for a legal-document assistant. After deployment, evaluation shows the model now performs well on legal tasks but has significantly degraded on general reasoning and instruction-following tasks it previously handled. You must re-run the customization to preserve general capabilities while still gaining domain accuracy, using minimal compute. Which approach best addresses this problem?

You fine-tuned a GPT model in Azure AI Foundry for a customer-support workload. The model performed well on offline evaluation, but leadership requires evidence that it behaves correctly on real production traffic before it replaces the current model, without exposing customers to the new model's responses. Which promotion approach lets you validate the fine-tuned model against live traffic while ensuring users continue to receive responses only from the current production model?

Your team is deploying an Azure AI Foundry hub and project for a regulated financial workload. Security policy requires that no data traverses the public internet: both inbound access to the hub and the hub's outbound connections to its dependent Azure resources (Storage, Key Vault, Container Registry) must stay on the Microsoft backbone. You need to configure the Foundry hub's networking so these requirements are met with the least ongoing management overhead. Which configuration should you choose?

Your team has deployed an Azure AI Foundry hub with a managed virtual network and configured a private endpoint so that all inbound access is restricted to your corporate network. On-premises developers connected through ExpressRoute report that they can reach the Foundry portal by IP address but consistently get name-resolution failures when using the resource's fully qualified domain name. What is the most appropriate action to resolve the connectivity issue?

Your team is deploying a foundation model as a serverless API endpoint in Azure AI Foundry for a new customer-facing application. The application must run in the East US 2 region to meet data residency requirements. During deployment via the Azure CLI, you discover that the specific model you selected cannot be deployed as a serverless API in that region. Business stakeholders require the exact same model family and version, and they cannot change the residency requirement. What is the most appropriate action to satisfy both the model and residency constraints?

Your team deployed a loan-approval model to a managed online endpoint with inference data collection enabled. Business stakeholders report that predictions seem to be shifting even though the overall input distribution appears stable. You want Azure Machine Learning model monitoring to specifically identify which input features are contributing differently to the model's predictions over time, so you can pinpoint the cause. Which monitoring signal should you configure?

You deployed a fraud-detection model to a managed online endpoint. Actual fraud labels (ground truth) only become available 30 to 45 days after a transaction is scored, once chargebacks are confirmed. Your team wants to monitor the model's classification accuracy and F1 score in production and alert when they degrade below a threshold. Which approach correctly configures Azure Machine Learning model monitoring to compute these performance metrics?

Your team is standardizing Azure Machine Learning workspace deployments through Bicep templates checked into a Git repository. Security policy requires that each workspace use a pre-created user-assigned managed identity (rather than a system-assigned identity) so that the same identity can be granted permissions on shared storage and Key Vault resources before the workspace exists. In the Microsoft.MachineLearningServices/workspaces Bicep resource, how should you configure the identity to meet this requirement?

Your production RAG system uses a general-purpose embedding model with 1,536 dimensions, and its vector index in Azure AI Search contains millions of chunked documents. Evaluation shows poor retrieval relevance for your specialized medical terminology. You fine-tune a domain-specific embedding model that outputs 768 dimensions and it scores significantly higher on your relevance benchmark. What must you do before the fine-tuned embedding model can serve production retrieval queries?

You operate a RAG solution for a legal firm. The knowledge base contains contracts full of specialized legal terminology and clause references. Users report that retrieval frequently returns generic passages instead of the precise clauses they ask about, even after you tuned chunk size and similarity threshold. Analysis shows the general-purpose embedding model places semantically distinct legal terms too close together in vector space. You want the most durable improvement to retrieval accuracy for this domain. What should you do?

Your RAG solution for a legal research assistant retrieves the top 20 chunks from an Azure AI Search hybrid index, but users report that highly relevant passages often appear far down the passed context and the LLM misses them. Retrieval recall is high, but the final answers lack precision. You must improve which chunks the LLM actually receives without re-indexing or changing chunk sizes. What should you add to the retrieval pipeline?

Your team is deploying a new Azure Machine Learning workspace with Azure CLI (ml extension). You must create a datastore that connects to an existing Azure Data Lake Storage Gen2 container using a service principal. Company policy forbids storing the client secret in any YAML file, Bicep template, or command-line argument that could end up in source control. How should you supply the service principal credential when running 'az ml datastore create'?