Databricks Intelligence Platform

A data engineering team runs a nightly production ETL pipeline that is triggered automatically by the Databricks Jobs scheduler. During code review, a teammate notices the pipeline is configured to run on an all-purpose cluster that stays running 24/7 so analysts can also attach notebooks to it interactively during the day. Management wants to reduce compute cost for the scheduled pipeline without affecting the analysts' interactive work. What is the most appropriate change?

A data engineering team runs a streaming job that continuously appends records to a Delta table while an analyst simultaneously runs a large aggregation query against the same table. The analyst is concerned that they might read a mix of old and newly written data mid-transaction, producing inconsistent results. Which characteristic of Delta Lake ensures the analyst's query returns consistent results?

A data engineering team is evaluating whether to standardize their lakehouse tables on Delta Lake. A senior architect raises a concern about vendor lock-in and asks how Delta Lake tables physically store their underlying data in cloud object storage. Which statement correctly describes the storage format used by Delta Lake?

A data engineer notices that a Delta table used for hourly streaming ingestion has accumulated thousands of tiny files over the past week, slowing down downstream read queries. The engineer wants to compact these small files into larger ones to improve read performance without changing the table's data or partitioning scheme. Which command should the engineer run?

A data engineer notices that a critical Delta table in the silver layer has produced unexpected row counts after a series of overnight jobs. Before deciding whether to roll back, they need to review a chronological log of every operation performed on the table, including the operation type (WRITE, MERGE, DELETE), the user, and the version number for each change. Which approach lets them retrieve this information directly?

A data engineer inspects a Delta table's storage location in cloud object storage and finds a subfolder named _delta_log containing several JSON files and one Parquet checkpoint file, alongside the data Parquet files. A junior colleague asks what the _delta_log directory is responsible for. Which statement best describes its purpose?

A data engineer accidentally ran a DELETE statement that removed thousands of rows from a Delta table named sales_transactions 20 minutes ago. No VACUUM has been run since the mistake, and the table retention settings are at their defaults. The engineer needs to restore the table to its exact state before the erroneous DELETE. Which approach will correctly recover the data?

A retail analytics team has a medallion pipeline. Raw clickstream and sales events land in bronze, and cleaned, deduplicated, and conformed records are stored in silver. The BI team now needs a table that provides daily revenue by product category and region, pre-aggregated so their dashboards load quickly without scanning millions of rows. Which layer should this new table be created in, and why?

A data engineering team is designing a medallion architecture for streaming IoT sensor data. The raw events arrive as semi-structured JSON with inconsistent schemas and occasional duplicate records. The team wants the first landing layer to preserve the data exactly as received from the source, including any malformed rows, so they can reprocess if downstream transformation logic changes. In which layer should this raw data be stored, and why?

A data engineering team ingests raw JSON clickstream events directly into a Delta table named events_bronze, preserving the exact source data including malformed records and duplicates. Downstream, a business analytics team needs cleaned, deduplicated, and validated event records with standardized schema for building aggregated dashboards. According to the medallion architecture, which layer should the team create to hold the cleaned and conformed data before it is aggregated for the dashboards?

A data engineer manages a Delta table that receives frequent small streaming writes throughout the day. Analysts complain that their queries filtering on the customer_id column have become slow, and they notice thousands of tiny files in the table's storage. Which single command should the engineer run to compact the small files AND co-locate data by customer_id to improve query filtering performance?

A data engineer runs OPTIMIZE on a Delta table daily to improve query performance. A colleague asks to be able to query the table as it existed 10 days ago using time travel. However, the engineer recently ran VACUUM my_table RETAIN 48 HOURS on the table. What is the outcome when the colleague attempts the 10-day-old time travel query?