Hard — Databricks Data Engineer Associate Practice Quiz

Question 1

A data engineering team manages a Databricks Asset Bundle that deploys a job to both a staging and a production workspace. When they promote a new release, they want the production job to always run the code from a specific, immutable Git tag rather than from files copied into the production workspace. Which approach best supports this requirement?

Accepted Answer

Configure the job task to use a Git source in databricks.yml, pinning it to the release tag, so the production deployment references the versioned Git commit directly.

Answer

Manually export the notebooks from staging and import them into the production workspace before each release.

Answer

Set bundle mode to 'development' for the production target so the workspace files are prefixed and isolated per release.

Answer

Store the code only in DBFS and reference the DBFS path in the job definition to guarantee immutability.

Question 2

A data engineer is writing a MERGE statement to update a Delta table of customer accounts. Business rules require that when a matched record has a status of 'CLOSED' in the source, the target row should be deleted; for all other matched records, the target row should be updated with the new balance. Which MERGE clause construction correctly implements both behaviors?

Accepted Answer

Use two WHEN MATCHED clauses: the first with condition `s.status = 'CLOSED'` THEN DELETE, followed by a second WHEN MATCHED THEN UPDATE SET t.balance = s.balance.

Answer

Use a single WHEN MATCHED THEN UPDATE clause and rely on a separate DELETE statement run before the MERGE.

Answer

Use two WHEN MATCHED clauses ordered as UPDATE first, then the DELETE with condition `s.status = 'CLOSED'`, since order does not affect evaluation.

Answer

Use a single WHEN MATCHED clause with a CASE expression that returns DELETE or UPDATE based on `s.status`.

Question 3

A data engineer defines a Lakeflow Declarative Pipeline (Delta Live Tables) table with two expectations on a single dataset: one uses @dlt.expect_or_drop for a 'valid_amount' rule and another uses @dlt.expect (warn only) for a 'complete_customer' rule. During an update, a record fails the 'valid_amount' rule but passes 'complete_customer', while a second record passes 'valid_amount' but fails 'complete_customer'. What happens to these two records in the target table?

Accepted Answer

The first record is dropped; the second record is retained and its violation is only recorded in the metrics.

Answer

Both records are dropped because any expectation violation removes the row.

Answer

The pipeline update fails immediately because two conflicting expectations cannot be applied to the same dataset.

Answer

Both records are retained because expect_or_drop and expect only log violations without removing rows.

Question 4

A data engineer maintains a Silver customer table in Delta Lake. Each micro-batch of change data contains one or more rows per customer_id, and the goal is to keep only the latest state per customer: update existing customers, insert new ones. Which approach correctly and idempotently applies these updates in a single command?

Accepted Answer

Use MERGE INTO with the target matched on customer_id, WHEN MATCHED THEN UPDATE SET *, and WHEN NOT MATCHED THEN INSERT *, after first deduplicating the source to one row per customer_id.

Answer

Use INSERT INTO to append all change rows, then rely on Delta to automatically discard older duplicate customer_id values.

Answer

Use MERGE INTO directly on the raw change batch without deduplication, matching on customer_id.

Answer

Use INSERT OVERWRITE on the entire target table with the incoming change batch each micro-batch.

Question 5

A data engineer runs OPTIMIZE on a Delta table daily to improve query performance. A colleague asks to be able to query the table as it existed 10 days ago using time travel. However, the engineer recently ran VACUUM my_table RETAIN 48 HOURS on the table. What is the outcome when the colleague attempts the 10-day-old time travel query?

Accepted Answer

The query fails because VACUUM removed the underlying data files older than 48 hours that the older version referenced.

Answer

The query succeeds because Delta always retains 30 days of history regardless of VACUUM settings.

Answer

The query succeeds because OPTIMIZE preserves all historical versions in the transaction log permanently.

Answer

The query fails because time travel is disabled once VACUUM is run on a table.