Question

A data engineer defines a Lakeflow Declarative Pipeline (Delta Live Tables) table with two expectations on a single dataset: one uses @dlt.expect_or_drop for a 'valid_amount' rule and another uses @dlt.expect (warn only) for a 'complete_customer' rule. During an update, a record fails the 'valid_amount' rule but passes 'complete_customer', while a second record passes 'valid_amount' but fails 'complete_customer'. What happens to these two records in the target table?

Accepted Answer

The first record is dropped; the second record is retained and its violation is only recorded in the metrics.

Answer

Both records are dropped because any expectation violation removes the row.

Answer

The pipeline update fails immediately because two conflicting expectations cannot be applied to the same dataset.

Answer

Both records are retained because expect_or_drop and expect only log violations without removing rows.