Question

A data engineer is building a Lakeflow Declarative Pipeline (formerly Delta Live Tables) for a silver customer table. Business rules require that any record with a NULL email be excluded from the resulting table, but the pipeline must continue processing all other valid records without failing. Which expectation clause should the engineer use?

Accepted Answer

@dlt.expect_or_drop("valid_email", "email IS NOT NULL")

Answer

@dlt.expect("valid_email", "email IS NOT NULL")

Answer

@dlt.expect_or_fail("valid_email", "email IS NOT NULL")

Answer

@dlt.expect_all("valid_email", "email IS NULL")