Question

A data engineer uses Auto Loader to ingest JSON files into a bronze Delta table. The stream is configured with a schema location, and the default schema evolution mode is in effect. One day the upstream team adds a new field to the JSON files. What happens to the Auto Loader stream when it encounters the new column?

Accepted Answer

The stream fails with an UnknownFieldException, updates the schema in the schema location, and succeeds automatically when restarted.

Answer

The stream silently drops the new column and continues running without interruption.

Answer

The stream places all values of the new column into the _rescued_data column permanently.

Answer

The stream ignores the change until OPTIMIZE is run on the target Delta table.