Question

A data engineering team is streaming clickstream events from Apache Kafka topics into Snowflake. They require the lowest possible ingestion latency (seconds, not minutes) and want to minimize per-file overhead and storage of intermediate files in a stage. Which configuration of the Snowflake Kafka connector should they use?

Accepted Answer

Configure the Kafka connector to use Snowpipe Streaming, which writes rows directly into target tables via the streaming ingest API without staging files

Answer

Configure the Kafka connector to use file-based Snowpipe, buffering records into files in an internal stage that are then loaded with COPY INTO

Answer

Use the Kafka connector with a scheduled task that runs COPY INTO from an external stage every minute

Answer

Replace the Kafka connector with the Spark connector and write micro-batches using overwrite mode