Question

A compliance auditor asks a Generative AI Engineer to demonstrate which source Delta tables ultimately feed a production RAG chatbot's Vector Search index, and to prove that a specific customer table was used to build the embeddings. All assets (source tables, embedding function, Vector Search index, and the registered chain model) are managed in Unity Catalog. What is the most reliable way to produce this evidence?

Accepted Answer

Use Unity Catalog's automatically captured data lineage graph to trace the upstream tables connected to the Vector Search index and downstream to the registered model.

Answer

Manually inspect the notebook code that created the Vector Search index and record the table names in a spreadsheet for the auditor.

Answer

Enable inference tables on the model serving endpoint and parse request payloads to infer which source tables were involved.

Answer

Query the vector index directly and match returned chunk text against rows in the customer table to establish the connection.