Question

A Generative AI Engineer at a healthcare startup needs to integrate OpenAI's GPT-4 into a Databricks-based application. Company policy requires that the OpenAI API key never be hard-coded in notebooks or application code, and that all model calls flow through a governed, centrally-managed interface that supports rate limiting and usage tracking on Databricks. What is the recommended approach to configure this integration?

Accepted Answer

Store the OpenAI API key in a Databricks secret scope and create a Mosaic AI Model Serving external model endpoint that references the secret

Answer

Hard-code the OpenAI key in a shared cluster environment variable and call the OpenAI SDK directly from each notebook

Answer

Provision throughput for GPT-4 through the Foundation Model APIs pay-per-token endpoint and pass the key as a query parameter

Answer

Download GPT-4 weights from OpenAI and register the model in Unity Catalog for local serving