Question

A data engineering team maintains several notebooks across a Fabric workspace that all require the same set of public PyPI packages and a shared Spark configuration (executor memory and dynamic allocation limits). Currently each engineer installs libraries with %pip inline commands at the top of every notebook, which slows session startup and causes version drift. The team wants a governed, reusable way to apply the same libraries and Spark properties to all notebooks in the workspace without editing each notebook. What should they do?

Accepted Answer

Create a Fabric Environment item, add the required libraries and Spark properties to it, and set it as the workspace default environment in the workspace Spark settings.

Answer

Enable high-concurrency mode in the workspace so notebooks share a single Spark session and inherit each other's installed packages.

Answer

Configure a custom Spark pool with a larger node size and instruct engineers to keep using %pip install in each notebook.

Answer

Add the packages to a requirements.txt file in the attached lakehouse Files section and reference it from each notebook at runtime.