A data science team needs a cloud analytics platform where they can write collaborative Apache Spark notebooks in Python and Scala, run large-scale data transformations on distributed clusters, and train machine learning models. The team specifically wants an optimized, managed Spark environment. Which Azure service best meets these requirements?