

What we’re about
We are a User Group, passionate about data and how the Microsoft Data Platform helps you store, transform and get insights into your data.
Centered around bringing people together, to share our passion and enthusiasm, we have loads of fun while doing so.
Upcoming events (2)
See all- Outperform Spark with Python Notebooks in FabricLink visible for attendees
Detailed information and registration can be found here :
https://dataminds.be/outperform-spark-with-python-notebooks-in-fabric/# Outperform Spark with Python Notebooks in Fabric
Session:
When Microsoft Fabric was released, it came with Apache Spark out of the box. Spark’s ability to work with more programming languages opened up possibilities for creating data-driven and automated lakehouses. On the other hand, Spark’s primary feature to scale out and handle large amounts of data will, in many cases, be over-dimensioned, less performant, and more costly when working with trivial workloads.
With Python Notebooks, we have a better tool for handling metadata, automation, and processing of more trivial workloads, while still having the option to use Spark Notebooks for handling more demanding processing.We will cover:
* The difference between Python Notebooks and a Single Node Spark cluster, and why Spark Notebooks are more costly and less performant with certain types of workloads.
* When to use Python Notebooks and when to use Spark Notebooks.
* Where to use Python Notebooks in a meta-driven Lakehouse
* A brief introduction to tooling and move workload between Python Notebooks and Spark Notebooks.
* How to avoid overload the Lakehouse tech stack with python technologies.
* CostsSpeakers:
Christian Henrik Reich | Principal Architect @ twoday Data & AIAgenda (CEST, UTC+2):
18u30 Welcome and introductions
18u30 Outperform Spark with Python Notebooks in Fabric (60 minutes)
19u45 Session End