My issue with notebooks is that it takes like 1000 years to spin up - 5 minutes for spark warm up is uncool. There are not many videos or info how to tune or even resolve it. I'd be interested in that.
If you're using the starter pools with no custom libraries installed then it should start in 8-9 seconds, it's only if the defaults don't work you have to wait for machines to spin up!
I@@matthiask4602 hehe, I still use those old days - prices of those "data delivery to your door" tools are immense - good ol' SQL with some knowledge about db engine or two and you can simulate those functions for cheap
Yes, Simon, please share your opinions about the layer concepts of data lakes!
czcams.com/video/fz4tax6nKZM/video.htmlsi=KFgv5hQPM3hdP8f4
I want to hear Simon's opinions on Bronze, Silver and Gold layers!!!
Me too!
there is a video on that :)
I like this platform videos, support to you guys from India. 🇮🇳
Yes to thoughts on medallion 🥇 Thank you both for this video!
Thank you for it. I always wanted to start with spark and that is a great starter :)
Amazing crossover :)
Simon is super-cool!
This was a great vid. Is it possible to use the sql cell to add columns to an existing Lakehouse table?
6:49 I want to hear Simon's opinion on Medallion. It's just ETL to me.
First comment :), joking, but Simon and Adam thanks for sharing, hope we can bring PySpark and Fabric in a Saturday live
I'd like to hear Simon's perspective.
I love him. Bring him back. Make him show off his DELTA tattoo!
My issue with notebooks is that it takes like 1000 years to spin up - 5 minutes for spark warm up is uncool. There are not many videos or info how to tune or even resolve it. I'd be interested in that.
If you're using the starter pools with no custom libraries installed then it should start in 8-9 seconds, it's only if the defaults don't work you have to wait for machines to spin up!
Simon what are you doing here!
I liked the old days when we had only sql, M and DAX 👵
and we had datasets instead of semantic models
I@@matthiask4602 hehe, I still use those old days - prices of those "data delivery to your door" tools are immense - good ol' SQL with some knowledge about db engine or two and you can simulate those functions for cheap
And just Power Pivot pivot tables for visualization.