Azure Data Factory, Azure Databricks, or Azure Synapse Analytics? When to use what. - Lisa Hoving
Vložit
- čas přidán 4. 10. 2022
- Presentation from Sytac Azure Night www.meetup.com/sytac-dev-jam/...
Have you ever found yourself at the start of a data (engineering) project, unsure about what tool to choose? Speak no more! In this session we will discuss three often used (data engineering) tools on Azure:
- Azure Data Factory
- Azure Databricks
- Azure Synapse Analytics
First, I will give a brief introduction to each tool, including for what use cases they are commonly used. Next, we will talk about what makes these tools different, and where they overlap (and believe me, they do). Lastly, we will arrive at our conclusion: when to use what. Of course, a slide on differences in costs will be included as well.
Lisa is a friendly Data Engineer from the neighbourhood. Her background comes in Finance, but she has had the pleasure to be working in IT for the past 5 years. At Intercept, she helps customers in migrating their data applications to the Azure cloud. - Věda a technologie
One of the best CZcams videos I have ever watched. Now, I understand the difference between these 3. Great job, teacher!
very nicely explained , thankyou so much...
I did love your explanation about the differences between these three environment, Lisa. It clarified a lot of doubts that I had in my mind about it. Thanks a lot.
Just the content I was looking for, thanks for sharing in such a clear and concise way!
Awesome explanations. Thanks, Lisa Hoving
thank you lisa from india🙏
Very helpful. Thank you so much
Thank you for sharing It really helped me to understand the difference between these 3 ☺️☺️
Well donde. Thank you Lisa
Great presentation. Thanks
Hi, my name is Henk, nice to meet you
Thank you Lisa, that was essentially a master class on the differences. Excellent. How does Snowflake compare? (separate video probably and also depends on the use case of course). Thank you again for this information. Job well done.
nice course
Isn't it true now that it's not necessary to know Spark for Databricks? I thought the transactional layer abstracts away the need to write Spark code and instead provides options to use SQL or Python.
Hi Dan, in many ways, you can view Apache Spark as a package. We can program it with Python, Scala, and many other languages. So, even though you can program Python in Databricks, just Python is not enough, unfortunately. We also need the powers of Apache Spark! However, if you already know Python, Apache Spark is just onee step away :)
And, even though it's technically true you can program SQL in Databricks, I do find that the SQL language by itself does not offer enough flexibility for most use cases. But, to be fair, some people are much more handy with SQL than me, haha!
What about using a VM with SSIS inside? Thanks.
Isn't Intercept a pharma company?
it's a Microsoft partner 😄
Isn't it true now that it's not necessary to know Spark for Databricks? I thought the transactional layer abstracts away the need to write Spark code and instead provides options to use SQL or Python.
I think Spark is very similar to python in Syntax... I think people use a combination of Spark and SQL in a given Notebook in Databricks.