Best Data Engineering resources
Vložit
- čas přidán 9. 06. 2021
- Many of you who want to enter the data engineering world have asked me for guidance and mentoring. Due to the overwhelming number of such messages, I’ve decided to create…
***
The four-step guide to becoming a data engineer in 6 months!
You can find it here:
karolinasowinska.notion.site/...
and it’s completely free!
It contains:
-market analysis
-explanation of what data engineers do + a case study!
-the 4-step study plan to becoming a data engineer in 6 months
***
In this video I'm elaborating on the 4-step plan. Please find the links to all the courses that I've mentioned in the video below:
1. Learn Python
I recommend following the Python for Everybody specialization course on Coursera, which is one of the most popular courses there:
imp.i384100.net/x9gVO3
2. Learn SQL
SQL is still the lingua franca of data. I recommend going with Learn SQL Basics for Data Science course, because it contains some chapters which are very releavant to data engineering in partcular, e.g. distributed computing with Spark
imp.i384100.net/QOMZ09
3. Learn Bash scripting/Linux
I wouldn't take a full course on it, but at least read a good article.
if you do prefer to take a course/guided project, I think this one is short and good:
www.coursera.org/projects/com...
4. Learn how to develop on the cloud, e.g. on AWS
There are a few good courses around there, but I think the Coursera one is the most comprehensive
imp.i384100.net/P0MJBM
Discloure: The above contain affiliate links, meaning when you click the links and make a purchase, I receive a commission!
I'm linking many more useful resources in the free pdf (dataappreciationsociety.com/) so make sure to check it out!
----
Join Data Appreciation Society on Discord:
/ discord
----
Data Appreciation Society website:
dataappreciationsociety.com/
----
Find me on Instagram:
@karo_sowinska
----
If you'd like to get me a cup of coffee, I'd be delighted ko-fi.com/karolina_sowinska. :)
----
Don't forget to subscribe if you're new here! ;)) - Věda a technologie
Thanks Karolina, I really need this, shifting from chemical engineering to data engineering as the demand is sky-rocketed in Malaysia. Definitely going to follow your suggestion.
Omg, I have a similar video coming out soon, nice to see overlap between our findings!
I just discovered your channel today, I immedialty subscribed and already watched half a dozen videos, then I noticed you haven't posted in a while... and *ding* here is a new video ! ;)
I am considering switching to Data Engineering and wish to know more from people in the field, so thanks a lot for your precious help !
Haha I timed it very well then! I'm happy to see you here - welcome aboard!
As a data engineer myself, I always feel good whenever someone suggests the course by charles severance to others since this is my favorite too.. I started my python journey from this course! Very well articulated video which will definitely help anyone who is willing to come into data engineering field.
I'm super glad to hear that I'm not the only one!! :) Thanks!
He is a legend
Thank you for the video
I manage to try for a fresh start as data engineer
And your videos always there for me
Very condensed and useful information.
Thank you Karolina, you're definitely helping us a lot.
Thanks so much! :) I'm glad to hear that I managed to make a nice summary!
Thank you so much Karolina! This is the most useful instruction video I've ever seen!! Very practical!
Besides, can you please share some advices on how to start trying out a data engineer project? I plan to do a project includes data collection(maybe web crawler) + data cleaning + ETL + data analysis(dashboards). Any good resources for the hand-on projects? Thank you!
This is amazing, thank you so much!!
I see that you've read and replied to almost all comments . Shows your commitment. I wish your channel grows exponentially.😊
Thanks! I do try to reply to the majority of comments :)
Big big thanks to you for such a quick guide on data engineering career.
No worries! :)
thank you! these resources are amazing!
Thank you. It is a good way to start especially when you have everything in one place.
I'm glad this is helpful :)
You are amazing! The guide will surely be helpful to aspiring data engineers.
Thank you so much! :)
Thank you kindly, you bring hope and light were almost none remains.
I'm glad that I could help. All the best!!;)
Thank you for the summary and the resources!.
I think I am following the path even without knowing it. I started with Python a year ago, now I am focusing on SQL databases. Still haven't learned any NoSQL yet. Linux... I had an Ubuntu in one pc, but all I remember is sudo apt-get install. So that will be the next step. And basic git, then other cloudy stuff ☁️. That seems to be mandatory to get a job in IT nowadays.
I think you're right. All the best with your learning! :)
Thank you very much. I am overwhelmed by what to learn to become a data engineer. Now I can focus in what is important.
Don't be overwhelmed;) step by step, you'll get there! :)
and wtf is that important thing?
This video came at the right time,
I been in the web develop field for quite some time (11 years) and with my humble experience I was surprised when my sister start a course in data engineering and u know how its look like when u wanna show some muscle for a fresh family member when start asking u about what u r good in..I helped her with sql and py but all of sudden a Hadoop , scala and few other things jumped to the middle..and I really didn't understand whats going on!
I hope if u can make a video to share some information about that, if that possible 🙏 bcuz I am feel useless here
Btw nice nails polish 😊
Haha I understand how you want to help your sister! I might make a video about the technologies you mentioned. Good luck to her :) and thanks for the comment! :)
I'm a freelance data scientist. And this video says a lot of important things that I need. Subscribed!
I'm glad it was useful! ;)
My English is a little bit but I like so much your channel and I would like to learn a lot with your advices.
Yay thanks for making these types of videos their really helpful to a beginner like me
I'm so glad! :)
You're amazing! thank you!
As alwalys with your videos, excellent subject and resources. Me, like many people who likes your videos, I'm starting this career path and this resources are indeed very helpful.
God bless you and I hope some day you can arrange a zoom meeting to meet you in "person", ahahaha.
Cheers Karo.
It's great to hear from you again! I hope you are doing well on your learning journey! :)
Thank you. This is very helpful.
You're very welcome! :)
Thanks so much. I'm grateful
My pleasure! :)
Thnx for sharing this important informations for start data engineer carrier
My pleasure!
I have learnt Python and SQL
Now time to do some Bash and Clouds
Awesome!!
Thank you karolina I'm from egypt and you make great
Thanks! I'm very glad to hear!
Thank you for this video 🙏🏻
Thanks for watching! ;)
Much needed video... 😇
thank you! You helped me a lot
My pleasure! :)
Thank you so much for these resources. I'm trying to learn how to build a data lake.
My pleasure! Setting it up from scratch might be a little complicated, but I'm sure there are some good resources out there for that!
Indeed! It's complicated and I always wonder if I am focusing on the right path. I'm the only data engineer at my work place, and I just started learning about it a few months ago. I'm comfortable with Python and I have learned a lot about SQL, Bash Scripting and Databases from DataCamp. I'm currently experimenting with AWS, and it would be helpful to have some experienced that I can gain insight from when I get stuck.
@@joannwatu7603 Oh wow, it must be really challenging to be the only data engineer at the company! But on the other hand, I'm sure you'll learn loads! Feel free to join our Discord server, perhaps someone can help you out there when you get stuck!
@@karolinasowinska I will do that. Thank you 😊
OMG, you helped me a lot!!!
I'm glad to hear that! ;)
With all respect. And I really like the video and information.
Isn't it recommended to do a bachelor/master or similar to understand better how computers/ programming works?
I think the video is more for those who want to take a shortcut.
Thank you for this great video
My pleasure!:)
Great content! new subscriber
Thank you very much.
Excellent video as always! In 4k as well 👍 Did you end up getting the Sony a6400?
Thanks so much! I did indeed! :) with the Sigma 16mm 1.4f lens
@@karolinasowinska I love that combo as well. In my case I bought the sigma and I was using it with an older a5100 and upgraded the camera but kept the lens. Greats lens.
@@iamchesco agreed, it's great, especially for architecture!
Oooh I have been waiting...
:)!
From my perspective, an average of 2.53 applicants per place is quite high. It means that companies can threaten you if you refuse when working environment worsens.
Other jobs have much higher average though!
Hello! Thank you for your invaluable video! I find it extremely useful for beginners! I would like to ask about one thing regarding this map. I learnt Pandas in terms of Data Wrangling and Transformation. Therefore, how about Pandas for Data Engineers? Is it useful tool for ETL/ELT transformations? Obviously, the next step will be PySpark, but I would like to start learninig Pandas. It seems it is a good path for the next one. What do you think about it ? I would appreciate it if you could share your views about it.
Thanks for video and Pdf.
youtube recommendation algorithm brought me to here, and it's the best thing he's ever done
Thanks for the information. The links in the description & pdf are not opening (linksynergy). Please provide alternate ones.
Thank you for the resources. Are there any resources for learning data mining you recommend?
I'll look through my resources!
Yaaaay!!! Finally we got the gold !!!
My mine isn't closed yet haha ;D
Thanks for this vid!
Sorry for off-topic question but being a Slav my self I can see where your name originates just by looking at it :D But also your English accent is very very clear and pleasant. How long have you been living in the UK? If it's too personal, just ignore :) Love your vids by the way! Cheers!
Hello fellow Slav ;) Indeed, I am Polish, I moved to the UK circa 6 years ago. Thanks so much, I definitely try to articulate words carefully to make sure that I'm understandable despite my accent :)
Waheyy welcome back!
Oh hey, nice to see you here! Are you coming back? ;)
@@karolinasowinska Ha.. hey you may have inspired me ;)
@@KingsTalksAI I hope so! and I can't wait to see your next video ;)
it was super useful thank you
Thanks for the video! I'm not able to receive the PDF of the guide on my mail, is it region locked or something? Is there any other way to get it? Thanks again!
It's defiitely not region-locked! If you've got a gmail inbox, try searching in the 'Promotions' tab
I am current learning skills to be a Business intelligence analyst. I like SQL and databases specifically rdbms. Data engineering seem very interesting to me. I have a windows computer do I need to purchase a linux based computer(please tell me o dont have to get an apple) to these shell commands?
Also for python are their any udemy courses you know that can be helpful as well? I was thinking of doing Business intelligence analyst then migrating ove to data engineering in time.
Side note: I am currently an ad operations analyst. I realize BI analyst overlap with my current field so that y I choose becoming a Business intelligence analyst first.
I'll share with my friends, I'm a data engineer from Brazil o///
Awesome, I hope they liked it!! :)
Hi Karolina, do you reckon someone who don't have any experience in date eng. field after 6 months of studying can have enough knowledge to get a job in this industries ?
Great , thanks a lot
You showed 5 books in the thumbnail but didn't mention any in the video. Maybe I missed that part. Can you explain why please, books are very helpful. Thanks
I'll add them too :) good point
The video was awesome and you were looking more beautiful than your last video. ❤️ I wish I had a 4 k monitor!!!😅
Haha thanks, lighting is king :) I realised that it's better to record during the day! :)
@@karolinasowinska That's great. The day thing made my day 😁.
you rock!
Thanks for the video! Do data engineering interviews typically involve algorithm style software engineer interview questions? Or is it more SQL/ less algorithmic style questions?
Its mostly complex sql and easy to medium Algos/Datastructures. It’s not as intense as Software engineer coding questions. That being said, they mostly focus on lists, Strings and Problem solving. Faangs may ask BST and Dynamic programming if the role is inclined towards SWE. I hope this helps😅
From personal experience, I've done quick simple Python coding tests, long project based Python coding tests, SQL tests, data engineering related techstack multiple choice tests, designing an end to end DE solution in the cloud tests, but never specifically data structures and algorithms. You know, in most jobs, a data engineer job in Python is only for simple scripting purposes anyway.
Thanks a lot, Beauty.
WoW Thanks
its just perfect..!!
Do data engineers use big data technologies such as spark, hadoop or use scala? The platform could be on databricks or on cloud provider
Can you please put links to the 5 books in your thumbnail?
I don’t agree that python is the most important language in data engineering. People tend to forged the importance of SQL.
As a senior data engineer, I can tell that SQL is more important than python.. so I’d say 1. SQL 2. Python
Fair enough, I said in the video that SQL is equally as important as Python. I agree.
Sql is much faster for any data manipulation, and should be first in use.
I hope you to make video about data warehouse jobs and learn path
I'll think about that, thanks!
I never learnt python but i am good with Java, would it still require me two months with Python to start
Python is easier so maybe you'll be able to go through the course quicker:)
Ok, looks like I have everything to transition from BI Developer to Data Engineer except for cloud. GCP cert, here I come! Ty for the vid :)
My pleasure, good luck! ;)
@@karolinasowinska thanks, good luck for you too
Real time project for freshers on data engineer.... Hadoop
Good idea, noted!
@@karolinasowinska when we can expect...
Step to step we need...
With this project... DE Should understand
mercedes benz vs Boston consulting group
which one to choose for data engineer role?
Hard to tell, perhaps ask yourself: are you more interested in cars or business consulting industry?
The links to python, SQL and Cloud don't work, a bad merchant page shows up,
Thank for the content, but the links are broken
The first, third and forth link doesn't work, pls make sure you put the correct link,
Im still waiting for your pdf and im already subscribed
Luna is that you? 😮
Thanks for the valuable info btw!!
Luna who? :D
Ms Lovegood of the Quibbler ⚡🧙♂️
There are people wanting me ...
*gets out of the bath ...
*wipes the tears ... D
Very helpfull !!
The amount of efford put in this video and resources is on another level
What do you think about the Google Course for Data Engineering on Coursera ?
Haha don't worry, you'll soon be a hot cake on the data engineering market! :D I think that course is good too, but I personally prefer to recommend AWS given its current dominance
could you add the translation? I look at all your videos and they have served me a lot, greetings from Argentina
Thanks! If someone wants to submit a translation, I'm happy to add it
Thanks, can you become a Data Warehouse engineer without learning programming? I just want to learn SQL
for Data engineering course no need java & scala. Python is enough? just asking
depends on the company, but in my experience it can be enough
@@karolinasowinska Thank u so much
Can't find the git ressource mentioned in the description
So you mean it is better first make some projects, then apply yea?
or vice versa?
First do some projects :) Then apply
@@karolinasowinska Thanks.
very important to me :)
You are beautiful Data Engineer. I like your charisma.
Oh thank you, I do like to think that I've got some charisma :)
I am not an software background. But I want to shift to software and become a software developer. Can you suggest from where should I start and finish to and what to learn to become Software developer
You can follow the guide I mentioned in the video. You won't be too far off from software engineering. :)
@@karolinasowinska Thank you Karolina. You are such a goddess from heaven saved me from going to hell
This was useful. (You asked it =)
How to get the pdf? there's no sending pdf in my email yet and i used 2 emails also
Links to courses are no longer working Karolina :(. Could you update them?
本当に綺麗な人
Whatever you said haha :)
The Queen Karolina
My friends call me Arctic Queenolina ;>
@@karolinasowinska hail artic queenolina
@@AzulonKaiba Sounds perfect haha
So, how if we just have a macos? It is ok?
I'm sure you'll be fine:)
hello, I'm starting in data engineering, is it possible to create a startup/company, so I can earn more? I see salaries from 100 to 180k as an entrepreneur it would be possible to reach 250,350k??
Anything is possible as an entrepreneur!
Hi please share power bi project ideas
I haven't worked with PowerBI, so might be hard
@@karolinasowinska no problem
What about Hadoop, Hive and Spark ? #Karolina
Hadoop is getting slightly obsole!
@@karolinasowinska Any Popular alternative for Hadoop in the current market ?
links not working :(
Can you add subtitles to help more people understand your videos?
drive.google.com/file/d/12_ID1RExNKbg-3pa59fqWFLUYLLe83jz/view?usp=sharing
I send subtitles. Thank you very much for your video. I'm new to DE, I hope you make more videos about the project for beginners.
Thanks so much. I'll try to add them!
❤️
Cześć Karolina!
I apreciate your videos. Here are some topic I would love you to cover:
What are a DS algorithms every DS should know on a Medium level. I know some DS are only focused on CV or NLP and don't know much about the other. That might be a mistake.
Where to find a good mentor if you already know all the basics?
Would love to ask for more, just it's difficoult to make it in a one comment.
Cześć! I'll definitely add these topics to my list and I'll try to cover them in the near future! Thanks for the suggestions! :)
I did not get the PDF . NO way i want that
4K makes you come out of my screen.....
Haha hello there!
"One month should be enough" -- Hey! Don't underestimate my stupidity.
Hahah it isn't that hard!
You are so adorable and beautiful I am also an data enthusiast just want to work with you and listen all day to you😌
SQL ok, Linux ok, Python not ok and Cloud not ok... thank you for the directions. Let's move on