These 3 Things Can Help You Understand Data Engineering Roles
Vložit
- čas přidán 26. 07. 2024
- Join My Data Engineering Courses - datavidhya.com/courses
Who are Data Engineers? What does a Data Engineers do? What do you need to know to become a Data Engineer?
These are questions many of us have when we get started with Data Engineering but the answer is not that simple!
In this video, I have explained the different roles of data engineers and what they do!
👇🏻 Check out our today's sponsor for this video 👇🏻
ProjectPro - bit.ly/3hsvNrH
ProjectPro CZcams channel - / projectprodatasciencep...
Video Ref -
www.confessionsofadataguy.com...
⭐️TIMESTAMP⭐️
0:00 Introduction (VERY IMPORTANT)
1:00 ProjectPro - Do Hands-On Projects
1:55 Data Engineering Category 1
3:41 Data Engineering Category 2
5:14 Data Engineering Category 3
Join this channel to get access to perks:
/ @darshilparmar
===================Follow Me Here===================
👦🏻 My Linkedin - / darshil-parmar
📷 Instagram - / darshilparmarr
🎯Twitter - / parmardarshil07
===================About Me===================
Freelance Data Engineer and Solution Architect.
I love learning about technology and teaching thing I know to other people. The goal of creating this channel was to teach everything I know to the world, I have spent most of my time learning things online and I want to share all my learning on this youtube channel.
I will be posting content related to data engineering, architecting systems, data science, machine learning, and interview preparation.
Thank you for watching the video, see you in the next video :)
My Accessories
🎙 Mic -amzn.to/3Bnu53n
📱 Camera (I use my phone) - amzn.to/3wOogIU
🎧 Headphone - amzn.to/3BsJxLB
⌨ Mouse and Keyboard - amzn.to/3zeF1yq
🖥 Monitor - amzn.to/3zk0Dts
tags
data engineer salary, data engineer jobs, data engineer salary, data engineer job description, data engineer vs data scientist, data engineer interview questions, data engineer skills
#dataengineer #projects #coding - Věda a technologie
Let me know what you want to see next and don't forget to hit Like button if you this vidoe helped you!
How to showcase experience and approach in bigdata for non it background please upload a vlog on it.Thanks
full video of a simple freelance order (for beginners)
Thank you this is what i was looking you have explained the roles in very simple way
Great content. Subscribed. Thanks
Your my hero of the day. You calmed my nerves so much
You simply AWSOME brother....Confession of a DATA guy !....Very well presented bro.....Thanks a Ton..
Please don't reduce your speaking speed, it's on point!
This categorical segmentation is rarely mentioned and vital for beginners to see a realistic roadmap.
Thanks!
Thank you for such a nice informative video...
Fantastic video Darshil!. Your videos are extremely valuable. Thank you so much.
Request you to do more videos on -> Real time Data Engineering projects and if possible granular level video on -> Day to day work in companies as a DE.
Thank you for the comment, I will put this on my list
This is great framework for looking at DE.
Valuable video.. 👍👍
Very Precise video. It really clears so many doubts. 👍👍
Glad it was helpful!
BRO!! this was soooo informative. Ill definitely start look for a JDE job next month.
I am definitely category 1 looking to transition into category 2
All the best bro
thx for your content
Anyone who stops learning is old, whether at twenty or eighty. Anyone who keeps learning stays young. The greatest thing in life is to keep your mind young. - Henry Ford
Wow mind blowing ❤
B.tech Mechanical guy here. On a way to career transition. Surely your videos helps in a big way. Thank you.
I am glad that this helped
Did you transition your career?
Another excellent video Darshil.
Thank you
the video was very informative thanks
Hi Darshil, great insights on Data Engineering, Thanks a lot. A small advice - reduce the speed of the speech a bit, may be my brain is too slow for processing so many words.
Hey Pradeep,
Thank you for the feedback, I am trying my best to reduce the speed it's just I have habit of speaking fast from childhood so I am trying my best!
I did get lot of comments on this before :)
Slow speed pr dekh le bewakoof insan
@@Icelander00 pahle tu bolna seekh le.
Hi Darshil - Thank You for delivering such amazing videos. This gives confidence to people who are aspired to take-up a career in Data Engineering. A request that if you could help share videos related to below topics based on your working experience. 1. How Data Engineering and Big Data is delivered in real-world Agile Scrum Projects? 2. End to End Project Videos of Big Data Technologies like Hadoop, Spark implemented in the Cloud platform/s. Thank You.
Hi,
Thank you for suggestion and I am glad this video helped you
1. I will work on this video soon
2. I already have project series on my channel that is similar
czcams.com/play/PLBJe2dFI4sgt-9GR2j-rTeKtimE9pfqyt.html
czcams.com/play/PLBJe2dFI4sgukOW6O0B-OVyX9c6fQKJ2N.html
I'm working under category 2( Airflow and Dbt) i would like to work in analytics part ... I'm planning to learn tableau, advanced SQL and python for analytics is this enough to work in analytics part?
Make a video on online training and Placement program
Hi Darshil, could you talk about snowpipe and cost calculation and if it's a viable solution for batch processing.
Hi Karan,
Sure I will try to talk about this in future video
As a software dev knowing python and SQL . What would you all say is next steps for me to land a data engineering job
Hii Darshil , How can data engg contribute in open source..plz reply
Hello Darshan thanks for this wonderful video 🙇. So darshan as you know that in India there is no well structured data engineering course available in offline classes... So i decided to do separate classes of each subject like MySQL , Python for data science , Hadoop and spark or AWS solution architect certificate. So can you tell me what should I do next ?
If you have already done certification then I'd say don't do more certificate instead try to do projects and get internship anywhere to learn more
@@DarshilParmar Ok Thanks 😁
At the time of watching this video I know SQL, Kafka, MQTT, Django, Databricks, Kubernettes, Tensorflow, Dash and power BI. I am so confused where do I fit in
Another informative video thank dear love from 🇵🇰 🙏
My pleasure 😊
I am college graduate and have got a job of data engineering, it has a role of designing ETL pipelines, should I switch to data science profile??
If you don't enjoy the work you are doing then think about switching
Although the confessionsofdataguy writes this category3 as category2+distributedcomputing, we rarely see them and see category4 thats category3-category2 (They are good at distributed computing and can code well in spark, java but they dont do well in SQL). One thing that you didnt touch and often most ppl fail to understand is the importance of right data modelling skills. Category1,2 possess it well and Category 4 doesnt. I havent come across any category 3 guys in my 13 yrs of experience.
What about cloud data engineer
Is DS/Algo very important for data engineering interviews??
No
Darshil do you know scala ?
Hi bro i needed a help. I'm unable to do project. I finished course on deep learning in Coursera i know theory but when i try to do any project I'm unable to write code and think of approaching a problem like i know I should predict given figure is cat or dog but i don't know wt all coding or data cleaning etc etc steps to follow to finish the project. So can u help me in this like after watching course how I make my own projects
Hi Rohit,
Divide your problems in small task
1. Collect Dataset such images of Cat/Dog
2. Find online how to read image dataset using code
3. Find algorithms that are used for classification on image
4. Build/Train/Test model
This process is not straight forward, you will have to watch more videos on youtube, read blog and try out many thing to finish it.
It's not that you are unable to do, you need patience when doing things, don't keep goal of finishing one project and jumping to next one, give one project 1-2 months and build that first
@@DarshilParmar thanks for guiding me bro
Please suggest me I know Basic Python upto loop but I have no interest in JAVA. Should I Learn DSA stuff in Python or JAVA is a must?
Hi,
I have these two videos on my channel that might help you on this
Best Programming Language To Learn In 2022 - czcams.com/video/Y6Yv01aPHz8/video.html
Best way to learn Data Structures and Algorithms in 2022 - czcams.com/video/fHA_JYECg0c/video.html
What is the salary difference between these 3 roles and what is the difference between software engineering-data and data engineering?
I will make a video on your first question in the future but for 2nd question
The title might be based on different companies so if it is a product company and you are working on product + database side then company might call that role as software engineering-data
If you have job description for this roles and you can link it here I will check it and let you know
@@DarshilParmar youtube removing comments with links so adding spaces in between
Emailing you since youtube is deleting links here
nice....
Thank you! Cheers!
👍👍👍❣️
Is talking as fast as this guy also one of the job requirements of being a data engineer?
My talking pace is fast from childhood, working on that ;)
Great 👍..
Thank you
o.o there’s no momentum to the speech… why..?
I’m Asian as well and grew up here. After awhile, it just feels like sound buzzing around.. Makes me start pondering if word spoken is about comprehension or just jamming “smart sounding words”.
English is my 4th language and I am aware I am not perfect
I speak in English so that people around the world can understand what I am trying to say
It’s only about me communicating my experience through whatever I know, I can do this in my 1st or 2nd language (Hindi) but not everyone will understand it
It does take lot of effort to research, record, edit, and publish these things
Taking time from my busy work and doing this
I hope you understand this because my job is to share what I learned
Thank you for your time
@@DarshilParmar Your English is awesome, just going on 0.75x speed and putting in some pauses to digest the graphs and the smart words and it would be perfect. ... in short dont try to speak fast to appear smart like Ben Shapiro.
✌️
if u know all these thing you can expect a salary of 1 - 2 cr per annum
Darshil my friend, shave the unibrow.
Video make slow it's very fast . otherwise nice.
Ok next time I will keep this in mind
This video just feels like a big advert..
Hey,
There is a segment for sponsor in this video and that’s how I make my earning as creator, it’s not that I just added it, I asked my audience and then took this decision
Thanks for your time
Please don't subscribe to project pro. You will get better projects in youtube, Darshil has better projects than project pro. I wasted almost 80k with almost 0 good project.
this video just gives me anxiety
Why???
Please l sent you a friend request on LinkedIn