Stock Market Data Pipeline Project on Google Cloud Platform
Vložit
- čas přidán 25. 06. 2024
- Join Data Engineering Courses Here - datavidhya.com/courses
In this video, we'll be discussing the basics of data engineering, and how it relates to the stock market. We'll be building a stock market data pipeline from scratch, using the Google Cloud Platform (GCP).
If you want to learn more about data engineering or build a stock market data pipeline using GCP, watch this video! We'll cover everything you need to know to get started, from choosing the right platform to understanding the basics of data pipelines!
GCP Data Engineering Course - coursera.pxf.io/4eMjNG
👦🏻 My Linkedin - / darshil-parmar
📷 Instagram - / datawithdarshil
🎯Twitter - / parmardarshil07
🌟 Please leave a LIKE ❤️ and SUBSCRIBE for more AMAZING content! 🌟
3 Books You Should Read
📈Principles: Life and Work: amzn.to/3HQJDyP
👀Deep Work: amzn.to/3IParkk
💼Rework: amzn.to/3HW981O
Tech I use every day
💻MacBook Pro M1: amzn.to/3CiFVwC
📺LG 22 Inch Monitor: amzn.to/3zk0Dts
🎥Sony ZV1: amzn.to/3hRpSMJ
🎙Maono AU-A04: amzn.to/3Bnu53n
⽴Tripod Stand: amzn.to/3tA7hu7
🔅Osaka Ring Light and Stand: amzn.to/3MtLAEG
🎧Sony WH-1000XM4 Headphone: amzn.to/3sM4sXS
🖱Zebronics Zeb-War Keyboard and Mouse: amzn.to/3zeF1yq
💺CELLBELL C104 Office Chair: amzn.to/3IRpiL2
👉Data Engineering Complete Roadmap: • Data Engineer Complete...
👉Data Engineering Project Series: • Data Engineering Proje...
👉Become Full-Time Freelancer: • Best Freelancer Series...
👉Data With Darshil Podcast: • Podcast Series - Data ...
✨ Tags ✨
gcp data engineering guide, road map to become gcp data engineer,
✨ Hashtags ✨
#dataengineering #gcp #bigquery
Working on End-To-End Azure Data Engineering Project, don't forget to LIKE the video :D
Waiting
Much awaiting brother
waiting
Why Azure? Why not GCP
bring with some complexity not like simple one please.
Working on Azure End-To-End Project, will be uploaded soon :)
Don't forget to hit LIKE and Comment what you want to see next?
Super excited to try this out along with you! As someone interested in learning data engineering, nothing beats getting involved by building projects. Big fan of your work!
It would be really nice if you will create etl project with i.e. this flow sources->Kafka-> S3->AWS Lambda->Airflow->Snowflake or maybe tutorial of Spark or more AWS
Great information. Glad if a video will be out. On the basic data engineering tools to learn . Such as spark, Hadoop, snowflake.
Taking this opportunity to let you know that you are too good as a mentor for a beginner like me in Data Engineering as a whole . Thank you so much sir !! Would be glad to join any DE end to end full fledged courses you would start (if) for a beginner no matter the cost !!
Thank you Shweta, courses are ongoing already done with Python and SQL and next is Data Warehouse with Snowflake
Thank You , keep it up 👍🏼
You are the best guide ever !
Hi, this is just high level overview. Please make a complete video along with codes. That will be helpful.
Thank you for this.
Very insightful
Bro, kindly pls create this project end to end...ur azure end to end data engineering project is awesome and thank you so much, you are my only go to channel for data engineering
Love you brother ❤
I want to know if I want to practice, how to write pipelines and ETL processes where I can practice, from where I will get this data.
I would liketo learn this project with pub/sub, dataflow and dataproc for both batch and streaming data, can you please share more details on them?
Thanks
can u make one end to end project on gcp?
Hi Darshil. Please guide us On azure data engineering certification
Why are we using dataproc (apache spark) in this pipeline? I mean dataflow can do the same as it has both the real time and batch capability. Is there any specific reason for using dataproc?
Thanks for the information Darshil , recently cracked Google professional data engineer exam 🤝
Congratulations man
Thanks for the information. Can I know the basic code I need to learn. Do I just need SQL and basic python or I need to understand fundamental of python. Or I should just focus on libraries?
Yes, start with Python and SQL basics and focus on fundamentals then you can forward learning about Cloud Services and implement project step by step along the way learn packages that required to do the work
Bro, I hav enrolled to 'Google' Business Intelligence Professional Certificate' programm on Coursera... I want to know how it can be helpful for an aspiring data engineer.... Can you please make a video on this?
Bro cn u do a project on stock market. If u need any help let me know
If i enroll in Coursera of 1 months fees for GCP data engineer..i will get only one months access even if i purchased the course?
Will not get lifetime access even if i purchase?
Bhai tumne LinkedIn pe post dala,
Data warehousing and snowflake ka koi course la rhe ho,
Mjhe woh post samjha nhi...
Mai snowflake seekhne ki planning hi kr rha tha,
Can you tell kab Tak aayega aur kya in conclusion of post yeh samjhu ki it's a end to end snowflake course??
Snowflake ek Data Warehouse hai
2 cheeze hoti hai Data Warehouse fundamentals jo common hoti hai sab tools ke liye, agar vo agaya toh koi bhi tool pe appyl kar sakte ho
Snowflake ka use karke hands-on practice karnege
End-To-End Snowflake toh nai hoga par but itna cover ho jaega ki Data Engineering ke liye kaafi ko
Snowflake alag se sikh ke matlab nai agar basic concept ka knowledge nai hai
Hey @DarshilParmar, will you be making data engineering projects on demand in the near future?
Yes, I have plans for this
Which one is easy to understand as a beginner GCP vs Azure vs AWS??
It’s subjective but for me AWS is easiest
Which is most used platform in data engineering jobs in india ? Aws , azure or gcp
Azure is growing at a rapid pace so I feel it should be it but a lot of major companies like Dream11, Cred, and other start-ups use AWS
I feel it is divided into roughly two-part, big or traditional companies generally prefer Azure as they use other Microsoft products and startups prefer AWS
There is no straight answer but I feel there is a place for every platform so AWS, Azure, GCP are top
How much would it cost to use GCP for building and running this project? Do they have any discounts for students?
They offer 300$ initially and that’s enough to execute this project
Have you implemented these scenario in hands-on in another video?
No
Hi darshil
I have a question for you.
Is BCA student eligible for data engineer or only engineers are allowed.
You can always do it, no one can stop you!!!
@@DarshilParmar thank you so much
Hi Darshil, which is your favourite cloud service?
Cloud Platform is AWS but I like working on BigQuery, currently working on Azure and DataBricks is good too
@@DarshilParmar Azure is much better than AWS
Do you have github ?
👆🏽 7:16 that part @DarshilParmar
This theory won't work until you show the development in real time rather than reading things..I don't know why people using gcp and Aws don't show any tutorials but then for azure there are so well defined tutorials that ppl can learn and implement
You haven’t really checked my channel properly, you’ll find many AWS and GCP projects tutorials on detailed
Thank You!!
Waste!
Can I get this project ? @DarshilParmar
Doesn’t exist, you have to build
It