Video není dostupné.
Omlouváme se.
How to Clean Data Like a Pro: Pandas for Data Scientists and Analysts
Vložit
- čas přidán 16. 08. 2024
- In this video, we will explore data cleaning techniques in Python with Pandas specifically tailored for data scientists and analysts. Whether you are a beginner or an experienced professional, these techniques will help you streamline your data cleaning process and enhance the accuracy of your analysis.
📖CHAPTERS
00:00 Intro
00:39 Data Walkthrough
03:48 Dropping Data
07:19 Dropping Duplicates
09:30 Cleaning String Data
18:29 Imputing Numeric Data
26:29 Imputing Categorical Data
31:53 Key Principal in Data Cleaning
35:18 Outro and Thanks!
UP NEXT:
- More Advanced Data Cleaning: • Master Missing Data wi...
🔗LINKS
- Data on Github: github.com/tre...
💵AFFILIATE LINKS (HELP SUPPORT THE CHANNEL)
- O'Reilly Media (Books courses and more): oreillymedia.p...
Fantastic easy to follow data cleaning video. I also appreciate you blatantly saying yes it's 50 or so rows but it could be 10k, but the same techniques apply.
This is great. ❤❤🎉
Very helpful and super simple explanation. Looking forward for your next advance pandas with larger dataset videos. Thank you for this video.
Actual actionable real life skills not fluffy fun python skills but actual valuable stuff we need to know!
Dude I cannot explain how helpful this was, man! Seriously, you literally allowed me to pickup any datasets I download and immediately gave me the practical guidelines to clean/analyze it. Thank you!!
You're very welcome!😎
nice tutorial
wow your are the real goat
the best video so far
please more video like this
Keep doing good work. Big help
I appreciate the kind words 🙏 thanks for the support!
This is the 1st video I watched that actually seeing the python libraries in action.
Thank you for this.
You're very welcome! I'm excited to hear about what you will build with them 🙂
Just like Thor said: "Another"
What are some data cleaning techniques that you have used? 🤔
You misspelled Tidyverse 😮
🤣
Hi ! Thanks for this video. I wanted to know, as a data scientist/analyst, why did you choose to use Jupyter and a .ipynb cleaning file ? Why not using pycharm and a .py for example ? Is that just a matter of personal preference ? Sorry I am new to python, proficient on Stata but trying to make a shift
Hi @kikiboy2545 🙂 thank you for your question.
TL; DR - I chose to use jupyter as it is easier for me to demo with and record the video with.
To your point on creating a .py file - I would recommend this if you are creating cleaning logic that is going to be re-used and shipped to 'production' as it is easier to test and maintain a straight Python script IMO.
That being said, there is increasing support for the use of notebooks as the preferred environment - as examples, Snowflake, Databricks, Azure Synapse and more all support the use of re-useable notebooks to contain all of your logic. I've worked in teams where notebooks are preferred for all data pipeline code due to how intuitive and approachable they are - but as I say my personal preference is: use notebooks for exploration, and .py scripts for your production code 🙂
No need to apologize! I am glad to be part of your learning journey - keep pushing man! 😎
Cool, thanks. Is Polars making much of an impact in your world? I've used it a bit and I think I prefer the more explicit syntax - besides the potential for enormous performance gains it brings.
Hi tmb8807 :) I have followed a couple of tutorials on polars, but never used it on anything in a professional setting as of yet 🤔
I'll test it out more extensively.
Any good tutorials you'd recommend?
Typically, when I've worked on projects that needed high performance I've used Apache Spark - but Polars could be a nice in-between pandas and spark?
Thanks for the support!
@@trentdoesmath thanks for the reply. There are a few tutorials on CZcams, the one from Rob Mulla is what got me onto it.
Because Polars can work with larger-than-memory data via the streaming API I’ve seen it suggested it could replace Spark on a single node for some jobs, although I’ve not done that first hand! But it could potentially expand the 'in-between' area, as you say.
Main reason I like it is that I just find the syntax much more consistent and readable (and easier to write as a result). Your mileage may vary on that, though, especially if you're extremely comfortable with Pandas (it's a bit less "Pythonic", with more explicit methods for everything).
Lazy evaluation and the query optimisation engine are a big selling point of it as well - can greatly improve memory usage.
Awesome! I'll check out the Rob Mulla stuff, thanks for the recommendation👍
For sure! It actually reminds me a bit of Scala 🤔... Very 'to the point'.
Not sure if you have tried out Dask before? but it's yet another performance option out there.