Python for-loops are very slow so you should avoid them whenever possible. Luckily pandas supports doing math directly on columns. So to add the BMI column you just need data['bmi'] = data['height']/data['weight']**2 For example, for a dataframe with 300000 rows this is over 1000 times faster on my machine than the video's method.
You're absolutely right Phsopher!! 😀 I totally messed up on this one! it's not just about speed - but also resources! aaaand 5 lines of code that you just reduced into one! 😊 Thank you so much for your comment and for letting me know - I will film a quick correction tutorial shortly, my apologies for not catching that! (it's funny how I'm instructing folks to avoid for loops in "drop_duplicates()" but encouraging them in data manipulation... 🙃 I can't even use the "brain fart" excuse because of that hahahahaha 🤣🤣🤣) Sorry once again - I'll make sure everybody knows there's a MUCH better way! 😉
@@PythonSimplified I wish there was a Code Jam platform that is constantly offering the Best Pythonic Way to do something in real time, where somebody catches the flow of code while in production so it consumes less Brain resources, just like a computer, lol. Any ideas ? Do you know of any camps that nuture Best practices other than the typical ? I would enjoy more live interaction on the code.
Mariya, have you looked into the Polars library for Python? It's supposed to be much faster than Pandas? It might be helpful to do a comparison video on Pandas vs Polars - strengths and weaknesses. What do you think?
@10:13 if you can't remove the index column, The issue might be the default behavior of read_csv when reading from a file with a custom separator (in this case, \t). The first column name is being interpreted as the index column. To resolve this, you can explicitly set the index column in select_DF file. Adjust line 2 like this: data=pd.read_csv("bmi.csv", sep="\t", index_col=0)
I Made a app to store my recipes in pdf . But I cannot figure out how to make the recipe save in to a format that I want to create. Really appreciate ❤
I am already in love with you. 😂😂😂😂 You are an incredible and amazing teacher. The smile on your face and your sense of humor makes me want to learn more. I want to be your friend.
Thank you for your video, but the loud background drum typing is very very distracting. So much so that I can not focus on what you are explaining. If you will, please fix your audio or whatever must be done to remove this distraction. Thank you.
I wouldn’t have bothered with the for loop for the bmi you can calculate the bmi by using: data[“BMI”]=data[“weight”]/data[“height]**2 Great overview though, enjoy all the videos.
hahahahaha trust me, it's not gonna take you 45 years if you watch my channel 😜 Checkout this vlog of mine when I talk about the roadmap of learning Python: czcams.com/video/wFEC7VbWBZo/video.html I have so many videos explaining these concepts, but you can always take a quick "introduction to Python" course in an online platform like Udacity, Udemy, Idx, Coursera first. They often offer free introduction courses and it's a really quick way of getting up to speed with Python 😊 Best of luck on your programming journey - it's the most exciting profession in the world, you gonna love it! 😁
Thank you so much Charlie! 😀 I have one correction to the code I presented in this tutorial, it should have been: data["bmi"] = data["weight"]/(data["height"]**2) This saves typing, doesn't waste resources and runs much faster! (especially when dealing with enormous amounts of data)
Hi Mariya, This video helped me with creating a budgeting app for an assignment in a programming course. Pandas was used in my app for outputting tables of expense data. Thank you for posting videos. - Henry
In fact I was working in a project last night, automating an excel file that contains supply levels for Xerox printers, and reading about pandas, great video
Thank you so much for caring and teaching python which is so sweet , you are so professional and affable.I'm following your youtube channel and telling my friends to follow you, Good Luck :)
I learned a couple of tricks, that's good. I believe the documentation does not recommend iteration on data frames. You're better off to use the apply, map etc functions which I use regularly. They're quick and efficient compared to iteration, especially if you're dealing with million plus lines of data.
You're absolutely right Philip! 😀 The method I've shown is not only taking up waaay too many lines of code - but its also time costly and wastes lots of resources! It should have been: data["bmi"] = data["weight"]/(data["height"]**2) instead of the unnecessary for loop, the appending and all that jazz 😟 My apologies on the mess-up, I will film a short correction video very soon! And thank you so much for your comment! I really appreciate you letting me know! 😊
Every time you click your keyboard, and my ears are devastated by the thunderous roar coming through my headphones I am traumatized 😢 Great video otherwise. I’m glad these resources are freely available. Thank you for your contribution to the coding community 😊
really nice explaination.....when i was new to programming i write a code in 3 days to group the dataframe by using index and lists, after 3-4 months i came to know that pandas have inbuilt "groupby" 🤦 sometime i still laugh for what i did🤣
hahaha I totally understand! I had the exact same situation with the pd.read_html() method!! 😁 I have a 20 minute long video showing how to web scrape a Wikipedia table with Mechanical Soup: czcams.com/video/MkGQmZoMuRM/video.html And then in one of the comments, somebody mentioned read_html()... took me less than 5 minutes to get the exact same results!!! hahahaha 🤣🤣🤣 But the good news is - you'll never forget "groupby" just as much as I won't forget "read_html" 😜
Thank u.......The reason I am thanking is ,hitherto I am confused by the result which given by the Python is error, while I doing dataframe work .......because I don't know before watching this video It is important to put dataframe D and F in caps😁😁😁
Great video! Now I have a quesiton, when I run the line 25 at time stamp 10:26, it was report "read_csv() got an unexpected keyword argument 'index'", could you please tell me how to fix it? Thank you very much!
Doing something like data.iloc[1]["weight"] is inefficient and requires accessing the frame twice - the first access extracts a series at index 1 and the second one retrieves the value at the 'weight' index from the series. Instead, one should try to always use the multidim accessors, that is .iloc[*, *] or .loc[*, *]. There are also accessors - .ix[*, *] - that mix the integer-based acces with with the column-name based access but these should be used with extreme caution and are in fact deprecated.
It's a cloud IDE called Wayscript: www.wayscript.com/ Made by a bunch of really nice folks who were supporting this channel from the very beginning 😀 Also, Scott4WhiteRock as in White Rock, BC? We're almost neighbors! 😁 Cheers from Burnaby!
Great video, thanks... fun examples and attitude made it quite interesting throughout in a way that many tech tutorials are not, and paced at what I thought was exactly the right speed..
I am an experienced developer and Big Data practitioner and this kind of simple introduction is just what I needed to accelerate into Python/Pandas. It also helps that the host is very pretty :)
A quick tip for the face cam video problems ... record both on seprately . record the screen as you do and record your facecam with a seprate camera or use a seprate recorder for the facecma
Trying to think of a use case with pandas for my Open World Text Adventure lol. Not sure if I'd benefit from it or not since I don't need to know what data exists and where, nor do I need to see it printed out (it's printed but in a more user-friendly way). But I still enjoyed watching this tutorial and will try some things out in the future! Right now I just use for loops to detect what the game is asking for in the player inventory either by selection (viewing the inventory and highlighting the item with the arrow keys) or by action (like trying to mine for Iron, we know we need a pickaxe so a for loop will check for one equipped). I want to see if pandas has anything interesting I can do with my inventory dictionaries, but like I said I have no need to know what is where or how many the player has, it is just handled with my functions and for loops.
Binge watching your channel today getting solutions for my TODO list on my current project. (Refactor and Mobile version of my Clownfish Genetics Calculator) Now, I'll be able to refactor my results phenotype data with Pandas. It's current a nasty spaghetti if statement...uggh! Thoroughly enjoying your channel and learning a lot (more kivyMD and adb)!
Muito obrigado, sigo sempre seu canal, cada vez mais conteúdos interessantes e úteis no dia a dia, Your Channel is the best congrats for your hard work, thanks from Brazil
Good question. You can try mine and check the playlist for Python. Very easy to follow fundamental tutorials for Python. And source files can be downloaded. Another playlist in the channel is R Beginning course.
hello belle, thanks for all good information you shared with us. I am a new python learner and i would like to know how or the best way to extract data from a pdf file that has inside date, numbers and letters. i would like to see how we can sort them and delete what we don't need. if it is possible . thanks
Python for-loops are very slow so you should avoid them whenever possible. Luckily pandas supports doing math directly on columns. So to add the BMI column you just need
data['bmi'] = data['height']/data['weight']**2
For example, for a dataframe with 300000 rows this is over 1000 times faster on my machine than the video's method.
Thank you
or data['bmi'] = data.weight.div(data.height.pow(2))
You're absolutely right Phsopher!! 😀
I totally messed up on this one! it's not just about speed - but also resources! aaaand 5 lines of code that you just reduced into one! 😊
Thank you so much for your comment and for letting me know - I will film a quick correction tutorial shortly, my apologies for not catching that! (it's funny how I'm instructing folks to avoid for loops in "drop_duplicates()" but encouraging them in data manipulation... 🙃 I can't even use the "brain fart" excuse because of that hahahahaha 🤣🤣🤣)
Sorry once again - I'll make sure everybody knows there's a MUCH better way! 😉
@@PythonSimplified No worries :) Thanks for great content!
@@PythonSimplified I wish there was a Code Jam platform that is constantly offering the Best Pythonic Way to do something in real time, where somebody catches the flow of code while in production so it consumes less Brain resources, just like a computer, lol.
Any ideas ?
Do you know of any camps that nuture Best practices other than the typical ?
I would enjoy more live interaction on the code.
I hope everyone appreciates the amount of time and effort you put into your videos! I know I do! Thank you very much.
Thia was really helpful. I just joined an advanced deep learning course. But I found out i was missing some background.
Wow! I have no idea what Pythons is, or why I'd want to use it, but I'm going to learn now!!!!
Mariya, have you looked into the Polars library for Python? It's supposed to be much faster than Pandas? It might be helpful to do a comparison video on Pandas vs Polars - strengths and weaknesses. What do you think?
@10:13 if you can't remove the index column, The issue might be the default behavior of read_csv when reading from a file with a custom separator (in this case, \t).
The first column name is being interpreted as the index column.
To resolve this, you can explicitly set the index column in select_DF file. Adjust line 2 like this: data=pd.read_csv("bmi.csv", sep="\t", index_col=0)
u teach super good keep uploading data science videos in your plaYLIST I LOVE IT
I'm new to programming, but I can swear the way of this explains it works for me perfectly.
I love ur videos. They're well structured with great explanations and are very engaging.
I Made a app to store my recipes in pdf . But I cannot figure out how to make the recipe save in to a format that I want to create. Really appreciate ❤
I am already in love with you. 😂😂😂😂 You are an incredible and amazing teacher. The smile on your face and your sense of humor makes me want to learn more. I want to be your friend.
Thank you for your video, but the loud background drum typing is very very distracting. So much so that I can not focus on what you are explaining. If you will, please fix your audio or whatever must be done to remove this distraction. Thank you.
Most creators don't understand audio production, so...
There is no way you will be able to work in an office if you are this easily distracted.
I wouldn’t have bothered with the for loop for the bmi you can calculate the bmi by using: data[“BMI”]=data[“weight”]/data[“height]**2
Great overview though, enjoy all the videos.
I must learn a absolutelly basic of computer programming first. I will be back 45 yrs later. Cheers.👽
hahahahaha trust me, it's not gonna take you 45 years if you watch my channel 😜
Checkout this vlog of mine when I talk about the roadmap of learning Python:
czcams.com/video/wFEC7VbWBZo/video.html
I have so many videos explaining these concepts, but you can always take a quick "introduction to Python" course in an online platform like Udacity, Udemy, Idx, Coursera first. They often offer free introduction courses and it's a really quick way of getting up to speed with Python 😊
Best of luck on your programming journey - it's the most exciting profession in the world, you gonna love it! 😁
Not nearly that difficult. Learn how to get the computer to print to screen, then how to get text from keyboard, then learn conditional statements
2 minutes into the video I realize we think alike when problem solving. Look forward to exploring your content!
I'm new in Python, but with your video I got it so fast and clear.....Thank you so much. Keep the great work on!
This is such a great video thank you. I have been getting up to speed on pandas the past few days and this is one of the best I've seen. Thank you!
Thank you so much Charlie! 😀
I have one correction to the code I presented in this tutorial, it should have been:
data["bmi"] = data["weight"]/(data["height"]**2)
This saves typing, doesn't waste resources and runs much faster! (especially when dealing with enormous amounts of data)
Thank you Mariya. Your tutorials are really simplified and awesome!!!
Thanx and highly appreciate ❤
Create more content about ML and DL
You sold me on learning pandas.
Looks like a great alternative to ETL tools like informatica.
I'll learn it well.
Hi Mariya,
This video helped me with creating a budgeting app for an assignment in a programming course. Pandas was used in my app for outputting tables of expense data. Thank you for posting videos.
- Henry
Great video with details. I would suggest you to make the same kind of videos about Numpy, matplotlib and other visualisation libraries. Also SQL)
Thank you so much Rashid!
Will do! both Numpy and Matplotlib are definitley on my list! 😉
In fact I was working in a project last night, automating an excel file that contains supply levels for Xerox printers, and reading about pandas, great video
"There you go", another great tutorial ! Thx!
This is the tutorial I was after for some time. Thanks a million. Fantastic presentation (as always)
Thank you so much for caring and teaching python which is so sweet , you are so professional and affable.I'm following your youtube channel and telling my friends to follow you, Good Luck :)
I learned a couple of tricks, that's good. I believe the documentation does not recommend iteration on data frames. You're better off to use the apply, map etc functions which I use regularly. They're quick and efficient compared to iteration, especially if you're dealing with million plus lines of data.
You're absolutely right Philip! 😀
The method I've shown is not only taking up waaay too many lines of code - but its also time costly and wastes lots of resources!
It should have been:
data["bmi"] = data["weight"]/(data["height"]**2)
instead of the unnecessary for loop, the appending and all that jazz 😟
My apologies on the mess-up, I will film a short correction video very soon!
And thank you so much for your comment! I really appreciate you letting me know! 😊
Excellent! Clear, efficient, simple and well explained and demonstrated.
Every time you click your keyboard, and my ears are devastated by the thunderous roar coming through my headphones I am traumatized 😢
Great video otherwise. I’m glad these resources are freely available. Thank you for your contribution to the coding community 😊
really nice explaination.....when i was new to programming i write a code in 3 days to group the dataframe by using index and lists, after 3-4 months i came to know that pandas have inbuilt "groupby" 🤦 sometime i still laugh for what i did🤣
hahaha I totally understand! I had the exact same situation with the pd.read_html() method!! 😁
I have a 20 minute long video showing how to web scrape a Wikipedia table with Mechanical Soup:
czcams.com/video/MkGQmZoMuRM/video.html
And then in one of the comments, somebody mentioned read_html()... took me less than 5 minutes to get the exact same results!!! hahahaha 🤣🤣🤣
But the good news is - you'll never forget "groupby" just as much as I won't forget "read_html" 😜
@@PythonSimplified yeah true😅😝
An excellent way to remember- self taught might make mistakes in the beginning, but become some of the most well thought out for programmers.
Thank u.......The reason I am thanking is ,hitherto I am confused by the result which given by the Python is error, while I doing dataframe work .......because I don't know before watching this video It is important to put dataframe D and F in caps😁😁😁
Just adding my own appreciation for a well architected video, and well done too for taking on the suggestion of others for improving the code.❤
Great video! Now I have a quesiton, when I run the line 25 at time stamp 10:26, it was report "read_csv() got an unexpected keyword argument 'index'", could you please tell me how to fix it? Thank you very much!
Doing something like data.iloc[1]["weight"] is inefficient and requires accessing the frame twice - the first access extracts a series at index 1 and the second one retrieves the value at the 'weight' index from the series. Instead, one should try to always use the multidim accessors, that is .iloc[*, *] or .loc[*, *]. There are also accessors - .ix[*, *] - that mix the integer-based acces with with the column-name based access but these should be used with extreme caution and are in fact deprecated.
Your content is gold.
Thank you mariya for your lecture it was really helpful
Awesome!!
I liked this video and the girl as well. She speaks and move hands as my MSc supervisor.
Your for loop really helped me with some things that I couldn't do with standard pandas calculations! Thanks! Great video!
I didn't know that you can do querys in a database with pandas! Super-mega-ultra-useful jajaja Gracias jefa
Love the pandas tutorials! Thank you from bitterly cold Ontario, I hope you're keeping warm.
Thanks , there is a lot of good information in this video!!
Thank you so much! I'm glad you found it useful! 😀😀😀
Thanks for the tutorial. What terminal program are you using? Looks handy.
It's a cloud IDE called Wayscript: www.wayscript.com/
Made by a bunch of really nice folks who were supporting this channel from the very beginning 😀
Also, Scott4WhiteRock as in White Rock, BC? We're almost neighbors! 😁
Cheers from Burnaby!
And there you go, this is how a magnificent video is made. Thanks for so much all tricks.
Thank you for all the details in this video.
Jumping to the previous video and codes (GTA stuff) was confusing, but "pandas" material was awesome!
Really cool tricks Mariya.
Especially the bot removing trick for code-jam, that really got me... Great work...
You're the best mentor of all time 🎉
Your content is amazing. I really enjoy your channel.
You are a very gifted teacher Mariya.
Wow you are a hero, you saved my project. Thank you so much!
Thank you Mariya, i now who what iloc is😁
Nice video. This is the kind of guide that is handy in almost every situation with pandas. I've learned new tricks. Thanks you.
Love your videos. Well presented and clear. Thanks for taking the time to make this.
"on a good day" really got me lol
Great tutorial
I like the sound made by your keyboard when u were typing. It sound great too.
This is dope am digging these pandas 🐼
This is a great video! I use excel a lot, but I am trying to use Python more so I can work with larger sets of data :) Thanks for the great tutorial!
How am I just discovering you?! This is glorious!
Adding a new row:
df.loc[len(df.index)] = [list of added data]
Pandas is the best money making library I’ve seen so far.
adding rows also can be done via pd.Series and passing a list
just what i was looking for!!!
There will be still one ZeroDay-X entry, Drop Duplicate only remove duplicate one but keep orignal entry
Great video, thanks... fun examples and attitude made it quite interesting throughout in a way that many tech tutorials are not, and paced at what I thought was exactly the right speed..
now i love pandas :)
Hola, great job!.. thank you.
I recommend learn typing (mechanography). Bye!
Hey Maria, amazing job! very nice video..just a feedback the audio is out of sync with your lip movements
I am an experienced developer and Big Data practitioner and this kind of simple introduction is just what I needed to accelerate into Python/Pandas. It also helps that the host is very pretty :)
A quick tip for the face cam video problems ... record both on seprately . record the screen as you do and record your facecam with a seprate camera or use a seprate recorder for the facecma
I love you so much, I like a lot you're humor and you're sarcasms jajajaj
Stitch suits you very well :D
Trying to think of a use case with pandas for my Open World Text Adventure lol. Not sure if I'd benefit from it or not since I don't need to know what data exists and where, nor do I need to see it printed out (it's printed but in a more user-friendly way). But I still enjoyed watching this tutorial and will try some things out in the future!
Right now I just use for loops to detect what the game is asking for in the player inventory either by selection (viewing the inventory and highlighting the item with the arrow keys) or by action (like trying to mine for Iron, we know we need a pickaxe so a for loop will check for one equipped). I want to see if pandas has anything interesting I can do with my inventory dictionaries, but like I said I have no need to know what is where or how many the player has, it is just handled with my functions and for loops.
Binge watching your channel today getting solutions for my TODO list on my current project. (Refactor and Mobile version of my Clownfish Genetics Calculator) Now, I'll be able to refactor my results phenotype data with Pandas. It's current a nasty spaghetti if statement...uggh! Thoroughly enjoying your channel and learning a lot (more kivyMD and adb)!
Great! Thank you!
Mam pls tell which keyboard u use? Keyboard sound is awesome..
M ❤❤❤❤..
I'll watch it again to concentrate..
Your Stitch t-shirt is awesome! :D
The queen is back
you can use zip to creat a dictionary than use in to creat a row.
my doubt is why they kept all these as a library why cant't like built in function means without installing these library we shoud use thse library
Thank you!
Very nice video, and since I got to see this video I also checked your channel.... I subscribed
Keep up the good work! :)
What's this light projector device? It's awesome.
Muito obrigado, sigo sempre seu canal, cada vez mais conteúdos interessantes e úteis no dia a dia, Your Channel is the best congrats for your hard work, thanks from Brazil
Nice keyboard drums!
Can you explain if you need to save data to excel file?
Good question. You can try mine and check the playlist for Python. Very easy to follow fundamental tutorials for Python. And source files can be downloaded. Another playlist in the channel is R Beginning course.
Great video!
Greetings from Israel :)
ur amazing always , can u do any video on stock market data manipulation using pandas and numpy ?
Mariya, I I really thank you for your tutorials!
Great video. Congrats for your channel
Thank you ❤️🥰🥰
Great video, Mariya. Keep up the great work!!
You have a gift for explain, thank you, I would love to know about pandas and data analytics
Great video presented with so much joy! Thanks a million!
Thank you , for all the useful and informative content.
Thank you so much for your lovely comment! 😃
Great video. Makes me appreciate R so much more.
hello belle, thanks for all good information you shared with us. I am a new python learner and i would like to know how or the best way to extract data from a pdf file that has inside date, numbers and letters. i would like to see how we can sort them and delete what we don't need. if it is possible . thanks
Nice video. I am new to Pandas and this video has thought me a lot. Thanks!
Love the Stich pullover!
At last pandas!!!! Thanks for the video
Yeeeyyy!!! At lasttttttt! 😀
Thank you! 😊
What a great explanation!!! Thxxx!
Thank you so much Juan! 😊
hey you are amazing.. so much love from Germany....