Previously used Pandas but the speed and syntax are just so much better and more consistent in Polars (after the initial learning curve) I only switch back to Pandas if HAVE to (usually GIS related). Also, one update since this talk was given, it was announced that Polars will also be getting GPU support via Rapids (same project that brought GPU support to Pandas)
Lazy processing with scan_parquet and sink_parquet is pretty good, use it all the time. I usually have 10's, or even 100 GB's of csv's. Just scan through it dump to parquet (still bigger than I could keep in memory), then break it apart with sink/scan and duckdb.
I wish it could lazy scan from a database - the lazy frame function is excellent, but for data security reasons, I’m working from secure database and not parquet files.
I was trying to learn pandas just as Polars was coming out and I'm glad I switched to Polars. I was so frustrated with pandas nonsensical quirks and inconsistencies. It didn't seem to follow any conventions I'd ever seen. Maybe there was some statistics or math standard that I'm not familiar with, but as a programmer, Polars has a beautiful API that does what I expect it to do.
It's beautiful once you get used to it - I'd suggest reading the blog post "The Expressions API in Polars is Amazing", if you search for it you'll find it
Why did they keep interrupting??
Previously used Pandas but the speed and syntax are just so much better and more consistent in Polars (after the initial learning curve) I only switch back to Pandas if HAVE to (usually GIS related). Also, one update since this talk was given, it was announced that Polars will also be getting GPU support via Rapids (same project that brought GPU support to Pandas)
You're right! I've just seen the update regarding rapids-polars-GPU a few days after the lecture. Now I need to update the slides lol
Lazy processing with scan_parquet and sink_parquet is pretty good, use it all the time. I usually have 10's, or even 100 GB's of csv's. Just scan through it dump to parquet (still bigger than I could keep in memory), then break it apart with sink/scan and duckdb.
I wish it could lazy scan from a database - the lazy frame function is excellent, but for data security reasons, I’m working from secure database and not parquet files.
The polars learning curve was tougher for me than dask but performance is consistent 👍🏾
Great presentation. It seems some of the audience members didn’t quite grasp what Polars is from the questions. Either way, they now have heard of it
I was trying to learn pandas just as Polars was coming out and I'm glad I switched to Polars. I was so frustrated with pandas nonsensical quirks and inconsistencies. It didn't seem to follow any conventions I'd ever seen. Maybe there was some statistics or math standard that I'm not familiar with, but as a programmer, Polars has a beautiful API that does what I expect it to do.
I'll take Polars seriously when employers ask for certification. Till that happens, I see enterprises unable to uproot themselves from pandas.
Are they asking for pandas certification now?
@@ringpolitiet No of course not. There is no pandas cert, but cert in ML or Data Analytics using pandas.
Sectarianism aplenty!
Video is very basic. If you have tried polars already don’t waist your time
waste
Polars might be faster but the syntax is so bad, it's extremely uncomfortable.
Omg syntax in pandas is way worse
It's beautiful once you get used to it - I'd suggest reading the blog post "The Expressions API in Polars is Amazing", if you search for it you'll find it
Do you have an example of something you find uncomfortable in polars that is better in pandas?
😂 i think u did not know sql
Better than pandas, same shit as pyspark lesser than sql
Databricks, Spark Sql best
Wow, this is the kind of content, keep me visiting youtube.