25 nooby Python habits you need to ditch
Vložit
- čas přidán 14. 11. 2021
- Nooby Python habits give away your inexperience.
Improve your code and your prestige just a bit by ditching those habits and doing things the Pythonic way.
CONTEST CLOSED! The winners are: Lluc Cardoner, David Saint, Gaurav Verma 1412, and Alpe!
OFFICIAL CONTEST RULES:
1. All entries must comply with the CZcams community guidelines ( czcams.com/users/tcommunity_gu...) and CZcams Terms of Service (czcams.com/users/static?gl=US&t.... Entries that violate CZcams guidelines are automatically disqualified.
2. CZcams is not a sponsor of the contest and viewers are required to release CZcams from any liability related to the contest.
3. Privacy notice: no personal data will be collected for this contest.
4. In order to enter, you must (a) be one of my subscribers, AND (b) make a top-level comment to the video including #pycharm somewhere in the comment.
5. The contest is free, there is no fee required to enter.
6. Winners will be chosen randomly 1 week after the date the video went live from all users who have entered and not been disqualified.
7. Each winner will be notified via a comment reply from me that details what prize was won (e.g. "Congratulations! You have won XYZ. Please email me."). I will ask the winner to contact me by email, and I will reply through email with a random token which must be posted as another reply to the winning comment from the winning account in order to verify account ownership and prevent fraud.
8. Each winner will have 72 hours to respond AND prove account ownership or their prize is automatically forfeited and another winner will be chosen.
9. A winner can only win 1 prize per contest.
10. The prize pool for this contest is: 4 licenses for PyCharm Professional ("Free 1-Year Personal Subscription"), which is a product made by JetBrains. A prize consists of 1 license, which will be delivered in the form of a redeemable code that can be redeemed at www.jetbrains.com/store/redeem/ before May 01, 2022.
11. You may not enter the contest if doing so would be a violation of any relevant federal, state, and local laws, rules, and regulations, including U.S. sanctions.
― mCoding with James Murphy (mcoding.io)
Source code: github.com/mCodingLLC/VideosS...
SUPPORT ME ⭐
---------------------------------------------------
Patreon: / mcoding
Paypal: www.paypal.com/donate/?hosted...
Other donations: mcoding.io/donate
Top patrons and donors: Jameson, Laura M, Dragos C, Vahnekie, John Martin, Casey G, Pieter G, Krisztian M, Mutual Information, Sigmanificient
BE ACTIVE IN MY COMMUNITY 😄
---------------------------------------------------
Discord: / discord
Github: github.com/mCodingLLC/
Reddit: / mcoding
Facebook: / james.mcoding - Věda a technologie
1. 00:25: Manual string formatting
2. 00:36: Manually closing a file
3. 00:53: Using "try:" and "finally:" instead of a context manager
4. 01:07: Using a bare "except:" clause
5. 01:32: Thinking that "^" in python means exponentiation (it's a bitwise XOR, exponentiation is "a ** b")
6. 01:42: Use of default mutable arguments
7. 02:07: Never using comprehensions, or only using list comprehensions
8. 02:24: ALWAYS using comprehensions
9. 02:50: Checking for a type using "=="
10. 03:23: Using "==" to check for None, True, or False
11. 03:37: Using an "if bool(x):" or "if len(x)" check
12. 03:51: Using the "range(len(a))" idiom
13. 04:30: Looping over the keys of a dictionary
14. 04:48: Not knowing about the dictionary items methods
15. 05:01: Not using tuple unpacking
16. 05:11: Creating your own index counter variable
17. 05:21: Using "time.time()"" to time things
18. 05:43: Littering your code with print statements instead of using the logging module
19. 05:59: Using "shell = True" on any function in the subprocess library
20. 06:13: Doing maths or any kind of data analysis in Python
21. 06:23: Using "import *" outside of an interactive session
22. 06:33: Depending on a specific directory structure for your project
23. 07:07: The common misconception that Python is not compiled
24. 07:34: Not following PEP 8
25. 07:56: Doing anything to do with Python 2
Good man/person!
Thanks for this!
hey u can add 0:00 and put the text in the descriotion so we get the topics also when using the skipbar
I have to admit, some of the items in that list actually make sense.
gods work
Me, who confidently wrote Python as one of my skills in resume: Visible sweating
+1 :))
well the fact you were open to watching speaks well of you
enumerate(l)
And, how did you fare?
@@m.sierra5258 It was beside Java. So none cared😂
"you don't need to turn every loop into a comprehension"
I do not comprehend.
One of the things that I don’t see very often is using underscores when writing big numbers. Example: 1000000 vs 1_000_000. The former is much harder to get at first sight than the latter
Wow, u change my life of coding, but well not something u can save in an int
@@Absoluto777 Yes you can!
a = 1_000_000 * 2
print(a)
OUTPUT:
2000000
You can also format in the print function to get a formatted output.
I would add that for larger powers of 10, it's just simpler to use exponentiation as in 10**9 (or even 1e9, though this gives you a float) for a billion.
Tbh many companies still use old af compilers which not a big problem if in python but if you are taking the code to another language then it's rough
I dislike how the underscores look tho
watching this video really shows how far you can come in only 4 years of work. When I started learning to code I was way too worried about memorizing these things, when in the end all I needed was more coding experience. I'm glad I found some fun projects to work on while basically ddosing stack overflow with questions until the best way to go about things just got ingrained in my brain. We got this bros. My advise is to just find a project that looks fun and make it to the best of your ability, constantly pushing your ability little by little. The rest comes naturally. :)
o]
Can you give me some examples of projects a beginner can work on please? I'm looking at getting into Python soon.
@@bth1279 I reccomend building a web scraper. There's a lot of tutorials on here, don't overthink it, just pick the first one you see and finish it. Thats the important part
Sounds good. I will now redesign the Free CAD Gui using python and or C++. I am learning python just for this :) Im also trying to learn QT in C++.
I have just started a couple of weeks ago and every now and then I get a bit down because I feel like I have to copy everything from other sources of information like tutorials and stack overflow. I’m self-learning so of course I do.. how would I know these things? But it still feels like I’m an impostor and might just be too stupid for it. Like I had to watch a video on implementing jumping into my first mario like pygame project and when I looked at how it should be done I felt so down because I was way off to solve a problem that other programmers make seem effortless and straight forward. I guess I should not expect much after couple of weeks but still feels bad 😂 Will keep trying tho as it’s fun!
As an ex-lecturer (software engineering) I was reasonably happy with most of these. The "if len(x) != 0:" example is an exception though, as it explicitly mentions the important property (length). I find this more readable and I've had student code using "if x:" cause problems when they have passed the wrong type of data in and it's not clear what condition they are trying to test.
In general, a well-read programmer specialising in one language will be fine with compact styles of expression. A non-specialist reader (maintainer, manager, QC checker) can have real problems with some of the denser language-specific constructs. When I was a programmer I had to educate some members of my coding team on complex regular expressions and the trickier bits of Oracle's flavour of SQL. They just couldn't follow the code - they were not front-line programmers and it was above their more generalist knowledge.
Sometimes reducing eloquence for readability is a good call.
Yes. Even code you've written yourself can be hard to decypher a couple of years later and you often don't have the luxury of sitting down and untangling it all because you have other crap to be doing. These days, I write code with the intent that it be at least not difficult to understand and somewhat easy to fix or modify. Being concise and economical is for when you have 1k or less of memory and slow CPUs (though it is definitely possible to go too far in the other direction with that).
As Bjarne Stroustrup often says, the compiler doesn't read comments -- and neither do I :)
Using a less general construct that more explicitly delineates your intent and saves you from writing a comment is a definite positive in my view.
@@isodoubIet Not only that but the compiler will (if it's any good) optimize it to the same machine code (or byte code).
I personally agree that “if len(x) == 0” is more informative and readable, but Python’s official style guide PEP8 recommends using the fact that empty sequences evaluate to False and writing “if not x”
@@rrr-mi9kv IMO the purpose of any style guide is to make code more readable by reducing the amount of time you need to get used to different conventions. There's little value in the style consistency beyond that, which means if violating a style guide (which, by its nature, is "dumb" in that it can't account for every conceivable situation) improves readability, you should do it.
Notes for myself:
0:27 f string
3:51 range len
4:30 Loop dict
5:11 index counter
5:43 logs instead of prints
Some of these "mistakes" i make because i write in 5 program languages due to legacy. Sometimes it is diffcult to remember the "proper" shortcuts or style code of each of those languages. Still I'm not consider myself a noob programmer. And switching from 1 version to the next version is sometimes hard to get some of the habits removed. That's why i like to watch and listen to these kind of videos in my freetime.
Tell me about it. I will work with JavaScript for 2 or 3 weeks then onto maintaining some code written in Python 2, then I will do some other job function for a few months, a bit of bash scripting, then to Python 3. I have they general syntax of 5 or 6 languages bouncing around in my head. Every project I go back to, I spend a few hours picking up the nuances of a particular language again.
Each ide has code formating you don't need to remember everything.
1. 00:26 - Don't concatenate strings, use f"{string}"
2. 00:36 - Don't use f.close(), use with open
3. 00:54 - Don't use finally as context manager, use with
4. 01:08 - Don't use bare except, use except Exception or except ValueError
5. 01:33 - ^ is bitwise XOR, not exponentiation
6. 01:42 - Argument defaults are defined when the function is defined, not when it's run! Don't use l=[] as empty list, use l=None
7. 02:07 - Use comprehensions
8. 02:24 - Don't overuse comprehensions!
9. 02:49 - Using == to check a variable's type of a Class is tricky!
10. 03:22 - Instead of ==, use is
11. 03:39 - Using booleans with if, Don't act like a noob!
12. 03:51 - Using the range length inside the loop, instead, learn to use enumerate and zip
13. 04:31 - Looping over the keys() of a dictionary
14. 04:49 - Use items() to get key, value pairs of a dictionary
15. 05:01 - Use tuple unpacking to get all its values
16. 05:11 - Creating your own counter variable (enumerate)
17. 05:21 - Don't use time.time() to time your code, use time.perf_counter()
18. 05:43 - Instead of using print, use logging
19. 06:00 - Don't use shell = True
20. 06:14 - Learn to use numpy for math operations
21. 06:24 - Don't use import *
22. 06:34 - Importing files from other directories are tricky! Learn to Package your code & install it into your current environment
23. 07:08 - Python codes are compiled! (pyc, pycache)
24. 07:35 - Use pep8 to avoid nagging
25. 07:57 - Python 2-3, Notes about range() & keys()
Thx!
Thanks!😁 Legend😊
The "default gets evaluated on module loading and not during call with undefined argument" is insane.
You are greatly underestimating the sanity of job security.
@@DemPilafian , you greatly overestimate how much the people who design Python care about job security.
i 100% agree. is it really even a default at this point? i think not.
@@willpeterson3943 And for when people do want it to be a reference to a mutable global, the default expression can, just, well, reference that named mutable global.
I can't imagine how this happened. This global they are introducing is literally anonymous. What could be worse?
The parameters passed to a function are outside the scope of the function, even if they're the default parameters. If you want the default to be within the scope of the definition then you have to put it inside the body of the definition. It's a consistent and reasonable choice. More so if you're using a mutable data type it implies you want it to persist. If not use an immutable type.
This is probably the most immediately useful (to me) video I've seen on CZcams. I watched this two days ago and have since gone through two scripts I've been working on and edited them to incorporate 7 of these suggestions (yes I counted). There's at least like 3 other nooby mistakes I think I've made in the past but I'm not going back to check lol, so probably will just reference the video in the future. This also made me watch the video about looping in python, which motivated me to rework a loop in one of the scripts, and the 21 other nooby mistakes video, which I think is much less applicable to my current habits but may also be relevant later. Thanks for making this video and the whole channel!
Had this pop up in my recommended, figured I'd watch it for fun.
It is now several hours later and I've incorporated #1(f-strings), #16(index counters), and #17(time.perf_counter()) into all of the scripts I use for work.
I though "I'm not that much of a python noob" before getting absolutely humbled by this video
#pycharm
if your code works, it works. this is more for someone working on long term project or team project when readability it very important
@@Graverman "If it works, it works" is a very low bar to set
@@hughmungus1572 some of those mistakes actually slow down the computational time, then they are not good in any case, but when it’s really just readability and aesthetics, then does it matter if you can read it and it’s not colab?
@@hughmungus1572 its the necessary bar^^
same but I am far off coding something other people will read or use. I hope I know all this when I get a job. (I am only 15) so I have plenty of time.
After many years of python, I still didn't know that socket had a context manager, and also that range had a custom __contains__. Great vid! Also great pacing
Same here. Reinforcing my imposter syndrome ;)
@@StoneyVintson sus
@@kenonerboy What does "sus" mean?
@@StoneyVintson suspicious
That's because you'll never need to check if a number is in a range (I can't imagine even one case to use it in). I have 7 years of python programming experience and all this time I knew "x in range" feature. How many times I used it? ZERO
Number 26: Usage of single character variable names. Single character variable names lead to confusion and bugs in your code as it's hard to discern beyond the definition point of a variable what it is being used for. Opt for a more descriptive name as, after all, variable names are free. (As a SDET by trade, this is my biggest pet peeve)
i honestly think people who only use 1-2 letter variable names are just flexing and don't want anyone to be able to read or manage their code down the road. I've straight up just deleted an old script and rewritten it just to avoid dealing with understanding the previous coder's impatience with adding 3-4 extra letters.
@@RobGodMC If you're looping over the indices, it'd be "index". Never end a variable name with "_variable", it's always redundant.
`for i_hate_long_names in range(10):`
@@itellyouforfree7238 First of all: Not a descriptive name, refactor.
Second of all: Sorry not sorry. Quality goes beyond just finding bugs. It also is about preventing bugs and single character variables are a gateway for bugs.
Number 27: thinking that long_names and 1-2 characters are the only options.
Solid video - thank you for making and sharing!
2 things on this list that I need to do from now on:
1. replace time.time() with time.perf_counter() for measuring program execution time
2. learn how to install my modules as a package
3:45 This advice is *questionable*, sometimes is good to use "if len(x)" just to clarify that x is a list/set/tuple, or use "if x > 0" just to clarify that x is a num, not always obviously, but when you're too deep that you can't verify what your variable is supposed to be it's good to have something that explicitly shows what the variable mean.
Not to mention that this advice would fail on numpy arrays, which are recommended in the same video.
one could argue that if "you're too deep that you can't verify what your variable is supposed to be", your code is already unreadable. However, I also think that "if len(x)" is not that bad. Keep in mind that "if x > 0" actually has a different (and in some cases more useful) meaning than "if x", at least for negative x.
I think it’s called type annotations. The something that explicitly shows what the variable mean.
All your assumptions with this checks may fail if X implements appropriate magic methods for this operations.
@Jazzy Jones it's because an empty array is a thing that exists and is defined, so it can't be false. If anything, it's the [] is False that is weird in python. And none of them are are treated as None, obviously. Nothing is None except for None.
@Jazzy Jones Here's the result of evaluating an empty array:
DeprecationWarning: The truth value of an empty array is ambiguous. Returning False, but in future this will result in an error. Use `array.size > 0` to check that an array is not empty.
#pycharm, by the way :D
I was very happy when I first found out that you can use comprehensions in python because I am absolutely obsessed with the set builder method. I have been using comprehensions since I was a newbie.
Numbers 6, 7, 9, 12, 15, 17 are really useful and well illustrated. Thanks for the explanations & comparisons
3:37 Honestly, writing "if x" to check if "x" is a non-empty list hurts readability quite a lot. I always like to write "if len(x) > 0", which reads like plain English (#pycharm by the way)
Came here to say this. You shouldn't be trying to prove how smart you are with your code, you should be making it readable to even the dumbest person
Of course, but the _real_ problem is that you're using x as a name for a container. _That_ is what you should change, and then "if errors:" or "if tasks_remaining:" should become _a lot_ more natural.
Vedran is right. "if errors" reads much more like english than "if len(errors) > 0".
@@Jeyekomon Yeah sure, but at this point wouldn't it be better to have a boolean variable called "errors_occurred" (defined as len(errors) > 0) and directly check that? In general, using a non-boolean expression in an if statement is fundamentally wrong in my opinion
@@francescocariaggi1145 Opinions are nice, but have you ever coded like that? My opinions are based on decades of experience, working on systems of various complexity -- and they align with the opinions of people who have written PEP 8. I've seen, and written, "if errors" or analogous constructs, many times. I've never seen your "errors_occured" approach.
I can even guess why: because it's very easy for such a separate piece of knowledge to get out of sync with the knowledge already present in "errors". Having a single point of truth is a powerful concept, especially in complicated systems.
About your "fundamentally wrong", not many such ivory-tower opinions survive the collision with the real world. We do think of various operations polymorphically, and denying that in the name of some simplistic type alignment is not useful. It is the same idea as "using a non-iterator in for-loops is fundamentally wrong", or "using a float in floor division is fundamentally wrong", yet we really want to write (and think) about "for error in errors", not "for error in iter(errors)", and "n_frames // rate" instead of "n_frames // int(rate)". Practicality beats purity in many occasions. It's exactly the same with "if errors" instead of "if bool(errors)".
All excellent tips, except one: "if x is True" is proscribed by PEP 8. Doing this violates LSP, which you had just mentioned. This is because a Boolean subclass should work just as well. Just write "if x"; never write "if x is True".
but that doesn't imply if you are checking for a False boolean, or a 'falsy' value, like None, 0 or an empty data structure
@@cristian-bull then you use "if not x:" instead of "if x == False:"
In dynamic languages, people keep making silly "overloaded" apis where
True => {a_key: True}
False => {a_key: False}
but you can also submit a whole object
{other: 8} => {a_key: False, other: 8}
{a_key: True, other: 8} => {a_key: True, other: 8}
to supply more advanced options for the same argument
If x is not the same as if x is true. And if not x is not the same as if x is false.
Christian is right.
@Certyfikowany Przewracacz Hulajnóg Elektrycznych That's because many C programmers have a bad habit to compare boolean things with true or false.
Awesome video. While I started writing python nearly 6 years ago I really didn't do much for 3 of those years. I'm working on a big project rn and a few of these habits are present and should be corrected. Thanks
Thanks a lot for this. I'm an engineer with a lot of experience in other languages but Python is pretty new to me. This helped me easily understand most of the more niche concepts and the dos/dont's. Thanks!
excellent video, yet laconic, thanks a lot! I'll add to your #22 that usually the most convenient way is to install a package you're working on as "editable", with pip install -e . (dot)
took me a while to wrap my head around package/module importing rules
I think doing a full "Python 2 + 3 in the same module" video would be pretty valuable, you'd be surprised how many programs still use Python 2 to some capacity for scripting
exactly. i am stuck with python 2.7 at work because the program that we use has not yet updated their API to be compatible with python 3...
Bit of a necro, but I actually came down to the comments to make almost exactly this point. In my day job, I deal with ArcGIS, but being a municipal government job with the usual attendant budget and manpower, we are still realistically a painfully-long way away from the main IT department completing their migration to ArcGIS Pro, which is when the developers finally moved to using Python 3. So I write almost exclusively for Python 2.7, because that's what's supported in the older ArcGIS Desktop.
Constant references to "Why are you still using Python 2? Upgrade already!" when I'm chasing a problem are more than a little tiresome, as that decision is many, many levels away from being close to being my call.
@@altasilvapuer Came here for exactly this comment. At my job I just ArcGIS Pro on my machine, so most of my work is done in Python 3. However we use ArcGIS Server instead of Enterprise and all the environments use ArcGIS Desktop. So if I want to write any tools to sit on the server side I write in Python 2! Enterprise is in the works though...
Awesome, thankyou! These were really easy to correct with refactoring tools, or at worst, search/replace. The surprising thing was, how often some of these nooby habits occurred in my very well supported dependencies!
One of the most useful python oriented channels i came across youtube. Watching this video for the nth time too. Thanks for the continued great content!
I'd love to hear more detailed explanation for 3:23. Since it's for noobs, people won't get why they shouldn't use == instead of is (or vice versa). AFAIK there are at least two reasons (1. speed--since "is" checks reference while "==" checks value, 2. custom equality definition), but I'm not sure if there are more.
The reason for using "is" with None instead of "==" is ostensibly speed, but the speed increase would be totally unnoticeable under normal circumstances. Another reason is that None is a singleton, so using "is" acknowledges it as such. This is one of those things that in actuality makes basically no difference, but people will nag at you anyway, so it tends to be noobs that still do it.
@UCAl4YW8UbOfjz9ZxaQcXlzw I think you are talking about the example prior to what the commenter was mentioning, and I can assure you LSP is relevant there. Furthermore, LSP absolutely does NOT say anything about the relationship of two children of the same parent. Substituting one child class for another from the same parent is only safe to do if the function/method/etc was expecting the parent class, not if it was expecting a specific child type. In general you cannot substitute one child class for another, regardless of their common parent. E.g. in a print_id function that expects an object, it's fine to pass a list or an int because they are both objects, but in a print_all that expects a list, you certainly cannot expect it to work if you pass an int just because int shares the same parent type (object). I think you are also mixing up LSP, aka "behavioral subtyping", with Python's notion of structural or "duck typing". LSP is a general principle not specific to Python. It is a stricter requirement than duck typing. I am definitely not saying you should always try to adhere to LSP, or even to duck typing for that matter. However, checking a type for equality is a violation of both LSP and duck typing (well, assuming you actually do something different in the two branches of the if statement). The "Pythonic" answer is to only enforce duck typing, not the stronger LSP, but in practice this becomes very difficult to maintain on anything but small projects. Many developers therefore choose to abide by the stricter LSP most of the time. Moreover, if someone (especially a noob) tried to check a type for equality, say to overload behavior if they got a str vs bytes, an isinstance check is morally probably closer to what they wanted than "just see what happens if you call quack()".
If you start to worry about speed then Python is not your language.
I always loathed that there is the “is” statement and the == it’s poor design. After all, functionally you do the same thing.
A trademark of a bad language IMO.
@@CallousCoder There are subtle differences, but I do agree; even the reference comparison is easier to just do it like what one would do with C (pointer dereferencing).
@@cucen24601 exactly!
Love all the tips except for a couple. I like writing things to be crystal-clear, and I find that `if len(x) == 0` takes just a little less mental processing power to understand. Same for `for k in my_dict.keys()`.
yes, I thought the same thing with both examples
And the fact that you "code highlight"ed your code passages even though they are not rendered on YT is oddly satisfying :D
It's also worth noting that "if len(x)" is necessary for some data types. Pandas famously throws an error if you check the object itself: "The truth value of a (Series/DataFrame) is ambiguous." (Thanks for the headaches, Jeff)
Agree with the .keys() one, save for specific scenarios, e.g., codebase used mainly by python senior devs.
In general, I'd vouch for explicit intent, as I could see reasonable arguments and counter arguments behind multiple conflicting answers to "what should be returned by default when iterating over a dict"
Yep
You can tell m has never passed 0 instead of [0] into a function they wrote and spent half an hour trying to figure out why their iterator wasn't working
As for PEP8, please do read the intro: "Many projects have their own coding style guidelines. In the event of any conflicts, such project-specific guides take precedence for that project."
This actually made me feel a lot more confident about where I am in my Python learning journey. I expected to fail all of these but I actually consistently do most of the things you recommend!
I thought I was ok, but you got me with perf_counter. Though I'm usually measuring time in minutes not microseconds, I had no idea I was being so inaccurate this whole time. Great video!
Can’t believe you are naming variables the lowercase “L”. That is one of the things that was stamped into my brain when learning programming. Now I feel weird if I use it lol. Anyways, super helpful video. Never knew about enumerate(). This is gonna be a total lifesaver!
@meme why single character vars are bad? I'm newbie
@@gustanoid shortened var names make you memorize stuff. It's fine if it's something important that will be seen a lot, but for most purposes variables should have done sort of description of themselves. You don't want to have to remember a new encyclopedia for each member of your team when doing code reviews.
@@kidneybean5688 I'd like to give an example to extend your argument;
For example, you could have a list named illustrations and would like to use map function to map into another list. While using lambda/anonymous functions/closures/whatever you name it, using the name illustration as parameter makes the code too long and hard to comprehend. It is clear that you are iterating over illustrations, so simple single letter like x or a would come in handy.
names = map(lambda illustration: illustration.name, illustrations)
names = map(lambda x: x.name, illustrations)
This is also applicable to other languages, don't duplicate the obvious naming. Adding player_money field to Player class is usually pointless, don't name it over explicitly unless it is needed to be.
@@yokunjon I've been writing a lot of code in Elm lately (great language), which definitely loves its newlines, so I can get a little more spacey with my names. My solution to this in a language that takes up more horizontal space would either be to shorten "illustration" to a mnemonic like "ill", "iln", "ilstn", or to make some newlines w/in the function call.
@@kidneybean5688 I definitely agree. I also started to use Rust for almost a year and libraries in Rust tend to use long names for sure, as being able to vertically space out code helps greatly. Though, I still prefer shorter names. It wasn't fun to use InteractionApplicationCommandCallbackDataFlags tbh :D
Thank you so much for these tips. This video was so clean cut and well explained.
I had no idea about the issue with mutable defaults. That's wildly counterintuitive.
Of course, they probably can't change that behavior because some major library somewhere is probably using it as a core feature.
this is one of those videos where I learn about a bunch of things I'll immediately forget without usage. I reckon I ought to save it so I can reference it the next time I'm pythoning
Now even if you save it, you're just going to let it sit there and some things will change by the next version of python. Then, after you review it, you'll still forget it later. And the likely scenarios where you do need it won't come up. Info dumps, man. It's all the same.
Excellent video, James, thank you. Commenting time stamps for myself and hopefully some others:
00:26 Manual string formatting; use f strings intead
00:36 Manually closing a file; with blocks close them
00:53 Using try-finally blocks; instead, use context managers
01:07 Using bare except clauses
01:33 Using caret exponentiation; use ** instead
01:41 Using default mutable arguments; check video for correct usage
02:07 Never using comprehensions
02:23 Always using comprehensions
02:29 Checking for type using "=="; use isinstance(object, objecttype) instead
03:23 Using "==" to check for identity; use "is" instead
03:37 "if bool" or ""if len" checks; instead use x itself
03:51 Using range(len(lst)) in loops; use the list itself, enumerate and/or zip
04:30 Looping over the keys of a dictionary
04:48 Not using the dictionary items method
05:01 Not using tuple unpacking
05:11 Creating your own index counter variable (Dang, this is the one that hit me the most.); use enumerate instead
05:21 Not using time.perf_counter() to time code execution
05:44 Printing excessively instead using the logging module
05:59 Using "shell=True"
06:13 Using Python natively to perform mathematical or data analysis; use numpy or pandas
06:23 Importing everything instead of the things you need
06:34 Depending on a specific path for your project
07:07 Thinking that Python is not compiled; it is
07:34 Not following pep8
07:56 Still using Python 2.x for no good reason
# pycharm
I always love these lists b/c I'm SURE I do 1 (or more) of them. Always wanting to do better. I'm not SO bad that I do the "string A " + "string B" (anymore) but I was doing more "Value of string A is {}".format(strA). I like the named interpolation better, to be sure. On the "with" statement, I'd read somewhere that you can omit "as " but I'm not sure how to get that TO work. I just use "as " and go about my life.
About the 19 tip: There's in fact a function which I use to avoid typing in manually the command inside a list. Instead of just typing the list, when using big commands, I just import "split" from "shlex" module, which can be used to split text into lists (i.e. subprocess.run(shlex.split("find / -type f 2>/dev/null")) and this will handle the problem
That solves many of the problems with 'shell=True', but not all of them - in particular you still have to worry about things like spaces in filenames if you're generating the command from strings. It's potentially more exploitable with malicious user input as well. It really is best to just bite the bullet and form your command as an array.
Also FYI your example doesn't work - it will execute the command "find" with four arguments: "/", "-type", "f", "2>/dev/null", which isn't what you meant. If you're not using 'shell=True', you can't use shell constructs like "2>/dev/null" or "|" or ";" etc.
There are 25 tips, not 19
@@csanadtemesvari9251 would seem they meant tip 19/25
I am learning Python and all your videos are amazing! Thank you so much for helping me in the python learning process. I really like your teaching style. Some of your python topics are above my python comprehension because they are intermediate or advanced and that is okay. I will come back to those videos in the near future when I gain knowledge in my learning process.
In many of your videos there are hidden gems. Those gems makes me realize that noobies need to ditch another bad habit: not reading the python "What's New in Python" documentation when a new python version comes out. I learn so much from reading the PEP documentation with example code that are referenced under the new features/implementations. Plus, I feel this will keep me in sync better with the python language updates.
Haven't even expected how "noob"ish I was.
The channel is underrated, because it's rare to se someone who digs deep into the topic with lots of details. Usually creators just run through basics of Python and that's it.
#pycharm
Absolutely amazing video. I'm very familiar with other languages but switching to python and your list is a perfect resource for that.
Thanks a lot but I gotta watch the video a few more times while working with python the get rid of my noob mistakes. Great Video and Im sure it will help me.
Fun story: I was hired as a PHP developer, but soon was tasked to overtake a Python project from a freelancer that was leaving in just a couple of months. Lambdas, list compression, confusing yet powerful multi-inheritance, wrapper functions and attributes, constructors and initializators, bytecode compilation, tuples, dictionaries. I quickly fell in love with the language. Now, I'm back dealing with legacy PHP code at another company and miss Python dearly. Still need to figure out a pet project, so I could finally scratch my nostalgia itch. #pycharm
Look at Ruby, it is Python done properly.
@@transientaardvark6231 Thanks! I'll look into it. For now I'm having fun with F#.
It's list comprehension, not compression
@@transientaardvark6231 It is also Python abandoned
@@itellyouforfree7238 yeah, looks like it got autocorrected, and I missed it
Love your format. Thanks for taking the time to put these videos together for us! I've decided to move over to a new career in python and data analysis after working for years with php. Every day is a school day👍 #PyCharm
This is the content I'm searching for, thanks!
wow thanks. i just startet learning python for school and got the hang for it. (even got schown some of these habits by teachers) and i think some of what you said really will help me for my current projekt. thanks
One more point that I think is a must in the list: defining class properties and expecting them to behave as instance properties. That is, defining properties on a class outside of init.
Seen this one to be genuinely confusing people that come from other languages.
Is it possible that you're confusing properties (i.e. @property) with attributes (e.g. self.x or cls.x)? they aren't the same thing, though I do believe you make a good point.
@@valorien1 no, it's about defining attributes in class definition vs. assigning inside init
@@kristobaljunta Which was precisely my point. you wrote "defining class properties..." when you meant "attributes". They aren't the same thing.
@@valorien1 what would i google to learn more of this?
18: No, that does NOT look a lot better. In logging format, just use {} formatting instead of %. It's more consistent with the earlier referenced f-strings and therefore much more readable.
Agreed. If data such as the time capture is required, or the ability to pipe to a file is necessary then logging is the way to go. Otherwise, just use modern formatting.
In either case, debugging through standard out messages alone is the real problem, and switching to logging isn't an improvement there.
agreed.
As an experienced Python user (5+ years, 10k+ lines experience), I am happy to know that I followed most rules mentioned in this video correctly. I did manually open and close files in REPL because you have to type 4 more space on every line afterward if you open a file using context manager (with statement). I also do this when I am writing a script in a hurry and need it to work right away. I don’t know that the perf_counter is the correct way of timing code (though I rarely measure the running time of Python code at all). For creating my own index variable, I have to say that it is sometimes necessary if you are using a while loop and the update condition is not as straightforward (for example, implementing some algorithms). Also, while I agree that numpy is powerful, there are a few cases that the built in Python math is better, such as big integer by default, and you never need to worry about overflow.
Generator comprehensions? Didn't know about these. Awesome video, thank you! :)
great video, I would love to see a video about logging module!
5:43 "number 18: littering your code with print statements instead of using the logging module" #pycharm
I felt that... I've been doing personal python stuff for at least 3-4 years ... yet I actually never spent the time to learn how to use logging and how to format stuff with it, etc.
same here! ive been writing python for almost 8 years now and i still use print statements rather than logging lmao, logging looks so much cleaner though
It looks nice but I can live without it
Meh, I've been using logging for a decade. Most of the time I find myself just relying on a mix of assert/break/print anyways. It's just quicker and easier and 90% of the time I'm just doing a temporary inspection. Logging is super great for end-user information, but for debugging break/assert is the right tool IMO. What's the point in all that pretty printing when you're just querying about internal state, etc. Obviously this changes a bit in collaborative projects, but even at that, I'd like to argue asserts are descriptive enough.
That was very clarifying. Awsome content!
Thanks so much for this. It's super helpful.
3:43 I find "if len(x) > 0: ..." more readable than "if x: ..."
4:27 Nested tuple unpacking is new to me though, thank you :)
5:51 I also didn't know about logging. I always rolled my own logger class.
It depends how you use x. Sometimes it makes more sense to use len in order to understand the code
@@Grivian Of course it depends on how you use x. But semantically speaking, the if statement expects some kind of condition to be evaluated, and x probably being a list in this context is not a very prototypical condition, unlike len(x)>0 which to me actually looks like a condition.
Woow, as a total newbie - started with python this semester at collage - this was an information overload, great vid though.
P.S. Loved the "smash the like button an odd number of times", but I'm at even and it is currently liked now, so I have to apologize for even momentarily disliking to achive this greatness
Funny to see that somehow I adopted your recommended habits naturally... Maybe reading some introductory books helped. Thanks for your efforts making these videos.
Thanks a lot for this video. All of them were helpful for me. I have become a subscriber of your channel now.
Great tips and I learned some stuff myself. For 6:14 / tip 20 (always use a math library like NumPy) though, I really need to caution against doing it blindly. For example, libraries like NumPy are mostly designed for handling large arrays. I don't want to get into the specific context where I encountered this, but I have seen situation where lots of people just blindly assume NumPy is the best and used it for doing math on 3D vectors using NumPy, resulting in really slow performance. From what I have found, NumPy is not always faster than just doing the math in Python directly if you need to do lots and lots of 2D / 3D vector calculations because in Python you could just represent them as a simple struct, and you don't have to wrap / unwrap them as NumPy arrays (which, again, is optimally designed for handling large arrays, not when length == 3). If you are doing data analysis type work with a long array of 100+ items, that's another story.
So bottom-line is: try to understand the performance characteristics of your library and understand if it actually fits your needs. Just because a library like NumPy is popular and fast for its intended use cases, does not mean you should just use it for everything math related without thinking.
The thing is, if you have to do "lots and lots of 2D / 3D vector calculations" you would probably stack them in a two dimensional `numpy.ndarray` and apply the same transformation to all of them simultaneously.
Python has so many little tricks that I'm unused to with my experience in other languages. Thanks for teaching me a few of them, particularly tuple unpacking! #pycharm
Can't wait to watch this video in a year and actually understand what's happening!
I'm glad I learned nearly all of these on stack overflow.
Somehow this channel is extremely underrated. I’m just a beginner in python but few of the videos have helped me to such a extent that has reduced my months of work.
Just to give an example the video on for loop Vs while loop is a gem.
Thanks a lot sir for sharing this valuable information.
#pycharm
Great video. I so much agree with 8, few times I stumbled across the projects that overused list comprehensions, sometimes it was painful to read such code. List comprehension is a fantastic Python feature, but people should learn when to use it and when to avoid it. Readability should always come first imo #pycharm
Well said!
readabilty can be improved by using multiple lines, in this format:
[ this
for this in that
condition
condition
]
@@kilianmio6243 Python is regarded as self-documenting code. As soon as you need to explain what your code is doing, it's not good code. Comments should only be used for why your code is doing something.
@@mCoding #26, not installing black and having it run on save...
With great power comes great responsibility. Python has a ton of features that can both make and ruin your code. Knowing what tool to use in each situation ensures the code is readable
[2:01] In Lua we do something similar: in Lua there aren't default values for function parameters. By default all parameters are nil, so we do something like:
var = var or some_value
Or first we can assert(var) to check if you placed the correct type.
it's important to note assert(false) will also fail, not just nil values
Returning to this video for 25th time, so helpful, thanks!
3:39 even if len will cause a little bit more execution time, it is much more readable compared to if something itself. Also, if you want a seperate *raise* call and/or logic for each case, you have to do a *seperate if ... is None* check and *if len(...) == 0* check anyways.
#23 mostly happens because people expect bytecode-interpreted implementations to work almost as fast as compiled implementations. But due to Python's dynamic typing and slow function calls, it lags behind a lot more than other languages
Holy crap it is soooo slow haha. Doesn't matter most of the time though.
3:50 I prefer `if len(x) == 0` over `if x` to handle the case where someone extends from `list` and modifies the `__bool__` attribute. It also will throw an error if someone passes in a type that doesn't implement `Sized`.
Ooh I just learned #9 independently only a few days ago!
I had two classes which inherit from a base class, and I wanted a function to check whether a class passed as an argument was one of those classes. I thought I would have to specify the two subclasses, but I was curious to see if isinstance(argument_class, BaseClass) would work, and it did! Then I tried type(argument_class) == BaseClass, and it didn’t.
I had learned isinstance from my beginner Python textbook, so I always used it anyway, but I saw a lot of code using type equality. I assumed the difference was mostly aesthetic, but I’m glad to now know that there was a good reason for what I was doing.
#pycharm I'm migrating from primarily C# and C++, and I really appreciate this video as a guide to get into good Python habits. Thank you for this!
why the migration ?
#pycharm It's always really interesting to watch your videos. It makes me realize that there's so much to learn and just how different things are outside of a university context.
For bool/len check, it is always a good idea to specify bool/none type checking. Eg: If a function returns bool and return value check is like 'if not x' and there is an exception in function, then 'if not x' will be false, which is incorrect.
Great insights, with at least one observation, and forgive me if I come off noobish, but @3:37, wherein the context of the function presented, for x is unclear both in terms of expectation, and the specified function's purpose. The function looks to be a validator, and depending on what it's meant to validate, simply refactoring the code to a single if x. may or may not do the intent of the presented code. Take for example if it's meant to ensure that X evaluated to true or false as a value, but more specifically true. The first test would pass if x evaluated to true, but not if it evaluated to 0 or false. However, the second test may pass x if it were a string value with a length > 0. But since it's not converting x to a type wherein a length attribute exists, the context will not evaluate to true for a numeric type (as is). so a value of 0 will not be given a length of 1 allowing it to pass. So a string or number evaluating to 0 will never pass these tests, and will not pass "if x" either by my understanding for 0 translates to false.
If this function were to have behavior based on x instead of validating, that's also a matter that a simple "if x" test would not address as the two tests can deduce type after a fashion and behave differently based upon which test "passes", and in both scenarios, what a pass does in that context. I mention that because the pass case doesn't seem winner take all how it's coded. so both tests should execute and do something, again depending on what behavior that pass is meant to do.
@Derek Youngman presented the following in his response, which highlights some of the nature of my point, that if x will evaluate to true if there's a non-zero evaluation of x and that includes its behavior with True and None, when not checking their identity with the is operator in those cases, but say x = None. if x will not evaluate to true, but is not specifically checked for with is in the case presented. @Derek Youngman response:
If x is True doesn’t do the same thing as if x.
x = 4
if x -> True
if x is True -> False
y = True
if y -> True
if y is True -> True
/end response
So, I would conclude that it would depend on what the objective of the code is on whether this case was noob goodness or not. But the case may have also been vague in the interest of the claim, but that further should have made the example more compelling to show where the claim had its validity in a more IRL context than a hypothetical one, for clarity sake as the claim of one not knowing the language was pretty strong, given the lead-in stating there was nothing particularly wrong with the statements themselves. If a bool or null check is what's called for then by all means they should be used, it's like the folks that hated the or tags, seems a bit personal and maybe a little pedantic. And to that end, the function name should have changed to reflect that drastic change of the functions purpose if no longer checking for bool or length, but if non-zero instead, which again, doesn't address if the original code was fit for purpose and the person "optimizing\
efactoring" understands they why of the original logic to conclude incompetence of the original Developer.
But I ain't one to gossip... .
#pycharm I'd really love an expansion on the logging and use of perf_counter. Timing stuff always seems like a pain
8:30 not necessarily true. *Only* works if the value you are checking is exactly of type int (or bool). Does not work for other int subclasses, any other object, and , most importantly, float. For those it will still fall back to one for one checking all values by iterating over them. That is very very slow for large ranges.
I think I need to watch this vid 3 times a week. Thanks a lot.
Almost 2 years into python, and i've programmed a lot of things in python.
Well,... there's some of these things I knew but didn't apply ( enumerate instead of range(len)), and some I simply didn't know.
I'm baffled. Finally a video of this kind, or the kind "things I wish I knew" that is actually helpful ! congratz ! (Also shame on me, for taking bad habbits, this isn't going to be easy to fix)
3:26
Using "is" to check for booleans is also a noob mistake or even a bug, as when using other libraries is will return false no matter what. So in the case of
a = np.array([True])
a[0] is True
returns False. Just use if a[0]: or if you want False statements, if not a[0]:
Oh yes. I just checked it. It's because
>>> type(a[0])
The 'is' keyword checks if two things are the very same. Two things which are not even the same type cannot be the very same.
>>> np.bool_(True)
True
>>> np.bool_(True) == True
True
>>> np.bool_(True) is True
False
This is not a bug but intended behavior!
If a[0] is something completely different than if a[0] is True.
A thought provoking list. I agree with most of your items, except PEP8. I will continue to use extra spaces inside parentheses. Power To The People
I had no idea about #6. I'm sure you just saved me from an eventual debugging nightmare; I can imagine a lot of cases where this subtly messes up things in a way that is hard to track down.
Yeah I learned that one relatively late in my Python education and was quite surprised at what a footgun it is. It may or may not be justifiable that it's there, but it's definitely a potent footgun.
Great vid, i didn’t know about a quarter of those. Some because I never even heard of like shell =true. Others were really silly, like I’ve been using list comprehension but didn’t think that would work for dictionaries too 😅
#pycharm Gotta love how you're channel's python tips consistently manage to surprise, been hooked since the video about the cache decorator!
Owo
#pycharm As always: thank you for the premium content! I've learned a lot from you so far :)
I just finished watching a video about not writing clever code because it will lead to bugs, and #6 is a perfect example of it.
Cool. Pretty much do all of this. Main thing I guess I never got used to was using enumerate
Most of those comes with a bunch of asterisks for them, an a couple of them are not necessarily what you want. For example, while you can often get away with "if x", you run into the issue the the boolean value is implicitly infered, but one of the basic zen of python states that explicit is better than implicit, and in this case you can often increase the readability of your code by explicitly stating what kind of property you expect, and it will tend to also give you better errors closer to the problem when you get an "you cannot use len on this thing", rather than it just performing the code for 0 lenth collections. That is a intermediate error, and not a noob one though.
Here are some examples of the asterisks (as in cases where it is not quite right):
1) When nesting string construction into each other, things quickly become complicated and you will want to start making use of more of your string building tools than just format strings.
3) When you are building context managers yourself or you want custom local behaviour.
4) The bare except can come up when dealing with dangerous multiprocessing, and you want to ensure that you do not leave zombies behind, though you would generally raise the exception again after doing graceful shutdowns. For the except Exception: case, it is especially useful for multiprocessing calls on the worker side, where you then have the option to gracefully send back information of the error.
6) There are some very rare cases very similar to singleton cases where you would want this. That said you always have to think really carefully about singleton cases, so this would be a super rare case where it would make sense.
7&8) They kind of already go there, and sometimes comprehensions are just not really good for the thing you are making, and sometimes you really should be using numpy instead.
9) Even isinstance is far from broad enough for a lot of cases, often you want to use hasattr instead to work better for ducktyping situations. Naturally there are also the rare cases where you need to make sure it has not be inheireted.
10) "==" has a different behaviour than "is" when the arguments are of different type, and especially the == True or == False has uses. Also be aware than the latter 2 can be vectorized with numpy, while the "is" format cannot (at least nicely).
11) See above about clarity/location of errors.
12) The idiom is used when you need to do alterations on a mutable collection and those alterations are not limited to operations on mutable internals. It also much more often happen as part of algorithms, though those are more commonly done with just range(n). An typical example (where it is double nested) is when you apply a local filter to an image, and you loop over the elements of the filter, and then do numpy operations to construct the image component from each part of the filter.
13) Okay this one is a bit far fetched, but the dictionary.keys() object can be a lot smaller (iirc), so if you want to build something based on those keys outside of the shared memory area, you could package it down and send that instead of the larger full dictionary. This would mainly be for multiprocessing.
14) Most of the time you actually want the dic.values() instead, if all you want are the items.
15) Tuple packing and unpacking can take time, especially if you have something long, and it will create the parts you ask for. If all you want to check is the first 3 elements of a 100+ length thing, then do not spend the time writing a, b, c, *_ = items, just write first_three = items[:3].
16) The case here is when you do not necessarily increment it every time as it refers to something slightly different, but I am unsure of whether that really is needed, and that kind of algorithm code is much more often written in c style languages.
17) Most of the time, you should run cProfiler on your code instead of checking manually this way, it will give you much more information.
18) Logging can be problematic in multiprocessing situations, whereas print is usually pretty safe there (logging can get into nasty deadlocks).
20) Numpy is nice, but a lot of good algorithms cannot make good use of it for at least some parts, and sometimes going with O(n log(n)) is better than vectorizable O(n^2). Neither numpy or pandas uses GPU acceleration, so once you start wanting that you also need to move away from them. Pandas is also kind of weird, in the sense that if you take a look at benchmarks, it tends to be quite poor compared to just base numpy, so you have to really want the other features of it for it to make sense to use, because a lot of your basics are just going to be slower because of it.
21) The general python style advice is to import modules and not component out of modules in most cases.
22) Making everything as packages makes it very hard to run parts of the code as scripts, so you should not just convert everything into packages.
23) This is blurring the waters a bit, as it is intepreter dependent, and you will not find compiled files for all python code. I for instance have not see such files for interactive sessions and other equivalent things. It is correct that python code is built into python byte-code at least im memory though.
25) In python 2, print was a statement and not function. Iirc generator comprehension also first came into python in one of the 3.x versions.
Of all those habits, the only one I would say I fall into is 17, as I was not aware of time.perf_counter.
#6 can be confusing for people coming from languages like C#. In C#, we call these "default arguments" (which have to specified at the end of the parameter list) and they are always that value if nothing was passed in. Unlike Python, they aren't stored in memory to be used every single time that method is called if nothing else is passed. I think that's a horrible way for python to handle it.
That's pretty much every language except Python, which decided to be weird. Python has a lot of quirks like this.
That's an extremely stupid way of Python doing it. I actually had to rewind and listen to #6 again, because I thought I misunderstood something because no way it can be THAT stupid. Apparently it can.
@@sgtGiggsyIt seems really stupid until you realize you can (ab)use it. It's a pretty neat way to do caching, for instance.
@joje86 Independently of mutables, I would gladly trade this trick for the ability to use dynamic expressions of previous parameters in the defaults instead of having to set default=None and have default initializations take half of the function body.
I have some python experience, but I picked up multiple things from this video, super helpful!
Although now these things look natural for me, it helped me a lot when I was a beginner, first time I saw this video. Thanks for making videos.
You are very welcome!
#pycharm
1. In my defense, print is a perfectly fine way to debug a code. It's simple and easy assuming it's only temporary.
Also, I'm surprised you didn't include using top-level programming as noobish. It's fine for simple code but some beginners still make their code looks like it's not an OOP language
After years of writing code professionally, I still can't be bothered to use the logging module for debugging. print statements all the way, they're easily added and easily removed.
@@jurgenhaan7652 One advantage of logging instead of printing is that you don't have to add or remove print statements manually - you can easily config logging to only print debug information in debug mode, but disable everything except critical errors in production mode.
I place at the beginning of my code: DEBUG = print
When complete, I do a line replace and get rid of all of them. And DEBUG gets its own highlight, and so stands out from needed "print" statements.
@@k.chriscaldwell4141 "when complete" is not always a hard statement. That's why I also do as you, but don't remove the line, instead replace DEBUG for #DEBUG....just in case.
" still make their code looks like it's not an OOP language"
Well, it's not.
Most of these things are really specific to Python. I’ve been learning this language by myself doing some personal projects and I see that I don’t know most of these pythonic stuff yet, but I can code many things following what I already know in other languages like javascript, C, C# and Java for example.
When looking for beginner courses I usually don’t get many of those features that were shown in this video, instead usually those basic courses teach about introductory things in a more procedural way and in some cases in object oriented fashion.
Yeah I learned python before java and for a long time most of my python code looked like java code. It's not the end of the world if you do this, but it is still very important to learn the "pythonic" way of doing things
I'll have to watch this a few times t o get it all..
Thank you!
I actually didn't know about the logging module and I legit learned something quite useful in this video. No more constant "if args.verbose:" checks. I can just set the logging level right in argparse. :D
6. 01:42: Use of default mutable arguments
9. 02:50: Checking for a type using "=="
17. 05:21: Using "time.time()"" to time things
18. 05:43: Littering your code with print statements instead of using the logging module
19. 05:59: Using "shell = True" on any function in the subprocess library
22. 06:33: Depending on a specific directory structure for your project
23. 07:07: The common misconception that Python is not compiled
24. 07:34: Not following PEP 8
I haven't programmed in python in a bit, this was a super helpful reminder!
I am no beginner - I started with python back in college circa 1991. But I do a lot of mathematics in pure python because I need the bignum integers and mpfr support for high precision floats.
Well I have now learnt that I am officially a noob lmao. #pycharm