Debugging 101: Replace print() with icecream ic()
Vložit
- čas přidán 22. 05. 2024
- Today we learn about the Python package called icecream, which you should use instead of print when debugging your code.
◾◾◾◾◾◾◾◾◾◾◾◾◾◾◾◾◾
📚 Programming Books & Merch 📚
🐍 The Python Bible Book: www.neuralnine.com/books/
💻 The Algorithm Bible Book: www.neuralnine.com/books/
👕 Programming Merch: www.neuralnine.com/shop
💼 Services 💼
💻 Freelancing & Tutoring: www.neuralnine.com/services
🌐 Social Media & Contact 🌐
📱 Website: www.neuralnine.com/
📷 Instagram: / neuralnine
🐦 Twitter: / neuralnine
🤵 LinkedIn: / neuralnine
📁 GitHub: github.com/NeuralNine
🎙 Discord: / discord - Věda a technologie
You could do this all much quicker by just inserting the built in python debugger with the line "import pdb; pdb.set_trace()" where you would normally put your print statement. It will then start the debugger in your terminal and you can view and even change all variables during execution. It also helps finding exactly where your code is breaking as you're stepping through the code line by line. I can't recommend it enough.
And recall in Python >=3.7 you can simply use `breakpoint()`
Ah! I didn't know that, thanks🎉🎉
Thank you. I have been debugging code for like 2 weeks using print and exit()
Don't get too cozy with that. ><
Professional environments often lack the ability to run debuggers. You are only guaranteed print logging. Debuggers come later. Ice cream as a better print starts to look real good in cutting edge environments.
They may(or may not) have caught up now, but during my time, cloud/embedded systems lacked debug support. Odds are, the next cutting edge thing will too.
@@gpeschkeWhat kind of environment doesn't have access to the terminal? The python debugger is a standard library included with python, and it starts in the same place where you would see your print statements.
You know, from 3.10 onward you can essentially do this without packages by doing print(f"{variable=}").
Putting an = after the thing you are formating makes it so that it tells you the name/calculation together with the value.
He covered this at the beginning
@@shubhamshrivastava5812 well he only hinted outputting with f function, but not the = suffix, which is interesting information.
I think the key advantages of icecream here are - being able to store the returned value, automatic context (line number, function name), ability to write to files rather than stdout, and the ability to disable all ic logging with a single command as opposed to tracking down every print statement you added while debugging.
I would compare icecream to the `logging` package in python's standard library and not to `print` with f-strings.
ok wow. I'm using string formatting for so long but never knew this. Awesome. But the ic looking like less writing required than that;
Thanks for this tip 🙂
One of the first things I do in any new language is create a debugging system. Helps to get you confortable with the syntax and environment, and gives you the tools up front to progress more efficiently.
This is nice, too.
Sounds like a good project, thanks
When you say that you create a debugging system, what does this entail for you?
@@JanB1605 Honestly, probably way more than I need. Usually, a sepatate console window that runs with the program, or if fullscreen, a separate screen I switch to that shows the state of current elements. Recently, I've been programming quite a bit directly on my phone in Pygame with Pydroid3, and it doesn't have a way to show debug print() statements in a console like you would normally have in an IDE, so I use pygame to draw the debug info direcrly to the game as it runs. I'll write my debug info to a txt file for later review while also displaying it in the program. The particulars depend on the application, but I usually get the variable name, its value, variable type, line number, filename, possibly a traceback, etc. The normal output you would need to swuash bugs.
Got any video on good debugging systems?
@@emmaaberg3566 There are ton of CZcams videos on debugging in VSCode. If they don't satisfy you, check out debugging in Visual Studio (similar). There are also a ton of CZcams videos on debugging in Pycharm if that's your thing. And of course, some for Sublime and others.
Learnt something useful today. Thanks, this is something I will probably use in my classes as debugging is what I do 99% of the time.
Use a debugger
@@Djellowmanas a viewer of bad code, I approve this message.
@@Djellowman vs-code debuger is still not very good with Django... So this ic seems a good option to me
With Django, I failed to use even the PyCharm's debugger 🤷♂️
(I haven't googled yet, though.)
I made a custom logger once with a decorator style function, similar to this and called "log_it", Tired of debugging step-by-step and wanted a complete debug during a full run.
Was like a swissknife that I could simple plug @ on any unknow function that somebody had written to "log_it" and check what was done. Printed the functions names that were being executed, the *args, **kwargs, variables values / types, total elapsed time, memory comsuption, functions results, and log errors aswell. Had a option to print only or save all this data every run in rootfolder /logs.json. There were functions in module to retrieve all the log history back to check what was done.
No complications, just put the module in root folder of any project, decorated the functions with @log_it and the magic happened
I'm not the IT guy (i'm financist) but damn, I felt like a python god when I was learning and wrote those 50-100 lines lol
Not going to lie, I use A LOT of print statement to debug. This is very cool , learn something new and will start trying TODAY!
Very useful. Don't worry about "professional" way. Log files and print statements are the way to debug large runtime programs.
@RobertLugg, Log files are very useful.
Interesting. Will check it out. The 'timing' of when the icecream output happens looks relevant and you didn't touch on the behaviour there. But a nice introduction. Thank you.
I would suggest skipping ice cream and going straight to python logging. It give much more control over how/what logging in emitted. I have found on many projects that starting from day 1 with using logging, instead of print is a great idea. The only thing I see that ic does that other don't is the quick ability to wrap a lines of code in ic and get a decent print that can help quickly debug issues. That said a debugger could also solve that use case.
Same 😊😊
I was going to post this. Using logging levels feels much cleaner and flexible.
Yes I agree
Thanks so much, I'm constantly learning new things from your videos and it's like this video was made for me. I was feeling specifically targeted when you were talking about using print(a) and print(type(a)), etc. 😄
I used at least print(f'(where): {var=}').
Or logging.debug(...)
(Yes, I know, it is not effective, and it is discouraged by the linters.)
I just found out about ice cream a couple of days ago. It was just what I needed when I needed it. Right now, I am developing in Flask, which doesn't print to the console, unless you flush. Ice cream takes care on that.
this is good to take debugging step by step, it reaches out to be complicated at the end, so I think it would be better to show the real level at the end, which is the built in debugger, where you can do all the debugging techniques ( breakpoints, tracing code line by line, viewing all variables..etc )
Great insight. Always love your videos!
Only after hearing it a dozen times did I realize that the function ic() is supposed to be a pun "I see" lol
This is very cool! Didn't know about it, but this looks super helpful. Thanks!
Sounds like I'm calling goodbye() on print()! Thanks for the video mate, super handy
I don't think there's anything more 'professional' about using a debugger.
And I'm really happy to learn about this new tool. It looks awesome. I sort of wonder if the enable and/or disable functionality can be used as a context manager.
@@skulver - I don't like them. They encourage the creation of code that can only be understood by stepping through it with a debugger, and they often don't actually work, or they alter the environment enough that the bug doesn't occur when you're running in the debugging environment. They're also often a huge pain to attach to things that are doing things like handling web requests.
As for IDEs, I only sometimes use a debugger in the context of one. Perhaps, more often than not. But I do frequently use a debugger from the command line, when I use a debugger at all.
Sometimes, they're a good tool, and I will use them if I think I can solve my problem more efficiently by using one. But, most of the time, a print statement is all I need.
@@Omnifarious0 I think debuggers truly shine when there's an external dependency. For example, in an ETL script, you will get the data from multiple external sources and no matter how well you frame your code, there's bound to be unexpected behaviour since the data itself might be misformatted. Adding print statements in this case can quickly become tedious especially if the parts of the data are correlated with each other.
Also, the problem with the prints is that it requires due diligence to clean them all up once you're done.
Another important use case of a debugger is the flexibility to poke further into the other instances based on the behaviour during runtime. Let say if i got some value as X instead of Y, I can quickly examine why that's the case when the program is suspended at the breakpoint. In case of a print statement, I'd have to rewrite them at a different level and rerun the program. And this would be a huge headache if you code requires a lot of time to setup up things.
@@HT79 - Of course, the flip side of the fact that print statements require diligence to remove is that sometimes the print statements become an excellent guide to what should be logged. :-) Especially at the DEBUG or INFO levels.
@@Omnifarious0 Debuggers are great when they work as intended, Pycharm has an excellent debugger in my experience. However you do run into situations where they become nearly impossible to use solely due to a dependency out of your control. For over a year straight there was a bug in Kivy that had no negative consequence for my application whatsoever but would instantly crash the debugger anytime I run my app, so the debugger was essentially unusable for me for that project for an entire year. I didn't force a solution to the issue, because I'm perfectly comfortable debugging without a debugger, but it just highlights how fragile they can be sometimes.
Nice tip, I tried it and enjoy the results. I believe it will become my tool of choice.
Great! Thanks for all your work and great videos! 👍
This is a nice showcase of icecream IC. Looks nice for some smaller projects. I do think, however, that there are better options to replace a good old print() statement, some of which are named in other comments. As for the pretty printing, if you add a bit more depth to some of those dictionarie, it could become bothersome to find what you are looking for. It is nice to be able to write the output to a file. Alternative for this includes the logging module which allows you to log on certain levels. Using logging together with some of the other builtin features might give you a nice solution that depends on standard library only.
It is convenient, however, to have everything with a simple function call.
Very useful! I will be using it from now on changing my 25 year old habit of using print()
Omg I just finished a project with lots of loops to manage some big dictionnaires and 2d array. Debugging with print and debugger was such a pain in the ass.
I wish CZcams recommended me this video 2 weeks ago. It could have saved me from lots of troubles
Should have used pandas. You basically reinvented the wheel, but as a cube...
@Zorcky-2C, when dealing with embedded dictionaries and lists like you would get from a non-trivial YAML file, I find the pprint library to be invaluable.
Thanks so much. As a Python Data Engineer, this is incredibly useful
I think you should try logging
While I can appreciate the tool (I'd written myself helpers that do similar things, so good to know there's a fully developed, feature-rich package for it), I feel that yous examples here made me go "akshually, that's what proper unit tests should do".
very nice library, thanks for sharing. Exactly what i am looking for!! thanks again!
The best part is that it returns the value from a function. Makes it easier to throw in print statements.
I found this video earlier today. Tried icecream out. Love it. Bye bye print(). An idiom is to use ic();ic(foo,bar) so as to print out the line at the same time. It would have been nice to have some way to tell ic() to always print out the line number. It shouldn't be hard to hack it in, so I'll poke around. I'm actually quite curious how it works.
I think with the ic.configureOutput, just put the prefix as ic() so it'll give the line number every time? Didn't try but could work
@@shu7bh Thanks, I'd missed that.
ic.configureOutput(includeContext=True)
People will move mountains so they don't have to use the logging package
Or use an actual debugger. It's so much better than anything else.
That’s me. I have been putting off learning how to use it
Proper Logging and a real debugger are the most valuable tools for debugging
Nice package, didn't know about it. But I expected more...
The introduction with the 3 prints (a+b, a and type(a)) made me expect that it would have a simple way to print the type, maybe some statistics about the number of times some line of the code was reached and other more fanciful common debugging strategies.
Also, I didn't like that it returns back what the function would return. This just makes it easier for some `ic` call to go into production by mistake.
For stylized output I used "rich" library, usually with "rich.inspect", otherwise "rich.print"
I just quickly checked "ic", and find one big downside, it doesn't work with None values at all.
Usually for this sort of debugging I need to check if value is set and not None, I do this kind of check all the time, but because ic behave differently, it does not work. And those kind of checks would not work at all in REPL. A short example:
>>> ic({"a": None}["a"])
ic| Error: Failed to access the underlying source code for analysis. Was ic() invoked in a REPL (e.g. from the command line), a frozen application (e.g. packaged with PyInstaller), or did the underlying source code change during execution?
let's just say that dict is some variable that I'm working with and it have optional value, with None as result it do a different thing.
Very good video …. Thanks for letting us know about ice cream …. I feel like the actual ice cream , this will be one liked by all 😀 …. Thank you so very much for such a great video ❤
Lovely! I will use this in the project i'm working on currently.
Brilliant! Thank you for a really useful tip.
Good to know.
Right now, I do most stuff like that:
```Python
pt = print
def print(msg):
if sys.gettrace() is not None:
pt(msg)
```
But I also use BP a lot, since VS starts python in debugging anyway. And if you set a BP, you can then use F11, to step through the code (or start it with F11).
try logging library. Its... 10x better than any print.
I tend to do a lot more poking around at the >>> prompt, copying and pasting between an editor and the python prompt while trying things out. You don't lose state that way, files and sockets stay open, etc. which makes it easier to test more complicated things. And the objects are all there if you want to see some details. Depends how comfortable you are with IDEs, but I try not to rely on them much because a lot of the time I'm logged in to a system through 3 successive ssh sessions and just won't have many tools available. I can see icecream becoming part of a workflow once it's installed by default. Most systems I can't even run pip on since they lack access to the Internet.
If you use the -i flag when running a script (python -i main.py) the script won't exit when it is finished, and you can still access variables etc
An interesting alternative would be to use the built in `breakpoint()` function which will set you up using the inbuilt debugger (pdb). You can think of it as a supercharged >>> prompt as it also will allow you step through functions!
@@amminamaromind elaborating for a debugging novice?
rich inspect and rich logging handler are a little bit more sophisticated and better, but icecream is also nice.
When I started programming back in 1980/90s we didn't have debuggers. So never really got out of the habit of print statement debugging :D
this video changed everything for me. thank you 🍦
Great tour of Icecream. Thanks!
Thanks for sharing! Looks great.
Excellent video. highly recommended.
5:22 Why the hell does the print execute before the ic? or is the ic running in like a thread seperate from the main thread and just takes longer?
or does it only print normally when you assignt it to a variable and the ic only gets printed when printing the variable later on? not on pc so cant test it rn but I think what I said above makes way more sense
It uses stderr by default, look into the source it is pretty simple library
@@TheRealChicken it uses stderr
I am using the same method in order to reg logs but without icecream, just adding what is needed using strings to a txt file among with the date stamp. Wondering which version is more memory effective.
This is way better than logger, nice content
Thanks for sharing this python package - icecream. nice tutorial
Thanks - great library !
Can you please teach us how to use keyboard shortcuts in the editor like how you were doing?
Great alternative without any imports is f-strings with Python 3.8 and up.
For example:
```
def is_odd(a: int) -> int:
return i % 2
num: int = 123
print(f"{is_odd(num) = ") # prints: "is_odd(123) = 1"
mylist: list[str] = ["a", "b", "c"]
print(f"{mylist=") # prints: 'mylist=["a", "b", "c"]'
```
Always with something amazing thanks
Of course py is mainly scripts, but I have an issue with the way icecream works - you sort of have to build it into your program. Just because you call ic.disable you're still adding a built in overhead to your entire program, and stripping it can become an exponentially hard task if you rely on it too much. Not to say that it's not useful, but there is definitely a trade-off between development time and performance overhead and you need to consider.
I don't think you should be leaving prints in your code anyways. If you need to hardcore some kind of feedback that's what logging is for.
I Agree with Willman. Icecream is for developing debug, not for production logging. There are other libraries for that.
You can have a conditional import that replaces ic with a no-op if you don't want to have it as a dependency. That's probably what I would do; import ic if it's available, and replace it with a simple print or no-op otherwise.
@@nio804 This is not what OP's talking about.
Thank you for sharing this very useful tool.
Why the output for result 30 appears/printed before the output for IC? @5:22
[EDITED] I just saw that this happens on every run, the IC output always comes after the program runs. I'm curious to know why.
Thank you. It is really useful even for beginners.
wow! So nice and really useful! Thanks bro!
The interesting question is, how does this work. If you are familiar with python, you know, that if you see a line like func1(func2(value)), the order of execution is this: evaluate 'value', call 'func2' on 'value', call 'func2' on the result of the func1 call. So func2 will never see that it's argument was generated with func1. So how can the function ic examine it's arguments BEFORE they get evaluated? The answer in case of icecream, it uses a package named 'executing' which let's you inspect the current frame and the related AST. I would like to see a video on that topic.
Personally, I think f-strings are almost equivalent and prevent me from adding another dependency to my project. For example,
print(f'{add(30, 70) = }')
will result in printing: add(30, 70) = 100
This is essentially the same thing that printing with ic will do, just using an '=' instead of a ':'.
I think the nice dictionary printing is a good feature, but I usually just use pretty print (pprint) for this. E.g.
import pprint
pprint.pprint(data)
This does still need to be imported, but only if you need to be printing dictionaries this way AND it's a built-in module so still no extra dependency.
Do you really need it as a project dependency? Have you added pprint to your project's dependencies? Can you use it as a development tool only?
+
Thankyou. this is really useful to me.
Thank you bro this video is so informative
In Julia you have the built-in macro @show to do that. I love Julia :) and I wish it was more widespread.
Really good info. Like the little niceties ice cream adds
great video learnt something new thanks ✌✌
not a bad place to start, was just curious on an idea that hit me earlier for a print-based debugging library just as a little project idea and wondering if one's been built / how well it works. even if nothing more than a small build for the fun of it, it would be kind of cool to have a whole suite of functions uniquely designed for capturing, tagging, storing, parsing, and comparing variables at common intervals. haven't thought nearly enough about how it would be done optimally (if at all) but it would be nice to have any given value stored alongside different "distilled" states that you set, so that an algo can detect "this is how this data value should transform under conditions A, B, C, and D within these states" and flag + auto-track deviations over time. if the library was robust and clever enough it might be able to catch your own fuck-ups well before you do in those cases where breakpoints and typical debugging isn't ideal.
As a side-note, you *can* capture a function's return data while simultaneously calling it within print, but you'd have to use a walrus operator.
print(result := add(10,20))
print(result)
would print 30 twice.
This is amazing! Is there something like this for C++?Regardless, this would be helpful for a couple projects I'm struggling with in python.
In C++ fmt library can print lots of things by default - saves your time to avoid manual looping to print a vector for example. You can extend it too, maybe there are more advanced printing libraries idk. Fmt will be used in regular print in C++23.
I personally prefer to debug but it’s not often possible in multithreaded or other kinds of environments. Ideally you unit test and keep your code clear so you don’t have to debug 😅 but it always comes down to it anyway.
I noticed at around 8:30, you have the ic() statement before the return value, but in the console output, the return value comes before the ic() output. Is there a reason for this?
print prints to stdout, ic prints to stderr, and they can get interleaved due to buffering.
@@vytah Thank you
The functionality described at 4:45-4:55 could be achieved by print by using the walrus to assign the variable in question
¡Excelente video! muy buena idea para no tener que llenar de print todo el código
Very nice library! ❤
seems like a good solution when using terminal editor, it’s kinda annoying to download debugger
Is there a way to configure ic so that it is virtually removed, so that you can leave them in the source code but not only does it not ouput anything, but it even doesn't slow it down?
Very useful. Thanks
Liked. Will need to try ic.
Cool. Seems like it would be smoother for the 1st example to just decorate the add function with ic. Could that work?
Logging is great, but dropping in a "breakpoint()" with pdb++ will still be my go to for actual debugging.
I think I will start using this module!
Which ide is this?,
Also is there a plugin which auto populate argument keys?
how did you do what you did in timestamp 4:11? that's pretty cool.
Ah, this is partially how it would normally work in at least some variant of Lisp I know: (setq result (print "foo")) prints and sets the result variable (called symbol) to "foo" but also still returns "foo".
usefull info 😀😀thank you bro...
How do you automiatcally get all the argument names added when calling a function?
I don't know if someone else ever told you. But I have seen plenty of your videos and they are awesome.
Me along with so many people watch would really appreciate if you could zoom in the recording a little bit. The text is kinda small, it could potentially harm our eyes to watch videos for hours. I am sure you don't want your viewers and subscribers to get eye glasses 🤓 ❤
This is really helpful, because I'm trying to come up to speed with Python. I used to use Basic, but I see Python is the future. So, I subbed, and please keep making this great content.
Thank you 🦋
Replace ic() by proper Unit Tests and use KISS... I'm not a Python developer but what in C# works should also be possible with Python.
Sweet! Thanks
Dumb question - @4:05 or slightly after.. you do a keyboard mash to select just that word (print in this case) and replace.. what keyboard combo are you hitting to do that so fast?
I'm familiar with double clicking with the mouse to select just the word but that's finicky and doesn't seem to always work as expected with selecting.
I assume you're also hitting arrow keys for going to next line and it's going to the same location because the lines are, at that place, the same...
But how are you rapidly selecting that word with the keyboard to the paste (or type) IC?
Ctrl-A would select too much and things like Ctrl+shift+end would grab the whole line..
I'm not familiar with a 'just select this word' combo
Appreciate the feedback!
also this IC thing is slick but do people have a rapid way to pull it back out, and leave code functional, when you're done debugging? Curious how much bloat or overhead leaving this in source code would create vs taking it out (at a bigger scale of course). As well as - the end product may just want print statements in a lot of those places but not necessarily all; so how do you rapidly replace all or a select few or comment out every debugging occurrence? (Similar to my above question as I find it tedious to swap words and comment debugging lines in/out.
Those are Vim bindings. Check out my videos on Vim or NeoVim. It is worth it :)
Cool.
Pro print for debug :)
I like icecream but I have seen a bug when using it to print a lot of data. In VS Code the output is truncated. It works fine if I run it without an ide. Can you replicate this error?
Hi, could you tell me.. Why it is better than logging or logger
What is that IDE? Doesn't look like PyCharm CE.
My bad, it is PyCharm CE with IdeaVIM plugin.
Maybe start looking into adding test coverage if you are actually doing this on a daily basis
Cool, thank you.😁😁😁
Can we use this in azure databricks notebook?
One hour after deciding to install icecream, I finally manage to reinstall python in a way where pip actually works.
For debugging how is it better than pdb ?
Seems like the IC output is stored until your program completes before being displayed. And therefore, you lose context of what messages happen at what stage of your program. This is trivial in smaller programs as demonstrated but as program complexity increases either a print-now or debug session will be more useful.
Excellent
Is it there a django compatibility as well?
this is gold.
Does this package work in AWS Lambda functions?
Thanks for a great tutorial. Can you also explain how to reset outputFunction to sys.stderr? ic.configureOutput(outputFunction = sys.stderr) errs.
Wow groovy, very useful