Crust of Rust: Lifetime Annotations
Vložit
- čas přidán 24. 07. 2024
- In the 2019 Rust Survey, a lot of people were asking for video content covering intermediate Rust content. So in this first video (possibly of many), we're going to investigate a case where you need multiple explicit lifetime annotations. We explore why they are needed, and why we need more than one in this particular case. We also talk about some of the differences between the string types and introduce generics over a self-defined trait in the process.
And don't worry, I know that what we're implementing exists in the standard library :)
0:00:00 Introduction
0:03:36 Start a rust project
0:05:20 Struct and method definitions for StrSplit and first test
0:09:32 How you decide between a library and a binary
0:10:58 Start implementing StrSplit
0:16:15 When to use match vs if let some
0:17:10 Doesn't compile! missing lifetime specifier
0:20:33 Can I be wrong by specifying lifetimes?
0:21:25 Anonymous lifetime '_
0:23:10 Order lifetimes based on how long they are
0:25:18 Anonymous lifetime '_ (with multiple lifetimes)
0:26:52 Compile error: lifetime of reference outlives lifetime of borrowed content
0:34:45 Static lifetime
0:41:27 Bug when a delimiter tails a string
0:48:07 What is the ref keyword and why not &
0:51:36 What's the * on the left of remainder
0:52:46 What is take() doing
0:54:48 Mutable references are one level deep
0:55:39 Solving a hang with as_mut()
0:57:49 Multiple lifetimes, implementing until_char
1:03:19 Difference between a str and a String
1:08:15 Multiple lifetimes (continued)
1:15:24 Generic delimiter (Delimiter trait)
1:23:14 char length utf8
1:25:30 Standard library split
1:27:39 Q&A
You can find the final code at gist.github.com/jonhoo/2a7fdc....
You can watch the live version with comments at • Lifetime Annotations (... - Věda a technologie
This is exactly the level of tutorials that we need. Not the "hello world" all over again.
helo ser now write brintln!("helo ser from india")
gud ser gud now we learned how to use brintln
in our next lecture we will learn what is == and !=
@@fgriane4589could have gotten the point across without the racist undertone.
Re-inventing parts of the standard library is such a good motivating example! I really can't click with contrived "foo/dog/shape" tutorials. Your explanations of the problems that come up are also very clear. These videos kept me from giving up on understanding rust, and I'm very grateful!
3:36 start a rust project
5:20 struct and method definitions for StrSplit and first test
9:32 how you decide between a library and a binary
10:58 start implementing StrSplit
16:15 when to use match vs if let some
17:10 doesn't compile! missing lifetime specifier
20:33 can i be wrong by specifying lifetimes?
21:25 anonymous lifetime '_
23:10 order lifetimes based on how long they are
25:18 anonymous lifetime '_ (with multiple lifetimes)
26:52 compile error: lifetime of reference outlives lifetime of borrowed content
34:45 static lifetime
41:27 bug when a delimiter tails a string
48:07 what is the ref keyword and why not &
51:36 what's the * on the left of remainder
52:46 what is take() doing
54:48 mutable references are one level deep
55:39 solving a hang with as_mut()
57:49 multiple lifetimes, implementing until_char
1:03:19 difference between a str and a String
1:08:15 multiple lifetimes (continued)
1:15:24 generic delimiter (Delimiter trait)
1:23:14 char length utf8
1:25:30 standard library split
1:27:39 Q&A
Wow, this is excellent, thank you! Mind if I copy this into the video description?
@@jonhoo Of course i don't mind! :)
One of the few videos online that not just a regurgitation of the Rust book. Love your work 👌
Fantastic content as usual. Your videos on Rust are one of the most valuable resources out there for those of us with a handle on the basics. i eagerly look forward to new ones. This focus on intermediate Rust content, along with its shorter length, is really really valuable IMHO. Keep up the great work!
Awesome stream! I've learned so much about Rust from your videos and I really hope you decide to make more intermediate content like this. I hope you're staying safe and healthy in these crazy times.
This was excellent. 90-ish minutes are perfect length for this kind of stream. I also liked the level of difficulty of the topic. I'm not saying stop making longer streams, just more like this! :) Thanks a lot!
I am grateful for the valuable information and knowledge you are sharing with the community. Thank you so so much, Mr. Gjengset.
Great video! In addition to learning on lifetimes, I also really enjoyed side notes you provided along the way, like str vs String, and your answers to questions.
Just watched it and it was awesome. This was probably the most practical and approachable way of getting a better understanding of lifetimes. Thank you for your time and effort in doing this. 🙇🏻♂️
Super grateful for these videos that provide so much value to me even after 3 years!
I once again come back to this video, I once again learn something new. As always, really great stuff Jon. Applicable teaching for various levels of understanding is quite a skill. Thanks again for making these
Finally, I've figured out what Rust lifetime annotations are, and how to use it. I've also liked the TDD way you have used here. Congratulations on your awesome job with this tutorial. Thanks a lot!
Great lesson. The build up towards the std implementation is logical and gives an impression of how one would come up with a good implemention by iterating towards more general concepts.
Would be awesome if you could find more examples that tie into the standard library in a similar way. One of the main hurdles after finishing some beginner course like the book is getting through the jungle of all the existing std traits. More so in rust than in other languages' standard libraries imo.
This content is gold! The people asking questions are reading my mind!
Awesome awesome :) Good explanation and with many small tidbits of other useful info while still staying on track till the end!
I almost gave myself a stroke trying to wrap my head around these lifetime things. This video literally saved me from it. Clear, concise, relevant.
@ Jon Gjengset
Thanks for your videos. I thoroughly enjoy them and they teach me a lot. I'm a seasoned programmer of nearly 20 years with a huge variety of languages that I worked in, but I am pretty new to Rust (only started to really get into it last December) and your videos are perfect for me. They fill the knowledge gap that I felt I have after reading through the official Rust book. Please keep up the great work. It is really appreciated. Thank you.
1:33:11 I'm very glad for the digestible length!!! Hoping you continue the Crust of Rust series :)
Excellent. Reading the book is so dry, but you brought the subject of lifetimes alive, and made it appear quite straightforward. Thanks
No matter how much experience you have, starting with Rust can be a struggle.
The Curst of Rust episodes are very nice to level up to the advanced beginner / intermediate level in a practical way, Kudos Jon for it. I hope you keep them coming in the future :D
This was super helpful. Thank you! I thought I had a pretty good handle on lifetimes before but I learned some pretty valuable things from this talk :)
Wow this video is full of great stuff. I appreciate you taking the time to make all these amazing videos!
Thank you very much. Intermediate level and shorter videos is exactly what I needed.
This video was fantastic. I tried watching Doug Milford's video on lifetimes, and it seemed adequate, but he was essentially just reading the chapter on lifetimes from the book and writing code examples. The book did a good job introducing lifetimes to me, but this video really solidified some confusing concepts about them (especially the part where you covered multiple lifetimes and when they might be needed), and served as a *supplement* to the book rather than a simple reiteration of the book.
I am seeing this from the future, omg, this is a gem. Thank you so much, I was really confused about managing lifetimes. Watching videos that explain lifetimes without going further away from the examples the BOOK shows is really not helpful. This has helped me a lot.
I thought I had a decent grasp of lifetimes and references. But I learned several things. Thank you! I appreciate your teaching style, and the level of difficulty of these Crust of Rust videos.
I'm glad you found the level about right! It's tricky to strike the right balance :)
Thank you for this! Really good example that helped me understand lifetimes :)
These tutorials are a real gem! Thanks for explaining all this
Thank you so much for this. An hour into the video, and I understand both how simple it can be to implement a custom iterator *and* cleared up a misconception of what lifetime annotations are doing. It's *not* saying that all struct StrSplit
Thank you! Great video, more than half way through the Rust book and this was very good to keep the learning going!
Amazing stuff, thank you for taking the time to rediscover the standard implementation. It's really helpful for the intermediate understanding of Rust. And you're right, Rust is not really that complicated to read compared to other language but, especially for the lifetime feature, things get completely crazy for a programmer coming from JavaScript ^^'
I’d like to have watched this video one year ago, it would have made my life much easier 😂
This is definitely going into my list of recommended material for learning Rust! Thanks a lot!
I used to just immediately panic whenever a lifetime-related error came up and make it shut up with Strings and whatnot, but I think your video made me really understand lifetimes for the first time ever, and realize that the errors weren't too scary after all. It feels a bit entitled to complain about Rust errors (which are notoriously clear compared to some other languages) but I guess the ones regarding lifetimes could be made somewhat less intimidating-looking. Anyway, thank you for this amazing video, this is the first video of yours that I watched, and I'm really looking forward to catch up with the rest of them :)
Thoroughly enjoyed this! Great explanations.
Jon, I just set up arch and copied a lot of your config files :) What an eye opening rabbit hole! Also, this is the content for me, very accessible with a lot I still need to fully grasp. I feel intuition forming with content like this. THANK YOU!
I'm very glad to hear that! Yeah, config files are a rabbit hole for sure, hehe. Hopefully you'll find upcoming sessions educational as well :)
Two of the things I struggled with in rust are references (like why we need *remainder) and dealing with Options especially ref mut , Thanks for your time and effort
I'm still struggling a bit with these myself, but it helps to think of refs like pointers, if you've ever worked in C / Go / Java. Options I think I am starting to understand, but still run into issues around generics.
love your videos Jon! I learned more here than using other resources
Thanks for the video, Jon! It's fun, pleasant and profitable to watch! : ))
This is a very helpful tutorial. Other than the length, everything about it is perfect.
Thanks man, very cool to see this types of tutorials.
This time I actually got lifetimes. I believe 😅
Honestly, this is the best explanation I've seen. And I've seen quite a few.
Do you still know what lifetimes are?
This is just fantastic content! Thank you for your time!
Best learning material for lifetimes I found. Thanks.
The explanation of why two lifetimes were needed (and the benefits of avoid String allocation) were great!
I hit this exact issue when I wanted to implement string split with regex, that also returned the delimiter groups in between (i.e. so the regex could be like "+|-|(|)" etc. and I could see which one it was). It's a pain that isn't in the standard library like it is for regex in Python.
Thank Jon Gjengset. This is what i looking . Nice explain and the example is very good.
Thanks, Jon! Fantastic video and explainations :)
This was so good I think I'm going to watch it again.
Good stuff, thank you for the stream, it was really helpful!
Awesome content! Thank you for putting this out! 🎉🎉🎉
Excellent! An hour ago, I didn't understand the problem lifetimes were trying to solve. After this example and explanation, I immediately see how I can use lifetimes to refactor my code to avoid a bunch of copies.
I think you meant to say "the problem lifetime *annotations* were trying to solve." Lifetimes exist whether we notice them or not. Also in C++.
@@Evan490BC This isn't actually true. Lifetimes are a self enforced rule about how memory can and can't be used. But memory is just 1s and 0s and a pointer is just a number. You can absolutely write a program which dereferences pointers after the memory has been deallocated or the stack frame has been popped. This is generally a bad idea because it will cause unpredictable behavior or segfaults. That is why rust chooses to impose lifetimes on pointers. But depending on the operating system, and the internals of the compiler, it is completely possible to write a deterministic program which ignores lifetime rules and dereferences a pointer after the memory has been deallocated.
@@Jesse_Carl I agree. I think you are confusing lifetimes, which is a property of an object with lifetime *annotations*, which are user-defined specifications and guarantees.
@@Evan490BCAn "object" is already a user defined specification. No such thing exists in assembly code or in memory. In the same way, no such thing as a lifetime existing, except for insofar as it is enforced by the language or enforced by the programmer. Lifetimes do exist in rust, but don't exist in C unless you choose to respect them. Lifetime annotations also exist in Rust, and don't exist in C.
@@Jesse_Carl The formal definition of "object" is as a formated area in memory; a bit sequence (see Stroustrup's books). It's different than the OO definition. We agree on the others.
Perfectly fit my need during my Rust journey
Amazing!!. Thank you very much for this content and keep up the good work!.
Awesome! Thanks a lot. This was well build up and really well done.
Thank you very much for making these videos!
Thanks for the tutorial, this was incredibly helpful!
Awesome video.. Thank you so much. I am learning a lot from you.
Thanks a lot for this live, it's awesome 🤩
i really like this video format thanks
Awesome stuff. Thank you for these videos.
Amazing helpful stream, keep up the good job👏👏👏👏👏
really nice explanations, keep going!
Excellent video! Thanks Jon.
TY. So much of Rust is explained linear (here is how it works, just do this in order). But iterating on a project is so much more helpful. Cheers.
One nitpick to your explanations - IMO explaining 'static lifetime as ‘living to the end of the program’ is misleading - eg. all owned values not bound by other lifetimes will satisfy the 'static lifetime - eg. if you accept some generic type that is bound by a trait and a 'static lifetime, then you’ll be able to pass owned values there (even though they’re owned and will be deallocated at the end of the block).
I think a much better explanation would be to say that 'static lifetime means that whatever you have that satisfies it will be valid *as long as you hold on to it* (but it might be deallocated later). If that thing is a reference (any &'static _) then it’s true it means that the memory behind it will be valid to the end of the program (because nobody can deallocate the memory pointed to by static references, it is always borrowed), but if you have T: AsRef + 'static, then you might have just owned String - you generally will deallocate the String when you go out of scope (you *might* as well leak it, making it valid to the end of the program but generally you won’t) - but as long as you hold on to the String, its memory is valid, and thus String indeed is valid type satisfying AsRef + 'static - and because of that actually satisfying AsRef + 'a for any 'a. So 'static on a variable means that if you hold on to that variable, you won’t see its memory deallocated, for as long as you wish (if it’s a reference, the memory will live forever, if it’s an owned value - it’ll get deallocated only after you drop it).
This misunderstanding (that 'static always means ‘living to the end of the program’) lead to this soundness issue in Libra: github.com/libra/libra/pull/1949
Yup, that's a good point! I agree that phrasing it as "'static means you are allowed to hold on to it for as long as you wish" is better.
Another great video 👏🏽, I really like the shorter format.
This is very useful! Thanks much.
Thanks for this video! It was amazing!
Also just regarding the comment about Rust being more difficult to read than other languages. I can see what the commenter means in that Rust does introduce some additional symbols that might make it look more "noisy" and this does take some getting used to and can take longer to parse. However, one of the things I like most about Rust is that it encodes so much more information than other languages do. As the programmer you get to encode your intent far more explicitly in Rust (if you want to), and as the reader of this code you get read access to this intent as well. I really liked your answer to the question. In practice reading Rust gets easier over time, and the expressiveness generally becomes more of a benefit than a cost.
Thanks, bro. Great work!😉
I would love to see a video on how you mock external dependencies. Or maybe unit testing practices in general when you have an application that does impure things like read or write to a database
Thank you for such a great video!
Thank you for making this video.
This really made lifetime annotations click for me, thank you :)
So glad to hear that! :D
Really love the intermediate level stuff.
Very nice video!
What might be a little confusing for people, is that Rust does auto dereferencing in some cases. Jon explains things perfectly but keep in mind the Rust changes remainder[ ] into *remainder[ ] automatically, so if we follow step by step the type of the &remainder[ ] expression, we get : remainder (&mut &'a str) --> *remainder ( &'a str ) (auto deref) -> *remainder[ ] --> (str) --> & *remainder[ ] ( &'a str ). This is why Jon says that the type of the right side is &'a str
On the left side, there is no auto defer, so we must use *remainder to get a &'a str
amazing, this is gold!
Thank you! Awesome video!
I had no idea until this video that you could put contraints on lifetimes relative to each other.
Great video, thank you
Up until now, this video has 610 likes and 0 dislikes. That's quite amazing!
Thanks for streaming this, Jon. I missed it live but it was fun to watch this nonetheless.
I do have a question if this is even the right place to ask - when you replaced the "&'a str" delimiter with the Delimiter trait, how is Rust able to infer lifetimes in that case? If I understand correctly, traits monomorphosize into individual implementations for each type they are implemented on. (&str and char in your case).
After that happens, does the situation not reduce to the same issue of lifetimes as before you had the traits? What information does Rust have now that it did not have before, allowing us to elide lifetimes?
Think of it this way: once we have `D`, we never need to name the second lifetime. `D` captures the type of the delimiter _including any lifetimes it may hold_.
I want to echo the request for common idiomatic structure implementations in Rust. :)
Could you elaborate a bit on what you mean by that?
@@jonhoo I think he's getting at the "rusty" way of doing things, so common tasks like error handling, destructuring, and stuff like that. In my experience there's often a few ways to go about most things, and in my early days I ended up doing things "wrong", meaning it worked, was safe, but was ugly and not the way an experience rust programmer would do it.
for a second i thought u we're talking about php
Thank youuuu!!!❤
Great resource!
Great, This Rust tutorial is great
so good!
I learned about how to Pattern
thanks for this!!!!
thanks a lot for the videos :)
I am at 29:00 and scratching my head about why the E0312 error didn't show up for me. Looks like it's because I am on compiler version 1.74.1 but the error code has been marked as "no longer emitted by the compiler" since 1.63.0 from a PR called "Remove migrate borrowck mode". Not sure what's the implication there yet but just wanted to point it out in case anyone else is also confused.
And thank you for creating all these wonderful content! I have only watched like
Great exercise, thanks!
Thank you so much!
It may sound kinda dumb, but i'd like to see something about common/conventional ways to break up a rust project possibly including multi-project workspaces. Also considerations for using CI/CD on a rust project.
Things i (think i) know :-
* examples directory - place to put source files showing how to use your library. I also use this in early stages to do micro-experiments that will eventually incorporate into the main project (e.g. how do i use libgit2 to clone a repository?). Not sure if there's a better way to handle this but this seems to work.
* tests directory - not sure what the intended difference between this and #[test] is but i know it exists...
* Workspaces - in a top level directory is a specially formatted Cargo.toml which references the child projects. Building in that directory builds all the projects.
Thanks for sharing.
It would be great if you can also cover "function pointers" in contrast with Fn, FnMut, and FnOnce traits.
Ironically, while you had a hard time finding an example where this was necessary, I ran into these problems in my very first rust program and had no idea how to fix them!
Huh, that's interesting. You had a case where multiple, independent lifetimes were necessary? Can you share some of the details of that API?
@@jonhoo Well, there may have been other ways to solve it; perhaps I should revisit it to see what I can do now. I was trying to write a FizzBuzz, but I wanted to see if I could get it faster than a Java version of the same program (never did btw). I realized that the console was affecting the speed drastically, so I decided to try to add all of the lines of output to a Vec instead of actually printing them. But I had the hardest time getting the str representations of the numbers to live as long as the Vec.
Ah, for that you'd probably need to use `String`, since you are _generating_ the strings, and then storing them for later. If you tried to use a reference there, then the strings you generated would be de-allocated as you moved on to later numbers, and those references would become invalid. I think in your example above, there should be _no_ references involved :p Of course, realizing that isn't always easy!
Around 23:00, the original statement was pretty much correct, you can write e.g. `Vec` to tell the compiler that you want a `Vec` and have it guess what the item type is, and that's basically what `'_` does as well.
`_` in *patterns* is a completely different beast, the compiler doesn't really guess anything, it asserts that there's an item there then discards the item.
Also at 1:06:20, memcpy is not free but it's pretty cheap and common, semantically Rust passes arguments using memcpy (whether Copy or !Copy). The copy overhead might be a factor for *very large* strings, but it's really the allocation itself which is deadly: some system memcpy will be faster than others, but odds are we're talking tens of GB/s, whereas allocations, we're probably talking millions in the very best case scenario (an allocator with threadlocal arenas and room in their size classes will probably need a few hundred cycles), and as low as a few tens of thousands.
Thanks. This has really helped my "spam 'a until the compiler shuts up" problem. If there is a follow up, it would be nice to also cover an example of traits with lifetime parameters.
Wow. This is just what I need at my stage of Rust. But in saying that I'm not saying I "now understand", as many commenters seemingly do. I grasp a bit more. `ref`! Wow, I'm not sure, in my fumblings, that I've ever had occasion to use that, despite having read The Book twice now.
What's fascinating to me is that even Jon, who "usually does longer vids on more complex Rust" (!) takes blinkin' ages, and lots of mental effort, to find out how to accomplish this (pretty trivial) utility. He himself is, there is only one word for it, **wrestling** with the compiler, and since Rust's ownership, borrowing and lifetime arrangements are stringently NECESSARY (for safe code for the 21st century) he is **wrestling with algorithmic reality**. More of this!