Crust of Rust: async/await
Vložit
- čas přidán 20. 07. 2024
- Finally, we tackle the topic of async/await in Rust, and specifically looking at how to use and think about async/await more so than how it works under the hood. My hope with this video is to convey the mental model and intuition you should use when using async/await in your own code, without getting bogged down in the details of Future, Pin, and Wakers.
0:00:00 Introduction
0:04:49 Fixing the video title
0:05:28 What is async fn?
0:10:40 awaiting futures
0:17:00 Yielding
0:20:03 Awaiting one of multiple futures
0:34:46 Cancellation
0:37:58 Executing futures
0:50:50 Select arms with side-effects
0:55:35 Cooperative scheduling
0:58:38 Selects with many arms
1:01:51 Fused futures
1:04:35 Overhead of async
1:07:56 Is select a future?
1:09:20 Awaiting multiple futures
1:19:17 Parallelism and spawning
1:30:37 Sharing across spawn
1:36:23 Runtime discovery
1:42:05 Stack variables in async
1:55:45 Async fn in traits
2:08:06 Asynchronous Mutexes
2:21:03 Do futures stay on a thread?
2:24:11 Async stack traces
2:33:04 Wrapping up
The async book: rust-lang.github.io/async-book/
Tokio's mini-redis: github.com/tokio-rs/mini-redis/
Live version with chat: • Crust of Rust: async/a... - Věda a technologie
I often feel I don't have enough time to watch a 30 minute video but spending 2+ hours watching your content is something I will make time for. Fantastic contribution to grokking Rust ... thank you.
I'm not even a Rust engineer, and i've still sat through most of the video. This is the kind of content I love watching.
Wow, this introduction is spectacular. Thanks for the beautiful content.
This was great! Even though you described it before, I didn't _get_ the "await/yield returns back to the thing that awaited" part until you answered the "do futures stay on a thread" question and then it clicked. And the async trait fn part was great because it always seems more intuitive when you describe why things that I expect should work, don't work. It also tied off why the stack variables part was important so nicely.
Thank you so much. I feel I finally getting into rust and start understanding why compiler complains on my code and how to fix it.
I finally understand the formal difference between concurrency and parallelism because of the 01:19:00
Your content is amazing. Thank you for putting it out!
i love you. it helped me SO MUCH! i'm gonna send this video to all my rust friends who struggled with this. awesome channel, i like that livestream idea.
This channel is the only one of its kind. Please don't ever stop! :)
Great content and delivery as always! Enjoying the new book too, interesting and challenging!
You are super good at explaining things! Been learning a lot watching this series. Thanks!
Man, was really hard wrapping my head around this, until I found this video.
Thanks for sharing.
Thank you, Jon, I love videos that convey the essence professionally.
Hey I’ve always enjoyed your streams but somehow never knew you were writing a book. I’m glad you mentioned it so I was able to preorder a copy!
thanks... it's great when we can see through the magic and the world makes sense again! I watch these things for start to finish...
This... is such a great introduction. Thanks for sharing with us.
excelent explanations on the hows and whys of async. Thank you! 👏🏻👏🏻
More of this is needed on CZcams. Thank you for comparing to other languages like JavaScript promises, that helped me personally.
Thanks for the effort you put into these videos and trying to help people understand difficult stuff. Great teaching, great presentation, great video :)
thanks a lot for this stream / vod. its amazing to see into detail, what a feature actually does. i hope we can see some more in the future (pun intended :-))
0:25 "[This] is gonna be one of the shorter streams..." 2.5hrs long? Surprisingly correct.
Thank you, I'm very excited to watch this. :)
Thank you soooo much for this. I Finally understand how this stuff works.
I am grateful to find this video. Thanks for awesome explanation :)
Useful? It was HUGELY valuable, especially given that it's one giant recorded live stream - simply amazing. Cheers!
this is the best lecture about async Rust. Thank you
Manually writing async FSMs in C really gives you appreciation for the niceties of this syntax sugar
Thank you alot for these streams, this one realy helped me in learning async 👍👍👍🙏🙏🙏❤❤
This was an amazing watch! The state machine unveil was an "aha" lightbulb moment for me, since I knew some of the background of how async executors work, but didn't really know how the macros worked and how the desugaring happened.
Also, now that async traits are a thing, and RPIT in traits work, it would be great to have a follow up video discussing the new way and what are the gotchas there.
Thank you so much for putting out amazing educational material like this for free! I only started learning rust like a week ago and I feel like I understand stuff so much more than I would expect to because of deep videos like this!
You may find czcams.com/video/CWiz_RtA1Hw/video.htmlsi=GcR0lZnT3tKds9UO interesting too :)
@@jonhoo wow, thank you so much!
For me the easiest way to understand async was writing it in C.
Of course that not everyone knows C syntax, but it really gives you a complete understanding of how simple async is and how you can leverage blocking syscalls
are there any resources out there about this? I kinda want a guide to implement this, thanks in advance!
Awesome videos, man. I'm not getting enough free time to learn Rust, nor getting a chance to use it at work. But I hope someday it will happen.
This is so awesome. I finally know how to handle the issues that I have met with async in Rust.
Just noticed it's your birthday today.. (what are the odds😅) Happy Birthday Jon! Thanks a lot for your high-quality videos! Super helpful!
I don't know any rust but I still watch these videos, idk why.. you're good
At 1:25:20, one of the `&mut connections` is in the "futures" section and one in the "matched handler block", where all other "futures" have already been dropped, so there wouldn't be any problem with the compiler.
This was incredibly helpful. Thanks!
Perfect timing!
Thanks for the video, really helpful!
This is very useful even though I don't use rust, only c++. This will definitely help for when I make my own multithreaded async yield system using c++20 co routine features. (With c++20 modules features, too. Let's see how many compiler bugs I will hit!)
Thanks Jon, learnt a lot!
That was very useful. Coming from scala I now have a better understanding and know what the differences are.
Great video. Thanks for sharing.
Lemme try to understand this video through the lens of a JS user, comparing Future with JS Promises:
*Similarities between js event loop and tokio event loop:*
- Operates like a state machine
- Single thread handles unbounded amount of "coroutines", or "callback", or whatever
- Single threaded = can be stuck dead on a compute-heavy function
- Single thread = no "real concurrency"
- Unwraps stack, VM/runtime will save stack somewhere on heap, resumes stack when needed
- Very good for I/O heavy high "concurrency", quite bad at compute-heavy
*Differences:*
- JS promise is just a different way to use callbacks an async fn with multiple breakpoints is just converted into a chain of callbacks, Rust actually inserts instructions to yield halfway in the function's binary.
- JS promise eagerly executes the executor, but tokio won't execute instructions unless the function is being awaited on.
- JS Event loop and queuing is in the spec, RS is implemented by some lib like tokio.
- Tokio give you a way to shoot your own foot (imho) with select!, since it allows for partial completion of an async function. In JS, once you enter an async, there is no way to stop it halfway, it will run its own callback chain without your help and will run to completion barring exceptions. Though this comes at the cost of you not being able to easily discard an async fn in the middle. (You can still have the fn discard itself upon conditions)
*Questions:*
- When an interrupt comes but the stack is still occupied by work, JS runtime queues it and waits till stack is clear, then it puts the relevant saved stack back in, IDK how tokio handles this.
- JS async/await compiles directly into usages of Promises, IDK how rust async/await syntax ties into tokio. Is it yielding?
- In Rust, if you await on an async fn, and this async fn has multiple breakpoints, do you await it once and it will run to completion, or do you have to await it as many times as it has breakpoints or else it will stuck at the first breakpoint, like in the select! macro?
Please someone inform me if my understandings so far is correct and if I've missed anything important. Help appreciated!!
As always, amazing learning material from Jon and I cannot subscribe to your channel more firmly!!!
CZcams comments aren't the best for these kinds of long-form questions, but I'll try to give somewhat concise answers. First, tokio is _not_ single-threaded. It can run with a single thread, but by far the most common execution mode is multi-threaded. Also, Rust Futures, unlike JavaScript Promises, have a lot of control over their execution. Specifically, a Rust Future must explicitly say if it can be run in parallel (e.g., with spawn) and concurrently (e.g., with select!/join!). Rust's Futures are also poll-driven, which is a fairly different internal driving mechanism to what NodeJS for example uses under the hood. It's also unclear whether it's right to say that the async stack gets unwound and stored in Rust. In some sense that's true, the effect is the same, but in practice yielding is more about updating the state machine represented by the large (generated) Future type to be in the right state for a future poll.
For the differences, I'm not sure the first one really applies. In both cases, what gets generated is a sort of "first this, then this" state machine. They may end up with different transformations, but the exact mechanism of that transformation I don't think is too important. You _are_ right that Rust Futures differ from Promises in that they do not implicitly execute though - a Future will never resolve if you just "have" one; it needs to be driven forward by an executor. And yes, in Rust the executor is a library that you explicit opt into, not an implicit global thing. I think you're right that select! is a foot-gun, but it's also a very powerful tool. I do know that there's some work trying to reduce the size of that foot-gun, but.. it's complicated. Especially because we really want to retain the flexibility and power of select!
For your questions: 1) I'm not sure what you mean by "interrupt", but in Tokio if a task becomes runnable, it does not interrupt a currently running Future (just like in JS) - instead, that Future is placed on the queue of runnable tasks, and will be run at some future point when an executor thread is available to pick it up. 2) all async blocks produce a Future, and tokio works just in terms of Futures (specifically, you pass it a Future and tokio runs it to its eventual completion). 3) Using .await on a value of Future type (like the return value of calling an async fn) means "don't run more code until the output value of this Future is ready" - it does not matter what that Future does internally; it gets to run all the way until it produces its value, at which point the awaiter gets the output value and continues running, much like with a regular function call.
Hope that helps!
Greate explanation, In Asynchronous mutexes you said that using standard library mutex for short operation is safe but I think it's true for single thread async app, But if we have more than 1 thread always there is a chance that two parts of code refer to same mutex that may lead to deadlock, certainly in short operation this chance is very low but it's still possible.
What an amazing video!
Thank you for your videos.
This was amazing.
Wow it’s the missing guide on async/wait I’ve been looking for. Watch the full video in one go, every second of it worth it.
A good way of picturing it is the number of awaits in your future equal to the number of states in the state machine it's being turned into. 15:00
Yes that's how it works. Each .await encapsulates a state.
async fn foo() {
//State1 begins..
let x = bar().await;
//State1 ends ..
//State2 begins..
let y = x.prop1.setColor("whatever");
let z = y.paint_screen().await;
//State2 ends..
// State3 begins ..
return z.status_code;
//State3 ends..
//End state begins (shouldn't be poll()ed after this).
}
brilliant explanations
really love your book, i am learn a lot.
Great talk! 👏
It's funny, coming from JavaScript, conceptualizing futures and async await feels natural. It's threads that confuse me yet it looks like most people in the comment section are in the opposite camp
Thanks for the brief
2 streams in 1 day? Christmas came early :)
fantastic content
5 stars, well done.
Fantastic talk - the only thing I would say is missing is more detail on how to deal with dynamically sized selects (i.e. the number of futures varies over time). Are Futures(Un)Ordered involved or tokio streams?
Great stuff, man!
Thanks for that
I like this very well. U have a new Abonnement! It is a good idea to use async to build a Gameloop ore should I use threads instead? Or what is the optimal Gameloop for a multi core system?
Man, i wish you run the programs you wrote. Content is great but seeing them in action makes me understand the process better.
I'm curious what you feel like that would add in this video? I usually do run my code, but here most of what I've written is more about the types, and running it wouldn't actually print anything 😅 For much of it, there'd also be a bunch of not particularly interesting extra code to write to get things to compile, which would make the video probably twice as long without actual contributing any further insights or understanding.
great intro, mate
Great channel. Found your reference from the Demikernel paper.
What is this Vim setup anyway? Looks great
great stuff
Would it make sense to have something like off-CPU analysis with bpf for futures on tokio executor? Some async-aware instrumentation might be able to hook into yield points and probe stack traces also saving timestamps.
Yup, I think there's been some talk about that already. Possibly by having a tracing subscriber that introduces labels that you can then refer to from bpf too. It's a tooling ecosystem that's very much still in its infancy though!
I'm interested in your vim configuration -- what extensions are you using & how did you set up your dev environment so nicely!?
So under the hood, rust, go, and other co-routine-like features are still using EPoll on Linux, kqueue on Mac, IOCP on Windows.
Never seen the "select" macro, the JS way would be to add .then() on each promise and then do Promise.all(). That way, you can still react to each individual promise resolving but you also have guarantee that the process will not continue until all promises have resolved.
Not sure how that would work in Rust. Event if you added a .then() function on a future, I guess there might be some complications with the borrow checker. I will have to try it out to find out.
Does anyone knows if there is any programming language that supports execution of async with multiple threads like rust?
In the "Stack variables in async" example, how does chunk2 access x?
Any variable that needs to be shared across states needs to be passed into the new state while transitioning.
Example,
// Inside poll() definition
match *self {
Self::State1(x) => {
// Do stuff
*self = Self::State2(x, some_new_state);
// That's it for State1's job.
},
Self::State2(x, y) => {
// ...
*self = Self::End;
},
Self::End => {
panic!("poll () shouldn't be called after future is completed");
},
_ => unreachable!(),
}
// End of match statement.
Each time poll () is called, you do work and potentially shift into a different state.
I know it's a big topic, but I want to dive deeper in how unix based systems actually handle async events and parallel operations. At the very least I want to understand what the basic level async facilities are (network calls, file operations, what else could be async??). Do you have recommendations for materials on this topic?
2:13:00 I feel there needs to be a way to get a warning when calling a blocking function in an async function. It would solve exactly these kinds of problems. I will have to look into it.
Thanks!
A Masterclass
Any reasonable threaded scenario would use a thread pool for handling clients, so it wouldn't be an issue and would (IMO) still be easier to understand. To me the async model feels weird, and hides what is going on more, which I don't consider really a good thing when there are multiple things going on. And it lets you be sure you have a thread per client (if you want that), whereas that's not the case in the async model, where I assume you are depending on some internal heuristics as to how many threads are being created and when. The thread pool based system has a much more controllable response time to connections.
A generic interface for waiting on things (and for things to make themselves waitable) of course is simple and obvious, and can be used in many ways, without the obfuscation of the async system. And for those things that you really are likely want to want to wait on already probably have OS level (and wrappable by the runtime) mechanisms to have them be done asynchronously and signaled, so you can easily make those waitable without any of the overhead of the async system.
As a cooperative multi-tasking, it's really not useful for latency sensitive operations, except when everything it does is truly async and very light weight, and if you have to carefully ensure such things, what have you really gained in terms of simplicity? And how easy would it be for code changes to introduce something not light weight without it being at all obvious from reading the code?
If you have a thread pool / green threads, you already are in the woods of cooperative multitasking. All the technical drawbacks of async/await apply, and the choice is simply about taste/style. If you want true OS threads it proves to be too much of an overhead (for example Apache) to be competitive against an async/await high performance server (e.g. Nginx)
And yes, it is true that async await and coroutines in general is only good if your application is really low on compute, and dead heavy on IO, but that is really what a web server is, for like 99% of the time
@@cat-.- And there are now many web servers in the world, meaning real ones that need huge I/O throughput, not simple ones in devices and such? They would be about 0.0001% or some such of applications. And could still achieve the same thing even lighter weight than green threads by directly using async I/O.
thanks a lot!
With C++ futures, its interesting to note that there is different behavior:
#include
#include
auto f() -> std::future {
return std::async(std::launch::async, [] {
std::cout
If my async function does an IO in a very deep call stack, does it have a lot of overhead everytime the future is polled? Because the poll need to check the state of the top of the call stack, down to the last call stack that actually perform the IO.
Awesome tutorial!
github.com/jonhoo/configs/blob/master/editor/.config/nvim/init.vim :)
The windows IO thing is IO completion ports IIRC
Good stuff for hardcore C++ coders other than just dummy hello worlds.. Big up!
I didn't get a YT notification for this :(. I would have liked to ask whether it was normal for tokio select! to poll without being explicitly notified by a waker? I have a delay_queue the impls Future and it gets several extra calls to poll by select! before select! actually sleeps and waits for the waker. It feels kinda like debouncing.
I would expect it the poll function to be called twice. Once on init and then finally when the waker notifies the executor and the future completes...
With the select! macro, lets say I have 2 things that I'm waiting on. The first is a TCP socket that gets data from the network in a highly variable manner. Sometimes I get packets from it between 1 second to 1 hour. (This also means that I have to keep the connection alive by sending it a ping to the server every 30 seconds.) The second is a UDP socket that gets data from the network in a much more consistent manner. I get a packet from there between 50 milliseconds to 6 seconds. Both of these packets are getting parsed out and saved to different SQLite databases on the backend. The TCP one is doing a lot of computation on that packet and indeed could make another TCP connection to another server, or send a HTTP request to another service this takes a lot of time. Is there a guarantee within the runtime that the if I get another TCP packet or UDP packet before that computation is done that it will also handle the next TCP packet as well? Or am I going to spill the next packet on the floor as it flows past what my program is able to handle. What about the UDP side that is waiting for TCP to handle its packet. Given that the TCP side could take several seconds to finish its processing of that packet, would I be dropping UDP packets on the floor because my TCP future is running and hogging the system?
Partially answered at 55:35 - You would need to call .await on these expensive functions call in the TCP socket to make sure that it yields back to the select call. What it does't answer is if another TCP packet comes in, would that TCP packet get processed as well, or would the new TCP packet clobber the handling of the previous packet that is still in flight, or would it drop that packet, or would it queue the packet for the next iteration into the select macro call. -- The solution (but not the answer) for that comes at 1:27:00 where they talk about giving that packet (or connection in their example) to the executor for them to handle and make progress on until it's done.
wouldve been epic if u did some basic rust playlist
Just bought your book.
I love the syntax highlighting here, is this a popular vim color scheme?
It's called base16-gruvbox-dark-hard
Is there much support for debugging with async in rust? I'm not able to get break points to trigger in async functions.
What’s the difference between tokio::spawn and spawn_blocking?
If a executor have 2 future spawned and are in epoll ready, one's f.poll() return Ready and done. How will the executor return the ready value to the .await. and will/if it continue after .await on that future?. or will it wait for 2nd future.
I clicked like faster than any other video ever. And I usually never do.
"rejiggered" ...i'm stealing this
I considered rust for writing a highly asynchronous I/O heavy application, however I ran into trouble when I was trying to await on a few lists of futures of different types. Additionally, at the end of a task from a list sometimes I would add some more to one of the other lists, meaning that I had to really select on these lists and depending on which task was finished, possibly modify these lists. I am new to Rust so I don't know if I was missing something but I found no way to do just that (selecting on multiple lists of different types of futures), do you know if / how it could be done?
The way I'd do this is to tokio::spawn each future, and then add all the returned join handle futures to a FuturesUnordered which you can then pull from.
@@jonhoo Thanks, I used std async just because I thought it's the standard having std in the name :)). But I see tokio is really the most used and fully featured one, I'll try again with tokio and see what results I get.
Is there a reason Tokio itself reimplements the std::fs functions directly for example? Rather than having an executor-agnostic async standard library crate?
Because in general the I/O resource types (like TcpStream) need to communicate with the executor in order for the executor to know what events to wait for when it decides to put a thread to sleep. And there is no executor-agnostic way to do that currently. We may eventually get something like that, but likely not any time soon.
So, with future and async/await, non blocking IO can be possible with only one thread? Or, is It possible without support of future and async/await ?
2:12:23 why does it have to be Arc if it's executing in a single thread? Also why does it have to be a mutex if since it's a single thread then no concurrent access will be happening?
The spawn() call requires it since it is allowed to handle it in a new thread, not only the current one. (It's not necessarily single threaded.)
In the functional programming world we put our "effectful" operations in an "effect" type (Like cats IO in Scala)... so is it fair to say that in the Rust world, corresponding idea is to use futures and async/await?
Ah, no, this isn't a way to describe whether an operation does I/O or not. Synchronous code can also do I/O. It is solely a matter of using a different execution model.
@@jonhoo thanks!!
So if you only ever use await and never spawn or join, the program behaves sequentially?
This is probably a very stupid question, but in foo2() why do you want to “sleep” until a future is resolved? I mean why not just block? What’s the benefit of going to “sleep” and being woken up by the OS if you’re in a non UI environment like a CLI? Is it because you’d save on computational cycles that would normally be wasted? Again, sorry if it’s a stupid question. I understand the use case with select!, but not in a “regular” code.
At 57:00, it's mentioned that if there's a future that's blocking on something that takes forever, (and therefore never yields), nothing ever gets the chance to run. But stuff can still run on another thread right? (Assuming the chosen tokio runtime is multithreaded?
Yep, I talk about that a little later in the video. The devil is in the details here though. For example, Tokio implements (at least, implemented) an optimization that makes it faster to schedule a future that recently ran, which in turn makes that future not possible to steal. So if you block an executor thread, you _also_ block that additional future. Which may in turn prevent other futures from running that depend on that future running. Basically, blocking the executor can have grave and unexpected consequences, so don't do it 😅
@@jonhoo noted. So what I'm gathering so far is that it's fairly easy to reap the full benefits of async/await as long as you adhere to a few simple rules
I feel really weird that the futures don't really execute until .await is called on them, while there is a background worker thread that could've taken that future and polled it to completion. Are there any executors that immediately start polling the future (in a background thread) as soon as they are made? (Eg: the future in its construction pushes itself to a worker_queue that the executor pops the futures out of and distributes them to worker threads).
This model of not executing a future immediately only works if you have a lot of futures to poll/execute so that the background thread isn't really wasting time when the future wasn't .await-ed .
No, there is no such executor. And there couldn't be - there's nothing special about the creation of a future that an executor could hook into. In fact, even await doesn't actually make a future run. It just says that *when* this future runs, and gets to this point, it must wait for this other future. It's only when a future is given to an executor that it actually runs.
@@jonhooWell yes, you are right in that Rust doesn't have some notion of "default constructor" that gets run, which would've made this elegant. However at the expense of more boilerplate, couldn't we write an extension method on Futures, say .run(), that spawns the future in a new Task/parallel compute, and returns some sort of handle that you can call .await on? Would it be possible to do this without too much synchronisation problems?
Hi, do you often do rust lives like this ?
How do panics work in async code? Can you catch panics from a spawn or something?
Depends on the executor. I believe Tokio will generally catch panics to prevent the executor threads from terminating, and then propagate that to join handles, but the documentation should say.