Avoiding long running HTTP API requests.
Vložit
- čas přidán 13. 02. 2024
- How would you design an HTTP API that needs to generate a transcript for a specific video (by id)? Deeper into that question, you can assume that won't be a fast request/response so how can we better manage this? Using asynchronous request/reply with HTTP.
🔗 EventStoreDB
eventsto.re/codeopinion
🔔 Subscribe: / @codeopinion
💥 Join this channel to get access to a private Discord Server and any source code in my videos.
🔥 Join via Patreon
/ codeopinion
✔️ Join via CZcams
/ @codeopinion
📝 Blog: codeopinion.com
👋 Twitter: / codeopinion
✨ LinkedIn: / dcomartin
📧 Weekly Updates: mailchi.mp/63c7a0b3ff38/codeo... - Věda a technologie
Want to learn more about software architecture and design? Join thousands of developers getting weekly updates!
🚀mailchi.mp/63c7a0b3ff38/codeopinion
Is there any sample code to show this pattern?
Another option, if available, is the caller provide a callback so the long running process can tell the caller when to fetch
Yes. "WebHook" style
Would have proposed the same, in finance, we use it a lot when integrating with partners and customers ...
Agreed. Totally wanted to mention this and forgot!
I’d only do this if client is within my control as it adds complexity if i have to handle in case of client’s “webhook” failure.
I like this because it hands off some of the responsibility of dealing with responses to the client. They must specify where they want the response to be sent
If it has to be some REST style API, I'd do both. 1. POST the id to the service to enqueue/start the conversion. 2. Have a GET endpoint like /status?id={id} to query the status and do polling or let the user refresh the status manually in the UI. Otherwise some kind of messaging via a queue could be nice.
The id in the GET could also be a job id the POST would return.
Polling is expensive. Have SSE.
@@morespinach9832Expensive how? Sorry if I'm a bit agitated but I am of the opinion that expensive should be a banned word in dev teams.
Expensive in dev time?. expensive in time to market? Expensive in Cheap resources like ram disk space and bandwidth? Expensive in resources I don't care about like the browser of the client?
Polling is stupid simple and can get you to market quickly and reliably. No mind bending needed for a dev team to adapt from request response to publish subscribe thinking. No additional technology integration.
At the end of the day no one cares if a v1 feature was done optimally. If the feature is worth optimizing the business will request it.
Not to mention SSE can be blocked on some of your customer's machines. Antivirus perhaps. Same with raw web sockets.
So using web sockets or SSE is a bad architecture for a LOT of web apps. You're far better off using something like signalr or socket.io
But it's never a bad idea to opt for good old polling for v1.
One thing that I would change here is to use server sent events for updates and http for initiating processes. In my experience websockets tend to get very chatty and maintaining one way communication from server to client and easier and simplier
I agree.
Absolutely. SSE better than web sockets. But how about web hooks?
Very insightful, thank you.
You're very welcome
Any long running process is screaming for a job queue. So the HTTP handler should validate the request, add it to the queue, and return the job id or maybe a url where the status can be fetched.
I like the original question posed in the video. Different scenario: If the url to the transcript a canonical url that should be cached, and you want to create the transcript lazily (on demand), e.g. If the url is /video/{id}/transcript, you'd probably want to have it long running until it's done and cache the result for subsequent requests.
Ooooh love the take! 100% agreed on long standing requests. One of my issues is trying to solve this problem in serverless where I don't have access to process queues, cron jobs, etc. Thanks for doing this take!
Hey, @JamesQQuick, a bit confused by your take. Why wouldn't you have access to all of those things? If you're on AWS for example with a API Gateway + Lambda serverless setup you can still push to SQS or SNS and you can have scheduled Lambdas acting as CRON jobs.
If anything, you should be using queues *more* in serverless to avoid blocking the resource thats billed by the hour.
Also, make your UI reflect the processing. Don't be afraid to show a queue and status to the user vs trying to pretend its really a blocking req/resp interaction.
Great video. Thought provoking! :)
Hey, thanks!
Option 1: Queue/topic (pub/sub)
Submit the request with a traceable/trackable id to a queue. (caller become the publisher)
transcript generation process subscribes to this queue.
then the caller subscribes to a transcript completed event from the transcript generation process.
use the traceable id to correlate the incoming messages to grab the transcript.
Option 2: Request/WebHook
Make a fire-and-forget request to the transcript generation service (with a traceable id).
The transcript generation service then posts the result to a webhook exposed by the service that made the request. Use the traceable id to correlate messages.
Your are completely right about the solutions, but I will say I have seen plenty of examples of people making overcomplicated asynchronous architectures for sometime they can resolve synchronously well within the 60 seconds timeout. Expecially combined with good explanations in the UI. Generally I always ask my self 3 times if I REALLY need to make a asynchronous solution in this case, since it bring so many problems with it.
If you're not using websockets and there's a reasonably accurate way to estimate the task duration (in this example, video file size) you can provide an ETA for completion to the client. This helps e.g. sizing the progress bar on the client and avoids incurring server load by starting to poll for a result that isn't available yet.
Or SSE. No need for web sockets.
Great video, thank you
Glad you liked it!
If any replica of the consumer is able to pick up the result later then 202 accepted followed by an event on a message broker, which could then go to a webhook if the consumer is external, but if you want it to go back to a specific consumer then you might have to deal with asynchronous request/reply which can add complexity e.g. you want the response to go back to a specific consumer and that consumer may disappear before the response is published, and then you might have to clean up any claim check payload when it eventually dead letters etc.
Wouldn't use polling though.
If it really has to go back to a specific consumer and if they disappear you can just stop then just make it synchronous. Could use gRPC here and maybe even stream the response back as it becomes available.
Maybe even RTMP depending on the use case (e.g. if you are streaming subtitles to a media player then don't reinvent the wheel)
what if in the business process we got an exception throw from the code, should we use email to communicate back to client about this ? any thought on this
53 seconds into the video, and I am going to suggest utilising a WebHook pattern to return the result to the caller once the process is complete. Initial request is a quick POST.
What do you think about the client subscribing to a message on a response queue
If we use websockets, what happens if the client misses the response due to connection issue? Should websocket messages be persisted and client can continue reading messages from last starting point like a chat session?
Sure, depending on the tech being used. You could require the client to ACK the message and have the server retry etc.
Isn't there some kind of synchronous request option to the server? In 99% of cases if we click something we want the result. Updating a div 5 seconds later asynchronously always sounded weird to me. There's nothing asynchronous about displaying a loading gif anyway. The User often can't click something else or should not while waiting. So I always wondered why some synchronous button is not the default. It's the much more common use case and would make development a little bit easier.
Sure you might want the result immediately but that's just not the case. I've shown this in other videos, but if you're in azure/aws/gcp for example, and you shutdown a VM. It doesn't just immediately turn off. Your request is sent (queued) and you're given feedback when it finally shuts down.
What about using javascript SSe instead of websockets for sending resource URI to the js client?
Yup, absolutely could use server-sent events.
@@CodeOpinion yes, SSE need more love
Yes! They're easy to handle also in server-server scenarios even if you don't have libraries for that. I guess in any language that lets you handle HTTP/TCP