56. What can you do with S3 presigned URLs?

Uploading and downloading files are some of the most common operations for web applications. But let’s face it, as common as they are, they are still challenging features to implement in a reliable and scalable way!
This is especially true for serverless environments where you have strict limits in payload size and you cannot have long-running connections.
So what’s the solution? If you are using S3, pre-signed URLs can help quite a bit!
In this episode of AWS Bites podcast, we are going to learn more about them, and… if you stick until the very end of this episode, we are going to disclose an interesting and quite unknown tip about pre-signed URLs!
⏱ Chapters:
00:00 Intro
00:56 Some upload/download use cases
02:06 The challenges with implementing upload/download solutions
04:00 What are S3 pre-signed URLs
05:27 How can we generate pre-signed URLs
08:26 How to do post-processing of uploaded files when using pre-signed URLs
10:22 Is it possible to use S3 Object lambdas for uploads?
11:02 Alternative solutions to pre-signed URLs
12:08 Some of the gotchas with pre-signed URLs
14:05 Recommendations and secret tips!
17:11 Closing
In this episode, we mentioned the following resources:
- Our illustrated article on S3 pre-signed URLs: fourtheorem.com/the-illustrat...
- Documentation for the pre-signed POST: docs.aws.amazon.com/AmazonS3/...
- How to upload files from a frontend app using pre-signed URLs (by Borislav Hadzhiev): bobbyhadz.com/blog/aws-s3-pre...
- Using pre-signed URLs for multi-part uploads: www.altostra.com/blog/multipa...
- Different architectures and tips for managing uploads to S3 (by Zach Charles): / s3-uploads-proxies-vs-...
- Using S3 Object Lambdas to generate and transform S3 files on the fly (By Eoin Shanaghy): / using-s3-object-lambda...
You can listen to AWS Bites wherever you get your podcasts:
- Apple Podcasts: podcasts.apple.com/us/podcast...
- Spotify: open.spotify.com/show/3Lh7Pzq...
- Google: podcasts.google.com/feed/aHR0...
- Breaker: www.breaker.audio/aws-bites
- RSS: ​​anchor.fm/s/6a3312a0/podcast/rss
Do you have any AWS questions you would like us to address?
Leave a comment here or connect with us on Twitter:
- / eoins
- / loige
#aws #s3 #upload

[@yuyokk]

s3 urls are handy especially if backend is built on lambdas and it is not that easy to handle media files via lambda / api getaway. We use s3 urls for letting clients upload documents and for downloading some "customer statements". Those "customer statements" are first stored on s3 and then download url is returned to the browser with redirect header so browser automatically triggers download (or preview for pdf).

[@AWSBites]

That's indeed a perfect use case for S3 pre-signed URLs. Thanks for sharing Iurii!
Is there any particular gotcha that you found out the hard way or something you wish would be easier with pre-signed URLs?

[@yuyokk]

@@AWSBites I think "gotcha" with cors setup on s3 might be slightly tricky for new comers. Also we had this gotcha with permissions. Our API user had access to this particular bucket within ip range of corporate network. It meant that presigned url failed for clients. It took us some time to figure out why it fails for clients.

[@kalingaranasinghe8601]

How can generate pre-sign URLs for multiple objects for example all user profile images in my system. I don't want pre-signed URL to be generated for each image again and again when users navigate through the system

[@AWSBites]

If you have a use case where these images need to be public you could use a CloudFront distribution rather than relying on pre-signed URLs. Pre-signed URLs are more when you want to control who can read or write a specific resource (maybe they need to be explicitly authorised or they had to perform a specific action like completing a payment before being able to have access to the file).
Let us know if this helps :)

[@Bala4433]

Typical claim check pattern..