Serverless Doesn't Make Sense
Vložit
- čas přidán 14. 10. 2020
- Every time I try serverless it doesn't make any sense to me and just feels kinda of meh.
links from video:
mikhail.io/serverless/coldsta...
mikhail.io/serverless/coldsta...
mikhail.io/serverless/coldsta...
#benawad #serverless
----
Follow me online: voidpet.com/benawad
#benawad - Věda a technologie
“I spent 5 minutes porting over my code and the next 10 weeks configuring API Gateway” ...so painfully accurate
Lmao almost made me spit out my water laughing when I heard that.
I can relate to that, things get worser if you configure API gateway with terraform.
Sounds about right. But once you get the hang of AWS SAM, the magic starts to happen and you realize how much more efficient serverless actually is.
You might be in the wrong job
@@jimbrown6422 you probably never used AWS, maybe you didnt even write a single line of code, lol
well my experience with serverless is positive, just like with girls: looks attractive and I'm done in just 5 minutes
😂😂😂😂😂😂
HAhahaha!
you're disgusting
That doesn’t sound very positive
you have a bruh
tl;dw: google cloud functions bad, azure bad, cold starts 1s extra, cloudflare workers ok but not for resource intensive funcs, serverless is a meme
Best XD
Actually more like, aws meh, gcf bad, azure worst.
Then let's build our own server
Thank you
Thanks
The problem people have with new technology is , they want to use new technology like the old technology, that is just not how it works. You also might need to revisit the definition of serverless .
Agree, there are several uses cases where either you keep your function warm due high loads or you dont care about 1-2 second diferences. Als serverless is not only about scaling, but also about not having to manage and support a server
Yeah the benefits of serverless cover a whole range:
* Speed of iteration
* less a priori Setup
* managed updates because one still needs to update Kubernetes, patch Docker images and Host OSes (also entails security implications)
* maintainability by writing less code, because that script shown in the video would never work anywhere other than in Serverless because you’d need a http server like express, logging, alerting, error handeling, request validation etc
@@marcelofm80 Exactly. Serverless computing has been a godsend to me as someone who uses the cloud almost exclusively for LOB/infrastructure - I get to have isolated processes running 3D processing/rendering/filing/scheduling and all kinds of business housekeeping without having to fuck around with Laravel and create a whole consumer app/API project for a business domain that requires two services as I used to.
Containers make it even better since I can reliably run legit 3D software in the cloud without having to depend on webserver shell calls, it sounds like magic sometimes.
Agree
Serverless doesn’t exist, and contrary to belief, it will have limited resources. There’s no magic here, it’s still maintained, still crashes, still needs updating. It just means your internal IT staff don’t have to do it. This concept has been around since the 90’s.
I'm using Serverless in production. Sure there are cons, but it is never as bad as some people describe. It has it's own purpose. It is not a replacement but an alternative.
a very good alternative if your use case allows for it
@@RandomGuy-co7rq What is your use case?
@@pjcollazo8318 need APIs to post data to a SQL DB frequently. And, access the data infrequently either through a Dashboard UI or through exported CSVs. We had to deliver the product in a week, so it made perfect sense for us. It is very cost effective, and no headaches to maintain the infra.
@@RandomGuy-co7rq If you’re posting requests to a db that often then it would have made more sense to do the opposite and have an in-house database server.
@@slapmyfunkybass Lambdas don't store information. They process a task and return a result. They do not maintain any type of state. Storage of information is a separate topic unrelated to lambda. Lambda can communicate with any storage you desire. Its the people speaking about lambdas in relationship to a database server that everyone will listen to though. Lambda has nothing to to do with databases or storage of state. Lambdas are throw away functions much like an ec2 server *should be. Something that be tossed in the trash and regenerated. Something that easily be distributed, balancing out traffic but thrown in the garbage after and respun up moments later without affecting performance.
For image resize you can cache back to S3, most of your images are re-requested a lot in the same formats like list view thumbnails and so on (unless you have an extraordinary use case). After the image conversion, save the file to S3 with the format __.png or something and make your lambda look there first. If it hits, it's superfast and cheap. Thanks for the video, very funny and informative.
You could even configure CloudFront to check the S3 bucket first, and then ONLY fire the lambda on a cache miss. More performant and also lower cost. It would probably also make sene to add a retention policy that deletes generated images after a period of time.
You might be expecting AWS Lambda too be a replacement for a static server but it's not. It's more for handlying async events that you don't needs results for immediately. Sure, you can bend it to pretend it's a replacement for an EC2 instance on a pay as tou go contract but that's not its purpose.
Async for sure, I can totally agree on that.
Also for something like that is hit rarely.
Lets say u have an api that generates a CPU intensive report but is called rarely or less often compared to ur usual login calls. In that scenario, I would prefer setting up a lambda with giant RAM/CPU instead of setting up an x2.large on aws.
People now days are abusing serverless thinking its cool. Sometimes, old school is always the way.
The AWS website DOES tout it as a good solution for backends though
@@gsb22 Yep. One of the things I tell my development teams who practice resume driven development is they must tell me how moving to a recent technology solves their problems and if it's worth the cost of throwing away a decade of the Ops guys VM experience. There are some things that Lambda has done really well but it's all event based, totally async, or just makes more sense from a pricing model. Everything else is still EC2.
@@WunderWafer It does work for backends. Just not traditional ones. I use Lambda w/ S3 and DynamoDB for a lot of processes that aren't utilized enough to justify hourly EC2 costs or can afford the latency of startup. If we're talking an old school jar file requiring MySQL and NFS then I'm going to use EC2.
"5 minutes configuring my code, and then the next 10 weeks configuring API gateway", 😂 Made my morning with that one!
THIS AGED WELL!
why?
I think you answered your own question. Serverless scales 10x more easily and the only downside (cold starts) can be fully removed with provisioned concurrency.
Pretty much this.
Yes, but provisioned concurrency is much more expensive... that's true too... I think that like I said in another post, it's about the right tool for the right job. Containers can be a very powerful solution for some things, while lambdas excel at others.
@ Another good point serverless can save developer time as well (ultimately saving money). There is not a need to make developers maintain servers etc...
@@kylekim5861 if course that's one of the main advantages, along with scaling...
Then. Uh. Why not just use a Docker container on a continually runner server? Much less cognitive load.
"If you know the number of requests why wouldn't you use a server?"
Cost my friend. Serverless is dirt cheap.
this right here.
Until you scale.
@@trustmebro703 depends on the use case, some never hit that kind of scale, you really have to be going 24/7 at a constant rate to make serverless cost-ineffective. plus there are many situations where having a cheap entry cost is worth the long-term trade-off of migrating your stack if you get big.
@@trustmebro703 That is true for literally everything though? Server computing is also cheap until you scale, cloud storage is cheaper until you scale, going from outsourced to in-house is just part of the natural evolution of a business.
The fact that once you scale high enough it makes more sense to have your own infrastructure is not a flaw, it's the point. It shows that without cloud computing you might never have scaled at all because you wouldn't have been able to afford on-premise when that was the only option.
@@anthonyskrzypczak9437 Most will never hit that kind of scale, and the fact that you can become extremely profitable way before having to worry about the scale economies of owning your cloud infrastructure is nothing short of miraculous.
I love how with every video I watch I have 0 idea what I'm clicking into but Ben's meme language just clicks and I come out having learned something in a non-mind-numbing way
Despite all the double-speak he's a lot more straightforward than most tech personalities that say a bounch of empty words either for clout or to sell you something.
Learning stuff is good, but make sure you actually verify this stuff is true. I'm honestly concerned with how much misinformation I'm seeing from Ben
@@wazbat1 Well I can tell from actually having used azure functions, that I can't share his views, either on azure, nor on serverless in general.
Cold starts are not an issue for the most part. Lambda containers will be reused for several hours so a cold start will just be
My main conflict with serverless is API Gateway and vendor lock in.
@@carlosjosejimenezbermudez9255 just use an ALB you can point it at a lambda instead of API gateway
@@carlosjosejimenezbermudez9255 #AppSync
@@carlosjosejimenezbermudez9255 I've been reading a lot of comments on the cons of serverless and this is by far the only one that actually holds. Tbh it's not very hard to convert the code, but biggest con is really spending all the time to learn their infrastructure. There is a serverless framework out there, but unfortunately you have to pay money to maintain that.
@Carlos, just go use gcp cloud run. Basically serverless containers. Gets rid of the vendor lockin problem
Two points you should consider
1) Provisioned concurrency makes perfect sense even in serverless architectures, because if you have a large complicated application decomposed into 10's if not 100's of lambdas then you can set the provisioning for each in isolation, this saves $$$ over having to scale a monolithic app. As for serverless vs Docker + Kubernetes there is very little difference as serverless is actually using these technologies under the hood, it's just abstracted away from the dev. If you go into the lambda settings you can see you can upload your own docker image to provide your own runtime even.
2) You don't even need to use provisioned concurrency to avoid cold starts in most cases. You can use a heartbeat method to pre warmup the function by hitting the endpoint and having it short-circuit and instantly return in anticipation before you are going to use it. You can even abstract that away using simple middleware to not convolute the function itself.
That is not to say serverless doesn't have it's problems, though each one is something that can be solved once and not have to be thought about again. AWS all the way, as long as you do the leg work to get your infrastructure setup once, repeating the process actually becomes easier than using any 3rd party.
The main advantage of Docker + Kubernetes imo is that you can run everything locally very easily and that you don't have any vendor lock in.
@@meamzcs That's true, AWS takes fiddling around with to get the entire infrastructure to run locally. I'm not sure about the vendor lock in as you can upload your own docker images to Lambda and your source is Version Controlled anyways. The only thing you are really buying into is the orchestration (Kubernetes part), it would be trivial to switch to Docker + Kubernetes for whatever reason.
@@Supersonicboom7 sam local start-api. sam local invoke... run tests in the provided container. Pretty easy.
@@DavidGraySailing Pretty easy if you only want to unit test, however becomes much more difficult if you want to locally run the entire stack. Also I believe the functions can only be invoked rather than deployed which means you can't run your application locally and trust me, your front dev's and testers will prefer to have a functioning stack locally than not in case of outages etc.
And 2 years later Ben was right and Amazon Prime moves away from serverless lol
I clicked just to see if this comment existed
Because of costs, not because of anything in this video which is just lack of experience.
Serverless is for devs that don’t understand trade offs and assume latency is free.
Lol.. Now I know why this video seemed familiar - it isn't new :/
@@beachbum868 serverless has it's purpose. Don't blame bad dev architectural decisions on the tools. Latency isn't even something to consider in lots of async scenarios.
"10 weeks configuring the API gateway" 😂😂I can relate brother
Underrated comment right here :)
Got mine running pretty fast but I was also using the serverless framework.
Just FYI: AWS has a function blueprint which creates a Lambda function integrated with API GW. Done in
@@johanlarsson9805 right on brother😆
@@optymystyc Never had an issue with API Gateway either. I also use serverless framework. Hell, I added a lambda authorizer in no time flat and deployed it.
Besides, he keeps talking about cold starts, but if your app is actually being used, its not really a big deal.
Funny how well this aged. Amazon Prime team surely watched this video.
I usually do the provisional thing, where i call my function every 15 minutes. Cause its only 96 calls per day which is 2880 calls per month of 2 million Free calls per month of gcp functions.
That way my server is completely scalable and warm all the time.
Wait, this seems interesting, do you have a setInterval API call set up in the front-end? But even that means you need to keep the website open and connected to internet all the time...
Serverless is pretty good. It's basically microservices with more of a functional aspect. Great for data processing. Also like it more for distributed processes and pub/sub models.
Cold starts are not a big deal for production scenarios. Convince me otherwise
@Juan2003gtr shut up "Juan"
@@glenkirkup6861 shut up “glen”
@@kevinpacheco8169 shut up "Kevin"
Why does "production" automatically mean you have consistent traffic from the start? You won't. *Plenty* of requests will be hitting a cold function endpoint. Over and over, new users will be waiting on that cold start and bail your service, or have a perception of "slow". Additionally, if you have "a lot" of traffic, every request above the threshold that 1 function can handle will spin up a new function instance. There is no avoiding this problem, even when your "production" service is succeeding, you'll be hitting cold starts left and right. Serverless functions do NOT make sense for general HTTP request/response endpoints. Mostly they make sense for async event handling/background work, like webhooks or Firestore events etc.
if you don't have the skills to manage and maintain servers, sure cold starts won't be a big deal for you, but for your customers/clients? I don't think so.
that half a second cold start will pretty much impact your throughput consistency.
that's why ALB exists specifically to counter this issue, to not having to deal with scaling issues and cold starts.
my man knew....
My experience with Serverless is actually extremely positive and we use it extensively at my company. However it forces you to take a different approach to doing things otherwise you will find similar struggles as described above.
To give a concrete example to the scenario mentioned in the video:
I’d start with asking myself if you really need ALL possible resolutions for an image? What about w=2&h=10000 ? So what we found is that there exist a handful of reasonable defaults (e.g. 5-10). We would then do the resizing after the image was uploaded and put the resized images alongside the original. Then hook up CloudFront with Lambda@Edge to do the dynamic routing to the resized images in S3 or CloudStorage. This way you got a solution that is extremely cheap and incredibly fast (even faster than servers)
How do you know what image size to send to the frontend? Or does the frontend directly request the image with the size specified from the backend?
@@gogoloco4769 You would usually request content metadata in your Frontend which includes available Image sizes with keys like "thumbnail", "preview" and "original" and then you request the image size that you need. For gallery type apps you would usually have a fixed width which serves as your key.
Ben just wondering if that was a bad design choice. I would approach it differently by creating an image factory where it takes on (upload) the image process it via lambda call and generates 3 (or more if needed) different files based on the resolutions and it would add these files to an S3 bucket which you can use as CDN to consume the images from.
" a T H I C C Kubernetes cluster" 😂
The first 19 secs can be applied to both (especially the "dip my toe" part)
I think you kind of missed the point. Severless isn't designed for applications where you're looking to optimize perf. It's for stuff where you don't want to deal with a ton of configuration, security, maintenance, etc. It "just works", and there are plenty of use cases, like some internal service for your company, and app's background telemetry, etc. where a 2 second call often isn't a big deal. This is the appeal
IMO scalability is just a side benefit and if that's your biggest concern, there are better options for you
> "scalability is just a side benefit and if that's your biggest concern, there are better options for you"
You mean just reaching for K8? Or other "better options"?
I think another use-case is building small side-projects/prototypes and don't want/have the time to set up the infra for even a managed k8 cluster (after all, managed != no management/config at all). When all you need is a PoC or something nifty that you've worked on to show potential employers, it doesn't have to be fast and saves a lot of money since 99+% of the time it will just be sitting around, scaled down to 0, waiting for someone to demo it. Whereas with something like K8, it'd be wasting away a lot of server compute time (after all, K8 needs to have at least the control plane powered at all times).
@@mateowang6570 thats another good point. Often times serverless isnt even used in production, but even when its not that doesnt mean it cant be useful. For prototyping it can speed things up noticeably!
@@ThatsPety Yea, for sure. As for better options, did you just mean K8? Curious about any other ways you would approach scalability and high availability.
@@ThatsPety not used in production? You're kidding right? A lot of large companies implement serverless processes somewhere in their system. Check out this is my architecture by AWS in youtube
@@mateowang6570 oh btw yeah K8s was the main thing on my mind, and sometimes these providers often have a sort of "app" service that kind of sits in the middle. It does all the scaling for you like serverless, but you upload an entire build like a more traditional system
Serverless is not just about scaling from zero to infinity, it's more about scaling between a to b, and a is not necessarily 0. If you think about that, serverless is pretty handy in case your b is too high and spikes at decent intervals. Apart from that, serverless also helps you create microservices without having to bother much about server maintenance.
I had a terrible experience with server-less when we decided our solution to slow start is to be periodically requesting our lambdas to keep them "warm". Some of the most insane programming I've done
Mostly with .net
Hmm.. Still think this is the wrong solution. You may as well provision a full on instance and azure supports that while maintaining the serverless nature of it. More importantly, if you are not running anything mission critical where someone will die, then an extra 900ms isnt the problem. Its probably a front end issue... Displaying some form of loading message etc. Serverless offers way more benefits than just getting rid of servers... Once you appreciate those benefits,you can not go back to procuring full instances
Yeah..our entire architechture is on aws and serverless having more than 1k lambdas and other services..thanks for the suggestion but every best practice or solution like provising u mentioned we habe imolemented it all :)
Some companies call this a 'Canary', it's pretty industry-standard to keep applications warm.
Yeah that absolutely defeats the entire purpose.
Everytime a new concept is created in software, I think of the french refrain "Why to make it simple when you can make it complicated?"
I've been seriously considering where to deploy. Came across holo. I'd be interested in some secondary opinions from pros.
I use Oracle Cloud, pretty good so far, I have been using it for a year, I started with the free tier :D
I’ve been working with azure for a couple months now and I can’t complain
Netlify offers a ton of features for free!
@@cameronthompson1275 I compared it to vercel, which seems better (money wise) for scaling apps.
That’s an interesting overview. I have experienced similar issues and ended up with writing a “keep-me-warm” lambda. Comparing performance and cost made this approach look better as opposed to “cold-start-it-is” one
CZcams @ May, 2023 : It's Perfect time to Recommend this video to everyone
Everyone: Lambdas are so cold
Lambda warmup plugin: '-'
The cold start is definitely something that really bothers most people when they start working with serverless, however, there are simple ways to overcome such complications like a keep-alive function, in AWS you can use a Cloudwatch alarm to send a request to your endpoints to that they continue to be warm forever. One of the main reasons to use serverless is the fact that you don't have to worry about server maintenance (OS updates), load balancing, etc...
A developer won’t maintain a server, your IT support staff do.
@@slapmyfunkybass bro devops was invented more than 10 years ago
we were using cloud functions as the main drivers, and it was a nightmare. even the warm starts aren't guaranteed because with traffic, some users will be moved over to new clusters. the main use case for these, as i see it, is to save costs on requests that aren't essential to the performance or presentation of your service. they can work for prefetching, uploads, and just having data handy for predictable user interaction, but even then cold starts would range from 3-20s for us sometimes. it was nuts
Hey Ben. Provisioned functions are not about predicting your traffic but instead predicting the spikes that you may have. In your example you will always have 30 extra functions and you will have warm starts unless you have more than 30 new requests before the new instances are provision.
On another note my favourite tool is Cloud run that is not Function as a service but Container as a service. Each container has a max of 80 connections and it solves the issue of having multiple functions that need to be warm eg. for Cloud Functions an API could have 20 cold starts where with Cloud Run only one because the container is reused!
well i cam here after amazon prime video blog posted about serverless, and yes ben you were right!
I don't think it's a serverless fault for poorly designed architecture and miss-uses. Serverless makes a lot of sense and it's very useful tool.
SLS works really well in an event driven architecture where you process data asynchronously. Besides, it is a good choice for running applications on a budget and if fault tolerance is important.
What is the benefit over a task queue where I can push stuff for work and then grab the result later
@@XDTuber
There is none because the queue system is not a replacement for sls and vice verse. You still need a compute unit to process queued jobs. AWS lambda functions are often used together with sqs, sns, and kinesis (in the aws environment). It's more about ec2 vs lambda vs ecs vs eks etc.
@@karolszymanowski518 Spot on. Would you rather run a ec2 cluster with loading balancing duplicating a bunch of crap. Would you rather manage a kube cluster with a bunch of convoluted configuration. Would you rather write a function in a familiar language and not worry about load balancing, scaling, kube architecture. Just write a node function push that out and have it available at scale. I rather use serverless framework to write functions in any language not worry about managing complicated configuration or legacy infrastructure requiring manual load balancing patterns / techniques.
Hey, love your videos, I just finished your basic React tutorial, i was thinking of creating some small projects on my own, can you give any suggestions on how to proceed or what to do to improve my understanding of React
It depends on the cost of having a server up and running 24/7. The Lambda container stays alive for about 15 minutes so it's warm. Depending on the traffic you got, in theory if you get 1 request each minute your Lambda should never go cold, also if it does get cold it is only impacting the first customer with the initial request and the rest will benefit as the Lambda is now warm.
I spent 1 month trying to figure out what the heck should I do to deploy my small project. When I saw what can be done with AWS Lamda + API gateway, I realized how tough can be a month of your life, if you are not aware of simple things like serverless.
Those poor girls will never get to hear about JavaScript…
I think use cases really matters. I used functions as a way to make background processes that don't need to be done in real time. I would queue messages in a pub sub and functions picks them up and processes them for something like email or SMS.
A proper engineer that tests and measures his services and makes decisions based on the results. I wish I had people like you working with me. Most of them don't do any of that.
If you are getting 30 requests all the time, then the first 30 cloud functions are gonna experience the cold start. Subsequent requests will be warm. So from my experience it is okay to use serverless in those situations. And the image resizer module would be a perfect candidate imo. However, the part where serverless sucks in my opinion is where you have to process long running tasks - say > 10 minutes. In such cases you have to figure out how to break up your logic into smaller pieces, communicate between them via a message queue etc.
This video aged really really well...
sometimes we just have to embrace the fact that cold booting is okay in certain situations. Like when I'm only building a really simple ecommerce site with the JAMstack and I'm only using like 4 different routes for checkout/order status stuff, in that case serverless seems more attractive that setting up a complete server.
Setting up a server is like 5min... install nvm (one command) and off you go.
Serverless makes easy things easier. But makes complex things impossible.
@@batmansmk the hard part that serverless abstracts is not creating a server but keeping it up, which is way harder than writing three lines of code to listen on a port
@@batmansmk love your line at the end, it pretty much sums up my use case. If I just want a couple of services up that execute CRUD transactions, I don’t see the need of mantaining and setting up a VPS or cluster when i can just upload my services alongside my front end on Netlify. Once again it is just my use case and what I personally think it’s the easiest way around.
Projects that require a bunch of services across multiple teams, faster response times as possible and overall a more robust architecture would get better benefits of a non-serverless implementation.
By the time we need more CPUs than what a single ec2 can provide, we are probably way past the simple CRUD on the product too. Serverless is a solution to a rare production problem.
With S3 and RDS, maintaining a server takes less time than just watching one video on YT.
TO SORT OUT THE COLD START ISSUE YOU NEED TO DO THE BELOW THINGS.
1. monitor how often you api get Hits.
2. you can use serverless warmup plugin
3. increase you lamda memory size.
Ben: "There are two things I don't understand, girls and serverless..."
Angular: "Am I joke to you?"
Serverless is amazing. Ever since I switched, my life has been 10x better as a dev
switch from or to?
@@sonmai3526 well be said, it was amazing, so I'm guessing he switched too
@@sonmai3526 LOL i meant switch to
@Juan2003gtr why? What's wrong with serverless?
switch from what?
Azure offer keeping their functions warm on one of pricing teirs.
Also, once a function is called it is warm for 15 mins, so a lot of people will just call their functions once every 15mins to keep them warm.
The biggest win for severless is the no worries about infastructure, just write your code put it on severless.
I talked about the cold start issue about a year ago with someone who worked on the sandboxing of cloudrun (like functions but for docker containers) and he said file and dependency indexing was the killer in cold starts he mentioned using precompiled binaries was the way to go.
Coming from the future, you were right
Hey there. I'm a member of the Serverless Framework team and found your video interesting. I thought I'd offer a little more info. First of all AWS's developer experience is not the greatest which is why the Serverless Framework is so popular. It makes Serverless a lot easier. But Serverless is not as affected by cold starts as you may think and most people do not go with Serverless for auto scaling. Also the use of permanently available kubernetes gets very expensive very quick. If you'd like to chat more let me know. I've built many serverless solutions that perform incredibly and worked with many other teams who experience the same.
Hello, I'm facing big trouble with cold start of serverless functions like google cloud functions or aws lambda, and gcloud function is very much slower then aws lambda. Is there any solution?
And one more BTW with consistent traffic cold-starts are almost non-issue on AWS. Most production deployments do have constant baseline traffic flow, so I wouldn't be discouraged by small-scale synthetic tests.
To get rid of cold starts I use Google Cloud Scheduler to hit my serverless function every minute. If each call is billed 100ms then it fits very well within the free quota.
While I've never been a fan of serverless architecture , I feel like your argument against serverless here is basically nothing more than "There's 900 ms of initialization SOMETIMES!" ... Which if your use case for the function is just resizing an image before sending ... what's the practical limitation here?
I could see the argument for this IF it was Sequential (If you have 20,000 requests simultaneously and each request has a 900 ms delay before the next request can execute, That's a practical problem), But the point of an Lambda function is that it's parallel , So this fundamentally won't be an issue in the first place.
If you have 20,000 requests, And they're handled in parallel all within 1s, I don't think you have an 'actual' problem there. You have a "This is taking 0.8 Seconds when it SHOULD be taking 0.5 Seconds" problem because ... Computer scientists are strangely obsessed with shaving all the ms they can from anything they do ever, regardless of whether the impact can be seen with the naked eye.
Granted, There are real-time applications where 'off by 500ms' can make a practical, real world, catastrophic difference, For example Vehicle/Aircraft operation systems (Self driving cars comes to mind), Medical Equipment (Think remotely operated surgical robot or anything exposing radiation or medication), Scientific Research Equipment / Programs etc.
Injecting a patient with drugs for an extra 500ms could kill someone . Stepping on the brakes 500ms too late could kill someone . Taking 900 ms to resize an image for the webpage to save 3Mb of network traffic isn't going to kill anyone.
Absolutely agree. He didn't explain why the 900ms for the first request is really a problem. There is nothing mission critical that will fail because of those 900ms🤦🏾♂️🤦🏾♂️
There are several fundamental flaws with this statement. Unsure of your experience in development, so apologies if it sounds patronizing.
Shaving ms is not a strange obsession. It's a legitimate requirement. This depends on the stage of the project of course. For example, if you're building a MVP or prototyping, then it doesn't matter. But in production, you have to consider UX, costs and other restraints or requirements. Obviously, there is a point when the cost-benefit is not worth it and you're just trying to squeeze blood from a stone.
Determining acceptable performance by measuring risk to life (i.e. if it isn't going to kill anyone then it's okay) is a, let's say, strange metric to be using. The key metric is usability, e.g. what is comfortable to the user and this depends on use case. For example, a user may think 900ms may be okay for resizing an image, but totally unacceptable for populating a dropdown box. Although it would be amusing to watch a product manager to react to someone saying "it's not like the extra delay is going to kill anyone."
@@Sasataf hmm... I still don't see anything you have explained that makes 900ms for the first request a problem. This isnt a constant 900ms for every request... Just the first one.if those additional 900ms are a show stopper then you definitely have bigger issues in your system. Most times this due to some bad UI/UX implementation where slow network requests are not accounted for and the UI hangs without any loading signal given to the end user. Besides all this, there are serverless options that cost a bit more but get rid of this warm up issue. FYI We have all built production systems.
@@nsubugakasozi7101 My response wasn't to you, it was to the OP. My bad, I could have made that clearer.
It definitely isn’t a strange obsession... as much as a 200ms change sounds small, it is very noticeable when you’re actually interacting with it. I work for a financial company and every year all the financial companies compete for the JD Power awards. It is a consistent trend that when response times go up (even if it’s only a few hundred ms) your JD Power rankings will suffer and when your response times go down your JD Power rankings will improve. This is true even for categories that aren’t directly tied to performance. Even if you’re not consciously recognizing a latency difference you will subconsciously. Making your website feel responsive and “snappy” will improve the user experience far more than you’d expect and that is impossible to do without fast APIs. In this case it may not be an issue. You should resize once and save the different sizes for distribution at a later date so it’s a one-time time sink. However, for APIs that get called on a more regular basis adding a few hundred ms to the response time would be completely unacceptable
I actually really enjoy serverless. Yeah it might not be the best in terms of performance, but it's pretty nice being able to write lambda functions and not have to worry about all of the other crap. I don't really want to spend my time reinventing the wheel by setting up and configuring servers. I've done that enough in the past decade. This just feels like the natural progression of back-end development -- let the IT guys handle the server maintenance and we write the application specific code.
I am an app developer and had no idea about backend infrastructure. Anyways with google search and chatgpt I built an app which does some processing on images through a firebase cloud function. And I am facing same issue. Some requests take insane amount of times like 5 seconds. Currently I have a traffic of around 150 users per day but if it scales I will need to get more into this. Thanks to Ben and youtube for recommending this
What's your opinion on not using containers at all? Or can you do a video on why you use containers and what benefits those bring you?
I just found your channel and I'm really interested in your opinion.
Serverless is a new paradigm and you seem to be examining it from the perspective of a different paradigm. You sound like a monolith developer arguing about how bad microservices are in 2012.
Microservices suck tho. Just like OOP, it was a funny experiment, but it's time to put an end to it. I'll never get tired of saying how bad those two things are.
@@juliansoto2651 Why would you say this? Could you expand and elaborate? I am legit curious. Thanks
@@mjphillips6619 having worked with microservises in enterprise, it's a really nice fit for big applications that handle a lot of different business logic. It's very efficient development-wise if that matters, you can hammer out code like on a factory line. Debugging though is a pain.
Even Prime Video is going back to monolithic architecture. So microservices are not the answer to everything.
In order to understand girls and serverless, you gotta learn to be a bit patient. 😂😂
Not really. Servereless framework. Write whatever function you want and push it out as an aws lambda.
I'm looking for the best option to deploy mobile app back-end with deep learning model (run on CPU). What do you recommend ?
Well you do have a good point, one of the things many people advocate is that servereless functions should be small and fast so use cases such as the one you have pointed out are better handled by a convensional provisioned server. However use cases that are less computationally demanding (e.g. updating a specific user entity) and are performed many times by users can be consided candidates for serverless lambda functions. This has one advantage conventional servers will not be bombadered by large amounts of requests for small computational workloads and thus can be utilised for more computationally expensive use cases. Whilst serverless functions can do most of the simple grunt work. This is also very evident from the timeouts lambda functions have. Cloud providers do not want you to use them for demanding use cases, you have services like ECS/Fargate for that.
Use the Serverless Framework, it'll spin up your Lambda functions APIGW and any other resources you want to use in less than 10 minutes.
So is there any chance we'll get another Flutter/React Native update. Love your unbiased opinion on these two!
his twitter bio is 'married to reactjs'. i dont think his opinions are unbiased. I come here for the angular jokes lol
@@aj.arunkumar I know. But he's already offered two videos on these techs and both are good NOT like appreciating the one because of fan boy and ignoring the other, so that's why I asked!
I doubt it will be unbiased lol. Obv he's married to ReactJS. You should look elsewhere.
What about Ionic?
@@martinn.6082 Ionic - Rendering Webview is not the talk here and also Ionic is not even close to Flutter/RN as far as I know.
can u do a video on how you were able to setup api gateway in only 10 weeks?
it's nice how you go through all this step by step of what serverless is and what to use or where to look. I kept editing my comment on where you could get more details but it looks like you did your research haha.
@s bright no I meant that my original comment was about different serverless languages, then x-ray, ect. But it turned out that all of them were discussed during the video. He didn't start filming at the first brick wall he hit, instead he made sure to research every problem first. I like that.
You are joking, but after my latest blasphemies and insults about serverless in the last 2 months, I just decided (2 hours ago) my self to port 130 functions (they take 1 minutes to start) to a classic rest apis, You know what they suggest you in order to fix the slow start? To enable Premium features, which means to have 24hr running server, well, then, where's the gain with serverless?
And yes, company forced me to use serverless, and yes, 130 functions are too much for serverless.
@@luxxa Idk man, i think serverless is good enough for notifications and simple mobile apps. Maybe is not the right tool for your project. There is some SERIOUS limitations with things like firebase.
Language, used framework?
@@luxxa So, now you need to maintain 130 functions (distributed system problem)?
@@fransafu no, I am migrating to simple rest controllers
Lambda and GraphQL is a great combination. I can’t speak for the other serverless providers, but I have had a great experience with Lambda and GraphQL.
You have to think differently when using Lambda as an api. Minimize bundle size, declare as much as you can outside of the handler, etc. There’s a ton of factors that go into the actual speed of your response. If your Lambda is 20mb, that 20mb needs be downloaded from S3, mounted in the Docker container, and ran. It adds up.
Using webpack is a huge boost to your bundle size. implementing tree shaking at work has helped me reduce Lambda sizes from 20-30mb to 200-400kb. Same functionality, but with the serverless first mindset.
GraphQL really shines because you keep invoking the same Lambda over and over. Ours are in a vpc which causes longer cold starts, but keeps the container alive a lot longer, sometimes 2x the life of one outside the vpc.
It all comes down to optimizing the code for the serverless platform. If you write static server style code, and throw it in a Lambda, you’ll have a bad time.
Are you using Apollo GraphQL?
People dislike serverless because they don't know it well enough, but it's understandable, it has relatively poor ressources online and we need to learn lot of things to make something serious.
But once you master it, all becomes easier
@@arvi8843 No, the Apollo Lambda package is over 1mb, and requires GraphQL as a peer dependency. I just use the plain ‘grpahql’ package.
If the Lambda doesn’t need an sql db package like ‘pg’, the bundle size is usually 100-200kb.
@@christophercaldwell6641 Thanks!
I mainly use lambdas for scheduled tasks. If I only need a bit of code running a couple of times a day, I don’t want to pay for an EC2 instance to be on 24/7
This was a long overdue video! thanks a lot! could not agree more!
You only spoke about one benefit, the auto-scaling. What about the cost vs time thing though? A lot of use cases don't need an instant response.
The wonderful thing about lambdas is that everyone can be in a different language. If you want to run a program in java you can. If you want to run another in .net you can. If you want to run another in python you can. Another in node js fine. Even though they are each in a different language they are able to communicate with one other. Not only communication with one another but integration with aws itself. Most aws services expose events that can be interacted with using lambda. For example, you can run a lambda in response to an upload to an s3 bucket. You can run a lambda as a side effect to many different aws service actions.
WELL WELL WELL SIR
I doubt there's really an issue with an image resizer function taking some additional time to initialise... In Azure you can keep it warm by selecting the app service plan consumption model instead and configuring always on in the function app. With the right app service plan you can scale-out the function automatically.
7:10 can someone explain this piece to me? I was under the impression that a serverless function won't spin up a new version of your function for each request, but will rather run the same function for multiple users until that function hits max load, then start up a new one. So if you get 30 requests per second, it should only call one function, right (unless your api is really bad and can't handle 30 requests/second)?
so in that example, wouldn't you just provision a certain number of functions for your baseline traffic, then let the functions scale up from there if you get more traffic?
So I get from this that server-less makes total sense to you ;-)
Out of curiosity, wouldn't it make more sense to create an endpoint that is designed to optimize your image then store it in something like S3 rather than re-optimizing an image on every single request? That seems like a lot of extra compute work that could be mitigated by using static assets more optimally.
If you were pretty set on optimizing the flow as you detailed, you could also create an endpoint that just returns 200 and a blank string in the same lambda. With that you could setup a scheduler to hit that once per minute to keep your function always hot to your cold start timing.
Exactly
Good point, but he would have to decide for each image the width and height statically and this is actually what he wants to compress dynamically. Maybe for some images the same dimensions are with a high probability requested and for those your optimisation is probably better.
I think he meant dynamically resizing images depending on the dimensions provided by the request.
@@romeostoll4056 You could still design the endpoint to accept parameters that would map to the expected or desired size. It's common to create a standard size to name convention and output not 1 but multiple versions of an image. For example to hyper optimize i might create a 2x, 1.5x, 1x, 0.5x, and 0.25x image pattern that outputs three versions of each image that have progressively better quality settings. In this way I can target 15 versions of the same static image programmatically. I get near instant loads on smaller lower quality images and I can choose to progressively load in images of increasing size and quality as needed to give a great user experience.
If we extrapolate on the honey/molasses analogy Ben used, I COULD squeeze the bottle until I am shaking and maybe have some small success. However if I take the time to understand the problem fully a different approach like heating the bottle correctly will make the fluid flow more freely and thus give me largely optimal results. Also, interestingly to that analogy i can store the honey in a cold state when not in use and only heat it up as needed to access the contents.
@@CitadelOfLukes Thank you for elaborating your answer in more detail.
You'll fail miserably trying to use serverless to build small webapp/API with constant load.
It thrives when stream processing, batch processing, pub/sub and asynchronous treatments in general, when you need to be able to scale by a few orders of magnitude in the blink of an eye and caching is not possible. Yes those type of apps exist, but it's nowhere near the web frontend world.
It really depends on what you're trying to do. I use AWS Lambda extensively for our data collection and processing pipeline. It works really well for that since it needs to collect data every minute and aggregate data every 5 minutes. We could spin up instances to run this code, but the server would regularly be idle or underutilized.
Dude... you are shaking trying to get honey out of the bottle... lol >.
The shake is just a side-effect of developer hand sqeezing things
It was crystallized, so he was squeezing the bottle hard. 😂
Dude... is this the only thing you took away from this video?
@@schlaus_kwab The discussion started from the most obvious point, honey
@@Greenthum6 I'm just joking, man. Don't take it too seriously :)
You should’ve used Google Cloud App Engine instead. No cold start, scales.
Even GAE have cold start unless you specify minimum number of instances running.
@@Abu_Shawarib Default instances is 1
@@Abu_Shawarib You can send warmup requests ahead of time, luckily.
Would love your opinion on Google's Cloud Run, which can handle multiple requests per instance, and you can eliminate cold start by processing the SIGTERM sent by cloudrun just before it terminates to quickly let the cloudrun call itself, thus provisioning another cloudrun to be in a ready state, while only paying when it is processing requests (assuming you allow cpu throttling when no requests are being processed)
you can keep warm a minimum instances of Cloud Functions with CGP. You can also increase memory and CPU usage while creating them. Just keep in mind that If you keep your functions warm, you'll get billed for it.
I would like to mention Cloud Run in GCP. It's specifically designed to deal with heavy tasks.
Well well well
I recommend use Webassembly with Rust, that's the right way into serverless, you'll notice the difference
Rust is next level for serverless!
Did you try uploading to s3 with multipart as I have not found it any different with upload speed between GCP / AWS S3.
Also Kubernetes is not the cheapest way to go.
You nailed it. I had a project that would sit and do nothing, then report hundreds of orders a second for about five minutes. That's perfect for serverless, right? The first lambda would get wrecked before the second one could even come up. We did some kind of always warm provisioning, but it would destroy any number of lambdas except the actual number we needed. So the on-the-fly provisioning was worthless. We came up with a way to warm up lambdas right before the data was going to hit, but sometimes it would get cold start anyway, like it either deprovisioned it in one minute or it started provisioning extras right away. We ended up using docker swarm or whatever it was called (this was before Kubernetes was popular). That did not have a cold start problem, was very easy and roughly the same price.
I really hope no one follows your computing advice, and this is all trolling
This video reminded me of another one where some guy tried to make the case that object oriented programming was bad, and his evidence were all examples of very simple procedural functions for which OOP was obviously overkill - which would be like saying cars are useless because bikes are much cheaper and perfectly enough for reaching a destination four blocks away.
The image resizing thing is especially bizarre since the reason why image resizing is such a typical serverless use case is that it can be asynchronous, making the whole cold start rant irrelevant. So instead of proving a legitimate downside of serverless that example just made it seem like he doesn't even get the point of it.
It's also kind of weird that these types are talking as though VPS-based computing is going anywhere? I don't doubt some devs may buy so much into the hype that they end up recreating a much less organized and responsive version of their full RESTful API on Cloud Functions and that is certainly silly, but serverless endpoints are fantastic for isolated pieces of functionality. I don't see why be bothered about an extra option when the previous one is still available.
@@DodaGarcia Was that the Brian WIll video "Object-Oriented Programming is Bad" by any chance? I am a pretty ignorant developer and that video started tilting me into functional programming, that sounds like a bit of context I could have missed from his argumentation. I would love to learn more about OOP but I haven't come across any solid intermediate deep-dives into it.
@@franksonjohnson Interestingly enough it wasn’t that one but another one by that same guy, called “object oriented programming is embarrassing” - I’ll watch the one you mentioned to see if he makes different points.
When it comes to tilting though, there’s really no reason to choose between them because OOP expands on FP rather than replacing it.
Functional programming has seen a big resurgence recently with serverless computing and the popularization of machine learning. It’s practical and intuitive in how it follows a linear set of instructions, and enough for a lot of uses.
OOP is a way to design your code that mimics how in the real world things belongs to categories and share properties and behaviors within them. It’s usually the standard for any project that’s expected to grow in complexity over time since it allows for more organized and reusable code by way of those hierarchical inheritances.
So it makes sense to get used to functional programming first and then familiarize yourself with OOP as needed, if nothing else because it’s how most external libraries are structured.
If this was about choosing a first language to learn it would make more sense to talk pros and cons, but when it comes to programming paradigms, most developers aren’t in “team OOP” or “team FP” any more than handymen divide themselves between hammer fans and screwdriver fans. They just use the more appropriate tool for the job at hand.
This is not specifically about OOP but my favorite development channel is @fireship, it offers a good mix of practical tutorials and language-agnostic design pattern lessons, some of which interview-oriented.
(I’m sorry if any of this was already obvious to you, I don’t know how experienced you are and erred on the side of thoroughness.)
@@DodaGarcia No very helpful, thank you! I've always felt tension with the functional gospel on no side-effects and whatnot as a web developer, because pretty much any DOM manipulation is a side effect right? I like the handyman metaphor a lot.
agree
3 things that will help with those cold starts:
1) bump up the memory. It might even be cheaper if your code doesn't run for as long.
2) compile with webpack
3) choose provisioned concurrency to keep your lambdas warm.
Wish you'd dived a bit deeper, but I know it's a lot.
Is there no way to schedule a timer so the the function is called again with null output if it's not used in 14 minutes?
Hey I am a fairly new developer and I have a question. When a function becomes warm, does it become warm for only that one user or can some other user come and use the function while it is warm if the first user is done with it?
Using serverless at scale with +800req/s in production.
You should investigate a lot.
You're also removing another benefit, which is that you don't have to manage servers. Regarding k8s, that's a lot of management overhead also.
There's managed k8s platforms, even so, you're still right. Managed != no managing.
@human bean I agree
I think the scalability benefits for small team/solo projects outweigh the 1s cold start if someone doesn't call your function for 15 minutes. If a userbase picks up the function should be running hot on a regular basis. Doesn't seem to bad to me. Am I missing something?
We exclusively use serverless on my team, but because cold starts, time limits, and other issues, we don't use a lot of lambda, preferring ECS Fargate instead for most of our services. It's great not having to worry about servers or cold starts, even if autoscaling takes a bit longer (have to wait for a container to spin up). We mostly use lambda for tasks that don't need to get done right away and can be batched to run infrequently.
Just run statistics on what sizes are most requested, resize them using serverless and cache them to s3
Try IBM Cloud Functions or IBM Code Engine
Wait... no server cache for the image requests? Was each image resized on the fly every time it was requested?
Some applications within our companies call function api's which we didn't have to maintain for years now, also billing $0 as they only get called every now and then. Having these as an application on our clusters would be more operation intensive.