You want efficient application scaling? Go serverless!
In the first decade of the 2000s, we saw a shift from using physical hardware to virtualization. This shift was driven by increases in clock frequencies and number of cores in modern CPUs. It was no longer cost effective to run a single application on a single physical server. Both open source and commercial hypervisor solutions started gaining tremendous popularity in the server market.
Today we’re seeing another shift from virtual machines to containers. Containers are virtual runtime environments running on top of the operating system kernel that emulates the operating system itself. The difference between virtual machines and containers is the virtual machines emulate hardware and containers—running on top of a VM—emulate just the operating system.
Operating system emulation has numerous benefits. Processes running in containers are isolated from each other. Containers can be started and terminated quickly as their footprint or size is typically minimal, making them a great choice for fast scaling.
Virtual machines and containers allow software to better utilize purchased hardware. Multiple applications can run concurrently on the same hardware. For finance departments, high utilization means effective use of capital expenditures.
That’s where the serverless model comes in. Serverless computing describes an application architecture designed as a collection of managed cloud services. Managed services like AWS Lambda, Aurora, or CloudFront provide high added value and allow you to build your applications quickly. You spend your time focusing on developing your business logic and not fiddling with infrastructure, like making sure your database operates as a cluster. The serverless application model helps you spin up your application stack and easily organize and manage the state of cloud resources associated with your application.
Serverless computing was made possible by significant technological advancements in container space. Cloud providers striving to reduce overhead and allow fast provisioning of resources in their platforms quickly found out it was necessary to develop new types of hardware accelerated hypervisors.
The combination of hardware-accelerated hypervisors like AWS Nitro and virtual machine monitors like AWS Firecracker allowed fast deployment and execution of container environments. Thanks to this new generation of hardware accelerated hypervisors, compute services like AWS Lambda now offer much higher CPU, disk, and network I/O performance.
A cost-effective cloud environment is all about high utilization of provisioned resources. Serverless computing promises that you’re always running your applications at 100% utilization. When running your applications on serverless, your provisioning is always perfect because you only pay for machine time you actually used.
Serverless compute scales well with your incoming traffic compared to classic compute services like EC2. Scaling with classic EC2 typically involves another service called auto scaling. Auto scaling itself is a complicated feature that handles tracking select metrics like CPU load or network utilization of your virtual machines. Auto scaling uses these metrics to trigger alarms to scale your EC2 instance fleet up or down. As you can tell, EC2 fleet scaling involves a lot of steps, and it’s really easy to miss something because every application is quite different.
With serverless, you can forget all that complexity. Serverless application scaling literally just copies the final container to select hypervisors, executes it, and routes all incoming events to it. If your serverless application fits into container size and runtime memory limits, you’re good to go and your application will scale just right. This is enabled by containers and hypervisors that run closer to hardware. Serverless has very little to no operating system overhead because it’s running just kernel code and the application in its containers.
Scaling is something serverless excels at, but it also has some limits. Serverless is a perfect compute model for event-based processing. Most workloads on the web are event-based or can be converted to it. Currently, serverless doesn’t work well with long-running processes because AWS Lambda and other cloud services have execution limits. Classic VM compute or other types of container-based compute are still better suited for long running processes. As long as your workload can be event-driven and processing an event is relatively quick with low resource requirements, serverless should work well for you. None of the current serverless limits seem to be permanent and service offerings should develop and improve over time. A limit that’s been a problem yesterday may not even exist tomorrow.
Finally, the most important thing: cost. How much cheaper is it to develop and run serverless applications? I’d say it depends on the application. When your applications scale well, they will provide better value for you and your users. You’ll run them cheaper. Serverless helps by eliminating overprovisioning and overpaying for compute resources you don’t utilize. While serverless improves scaling, other billable utilization still applies—you should still think about how much bandwidth, storage, and database volume you use. Serverless will not help you better utilize those resources at the moment, that’s still up to you and your application.
You may have read posts around the internet about serverless and vendor lock-in. I’d say you can still control those aspects in the architecture of your applications. Serverless computing has numerous benefits—great performance, easy setup, and quick development and deployment. This is why serverless became an important tool in my toolbox for building high performance applications that scale well while maintaining low cost. If you haven’t tried serveless yet, you should. Start by exploring the serverless application model.
Editor’s note: For those curious about the serverless offerings outside of those mentioned in this article, check out AWS Lambda, Azure functions and Google Cloud Functions. h/t to Chris in the comments.
Tags: bulletin, cloud computing, serverless, stackoverflow
26 Comments
Great
Blah blah buzzword blah blah buzzword blah blah
It is likely yet another marketing -ploy-.
I remember about 25 years ago, as software development of the future was envisioned as filling out forms, binding components together, behind a GUI-designed API. They never called it so, but the concept was some like a “code-less programming”.
MsAccess, some ERP systems still preserve the remembers of this failed concept, and many *Studio can still create some initial, -bad- example apps by wizards.
We will see, what will happen now.
It is not “serverless computing”, it is app generator wizards on remote hosted servers.
Editor’s note: Edits for language
I agree that ‘codeless’ platforms are garbage and I still see articles (ads) saying it’s the future every few years. Just saw one the other day, actually.
But that’s not what serverless is, at least not in general. It’s not about how the code is written e.g. AWS Lambda supports Node, Python, Java, etc. and you provide the code. Serverless is about the execution model.
I was under the impression the serverless model does not include Containers.
see:
https://www.cloudflare.com/learning/serverless/serverless-vs-containers/
has this changed recently?
So cloudflare seems to offer a competing service to Amazon Lambda. They are both using “serverless” which is a horrible term. Its bad because it doesn’t mean serverless but instead means “you don’t have to worry about the servers, we will”. It’s also bad because the technology they are describing is on implementation. Similar problem with “containers”, but I digress. It looks like they define, build and manage container images. You make a config file saying what you want. Then the service goes off and builds a container and starts it running *somewhere*. You can assume that the deployment system will be build and scaled by them. Also see serverless.com.
not having to worry about the server is a feature, not a bug. I welcome the Serverless movement.
The blog author seems to imply you need to shift your concern from VMs to Containers. I was pointing out that with the Serverless I learned about in 2018 you skip both VMs and containers and focus on your application logic and learning how to code cloud native applications using Lambda, S3, SQS, DynamoDB among others. Leave the server based application model as legacy and skip over containers and to take advantage of low cost, HA cloud based apps that don’t require servers to monitor, patch and secure.
After searching, I see that AWS is offering Fargate as a serverless container service. So it is evolving and improving.
https://aws.amazon.com/fargate/
“With serverless, you can forget all that complexity” – hahahahahahahahaha
“Multiple applications can run concurrently on the same hardware.”
Finally I can run more than one application on my PC!!!
if you’re going to do this go serverless rubbish, at least have some examples of systems available as well as compare and contrasting different options…
Looks like another hype, and advertising for Amazon. They’re not the only people that provide this kind of service.
A lot of people in these comments seem to have never worked with serverless applications. I want to clarify some things, in response to both those comments and the article itself.
For one thing, “serverless” isn’t necessarily tied to containers. You don’t need containers to run serverless. They work well together, though.
You can run non-serverless containers (serverful containers?) on something like Kubernetes. Or, you know, just running containers by themselves.
The term “serverless” is naturally a little confusing, and most developers’ initial response to it is skepticism. And rightfully so; there is, of course, a server involved. It’s kind of misleading, yeah. But it does *not* mean just that “the server is handled for you” — otherwise all managed infrastructure would be considered serverless, which it is not.
Serverless typically refers to short-lived transient processes handling incoming events. When you think of a “server”, you probably think of an application running for weeks and weeks, maybe longer, listening for requests and spitting out responses. Serverless moves the “listening” part further down the stack, so instead your application runs for, ideally, the length of one request.
I’m gonna say that again. Serverless means, ideally, your application spins up when a request comes in, and it’s terminated after the response has been sent out.
For some serverless platforms, this ideal notion is reality. Google Cloud Functions is a great example. You give GCF a single function, and it will provision, run, and deprovision your function for every single incoming request. Your function isn’t a full application (much less a container) so it’s very cheap to spin up and tear down. You pay nothing when there are no requests coming in(!), since your code isn’t running when there aren’t any events.
This optimistic world gets clouded when you face more realistic situations, though. Very few applications can be expressed in single functions. So instead, you can try to do roughly the same thing, except with a full app — containers are perfect for this scenario, because they guarantee a static environment and they’re super lightweight. An example of a serverless platform using containers is Google Cloud Run. I highly recommend checking it out yourself, it’s got a generous free tier and it’s super easy to use.
But containers aren’t as lightweight as functions! You can’t spin up a container and tear it down between each and every request, that would be super wasteful. So what Google Cloud Run does is strike a more pragmatic middle ground: at first when there are no requests coming in, there are no containers running. But when a request comes, and a container spins up, it doesn’t get killed after the response is sent. Instead, the container is kept alive for a while to see if any more requests come, and it can take care of those, too.
You specify how many concurrent requests your container can handle, and if there are more requests than that limit, Cloud Run spins up another container. It makes scaling a dream. So with this approach, you kind of get the best of both worlds: the portable, predictable nature of containers, along with the SUPER intuitive + fast scaling (and scale-to-zero) of serverless.
As far as vendor lock-in, I can’t speak for AWS, but Google Cloud Run is running the open-source KNative under the covers, so you can run it on-prem if you want (just without the pretty GUI). That’s an example of running serverless *without* paying someone else to manage your infrastructure.
tl;dr: serverless doesn’t mean containers, nor does it mean managed infrastructure. It just means short-lived, fast-scaling apps. Though like anything in the cloud, the definition changes depending on who you ask
So this comment is way more clear than the post. Thank you, sir.
Thanks for the detailed and objective comment. It puts a lot of things into light
This was way better than the post, thanks for the info!!
(I’m also getting a feeling that many of these posts are just glorified ads …)
That seems like a shameless advertisement for AWS hidden as a blog post. If you wanted fairness, you should link not just to the AWS offerings, but also to the Azure and Google Cloud Platform offerings.
Add the links here and I’ll put them in.
Thanks for your reply! I’m not familiar with AWS SAM, but if I understand it correctly, it’s based on AWS lambdas (https://aws.amazon.com/lambda/), which are similar to Azure functions (https://azure.microsoft.com/en-us/services/functions/) and to Google Cloud Functions (https://cloud.google.com/functions).
I’ve heard this one before. Did you consider the complexity of serverless?
Serverless is deep and different between clouds. AWS is something I have experience with and that’s why the article talks about serverless in AWS.
Then you are vendor-locked to AWS. 🙂 Serverless has it’s advantages, especially serverless on edge (Cloudflare workers, lamba on edge) but it is not magic to solve everything. I can see a static website hosted there or a link redirector tracking clicks. It can serve content with low latency and in some cases be very useful. But generally, I don’t like it. For anything more complex you still need to deal with complex databases and services around. And saving 10ms of latency may not be a big deal. In other words – I gave up on serverless, it wouldn’t help me in any way.
Serverless can be seen as a tool to deploy edge or CDN computing. You can develop quite complex applications with serverless that have database and complex set of services around them. In AWS serverless is a way to deploy entire application stack and get efficient compute with easy scaling. Even for small applications it is an easy way to skip VMs and devops around it.
What about debugging? What about language support? What about running your stack locally? You say “Finally, the most important thing: cost”. Honestly I think one of the key aspects to take into account when selling technology is the user experience from a developer perspective.
This is true. That’s why there is a link to Serverless Application Model from AWS in the conclusion of the article.
If you want to develop serverless functions without vendor lock-in, it might be interesting to look into the serverless framework: https://www.serverless.com. The whole configuration for the serverless setup can be checked in into the code repository and deployed based on the configuration.
Nothing, but absolutely NOTHING, about [warmup cost](https://dev.to/dandv/stop-using-aws-lambda-for-everything-4egc)?
I have specifically mentioned executions limits. These limits change and cold start is not a problem for every Lambda runtime.
Thanks for that nugget of wisdom.
Just before reading the article I had spent hours and hours trying to get a Docker container to run with some graphic processing code, and eventually gave up.
Got the same API endpoint running as an Azure Function in a matter of minutes.
Much obliged.