Why is Kubernetes getting so popular?
At the time of this article, Kubernetes is about six years old, and over the last two years, it has risen in popularity to consistently be one of the most loved platforms. This year, it comes in as the number three most loved platform. If you haven’t heard about Kubernetes yet, it’s a platform that allows you to run and orchestrate container workloads.
Containers began as a Linux kernel process isolation construct that encompasses cgroups from 2007 and namespaces from 2002. Containers became more of a thing when LXC became available in 2008, and Google developed its own internal ‘run everything in containers mechanism’ called Borg. Fast forward to 2013, and Docker was released and completely popularized containers for the masses. At the time, Mesos was the primary tool for orchestrating containers, however, it wasn’t as widely adopted. Kubernetes was released in 2015 and quickly became the de facto container orchestration standard.
To try to understand the popularity of Kubernetes, let’s consider some questions. When was the last time developers could agree on the way to deploy production applications? How many developers do you know who run tools as is out of the box? How many cloud operations engineers today don’t understand how applications work? We’ll explore the answers in this article.
Infrastructure as YAML
Coming from the world of Puppet and Chef, one of the big shifts with Kubernetes has been the move from infrastructure as code towards infrastructure as data—specifically, as YAML. All the resources in Kubernetes that include Pods, Configurations, Deployments, Volumes, etc., can simply be expressed in a YAML file. For example:
apiVersion: v1 kind: Pod metadata: name: site labels: app: web spec: containers: - name: front-end image: nginx ports: - containerPort: 80
This representation makes it easier for DevOps or site reliability engineers to fully express their workloads without the need to write code in a programming language like Python, Ruby, or Javascript.
Other benefits from having your infrastructure as data include:
- GitOps or Git Operations Version Control. With this approach, you can keep all your Kubernetes YAML files under git repositories, which allows you to know precisely when a change was made, who made the change, and what exactly changed. This leads to more transparency across the organization and improves efficiency by avoiding ambiguity as to where members need to go to find what they need. At the same time, it can make it easier to automatically make changes to Kubernetes resources by just merging a pull request.
- Scalability. Having resources defined as YAML makes it super easy for cluster operators to change one or two numbers in a Kubernetes resource to change the scaling behavior. Kubernetes has Horizontal Pod Autoscalers to help you identify a minimum and a maximum number of pods a specific deployment would need to have to be able to handle low and high traffic times. For example, if you are running a deployment that may need more capacity because traffic suddenly increases, you could change
maxReplicas
from10
to20
:
apiVersion: autoscaling/v2beta2 kind: HorizontalPodAutoscaler metadata: name: myapp namespace: default spec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: myapp-deployment minReplicas: 1 maxReplicas: 20 metrics: - type: Resource resource: name: cpu target: type: Utilization averageUtilization: 50
- Security and Controls. YAML is a great way to validate what and how things get deployed in Kubernetes. For example, one of the significant concerns when it comes to security is whether your workloads are running as a non-root user. We can make use of tools like conftest, a YAML/JSON validator, together with the Open Policy Agent, a policy validator to check that the
SecurityContext
of your workloads doesn’t allow a container to run as a root. For that, users can use a simple Open Policy Agent rego policy like this:
package main deny[msg] { input.kind = "Deployment" not input.spec.template.spec.securityContext.runAsNonRoot = true msg = "Containers must not run as root" }
- Cloud Provider Integrations. One of the major trends in the tech industry is to run workloads in the public cloud providers. With the help of the cloud-provider component, Kubernetes allows every cluster to integrate with the cloud provider it’s running on. For example, if a user is running an application in Kubernetes in AWS and wants that application to be accessible through a service, the cloud provider helps automatically create a
LoadBalancer
service that will automatically provision an Amazon Elastic Load Balancer to forward the traffic to the application pods.
Extensibility
Kubernetes is very extensible, and developers love that. There are a set of existing resources like Pods, Deployments, StatefulSets
, Secrets, ConfigMaps
, etc. However, users and developers can add more resources in the form of Custom Resource Definitions. For example, if we’d like to define a CronTab
resource, we could do it with something like this:
apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: name: crontabs.my.org spec: group: my.org versions: - name: v1 served: true storage: true Schema: openAPIV3Schema: type: object properties: spec: type: object properties: cronSpec: type: string pattern: '^(\d+|\*)(/\d+)?(\s+(\d+|\*)(/\d+)?){4}$' replicas: type: integer minimum: 1 maximum: 10 scope: Namespaced names: plural: crontabs singular: crontab kind: CronTab shortNames: - ct
We can create a CronTab
resource later with something like this:
apiVersion: "my.org/v1" kind: CronTab metadata: name: my-cron-object spec: cronSpec: "* * * * */5" image: my-cron-image replicas: 5
Another form of Kubernetes extensibility is its ability for developers to write their own Operators, a specific process running in a Kubernetes cluster that follows the control loop pattern. An Operator allows users to automate the management of CRDs (custom resource definitions) by talking to the Kubernetes API.
The community has several tools that allow developers to create their own Operators. One of those tools is the Operator Framework and its Operator SDK. The SDK provides a skeleton for developers to get started creating an operator very quickly. For example, you can get started on its command line with something like this:
$ operator-sdk new my-operator --repo github.com/myuser/my-operator
Which creates the whole boilerplate for your operator including YAML files and Golang code:
. |____cmd | |____manager | | |____main.go |____go.mod |____deploy | |____role.yaml | |____role_binding.yaml | |____service_account.yaml | |____operator.yaml |____tools.go |____go.sum |____.gitignore |____version | |____version.go |____build | |____bin | | |____user_setup | | |____entrypoint | |____Dockerfile |____pkg | |____apis | | |____apis.go | |____controller | | |____controller.go
Then you can add APIs and a controller like this:
$ operator-sdk add api --api-version=myapp.com/v1alpha1 --kind=MyAppService $ operator-sdk add controller --api-version=myapp.com/v1alpha1 --kind=MyAppService
And finally build and push the operator to your container registry:
$ operator-sdk build your.container.registry/youruser/myapp-operator
If developers need to have even more control, they can modify the boilerplate code in the Golang files. For example, to modify the specifics of the controller, they can make changes to the controller.go
file.
Another project, KUDO, allows you to create operators by just using declarative YAML files . For example, an operator for Apache Kafka would be defined with something like this, and it allows users to install a Kafka cluster on top of Kubernetes with a couple of commands:
$ kubectl kudo install zookeeper $ kubectl kudo install kafka
Then tune it also with another command:
$ kubectl kudo install kafka --instance=my-kafka-name \ -p ZOOKEEPER_URI=zk-zookeeper-0.zk-hs:2181 \ -p ZOOKEEPER_PATH=/my-path -p BROKER_CPUS=3000m \ -p BROKER_COUNT=5 -p BROKER_MEM=4096m \ -p DISK_SIZE=40Gi -p MIN_INSYNC_REPLICAS=3 \ -p NUM_NETWORK_THREADS=10 -p NUM_IO_THREADS=20
Innovation
Over the last few years, Kubernetes has had major releases every three or four months, which means that every year there are three or four major releases. The number of new features being introduced hasn’t slowed, evidenced by over 30 different additions and changes in its last release. Furthermore, the contributions don’t show signs of slowing down even during these difficult times as indicated by the Kubernetes project Github activity.
The new features allow cluster operators more flexibility when running a variety of different workloads. Software engineers also love to have more controls to deploy their applications directly to production environments.
Community
Another big aspect of Kubernetes popularity is its strong community. For starters, Kubernetes was donated to a vendor-neutral home in 2015 as it hit version 1.0: the Cloud Native Computing Foundation.
There is also a wide range of community SIGs (special interest groups) that target different areas in Kubernetes as the project moves forwards. They continuously add new features and make it even more user friendly.
The Cloud Native Foundation also organizes CloudNativeCon/KubeCon, which as of this writing, is the largest ever open-source event in the world. The event, which is normally held up to three times a year, gathers thousands of technologists and professionals who want to improve Kubernetes and its ecosystem as well as make use of some of the new features released every three months.
Furthermore, the Cloud Native Foundation has a Technical Oversight Committee that, together with its SIGs, look at the foundations’ new and existing projects in the cloud-native ecosystem. Most of the projects help enhance the value proposition of Kubernetes.
Finally, I believe that Kubernetes would not have the success that it does without the conscious effort by the community to be inclusive to each other and to be welcoming to any newcomers.
Future
One of the main challenges developers face in the future is how to focus more on the details of the code rather than the infrastructure where that code runs on. For that, serverless is emerging as one of the leading architectural paradigms to address that challenge. There are already very advanced frameworks such as Knative and OpenFaas that use Kubernetes to abstract the infrastructure from the developer.
We’ve shown a brief peek at Kubernetes in this article, but this is just the tip of the iceberg. There are many more resources, features, and configurations users can leverage. We will continue to see new open-source projects and technologies that enhance or evolve Kubernetes, and as we mentioned, the contributions and the community aren’t going anywhere.
Tags: bulletin, containers, kubernetes, serverless, stackoverflow
33 Comments
Good article – accurate.
One thing to note is that many seem to believe K’s provides higher “availability” of a system.
This simply is not true in that, barring applications crashing randomly out of memory (bugs)…
A system (globally) does not “discover” working K-clusters without more work.
That is, K is not going to solve:
– Client cannot access the cluster (need many clusters, “deep” health-checks, DNS-resolution support, etc).
– Unit-of-order (UOO), at-least-once-delivery and other transaction contracts for a system of many clusters across regions (earlier point).
Otherwise, it does nicely for local cluster management and can be pushed to provide heuristic auto-scaling (though not without more magic).
I was excited when I first met Kubernetes but today I just think it is very complex and difficult to manage infrastructure and kinda obsolete. To be honest I am working right now with Server-less and AWS and whole Kubernetes to me now looks like a crap.
Let me name few things. Operators – you get something not so reliable and not so good if you are looking for Operators for databases cluster compare to Amazon RDS. It is very complex, difficult to set-up, in Amazon RDS, you set-up and manage RDS cluster just by 3 clicks.
Development. Let me tell you one thing, Server-Less is the future. I just do not see many reasons why I should take care of container images. Why I should care about OS?
Let me tell you next thing – networking. In Kubernetes, it is extremely complex. If you use AWS and Server-Less, you just define your VPC, Gateway and that’s it.
I just do not think that Kubernetes is a future anymore. You will end-up paying a big bucks for managing your infrastructure on Kubernetes anyway, you will end up with obsolete docker containers and very complex setups.
Kubernetes is simply definitely not a future. You can build with it a castle but you still have to manage LEGO bricks yourself and this will not only slow-down you a lot in future (you are going to find out that 90% of the work is just supporting current infrastructure not adding user requests) but it will be also costly.
The reason why Server-Less is not used more is that with Kubernetes you can take an old crappy code and crappy development and do a new painting to let it looks good. You can take 20 years old code and package it in Kubernetes. That’s the greatest thing about Kubernetes. You cannot do that with Server-Less. Server-Less is completely new paradigm, no backwards compatibility.
That’s the largest problem with Kubernetes. It is an old paradigm painted with a fresh new color.
I agree with your comment. This is also why I think serverless is much cheaper for new applications.
“Server-less” just means you don’t run the server. There are many negatives such as extreme vendor lock-in and more “magic” in the hosting layer. But there is still a host, running on a server, that requires a control structure.
To put it more bluntly, these toys aren’t meant for developers like you.
Tom you’ve somehow taken Martin’s comment as a personal attack. No need to discourage his ability as a developer. And if you are going to play that game, a sharp developer will tell you to keep a simple architecture.
For every complex problem there is an answer that is clear, simple, and wrong.
H. L. Mencken
So disparaging as it might be, this comment has some merit. I started programming when I was 12. That was oh, over 30 years ago and I STILL program every day writing applications for Fortune 500, startups, etc. Suffice to say what I have learned is that it is so easy to just setup a web-page and have no idea how it actually works (HTTP et al). Big AWS fan and think K’s is overblow and agree – overly complex for what it does but… this whole ‘serverless’ stuff is nonsense for anything but the most trivial of applications.
These are some of my opinions about this, would love to chat more!
– Serverless is a different paradigm: yes
– Kubernetes can get quite complex: yes
– AWS has over 100 people working on Lambda: yes
– This hides the complexity but it’s no simple task.
– AWS has hundreds of engineers working on other services to support Lambda (S3, SQS, API gateway, Fargate, etc): yes
– Lambda is cheaper: it depends.
– If for example, you are serving millions of API requests 24×7 spinning up a Lambda function it can get even more expensive.
– If you are doing a few one-off asynchronous operations it can save a lot of money since it’s pay as you go.
All of these technologies have their place. I never understand why so many conclude that technology-A is obsolete and technology-B is the new thing that will completely eliminate tech-A from existence.
Decades of technology innovation should have taught us that this is now generally how it plays out. Mainframes still exist, and still have a solid (if not more narrow) use case.
On-prem virtualization still has a place. So does IaaS. So does PaaS. So do containers. So does orchestration. Server-less also has a place.
Many of these technologies will continue to co-exist for years, until one of the other technologies can do the overlapping jobs more easily or more cost-effectively than some of the others. (Or, better marketing comes into play for one tech, or the right org jumps on the bandwagon of a specific tech).
“Old” is not automatically bad.
“New” is not automatically good.
It is best to ensure that you are using technology that makes sense for your business needs, without closing your eyes to everything else that doesn’t fit those needs right now, but might in the near future.
It never makes sense to turn technology into religion. Today’s new, is tomorrow’s old. Ask the “mini computer” folks.
-ASB
I thought you’re referring to https://www.serverless.com/ when using word “Server-Less”. Are you using this as a general term, or something third?
Server-less IS a software engineering concept: https://en.wikipedia.org/wiki/Serverless_computing
Kubebuilder(https://github.com/kubernetes-sigs/kubebuilder) is worth mentioning for creating operators.
The link to the insights survey should be https://insights.stackoverflow.com/survey/2020#technology-most-loved-dreaded-and-wanted-platforms
You absolutely miss the point that editing YAML-files is absolutely cumbersome. One never knows whether a value turns into a string (1.1 vs. 1.1.1) or where this snippet was, you came across last week.
Managing a sample project with two or 17 pods that run a simple container per pod are easy to maintain. Make it an Istio-config in a mesh of 100+ pods from more than 10 teams with about 5 programming languages and sufficient scripting/ node/ edge computing/ static content and you’re up and running in >3months.
This article is about how Kubernetes is where it is right now. Yes, YAML is a pain point that many have realized, it can be cumbersome when implemented at scale. It’s unclear where the industry will go forward but there are a few projects that are trying to address some of the problems:
– https://cuelang.org/
– https://dhall-lang.org/
– https://github.com/brendandburns/configula
– etc
I would love to chat more!
The reason is you write such stories and start to believe in yourself. Oh no now even others write about it too, we must be on something.
We don’t use it as it wouldn’t improve our business no need for it.
This is the only sensible approach. Regrettably, it’s very much the exception.
Please fix the the broken link to dev.insights
I have a problem:
https://dev.insights.stackoverflow.com/survey/2020#technology-most-loved-dreaded-and-wanted-platforms
The link above is not accessible.
Ooof. Sorry about that, fixed.
A nice article, but for anyone arriving here thinking they should immediately get on the K-bandwagon, there are opposing articles available, and before moving to kubernetes, one should definitely consider security (which Kubernetes has had significant problems with in the past).
Also, consider cost – in most cloud providers (for example), running a Kubernetes cluster is pretty expensive – and far more expensive than running a handful of containers in their container services (eg. ECS, App Engine, etc). You still get to write infrastructure as code there – just you’re writing Terraform instead of Kubernetes. It won’t earn you so much “cred” when you’re geeking out with your pals, but you’ll have more money to buy the beers.
IMHO, Kubernetes is the “big data” of containers/infrastructure – it’s pretty good if you know what you’re doing. However, it turns out that most people using it don’t really need it and could have used something else for a fraction of the cost. Caveat Architect 😉
Kubernetes has, hand in hand with Docker, become the latest fad – companies want Kubernetes because “KuBErnETeS gOoD”. Azure & AWS have been doing the same thing (but better) long before this pile of crap became the hype and will be doing it long after it’s as dead as Google Wave. I only hope that happens soon.
Why are you so pessimistic dude ! Why do you think it’s a fad ? Or are you just sitting in your corner which the world will be the same as you know it and nothing will change and hoping that your knowledge will be valuable 10 years from now, things are changing and evolving, that’s Tech. Azure have been created far after Kubernetes and they are doing a good job in some areas but not all of them, AWS has very good products but again “very good” not a “good” platform, i’m sorry i used AWS and Azure, they are crap. Kubernetes will stay around up to and probably while after you are gone
Shows how clueless you are! Kubernetes is the future!
Kubernetes is absolute garbage. No wonder soydevs love it.
In the beginning, being on Google+, being exposed to the daily hype flow of Google+, I was of course interested in k8s.
I waited, set up my own 1 node, tried to go deeper, but it’s just too complex. I’m not stupid, but I’m no IQ superstar either.
The only people who would actually benefit from k8s are people like Kelsey Hightower aka rockstar devs.
His job is promoting k8s. When he does it, everything looks simple and easy.
When you try that you’re met with landmine after landmine, stuff that has never been mentioned like requiring shared storage.
k8s is an absurd waste of resources.
Run your own database installation for every project?
k8s is for rich people, for big companies. For people who can dedicate a team on just k8s.
I’m not one of those. For me even docker is too much additional effort.
I’m also too poor for AWS,GCE,Azure and other public clouds.
I have 2 servers with 32GB RAM each that cost 100€/month combined and both has each 1gbit/s flat traffic.
If I have a project that takes off I put it on a separate server in a VM so I can move hosts every few years.
You guessed it, I have only 1 successful project that justifies paying an additional 50€/month for a server.
Sometimes I wish I worked in one of those rich companies so I could be hipster too.
But I’m not and I need to be cost effective and time effective. k8s is neither.
Sorry, Darko (and everyone else), I don’t know if I can ask here and it certainly has nothing to do with the Kubernetes subject but… you said that you “have 2 servers with 32GB RAM each that cost 100€/month combined and both has each 1gbit/s flat traffic”.
I’d really need something like that, since I’m paying more or less the same for a single server with worse specs. Can you tell me where do you have those servers, please? Thank you very much.
you comment is a reflexion of your choice of career, comparing simple servers to k8s is like comparing cars to dog, yeah maybe they have something in common (they can both move) but they are remotely similar to each other. There is a place and tool for each task, if k8s is not for you then fine but going out with such a useless comment to people, companies and communities working on delivering something bigger then what a single person could comprehend is remarkable. I which we lived in a world with every person in Tech could just shut the f**** up, do their job and stop lashing on other people efforts
Technologies like Ranchers, GKE, AKS, etc are more likely to become mainstream hiding the complexity of Control Plane and even various data plane quirks making it more developer friendly. Kubernetes is still the best way for Orchestration of hundreds of microservices. Combine it with managed kubernetes with cloud providers and you have a technology which is here to stay for sometime, IMHO. My 2 cents.
Word of blogger advice, why not balance information like this with the drawbacks and pain points of using Kubernetes? Why not give some context where using Kubernetes is useful versus where it’s overkill? This is overkill for a lot of organizations and fanboy, hand-waiving literature like this will confuse the less critical developers into thinking they need to use this.
I think Kubernetes is a great tool that permits most people and enterprises (very small to very large) to have an High Availabilty Infrastructure at very low costs (the costs depends on many factors not only software side).
But k8s implies a dedicated know how: a DevOps that spends some time to configure and maintain this infrastructure.
Anyone can use k8s into AWS, Azure, Google…and maybe many others including VPS or Bare Metal. All depends of the specific project and long term vision of the enterprise, project and so on.
In a precedent project, some year ago, i hav replaced an entire infrastructure with kubernetes with 70 microservices including databases (not so many ok… 🙂 ) with many benefits: scalability, observability, real time infra changes, cost reduce, more control and very less pains. The other side is that all developers and sysadmins (DevOps) must change it’s point of view: all services can fail at any time and must be replaceable in a few seconds.
For me: everything can fail, and we must known that is it!
Serverless for that i know is very usefull for specific tasks, but any serverless system is very specific, k86 aim to a standard for infrastructures. If you have a VPS whats happen if a service is under ddos attack? If you use a serverless how can install and manage a CMS?
Any tachnology have pros and cons and the choice depends on many factors: peoples, passions, know how 🙂
Lots of people get heated about Kubernetes taking over the spotlight, honestly it’s not taking over the spotlight. Serverless computing will still remain a good solution to start projects in the cloud, however it’s only very specific workloads with standard dependencies.
Containers and Kubernetes are lot more flexible in terms of the workloads, not only you can run you APIs in there but also crunch data with Dask or Spark. You can also host your front end servers, as well as your backend. That is not to say you have to replace everything you have on your stack, you can continue to use hosted Databases, and serverless APIs.
Kubernetes is not the future for everything, but is certainly the future of hybrid cloud computing.
Kubernetes is not my solution of choice for deployments. It’s great for monolithic, permanent applications, but for serverless/lambdas and atomic/functional microservices I have yet to encounter an application where it adds more complexity than it resolves.
Not serverless? How is the component inside a container server-full? The amount of complexity that simply vanishes inside a code inside a container is astonishing.