I'm looking for a bit of guidance or insight from teams who have adopted Kubernetes as a key part of their platform. Our team uses Azure exclusively and mostly use PaaS (Azure Web Apps) or VMs. The VMs run a couple of Windows Services and a few console apps. There seems to be a push in the industry towards hosting everything in Kubernetes. I suspect this is driven more by engineers being attracted to fancy new tech and higher salaries than a genuine business need. If you come at it from a business requirements perspective what is the justification for Kubernetes over using PaaS components? It would be trivial to deploy our web apps to Azure App Services and convert our services and console apps to serverless functions. These services are easy for developers to understand and there is no special skills required within the Platform team to administer them. Conversely, using Kubernetes first means the development team need to adopt containerization. This is a significant investment for a team with no current experience. Then the platform needs to build and maintain K8s clusters. Even when using a hosted solution like AKS still requires a significant investment in skilling up to properly support Kubernetes. I feel like Kubernetes is a solution to a problem that most of us don't have. Am I missing something?

I think there are a few things here. Depending on the number of microservices, the complexity of their interactions, and the development phase of a team (I hate the term team maturity, it's so judgemental). Also, considering the org structure (platform team, ops team, are they separate, do developers actually 'run' their environments), the actual business domain and environment, and several other factors. Kubernetes might be precisely the right thing or possibly the worst thing ever. If Kubernetes is 'just' a container orchestrator, then the position you laid out above might be the correct answer for many organizations. However, if Kubernetes is considered to be a 'posix for the cloud' with re-usable control loop components and a standard yet expandable API service. And can be deployed to any number of clouds, on-prem locations, or hybrid situations, where applications can be run and managed with the same pipelines and tools regardless, then the value proposition changes. I work as a consultant and have clients where Kubernetes is the wrong answer, even though it's what they ask for. I also have clients who could benefit from Kubernetes in many more ways than as a container runner.

I feel like Kubernetes and containers is about options to port between one cloud or another. If you are only in Azure it could be fine.

What Andy said, in addition the downside of serverless is it can be a bit of a blackbox at times in terms of troubleshooting, and testing it locally is a bit hard sometimes, although serverless functions is the future even if its running serverless on top of kubernetes 😂

Worth considering which

Controllers

and

CRDs

you might end up using. Implementing the same golden path for pipelines is a bit different, especially if you care about things such as encryption, fast onboarding to LB ingresses, Progressive Release (Canary), External Secrets, Observability based AutoScaling, etc... If all you need is to put a container live, that might not do much for you, but if you put the time into building a mature DevSecOps pipeline, you might consider some of the benefits.

In my org, we run everything in docker containers already, but just deploy to raw ec2. the advantage to k8s for us (which we are working on), is the ability to run multiple microservices on the same ec2 node to reduce costs. most of our microservices underutilize the ec2 resources they are on, so pooling them will reduce the number of nodes we will need overall. of course this is possible without k8s, but k8s makes it easier. in addition, we are building a new loccal dev environment which will also be k8s based so that local dev and preprod/prod deployments look more similar (including using our envoy mesh and everything)