We have our services running on AWS EKS clusters a...
# platform-blueprints
p
We have our services running on AWS EKS clusters and we are using ALB ingress controllers to expose those services. We want to scale these services based on number of HTTP requests the service receive (excluding ALB health checks). Is there a built in way to do this in k8s? If not, I need some suggestions how can we achieve this with minimal effort.
p
do you use datadog? or prometheus?
you can configure HPA to use external metrics
h
If you want something more flexible https://keda.sh/ also helps with that
p
We use prometheus in our cluster, the approach @Philip Clark mentioned is using External metrics of load balancer to scale but AWS load balancer has 5mins delay in adding external metrics to cloudwatch from where we can scale and thus us unreliable. Is there any other wasy way we can achieve this?
@Hugo Pinheiro Keda has an HTTP addon but it is in beta and is unreliable and does not give expected results
Any thoughts @Philip Clark ?
p
I’ve head of keda, haven’t used it though. I think it all depends on how flexible of a solution you need, and if you want to maintain yet another kube service
p
I thought it was a simple problem that should have been solved already and a lot of people might be using this but i guess it is not🥲
p
Autoscaling is simple yes, but measuring network traffic on cloud resources outside Kubernetes as an input to autoscaling isn't built in (for good reason, Kube only knows what's going on inside it), so you need to rely on something outside the Kube to monitor that traffic. More often, people use a service mesh or some kind distributed tracing.
But these are both well established patterns (Keda and HPA+External metrics from something like Datadog or Prometheus). It's not so much of the problems not being solved, more that Kubernetes is a Bring Your Own Tool mentalitity. It provides a framework, and you bring whatever tool to solve a problem that best meets your needs
a
@Prashant Banthia Instead of using an ALB, why don't you use an NLB in front of an ingress controller like nginx or istio? Then you can use the instantaneous metrics as scraped by nginx/envoy proxies running in your cluster to power the HPA. And you reduce your costs by not having an LB dedicated per application.
h
All depends "The NLB does not support SNI, which means you can only associate one SSL/TLS certificate per NLB" so it has some limitations.
a
Not at all, because you wouldn't be using the NLB to handle SNI or redirection to any of your services. The NLB is just a dumb L4 LB that directs EVERYTHING at the nginx/envoy proxies inside your cluster. Those proxies are the ones handling SNI and routing, and if you terminate at the proxy, loads of L7 routing functions.
h
True that's a good point
p
We’re actually looking at using this pattern to get around the timeout limitations with nlb, seems kind of weird to have two ingress controllers (nginx and aws lb controller, but 🤷🏻‍♂️
h
Ya that's our current setup as well but we're moving to just albs ( and some cloudflare magic ) to make things easier
a
@Philip Clark Just see it as an L4 proxy load-balancing between your L7 proxies. AWS even forces you to do this outside of k8s if you're connecting API gateway to e.g. ECS containers. API gateway only supports linking to NLBs, which obviously have no path-based routing. You have to route the gateway to an NLB which then points to an ALB in order to route different paths to different containers lol. Thankfully the new HTTP API type in gateway allows directly routing to an ALB.
p
Yeah that’s a really good way of thinking about it