https://platformengineering.org logo
Title
l

Lukas Mrtvy

04/28/2023, 8:35 AM
Anyone has experience with building a platform for delivering clusters? Gardener, and Kubermatic, both are Cluster-API adopters, but there`s probably no "control-plane" platform for vannila Cluster-API, of course, this could be doable in Kubevela, maybe even with https://github.com/kubevela-contrib/backstage-plugin-kubevela extension also with Backstacke. Ideas? Thanks
a

Abby Bangser

04/28/2023, 11:03 AM
I am curious what you want from the delivered cluster. For simple plain clusters and no more management / support I think CAPI is a popular choice, Crossplane is an option, Terraform modules is an option. One thing we have been hearing though is that there is a need for clusters, for those clusters to have some sort of baseline installs on them, and for them to be able to get additional installations / upgrades over time. With that in mind we just recorded a really small demo of how to create an EKS cluster with Kratix.io and include baseline tools (in this demo it is cert manager and otel). The cool thing is that this cluster continues to be registered with Kratix so can get more updates / installations over time if needed. The demo if that sounds interesting: https://asciinema.org/a/nPezcLqh4oMk9oabcg96HOli1 Back to the original question though, what is your use case for this cluster because there are all sorts of options depending on how you plan to use it and manage any lifecycle bits!
l

Lukas Mrtvy

04/28/2023, 11:11 AM
Hey, Currently, I am using CAPI + Projsvetlos for creating and bootstrapping clusters ( CCM, CNI, CSI, CA, etc ) via ArgoCD, thats quite flexible. Kube-state-metrics with CRDs support provides me with observability. Gardener ( not sure if Kubermatic too ) takes this far, it also implements OIDC, etcd backups, etc..
a

Abby Bangser

04/28/2023, 11:13 AM
Yea having it all tied into ArgoCD can help with the delivering additional pieces to the cluster on create / throughout lifecycle.
Given your set up it sounds like you are pretty well covered 👀 Was the original question just to see other options? or is there something you were hoping to improve on? 🤔
l

Lukas Mrtvy

04/28/2023, 11:17 AM
Never heard about Kratix, unfortunately it does not cover CAPI promise ATTOW, right?
I just looking for Gardener, Kubermatic alternatives, but its still edge case, so not much opensourced software were released
a

Abby Bangser

04/28/2023, 11:21 AM
Kratix is higher level than CAPI and the other options. It is probably more like what you are using Argo for. A way design a self serve API for your platform offerings and then schedule them. So in the Kratix world you would still do what you are doing with CAPI but you would wrap it in a Promise to allow for an API for requests from your users and a scheduling process for where to actually provision things. Sorta one of those “it doesn’t reduce the tools you use, but it reduces the boilerplate / heavy lifting for delivering your platform services” kinda thing.
l

Lukas Mrtvy

04/28/2023, 11:22 AM
Has Kubevela similar model then?
a

Abby Bangser

04/28/2023, 11:22 AM
I think it is clear but to make sure, I am a contributor so all caveats around that apply. The fact is that it is early days still, but the pattern of needing to create clusters and then lay down baseline apps as well as ondemand additional items is something we are solving for
l

Lukas Mrtvy

04/28/2023, 11:24 AM
The thing is, KV has VelaUX, which goes far more than API only, it provides also dynamic forms and bindings and thats quite useful just like Backstage does it for lower lever of abstraction.
a

Abby Bangser

04/28/2023, 11:24 AM
My experience is that a lot of tools can be used for a lot of different solutions. But often they are optimised for a certain use case. My understanding is that KubeVela is optimised for app delivery (like web app) where as Kratix is optimised for broader definitions of a platform (including infra, including off cluster things, etc). That doesn’t mean you can’t use both together or only one to solve problems. Just optimisations.
Yup, we definitely aren’t going the custom UI route (at least right now). Our ethos is that each organisation needs to customise (not just configure) their own platform because there are so many unique needs. Our customers are mainly using backstage and compass to create UIs when they need them.
As ever, if you can buy the solution, DO IT! If KubeVela or another tool has your use cases covered, it is WAY cheaper to build with that rather than build your own. Kratix is for if that becomes limiting. The joke is that everyone wishes they could have Heroku experience, except they can…it is very much still around 😅 The reason people say that but don’t often go that way is they need more customisation. But that is heavy lifting. Which is the goal of Kratix (to reduce that heavy lifting for if you need to create custom).
l

Lukas Mrtvy

04/28/2023, 11:31 AM
Got it, thanks 🙂 I am a solo dev pursuing opportunities and still learning stuff, so I know its doable for each company to make some customized solution. My end goal would be a opinionated PaaS that would mostly serve only my learning purposes.
Will check Kratix more in detail
a

Abby Bangser

04/28/2023, 11:38 AM
That sounds awesome and a lot of fun. As I said before, there are tons of tools around right now that can all solve the same problem in their own way so I hope you can share learnings and opinions on them all as you go. Also hope the thread uncovers more options as you asked 👀 Would love to hear more about how others are going through it!
h

Hugo Pinheiro

04/28/2023, 12:26 PM
https://github.com/arlonproj/arlon is a alternative that uses capi and argocd for cluster creation, it's still a young project though but promessing, not too many frameworks use capi yet, like you mentioned there is kubermatic, and Gardener, I found those a bit limiting since they use the vSphere provider for onprem clusters, Arlon has the potential to use providers like Talos and kubekey which offer more flexibility
I still need to dig a bit more into it but so far they only got the instructions for the aws and docker providers
l

Lukas Mrtvy

04/28/2023, 1:08 PM
Interesting, seems its like Argo-CD Autopilot, but on a higher level of abstraction for CAPI, and it actually also wrapping Argo-CD Autopilot functionality
h

Hugo Pinheiro

04/28/2023, 1:09 PM
Nods it's a quite clever solution
l

Lukas Mrtvy

04/28/2023, 1:13 PM
Wondering if there is some universal solution for bootstrapping "apps/clusters/whatever" via CLIs, but for YAML in general. https://projectsveltos.github.io/sveltos/ is better for bootstrapping, for example https://projectsveltos.github.io/sveltos/template/#deploy-autoscaler-in-the-managed-cluster ( it can create service account in management cluster per workload cluster for cluster-autoscaler ) for CAPI enabled clusters, but of course its not bootstrapping CLI 🙂 EDIT: • https://github.com/weaveworks/weave-gitops
h

Hugo Pinheiro

04/28/2023, 1:25 PM
Haven't found one yet 😂
l

Lukas Mrtvy

04/28/2023, 1:28 PM
weave-gitops + https://www.weave.works/blog/flamingo-expand-argo-cd-with-flux could probably do the trick for argo fanboys, intesting
h

Hugo Pinheiro

04/28/2023, 1:30 PM
Sort of but I believe that's more or a argo + terraform thing, if that's the case I would use https://kubefirst.io/ instead
Weave has the habit of abandoning projects that are not core to their product
l

Lukas Mrtvy

04/28/2023, 1:30 PM
weave-gitops supports CAPI
h

Hugo Pinheiro

04/28/2023, 1:31 PM
🤔 does it
Ahh so it does but it's a enterprise feature 😞
l

Lukas Mrtvy

04/28/2023, 1:37 PM
Oh, I didnt even notice.., Yes, Weave..
Honestly, it does not seem like a fully-fledged platform to me, just a wrapper with WebUI at the first glance
h

Hugo Pinheiro

04/28/2023, 1:41 PM
You mean kubefirst?
l

Lukas Mrtvy

04/28/2023, 1:41 PM
No, Weave-gitops
h

Hugo Pinheiro

04/28/2023, 1:43 PM
Ahh ya it's kind of a wrapper around flux with some extra stuff on top, it can become a platform if you buy the enterprise version 😂