In the context of platforms, a topic I rarely encounter, is about Kubernetes objects abstraction, in broad terms (simplified abstraction, yaml alternatives, yaml generation from code, ... ; in short, how to let devs focus on the app, and not have to understand kubernetes configs on the level of

Deployemnts

, `Service`s etc. Most (open source ) platforms do not really focus on this (except the use of existing abstractions like helm, cue, kustomize etc, but nothing that reall simplifies processes for devs on that level) I was wondering how this community thinks on this topic. Links to resources on the subject are also welcome.

That abstraction exist natively in Kubernetes and is IMO a crucial component in platform engineering to define an unified domain model of your platform : CRDs (+ custom controller)

I meant general abstraction, in the sens of how you would define anything, starting with base deployments etc. Things that nowadays typically are "solved" by using tools like helm, kustomize or with DSL's like cue, jsonnet. Those are what I'd call low-level abstraction.

Absolutely spot on in that code is not an abstraction which is why Helm and Cue and Kustomize fall a layer short. I do actually think that is actually getting a lot of focus right now. KRO is a new thing by AWS, Crossplane has compositions, Kratix (which I work on) has Promises which also tie in business processes. All of these are looking at ways to create Platform provided abstractions for users.

I can see how KRO could be a nice helm alternative, but the base abstractyion is still a complex bunch of yaml

Agreed, these are all kind of design decisions. KRO brings a level of simplicity in design, but may not meet everyone’s needs.

In order to reach this kind of abstraction, I'm creating a framework called HAPE Framework to use it to build automation tools. Once it's ready I'll use it to build a cli/api platform (InnoDP), where developers would only need to provide the deployment requirements like: programming language they're using, framework, databases, caching, message brokers ..etc, then based on the helm templates the innodp platform creates the deployment and configures the CI/CD. And later there will be a simple UI that connects to innodp platform, where the developer can modify environment variables and resources, and be able to check the logs or monitor the app from grafana. They won't need to know anything about kubernetes, helm, or any other tool. I don't believe this kind of abstraction can be done using a one-for-all solution, because each software development team has their own use-cases and development process. That's why I'm working on a Framework to build platform instead of a platform tool, to make it easy to build platforms and get the needed result based on each platform engineer's use case.

I don’t use Resourcely myself but it might be relevant

Helm is very easy and simple, you just need to: • use: https://github.com/bitnami/charts/blob/main/bitnami/common/Chart.yaml as baseline • create your own abstractions in your own custom helm library chart, eg: organization-common-lib • create all charts with the two above libs included Life is easy after that: name: xyz version: 1.1.1 include <deployment> include <ingress> Full control over abstractions.

helm is a text templating tool, without any knowledge of inner objects. It doesn't even know about yaml.

I think the abstraction would differ depending on the use case. K8s provides low-level building-blocks to deploy any application. Application developers should not be aware of these low-level constructs, instead think of what their application needs to run (which is their responsibility). For example, most applications/services requires a runtime, object storage, database and a message queue or event-system. So, the abstraction would be; Container CRD, ObjectStorage CRD, DB CRD and MQ CRD. The platform team would be providing these abstraction to the development teams. dev team shouldn’t care that DB uses a StatefulSet. For them, they need a service that persists data reliably and that’s it.

You say Container CRD, ObjectStorage CRD, DB CRD etc … but really escape your thinking about mapping basic k8s blocks to CRDs , with CRDs you really define your platform domain model as you want , it’s really the source of truth

Let me throw score.dev into the mix. I think, that this might be what you’re looking for @Serge van Ginderachter. Score provides a platform-agnostic and environment-agnostic abstraction for developers, that allows for a declarative description of their app and needed resources. Then take the right Score implementation for your target platform, e.g.

score-k8s

, to turn this into the target platforms native format -> tons of YAML manifests for Kubernetes 🙂 Another Score implementation like

score-compose

might turn the same Score file into a Docker compose file, that makes it more easy to run the same thing locally as a dev. Score is a CNCF project, OSS by nature and community driven. So if it is not completely what you’re looking for, you might be able to fix that.

In general, talking about very friendly and simple developer experience I find in the market many problems that many products try solving, but usually one at a time. 1. yaml is presented as a silver bullet, it gives no schema, so high error prone. IMO only real language can give a strictly typed schema and lower delivery cycle time 2. All the cloud resources integration is done manually, I haven’t seen one can get a database secret and automatically inject the secrets to the app, or create dns rules and automatically update ingress config 3. Configuration management is complicated for local development, in the best case team write short script to fetch data from vault and generate Env files, even rarely they use telepresence or mirrord, it doesn’t show devs how easy can it be to reproduce a bug from staging 4. Observability if a huge pain point every team solves on its own way, while I’m convinced it’s possible to give a default solution from service mesh + OpenAPI creating metrics group on every given api route and alert on designated error rate I think I listed not all the problems, but my biggest pain point I’m busy to solve in my current project.

Hey @Candyboober! A few things did spring to my mind 1. JSON schema will work for YAML as well - there is even a YAML specific addendum from draft 4 onwards 2. If you want those capabilities beyond the schema, then your platform that you address via the spec needs to support it. If you pair Score with the Humanitec orchestrator, then you have exactly that functionality OOTB. 3. Same as for 2 - fully functional local environments or ephemeral ones in the cloud. 4. Injecting cross-cutting concerns like observability or costing in the same way that you would work with meta-programming is also a functionality an orchestrator will give you. My best guess here is, that most OSS projects rather focus on the needs of smaller constructs / their own domain, while commercial solutions have those integration patterns needed for scaling already integrated.