This message was deleted.

What languages are you building for? What kinds of artifacts? A lot of folks are looking at Dagger for builds as well as driving tests and even deploys. Dagger is pipelines as code (e.g. Go, Node.js, Python) that run in containers the same locally as in CI. Caching works out of the box locally for a fast inner dev loop. There are some other great options for just build depending on your language/framework.

We are mostly building Go and TypeScript with a sprinkling of Python. Artifacts are by in large Docker images, or a lambda binary which is stored in S3. We distribute some binaries for CLI type usage but these are more rare and not the main focus. I've been looking into dagger and it seems intriguing. What has been your experience with hosting it and integrating it with other parts of your build system? Is observability of individual steps of the build process easy (things like duration)? If a certain step is slow or failing is it easy to immediately understand that from the system output? Has cacheing being easy to implement? Is the community support good? When you get stuck, are there easy resources to learn more?

@Andrew Marine thanks for sharing! For full disclosure, I work at Dagger 🙂 and would love to chat more. Sounds like you’ve probably learned a lot on your own, but I’ll put some resources in here. I’ll DM you to show you a demo of the observability features in private beta 🕵️ There are Dagger SDKs for writing your Pipelines as code in Go, Node.js (TS/JS), or Python. Since Dagger has a GraphQL API behind everything, it’s relatively easy to add new SDKs. Some folks are working on more SDKs for Rust, etc. (or vote in SDK survey!) https://docs.dagger.io/593914/quickstart-hello Since everything is code running in containers, you could choose the Go SDK, for example, and then use it to write pipelines that build images or binaries for Go, TypeScript, or Python projects (or anything, really). And you can pull in any libraries you need for interacting with clouds or secrets providers or whatever. You can also create reusable code libraries with your Dagger code, of course. https://github.com/dagger/examples/tree/main/go (some simple examples of Go SDK building Go, Java, Node.js projects) https://docs.dagger.io/544174/multistage-build (example of a multi-stage container build--published to a local registry in the case--using nice chaining dot syntax in the

builder

image and

prodImage

). Sticking with the Go SDK example, It’s super easy to run your pipelines locally as just

go run ci.go

(or compile that Go pipeline into a static binary your team can run from

npm

or `yarn`or build an internal CLI, or whatever. It’s just code!). Then you can run the same thing from CI. Whether it’s CircleCI, GitHub Actions, GitLabCI, or Jenkins (or whatever else) you’d just have your CI run

go run ci.go

. https://docs.dagger.io/768421/go-ci Caching “just works” when you run locally which makes for a super rapid local dev loop where you can get everything working without dozens of “fix CI” or “fix typo” pushes to git/CI with multi-minute wait times per push. Once your pipeline is passing locally, it’s going to run just the same in CI, so you can avoid the “it only works in CI” or “it only works on my laptop” problem. (Fun community video on this topic 😆 English dub coming soon). For caching in CI, you can have a build host with a single cache, or with ephemeral CI runners, you can save the cache between runs (e.g. to blob storage). Many users rely on automatic caching locally and then keep it simple and don’t cache in CI since they’re not wasting time on a lot of failures. The observability features will make this decision easier. The community Discord is super active and both Dagger staff and community members help out. https://dagger.io/ (Discord link at top) There are regular community calls with demos showing, for example, how a user has built an internal platform with Dagger. https://dagger.io/events https://www.youtube.com/playlist?list=PLyHqb4A5ee1tEgcr7KsNFzQSPN-R3fPs2 (just the demos from Community Calls) The docs and blog posts are getting better and better 🙂 https://docs.dagger.io/ https://dagger.io/blog Hope this helps! Happy to chat any time.

Wow this is great info! Thanks so much for sharing.