Hi, I was wondering have experience with other alternatives than using a pushgateway for collecting metrics from short-lived jobs?

You could try Graphite? Can hook that into Grafana too, instead of pushing to pushgateway into Prometheus (have run both at a previous job, but Graphite doesn't cover all the same use-cases so YMMV) https://grafana.com/oss/graphite/ https://graphite.readthedocs.io/en/latest/feeding-carbon.html

OTEL works dandy too btw, you can even insturment you scripts / jobs with it using Tracing 🙂