Any votes for preferred experimentation platform vendors?

Any idea what types of experiments you’d want to run? Is this mostly A/B tests or something else?

certainly including that but ideally the tool/platform would support more than just a/b -- multivariate, a/a, etc.

Gotcha. I’ll say we (Honeycomb) really like LaunchDarkly, which supports a/b and multivariate testing in addition to regular feature flag management. Their caching/reliability strategy seems to work very well, and we’ve basically never noticed downtime (the most common failure mode would just be serving stale flags) so we feel fairly comfortable using them for experiments in critical-path components. Their SDKs are easy to work with and we are able to use them for frontend, backend, and infra-y flags & experiments. I think the experimentation feature set was added later in the product’s history, and it doesn’t feel as rich as in some products, but the essentials are there. It’s also a tool that feels built for engineers, vs. lots of testing tools that feel built for e.g. marketing or a team that doesn’t deal with code very much.