Hey folks - sharing here in case it would help others too! we just open-sourced
Viteval, a JS/TS framework for running LLM evals 🚀
It works just like Vitest, so you can run evals locally or in CI without changing how you write tests.
We built it after getting frustrated with existing options:
• Framework lock-in (we didn’t want to live inside LangChain just to run tests)
• JS support always felt like an afterthought
• Burning tokens every run because datasets keep regenerating 🤦
With Viteval you get:
✅ Powered by Vitest (a runner you already know)
✅ Write evals like unit tests, locally or in CI
✅ Save & reuse datasets → no more paying for repeats
Hope this helps your workflow too! 🙌