Has anyone tried running some sort of NPS Slack bot to gather feedback from engineers on developer experience in the company?

Yes. Highly successful. https://getdx.com/

Yes. And several other methods such as imbedding NPS into the tools themselves and non-NPS surveys. The primary issue we ran into was a very low response rate, <20% for 1000-3500 engineers. It declined over time (quarterly) until some steady state of low single digits. What we found more successful was 1:1 customer engagement interviews. It gave more actionable feedback but cost the team more time to ensure we got a broad range of engineers.

We achieved 90+% response rate over the 4 quarters I ran the DX survey linked above. (thousands of engineers)

We used DX for a couple quarters. It works, and we got some good insights, but it is pretty expensive to keep it running year round (for us).

These are pretty broad which was our teams' attempts to validate intended direction (some questions are leading) and identify if there were any hotspot areas. I took a pass and I think they're still pretty general for any org going through a Platform journey.