Has anyone had experience implementing load balancing solution for RabbitMQ or Redis cluster using Nginx or HAProxy? What would be your preferred choice for a LB, Nginx or HAProxy and why?

Nginx can only do web/http traffic as far as I know. HAProxy can do many protocols. I don't know what protocols RebbitMQ or Redis use off the top of my head.

@David L thank you for the feedback.

Don’t try to load-balance in front of Redis for any other reason than geo-redundant HA failover. If you need high-performance beyond what Redis OSS can offer, then your best bet is to for Redis Enterprise, that is able to handle that Redis internally for you - while providing the single-node Redis interface and as such zero hassle for your devs.

@Clemens Jütte thank you the feedback. The main reasons why I would like to use LB in front of RabbitMQ and Redis cluster is to decouple client apps from RabbitMQ and Redis infrastructure and to distribute load across all nodes in the cluster. I am not yet sure how would load distribution work with LB in front of a RabbitMQ cluster because, by default, all requests are handled by the primary node. Regarding Redis cluster I already have proof of concept implemented that distributes read requests to all nodes in a cluster. In case of write requests they would be distributed to primary nodes only. Unfortunately Redis Enterprise is not an option for me.

@Zoran Radovanovic since you keep saying cluster, I guess you’re running an instance of this here for Redis OSS cluster: https://redis.io/docs/management/scaling/ That would normally translate into a strategy of sharding your keyspace in an elegant way to distribute your data across nodes. A cluster-aware client will discover that topology automatically and communicate with the right nodes to access the data they contain. In that scenario the introduction of a LB makes almost no sense. But what you’re stating as a test scenario makes me believe you’re running a Sentinel setup - which is no cluster. I am confused 🙂

Thank you @Clemens Jütte. I agree with you that a client library with cluster-aware support would be enough to utilize all nodes in a cluster. I am just evaluating different architectures, I am not running Redis cluster in production yet.