I have an application (run by a Windows Service) which connects to a TCP server (let’s call the service ‘Listener’ from here on).
[It’s important to note that the TCP server is out of my reach, and should be considered as an external component that cannot be changed in regard to the following question]
The TCP Server pushes data to any client that is connected to it, and this data starts a chain of complex business logic, which is very demanding.
In order to improve reliability, I would like to run several Listeners, and distribute the incoming data between them.
They share a distributed cache (Redis, if it matters).
How would you design an algorithm that is effective, fast (as time is of the essence) and the different Listeners should not be dependent on one another (should one of them fail, the algorithm should still work)?
It is also important that no server data goes down the drain (e.g. It shouldn’t be possible for no Listener to deal with server-sent data)
I was thinking in the direction of Leader Election based on random numbers – am I in the right direction?
[Listener is written in C#]
0
TCP is not a great protocol for this kind of thing.
I would suggest a middle layer. A single TCP listener that receives all messages and puts them on a queue, where “a queue” means any queuing system that allows distribution of messages across multiple subscribers.
RabbitMQ is pretty easy to implement in .NET and very flexible for your needs. Work Queues are the concept you’re looking for.
2