Hello, I have been monitoring the data traffic and I have notice that I'm loosing a lot of messages, What could I do? Using REDIS buffer could be a solution but it consumes a lot of memory and I would like to study other alternatives.
Thank you very much
How are you receiving the messages? What's your approximate message rate?
I'm receiving the messages via UDP (therefore is normal the message loss but I would like to minimize it)
And the message rate is not fixed but sometimes I have 5000 or 10000 msgs/s.
That's quite a lot. Setting up a broker for quick off-loading of messages should definitely help. I'd look for benchmarks of how many messages a single machine can receive under any circumstances, then add Logstash into the mix. If you can't receive at least 10k messages/s without doing anything with them (i.e. just dropping them on the floor) you have a bigger problem.