Prevent Elasticsearch from message flood


#1

Hello

We use Elasticsearch in our elk setup. Logstash is sending the logs via http to Elasticsearch. I wonder is there a way to protect Elasticsearch going down from unexpected high message load?

Cheers
Chris


Throttle incoming logs in a smart way
(Mark Walkom) #2

Use a broker - redis, kafkfa, MQ.
Send events to the broker and then to ES, that way they can queue up in the broker if there is an influx and you process them into ES at your own pace.


(system) #3