Avoid to lose data with full restart cluster

When a full restart cluster is needed, what is the best solution to avoid to lose data?
For example , if many server send syslog to logstash and logstash sends logs to elasticsearch, when I restart elasticsearch I lose data?

Personnaly I use a rsyslog before my logstash server and then ship the logs with either the file input from logstash or filebeat.

This syslog server can be used as a buffer for example if you have to reboot your logstash instance.

I would recommend to delete the log file from the server after it has been ingested by logstash.

Another option that I saw would be to use Redis but I don't know about this setup as I never used it.

You can also find a lot of example configuration on the net for this setup as is it pretty common and I would assume the best practice.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.