I see lots of tutorials like this one: http://www.everybodyhertz.co.uk/setting-up-a-relk-stack-a-how-to/
suggesting that you use logstash forwarder to send logs to redis before sending it to logstash and then to elasticsearch. I'd like to know if it is possible to send from rsyslog/syslog to redis directly without logstash forwarder then from redis to logstash and then to elasticsearch.
What I'd like to do:
What other people are doing:
The reason I'd like to send it directly to redis is I'd like to be able to cache data in a way so that if logstash crashed then I would still have the data sent from rsyslog to logstash in that short while that logstash was down.
Also is redis the right choice for this. Would it be possible to use varnish for this purpose instead?