Hi all, we are building an elasticsearch stack currently but hitting some
uncertainty whether to include Redis or not.
I am writing up an argument for it's inclusion and hoped to gain additional
insight from this experienced community.
We are using elasticsearch specifically for indexing various system log
files, and may generate as much as 1gb per hour.
So far, I have made points regarding improved scalability by segmenting
logstash and basically using Redis as a fancy buffer.
Also, I have made the point about flexibility should we ever bring
ElasticSearch down that we continue to capture logs during any downtime.
And the point that we can attach parallel services to either side of Redis.
Are there additional advantage or drawbacks that I should be aware of?
Thanks so much for any insight, thoughts or pointers to relevant existing
conversations on this topic.
Not sure how much you want to scale....but something like a rabbitmq
cluster seems, to me, far more scalable than redis, for this use case, and
with better data safety rules...
You could of course have multiple redis and hash across the cluster but it
is harder to scale up and down as needed.
Throughput will likely be higher with redis on a simple implementation...
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.