Hi there,
I have followed the standard way to try set up my ELK system, where in my configuration, I have 3 hosts:
a. HOST1 is where my LogStash Shipper is running.
b. HOST2 is where my Redis is running.
c. HOST3 is where my LogStash Index and Elasticsearch sever is running.
My questions are:
- Redis-server ( version 3.0.7) always stopped after a period time of running, and I could not find any valuable reason for that. Does someone have any suggestion what could be wrong ?
- If I had the Redis restarted, do I have to restart LogStash also ? on both Shipper and Indexer side ? Because otherwise, I don't see any moving in the shipper and indexer log, and no new index are being created.
- For my case, on shipper side, the shipper is trying to read the logs from a directory so called ".../logAnalysis/...", under which there are list logs dir from each failed job, and it will be there for some time, before it is completely deleted. So My goal is to find the EARLIEST failure from individual one, not necessary all the failure details, but the earliest piece that may cause the rest failure and have them reported.
My understanding is I can running the shipper command in this way, "logstash -f shipper.conf -w 10 ...", so that 10 jobs' log dirs are being analyzed in parallel, is that correct ? If so, what is the max value I may use here ?
Thanks,
Chun