I have set up a logstash with redis architecture to handle my logs. The way I have organized it is:
logstash ---> redis ---> logstash ---> elasticsearch
but the problem that occurred is that after parsing nearly 1.25 million logs a java exception is thrown.
In my logstash.err log file, the exception appears as
Exception in thread "<file" java.lang.UnsupportedOperationException
at java.lang.Thread.stop(Thread.java:869)
at org.jruby.RubyThread.exceptionRaised(RubyThread.java:1221)
at org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:112)
at java.lang.Thread.run(Thread.java:745)
I think that this exception might be thrown because of logstash unable to open/close a file. So what can I do to rectify this error? The way that my input configuration is set for my first logstash server to send the logs is:
input {
file {
start_position => "beginning"
path => [
"/var/logstash_logs/child1/nginx/*log*",
"/var/logstash_logs/child2/nginx/*log*",
"/var/logstash_logs/child3/nginx/*log*"
]
}
}
And the way output is sent is like this:
output {
redis {
host => "X.X.X.X"
key => "logstash"
data_type => "list"
}
}
There are no errors in the logs of logstash server with redis installed. Thanks in advance for your help! 