Facing error and warning messages with ELK stack

Elastic Stack Elasticsearch
1

I have a setup of ELK stack with redis as a broker and a cluster of 2
Elasticsearch Nodes. The stack was running good and suddenly I started
facing the following error and warning messages at different levels of the
stack.

http://pastebin.com/eG7p0PCc

If I restart the redis broker, Logstash and the Elasticsearch node showing
error message, the stack comes back, with Redis-broker performing well,
able to allocate memory but I have been seeing the same error messages for
E.S and Logstash after the restart. Because of which the redis-broker start
buffering the logs in memory and eventually goes out of memory as E.S and
logstash stop indexing the data.
I am trying out figure out what might be the possible cause of this, any
help will be really appreciated. Please let me know if there is any
confusion understanding the situation.

Thank you,
Shriyansh

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/99ac902f-c183-4700-9d32-9d23ea0d7c3c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

It looks like networking issues, lots of connection reset/closed/timeout.

It might help if you can out your configs into a pastebin too.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 20 September 2014 04:33, shriyansh jain shriyanshajain@gmail.com wrote:

I have a setup of ELK stack with redis as a broker and a cluster of 2
Elasticsearch Nodes. The stack was running good and suddenly I started
facing the following error and warning messages at different levels of the
stack.

Errors with Logstash:{:timestamp=>"2014-09-19T10:32:59.241000-0700", :messag - Pastebin.com

If I restart the redis broker, Logstash and the Elasticsearch node showing
error message, the stack comes back, with Redis-broker performing well,
able to allocate memory but I have been seeing the same error messages for
E.S and Logstash after the restart. Because of which the redis-broker start
buffering the logs in memory and eventually goes out of memory as E.S and
logstash stop indexing the data.
I am trying out figure out what might be the possible cause of this, any
help will be really appreciated. Please let me know if there is any
confusion understanding the situation.

Thank you,
Shriyansh

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/99ac902f-c183-4700-9d32-9d23ea0d7c3c%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/99ac902f-c183-4700-9d32-9d23ea0d7c3c%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624YD7tqLU2%2BuufpvhO%2BgzSZA%3DLyMFods1o5g2JxHzr2%2B-Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

Which configuration you will be interested to loo at.?

-Shriyansh

On Friday, September 19, 2014 2:17:16 PM UTC-7, Mark Walkom wrote:

It looks like networking issues, lots of connection reset/closed/timeout.

It might help if you can out your configs into a pastebin too.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: ma...@campaignmonitor.com <javascript:>
web: www.campaignmonitor.com

On 20 September 2014 04:33, shriyansh jain <shriyan...@gmail.com
<javascript:>> wrote:

I have a setup of ELK stack with redis as a broker and a cluster of 2
Elasticsearch Nodes. The stack was running good and suddenly I started
facing the following error and warning messages at different levels of the
stack.

Errors with Logstash:{:timestamp=>"2014-09-19T10:32:59.241000-0700", :messag - Pastebin.com

If I restart the redis broker, Logstash and the Elasticsearch node
showing error message, the stack comes back, with Redis-broker performing
well, able to allocate memory but I have been seeing the same error
messages for E.S and Logstash after the restart. Because of which the
redis-broker start buffering the logs in memory and eventually goes out of
memory as E.S and logstash stop indexing the data.
I am trying out figure out what might be the possible cause of this, any
help will be really appreciated. Please let me know if there is any
confusion understanding the situation.

Thank you,
Shriyansh

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/99ac902f-c183-4700-9d32-9d23ea0d7c3c%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/99ac902f-c183-4700-9d32-9d23ea0d7c3c%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/624cd024-cc69-4eba-8bf9-d05b21f8d41a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

4

which configuration's you will be interested to look at.?

-Shriyansh

On Friday, September 19, 2014 2:17:16 PM UTC-7, Mark Walkom wrote:

It looks like networking issues, lots of connection reset/closed/timeout.

It might help if you can out your configs into a pastebin too.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: ma...@campaignmonitor.com <javascript:>
web: www.campaignmonitor.com

On 20 September 2014 04:33, shriyansh jain <shriyan...@gmail.com
<javascript:>> wrote:

I have a setup of ELK stack with redis as a broker and a cluster of 2
Elasticsearch Nodes. The stack was running good and suddenly I started
facing the following error and warning messages at different levels of the
stack.

Errors with Logstash:{:timestamp=>"2014-09-19T10:32:59.241000-0700", :messag - Pastebin.com

If I restart the redis broker, Logstash and the Elasticsearch node
showing error message, the stack comes back, with Redis-broker performing
well, able to allocate memory but I have been seeing the same error
messages for E.S and Logstash after the restart. Because of which the
redis-broker start buffering the logs in memory and eventually goes out of
memory as E.S and logstash stop indexing the data.
I am trying out figure out what might be the possible cause of this, any
help will be really appreciated. Please let me know if there is any
confusion understanding the situation.

Thank you,
Shriyansh

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/99ac902f-c183-4700-9d32-9d23ea0d7c3c%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/99ac902f-c183-4700-9d32-9d23ea0d7c3c%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ca66b25e-77ab-4339-820e-74e25f5e9e7e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.