Monitor incoming data rates

Elastic Stack Elasticsearch
1

I have alerts configured on my Logstash - Elasticsearch setup which perform
regular queries to see what state the cluster is in etc.

Recently I had a situation where Logstash was running OK and ES cluster was
in Green state, but there was no data going into ES.
Is there a way to monitor this, e.g if the incoming data rate to ES drops
below 100 events a minute.

I can't do it by Logstash metrics because that side of it was running
normally, i.e. no change in data rates.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/2458951a-5c82-4003-841a-d634aa1c1dfd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

Marvel monitors these sorts of things out of the box.

But maybe _cluster/stats and keep an eye on docs.count? What is monitoring
things now?

On 9 April 2015 at 13:49, Bernie Carolan bernie.carolan@gmail.com wrote:

I have alerts configured on my Logstash - Elasticsearch setup which
perform regular queries to see what state the cluster is in etc.

Recently I had a situation where Logstash was running OK and ES cluster
was in Green state, but there was no data going into ES.
Is there a way to monitor this, e.g if the incoming data rate to ES drops
below 100 events a minute.

I can't do it by Logstash metrics because that side of it was running
normally, i.e. no change in data rates.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/2458951a-5c82-4003-841a-d634aa1c1dfd%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/2458951a-5c82-4003-841a-d634aa1c1dfd%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEYi1X8xhf2bKbH73eY-EMkoUXbShbBe%3Db%2Bg9n%2BaCyzoBqOvXw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

On Thursday, April 09, 2015 at 05:49 CEST,
Bernie Carolan bernie.carolan@gmail.com wrote:

I have alerts configured on my Logstash - Elasticsearch setup which
perform regular queries to see what state the cluster is in etc.
Recently I had a situation where Logstash was running OK and ES
cluster was in Green state, but there was no data going into ES.
Is there a way to monitor this, e.g if the incoming data rate to ES
drops below 100 events a minute.
I can't do it by Logstash metrics because that side of it was running
normally, i.e. no change in data rates.

How can Logstash be running normally when it halts the pipeline when one
output isn't able to accept messages? Are you saying that Elasticsearch
accepts messages but just drops them on the floor?

--
Magnus Bäck | Software Engineer, Development Tools
magnus.back@sonymobile.com | Sony Mobile Communications

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/20150409061248.GA13186%40seldlx20533.corpusers.net.
For more options, visit https://groups.google.com/d/optout.