Hi, I have a 5 node cluster that I'm using as part of an ELK system. Most
of the time it works great but today we saw a spike in writes for one of
the nodes and around the same time we saw indexing on that node spike up
too -- which makes sense, if you have more writes you'll need to do more
indexing. But none of the other servers were all that taxed, we don't have
anything else writing to Elasticsearch other than Logstash and normally it
does a pretty good job of load balancing. Any idea where I could start
looking for clues? I was looking through the logs but there doesn't seem to
be much information in there, most of it are just debug errors that say
something like this: [logstash-2015.02.13][3] failed to execute bulk item
(index) index {[logstash-2015.02.13] but they show up pretty consistently
so it doesn't seem like anything to be worried about. Where else could I
look to see why only one server is getting all the writes and how do I
mitigate it if this is happening because it ended up making Elasticsearch
unresponsive to information being sent from Logstash.
Hi, I have a 5 node cluster that I'm using as part of an ELK system. Most
of the time it works great but today we saw a spike in writes for one of
the nodes and around the same time we saw indexing on that node spike up
too -- which makes sense, if you have more writes you'll need to do more
indexing. But none of the other servers were all that taxed, we don't have
anything else writing to Elasticsearch other than Logstash and normally it
does a pretty good job of load balancing. Any idea where I could start
looking for clues? I was looking through the logs but there doesn't seem to
be much information in there, most of it are just debug errors that say
something like this: [logstash-2015.02.13][3] failed to execute bulk item
(index) index {[logstash-2015.02.13] but they show up pretty consistently
so it doesn't seem like anything to be worried about. Where else could I
look to see why only one server is getting all the writes and how do I
mitigate it if this is happening because it ended up making Elasticsearch
unresponsive to information being sent from Logstash.
On Monday, February 16, 2015 at 1:43:59 AM UTC-6, Mark Walkom wrote:
Can you link us to your LS config?
On 14 February 2015 at 08:56, rhea ghosh <rhea....@gmail.com <javascript:>
wrote:
Hi, I have a 5 node cluster that I'm using as part of an ELK system. Most
of the time it works great but today we saw a spike in writes for one of
the nodes and around the same time we saw indexing on that node spike up
too -- which makes sense, if you have more writes you'll need to do more
indexing. But none of the other servers were all that taxed, we don't have
anything else writing to Elasticsearch other than Logstash and normally it
does a pretty good job of load balancing. Any idea where I could start
looking for clues? I was looking through the logs but there doesn't seem to
be much information in there, most of it are just debug errors that say
something like this: [logstash-2015.02.13][3] failed to execute bulk item
(index) index {[logstash-2015.02.13] but they show up pretty consistently
so it doesn't seem like anything to be worried about. Where else could I
look to see why only one server is getting all the writes and how do I
mitigate it if this is happening because it ended up making Elasticsearch
unresponsive to information being sent from Logstash.
Hi, I have a 5 node cluster that I'm using as part of an ELK system.
Most of the time it works great but today we saw a spike in writes for one
of the nodes and around the same time we saw indexing on that node spike up
too -- which makes sense, if you have more writes you'll need to do more
indexing. But none of the other servers were all that taxed, we don't have
anything else writing to Elasticsearch other than Logstash and normally it
does a pretty good job of load balancing. Any idea where I could start
looking for clues? I was looking through the logs but there doesn't seem to
be much information in there, most of it are just debug errors that say
something like this: [logstash-2015.02.13][3] failed to execute bulk item
(index) index {[logstash-2015.02.13] but they show up pretty consistently
so it doesn't seem like anything to be worried about. Where else could I
look to see why only one server is getting all the writes and how do I
mitigate it if this is happening because it ended up making Elasticsearch
unresponsive to information being sent from Logstash.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.