We're deploying ES for logstash and I recently setup the cluster to migrate
older indexes to "slower" nodes for longer term storage. The nodes are not
being queried or doing any indexing but CPU is at 75% constantly. Here is
output from hot_threads , jstack and some settings
$ java -version
java version "1.7.0_65"
OpenJDK Runtime Environment (IcedTea 2.5.1) (7u65-2.5.1-4ubuntu1~0.12.04.2)
OpenJDK 64-Bit Server VM (build 24.65-b04, mixed mode)
I just realized while posting this my index buffers are set too high for
these nodes since they arent doing indexing, but just in case that's not
the issue, I'll post anyway. I'll report back if issue is still present
Looks like you have a monitoring tool running and it got stuck in the node
stats call while traversing a number of shards/segments.
How many shards/segments are in your migration? It seems to be very active.
Maybe the bloom filter format conversion is expensive but I am not sure.
Jörg
On Sat, Sep 13, 2014 at 12:10 AM, Justin Lintz jlintz@gmail.com wrote:
Hi,
We're deploying ES for logstash and I recently setup the cluster to
migrate older indexes to "slower" nodes for longer term storage. The nodes
are not being queried or doing any indexing but CPU is at 75% constantly.
Here is output from hot_threads , jstack and some settings
$ java -version
java version "1.7.0_65"
OpenJDK Runtime Environment (IcedTea 2.5.1) (7u65-2.5.1-4ubuntu1~0.12.04.2)
OpenJDK 64-Bit Server VM (build 24.65-b04, mixed mode)
I just realized while posting this my index buffers are set too high for
these nodes since they arent doing indexing, but just in case that's not
the issue, I'll post anyway. I'll report back if issue is still present
Thanks, it ended up being the Ganglia plugin that was causing crazy CPU
consumption. I've disabled it since we'll end up buying Marvel once we've
fully deployed.
On Friday, September 12, 2014 6:36:27 PM UTC-4, Jörg Prante wrote:
Looks like you have a monitoring tool running and it got stuck in the node
stats call while traversing a number of shards/segments.
How many shards/segments are in your migration? It seems to be very active.
Maybe the bloom filter format conversion is expensive but I am not sure.
Jörg
On Sat, Sep 13, 2014 at 12:10 AM, Justin Lintz <jli...@gmail.com
<javascript:>> wrote:
Hi,
We're deploying ES for logstash and I recently setup the cluster to
migrate older indexes to "slower" nodes for longer term storage. The nodes
are not being queried or doing any indexing but CPU is at 75% constantly.
Here is output from hot_threads , jstack and some settings
$ java -version
java version "1.7.0_65"
OpenJDK Runtime Environment (IcedTea 2.5.1)
(7u65-2.5.1-4ubuntu1~0.12.04.2)
OpenJDK 64-Bit Server VM (build 24.65-b04, mixed mode)
I just realized while posting this my index buffers are set too high for
these nodes since they arent doing indexing, but just in case that's not
the issue, I'll post anyway. I'll report back if issue is still present
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.