# High CPU usage of elasticsearch **URL:** \ **Category:** Elasticsearch\ **Created:** [January 1, 2015, 6:09pm UTC](https://discuss.elastic.co/t/high-cpu-usage-of-elasticsearch/21450 "2015-01-01T18:09:51Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Danishka\_Navin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danishka_navin/32/1013_2.png) [@Danishka\_Navin](https://discuss.elastic.co/u/Danishka_Navin)\ **Post date:** [January 1, 2015, 6:09pm UTC](https://discuss.elastic.co/t/high-cpu-usage-of-elasticsearch/21450/1 "2015-01-01T18:09:51Z")
Hi, I am new to elasticsearch and logstash. using elasticsearch-1.1.1 and logstash-1.4.2-1 with Kibana. Its a single node with 4 vCPU and 30GB of physical memory. Currently logstash (single node) receive logs from 40 jboss servers. Most of the time elasticsearch use almost all the CPU resource. Is there any way I can limit the CPU consumption by tuning? PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 12216 elastics 20 0 514g 19g 3.6g S 389.8 63.5 3224:32 /usr/bin/java -Xms15g -Xmx15g -Xss256k -Djava.awt.headles 11722 logstash 39 19 3443m 1.2g 6496 S 8.6 3.9 2037:27 /usr/bin/java -Djava.io.tmpdir=/var/lib/logstash -Xmx1g -X Any recommendations? I already followed following article. > **[ElasticSearch and Logstash Tuning](https://jablonskis.org/2013/elasticsearch-and-logstash-tuning/index.html)** > > I was slightly familiar with elasticsearch and logstash before at a very > minimum level. But just a couple of days ago I had a chance to play with both > toys at a larger scale. I was given a box with elasticsearch, redis and > logstash already running,... Appreciate your help. Thanks, Danishka -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/e623e1a9-2a59-4b2f-bce0-11c8d2006c54%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/e623e1a9-2a59-4b2f-bce0-11c8d2006c54%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [January 1, 2015, 9:49pm UTC](https://discuss.elastic.co/t/high-cpu-usage-of-elasticsearch/21450/2 "2015-01-01T21:49:03Z")
How much data do you have on the node? How many indexes? Have you checked the logs for GC issues? You can use nice on the OS level to manage CPU use, but it's not a good idea, and instead you should figure out why ES is using that CPU. On 2 January 2015 at 05:09, Danishka Navin [danishka@gmail.com](mailto:danishka@gmail.com) wrote: > Hi, > > I am new to elasticsearch and logstash. > using elasticsearch-1.1.1 and logstash-1.4.2-1 with Kibana. > Its a single node with 4 vCPU and 30GB of physical memory. > > Currently logstash (single node) receive logs from 40 jboss servers. > > Most of the time elasticsearch use almost all the CPU resource. > Is there any way I can limit the CPU consumption by tuning? > > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > 12216 elastics 20 0 514g 19g 3.6g S 389.8 63.5 3224:32 > /usr/bin/java -Xms15g -Xmx15g -Xss256k -Djava.awt.headles > 11722 logstash 39 19 3443m 1.2g 6496 S 8.6 3.9 2037:27 /usr/bin/java > -Djava.io.tmpdir=/var/lib/logstash -Xmx1g -X > > Any recommendations? > > I already followed following article. > [ElasticSearch and Logstash Tuning – Vaidas Jablonskis](http://jablonskis.org/2013/elasticsearch-and-logstash-tuning/index.html) > > Appreciate your help. > > Thanks, > Danishka > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/e623e1a9-2a59-4b2f-bce0-11c8d2006c54%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/e623e1a9-2a59-4b2f-bce0-11c8d2006c54%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/e623e1a9-2a59-4b2f-bce0-11c8d2006c54%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/e623e1a9-2a59-4b2f-bce0-11c8d2006c54%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X\_9VizVn4m7C61Xo8GBakRfa00iQV5WiAKhdGHRWAic1g%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_9VizVn4m7C61Xo8GBakRfa00iQV5WiAKhdGHRWAic1g%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![crimondi](https://avatars.discourse-cdn.com/v4/letter/c/f05b48/32.png) [@crimondi](https://discuss.elastic.co/u/crimondi)\ **Post date:** [January 2, 2015, 6:26pm UTC](https://discuss.elastic.co/t/high-cpu-usage-of-elasticsearch/21450/3 "2015-01-02T18:26:00Z")
I agree with what Mark said. Nice will be just masking a deeper issue. Have you tried looking at hot threads? > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. Also, if you are seeing CPUs sustained at 100% CPU that seems like old GCs that just are never finishing. So check the GC logs. Do you have any idea on the number of events per second you are trying to index and size of the events? If you are using logstash and Redis is the queue backing up because it can't index? On Thu, Jan 1, 2015 at 4:49 PM, Mark Walkom [markwalkom@gmail.com](mailto:markwalkom@gmail.com) wrote: > How much data do you have on the node? How many indexes? Have you checked > the logs for GC issues? > > You can use nice on the OS level to manage CPU use, but it's not a good > idea, and instead you should figure out why ES is using that CPU. > > On 2 January 2015 at 05:09, Danishka Navin [danishka@gmail.com](mailto:danishka@gmail.com) wrote: > > > Hi, > > > > I am new to elasticsearch and logstash. > > using elasticsearch-1.1.1 and logstash-1.4.2-1 with Kibana. > > Its a single node with 4 vCPU and 30GB of physical memory. > > > > Currently logstash (single node) receive logs from 40 jboss servers. > > > > Most of the time elasticsearch use almost all the CPU resource. > > Is there any way I can limit the CPU consumption by tuning? > > > > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > > 12216 elastics 20 0 514g 19g 3.6g S 389.8 63.5 3224:32 > > /usr/bin/java -Xms15g -Xmx15g -Xss256k -Djava.awt.headles > > 11722 logstash 39 19 3443m 1.2g 6496 S 8.6 3.9 2037:27 > > /usr/bin/java -Djava.io.tmpdir=/var/lib/logstash -Xmx1g -X > > > > Any recommendations? > > > > I already followed following article. > > [ElasticSearch and Logstash Tuning – Vaidas Jablonskis](http://jablonskis.org/2013/elasticsearch-and-logstash-tuning/index.html) > > > > Appreciate your help. > > > > Thanks, > > Danishka > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/e623e1a9-2a59-4b2f-bce0-11c8d2006c54%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/e623e1a9-2a59-4b2f-bce0-11c8d2006c54%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/e623e1a9-2a59-4b2f-bce0-11c8d2006c54%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/e623e1a9-2a59-4b2f-bce0-11c8d2006c54%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X\_9VizVn4m7C61Xo8GBakRfa00iQV5WiAKhdGHRWAic1g%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_9VizVn4m7C61Xo8GBakRfa00iQV5WiAKhdGHRWAic1g%40mail.gmail.com) > [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X\_9VizVn4m7C61Xo8GBakRfa00iQV5WiAKhdGHRWAic1g%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_9VizVn4m7C61Xo8GBakRfa00iQV5WiAKhdGHRWAic1g%40mail.gmail.com?utm_medium=email&utm_source=footer) > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- Chris Rimondi | [http://twitter.com/crimondi](http://twitter.com/crimondi) | [securitygrit.com](http://securitygrit.com) -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CA%2BqatLgnXX7Rp7D%2Bmn5XNOfZqe4Ko706EV9CxPYcFL%3DA11mR7A%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CA%2BqatLgnXX7Rp7D%2Bmn5XNOfZqe4Ko706EV9CxPYcFL%3DA11mR7A%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Danishka\_Navin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danishka_navin/32/1013_2.png) [@Danishka\_Navin](https://discuss.elastic.co/u/Danishka_Navin)\ **Post date:** [January 2, 2015, 9:18pm UTC](https://discuss.elastic.co/t/high-cpu-usage-of-elasticsearch/21450/4 "2015-01-02T21:18:00Z")
Hi, Here is stats of my single cluster [http://fpaste.org/165093/20232576/](http://fpaste.org/165093/20232576/) Elasticsearch configuration [http://fpaste.org/165092/14202325/](http://fpaste.org/165092/14202325/) I don't use Redis. There were 20+ million events per 10minutes (Attached screen-dump of events over time) On Sat, Jan 3, 2015 at 2:26 AM, Christopher Rimondi \<[chris.rimondi@gmail.com](mailto:chris.rimondi@gmail.com) > wrote: > I agree with what Mark said. Nice will be just masking a deeper issue. > Have you tried looking at hot threads? > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-nodes-hot-threads.html) > Also, if you are seeing CPUs sustained at 100% CPU that seems like old GCs > that just are never finishing. So check the GC logs. > > Do you have any idea on the number of events per second you are trying to > index and size of the events? If you are using logstash and Redis is the > queue backing up because it can't index? > > On Thu, Jan 1, 2015 at 4:49 PM, Mark Walkom [markwalkom@gmail.com](mailto:markwalkom@gmail.com) wrote: > > > How much data do you have on the node? How many indexes? Have you checked > > the logs for GC issues? > > > > You can use nice on the OS level to manage CPU use, but it's not a good > > idea, and instead you should figure out why ES is using that CPU. > > > > On 2 January 2015 at 05:09, Danishka Navin [danishka@gmail.com](mailto:danishka@gmail.com) wrote: > > > > > Hi, > > > > > > I am new to elasticsearch and logstash. > > > using elasticsearch-1.1.1 and logstash-1.4.2-1 with Kibana. > > > Its a single node with 4 vCPU and 30GB of physical memory. > > > > > > Currently logstash (single node) receive logs from 40 jboss servers. > > > > > > Most of the time elasticsearch use almost all the CPU resource. > > > Is there any way I can limit the CPU consumption by tuning? > > > > > > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > > > 12216 elastics 20 0 514g 19g 3.6g S 389.8 63.5 3224:32 > > > /usr/bin/java -Xms15g -Xmx15g -Xss256k -Djava.awt.headles > > > 11722 logstash 39 19 3443m 1.2g 6496 S 8.6 3.9 2037:27 > > > /usr/bin/java -Djava.io.tmpdir=/var/lib/logstash -Xmx1g -X > > > > > > Any recommendations? > > > > > > I already followed following article. > > > [ElasticSearch and Logstash Tuning – Vaidas Jablonskis](http://jablonskis.org/2013/elasticsearch-and-logstash-tuning/index.html) > > > > > > Appreciate your help. > > > > > > Thanks, > > > Danishka > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/e623e1a9-2a59-4b2f-bce0-11c8d2006c54%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/e623e1a9-2a59-4b2f-bce0-11c8d2006c54%40googlegroups.com) > > > [https://groups.google.com/d/msgid/elasticsearch/e623e1a9-2a59-4b2f-bce0-11c8d2006c54%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/e623e1a9-2a59-4b2f-bce0-11c8d2006c54%40googlegroups.com?utm_medium=email&utm_source=footer) > > > . > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X\_9VizVn4m7C61Xo8GBakRfa00iQV5WiAKhdGHRWAic1g%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_9VizVn4m7C61Xo8GBakRfa00iQV5WiAKhdGHRWAic1g%40mail.gmail.com) > > [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X\_9VizVn4m7C61Xo8GBakRfa00iQV5WiAKhdGHRWAic1g%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_9VizVn4m7C61Xo8GBakRfa00iQV5WiAKhdGHRWAic1g%40mail.gmail.com?utm_medium=email&utm_source=footer) > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > Chris Rimondi | [http://twitter.com/crimondi](http://twitter.com/crimondi) | [securitygrit.com](http://securitygrit.com) > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CA%2BqatLgnXX7Rp7D%2Bmn5XNOfZqe4Ko706EV9CxPYcFL%3DA11mR7A%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CA%2BqatLgnXX7Rp7D%2Bmn5XNOfZqe4Ko706EV9CxPYcFL%3DA11mR7A%40mail.gmail.com) > [https://groups.google.com/d/msgid/elasticsearch/CA%2BqatLgnXX7Rp7D%2Bmn5XNOfZqe4Ko706EV9CxPYcFL%3DA11mR7A%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CA%2BqatLgnXX7Rp7D%2Bmn5XNOfZqe4Ko706EV9CxPYcFL%3DA11mR7A%40mail.gmail.com?utm_medium=email&utm_source=footer) > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- Danishka Navin > **[Danishka's Diary](http://danishkanavin.blogspot.com)** [http://twitter.com/danishkanavin](http://twitter.com/danishkanavin) [![Imgur](https://us1.discourse-cdn.com/elastic/original/3X/4/e/4e1d7fd95311b3a0453cd3c7e3a83bfd359ec27d.jpeg "Danishka Navin") ](https://www.flickr.com/photos/danishkanavin/) -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKjxiWRAWCRVa0Y2OMN4vcscUzDqMvuH5daWhvnc9BMudcs5XA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKjxiWRAWCRVa0Y2OMN4vcscUzDqMvuH5daWhvnc9BMudcs5XA%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [January 2, 2015, 10:34pm UTC](https://discuss.elastic.co/t/high-cpu-usage-of-elasticsearch/21450/5 "2015-01-02T22:34:19Z")
You're just hitting limits of your node. Drop some data, add more nodes or more heap are pretty much the options you have. Upgrade to 1.4.2 while you are at it. Setting indices.memory.index\_buffer\_size so high probably isn't a good idea unless you know what it does, if you have such a high index rate then look at adding more nodes to spread the load. On 3 January 2015 at 08:18, Danishka Navin [danishka@gmail.com](mailto:danishka@gmail.com) wrote: > Hi, > > Here is stats of my single cluster > [http://fpaste.org/165093/20232576/](http://fpaste.org/165093/20232576/) > > Elasticsearch configuration > [http://fpaste.org/165092/14202325/](http://fpaste.org/165092/14202325/) > > I don't use Redis. > There were 20+ million events per 10minutes > > (Attached screen-dump of events over time) > > On Sat, Jan 3, 2015 at 2:26 AM, Christopher Rimondi \< > [chris.rimondi@gmail.com](mailto:chris.rimondi@gmail.com)\> wrote: > > > I agree with what Mark said. Nice will be just masking a deeper issue. > > Have you tried looking at hot threads? > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-nodes-hot-threads.html) > > Also, if you are seeing CPUs sustained at 100% CPU that seems like old GCs > > that just are never finishing. So check the GC logs. > > > > Do you have any idea on the number of events per second you are trying to > > index and size of the events? If you are using logstash and Redis is the > > queue backing up because it can't index? > > > > On Thu, Jan 1, 2015 at 4:49 PM, Mark Walkom [markwalkom@gmail.com](mailto:markwalkom@gmail.com) wrote: > > > > > How much data do you have on the node? How many indexes? Have you > > > checked the logs for GC issues? > > > > > > You can use nice on the OS level to manage CPU use, but it's not a good > > > idea, and instead you should figure out why ES is using that CPU. > > > > > > On 2 January 2015 at 05:09, Danishka Navin [danishka@gmail.com](mailto:danishka@gmail.com) wrote: > > > > > > > Hi, > > > > > > > > I am new to elasticsearch and logstash. > > > > using elasticsearch-1.1.1 and logstash-1.4.2-1 with Kibana. > > > > Its a single node with 4 vCPU and 30GB of physical memory. > > > > > > > > Currently logstash (single node) receive logs from 40 jboss servers. > > > > > > > > Most of the time elasticsearch use almost all the CPU resource. > > > > Is there any way I can limit the CPU consumption by tuning? > > > > > > > > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > > > > 12216 elastics 20 0 514g 19g 3.6g S 389.8 63.5 3224:32 > > > > /usr/bin/java -Xms15g -Xmx15g -Xss256k -Djava.awt.headles > > > > 11722 logstash 39 19 3443m 1.2g 6496 S 8.6 3.9 2037:27 > > > > /usr/bin/java -Djava.io.tmpdir=/var/lib/logstash -Xmx1g -X > > > > > > > > Any recommendations? > > > > > > > > I already followed following article. > > > > [ElasticSearch and Logstash Tuning – Vaidas Jablonskis](http://jablonskis.org/2013/elasticsearch-and-logstash-tuning/index.html) > > > > > > > > Appreciate your help. > > > > > > > > Thanks, > > > > Danishka > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > To view this discussion on the web visit > > > > [https://groups.google.com/d/msgid/elasticsearch/e623e1a9-2a59-4b2f-bce0-11c8d2006c54%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/e623e1a9-2a59-4b2f-bce0-11c8d2006c54%40googlegroups.com) > > > > [https://groups.google.com/d/msgid/elasticsearch/e623e1a9-2a59-4b2f-bce0-11c8d2006c54%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/e623e1a9-2a59-4b2f-bce0-11c8d2006c54%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X\_9VizVn4m7C61Xo8GBakRfa00iQV5WiAKhdGHRWAic1g%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_9VizVn4m7C61Xo8GBakRfa00iQV5WiAKhdGHRWAic1g%40mail.gmail.com) > > > [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X\_9VizVn4m7C61Xo8GBakRfa00iQV5WiAKhdGHRWAic1g%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_9VizVn4m7C61Xo8GBakRfa00iQV5WiAKhdGHRWAic1g%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > . > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > Chris Rimondi | [http://twitter.com/crimondi](http://twitter.com/crimondi) | [securitygrit.com](http://securitygrit.com) > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/CA%2BqatLgnXX7Rp7D%2Bmn5XNOfZqe4Ko706EV9CxPYcFL%3DA11mR7A%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CA%2BqatLgnXX7Rp7D%2Bmn5XNOfZqe4Ko706EV9CxPYcFL%3DA11mR7A%40mail.gmail.com) > > [https://groups.google.com/d/msgid/elasticsearch/CA%2BqatLgnXX7Rp7D%2Bmn5XNOfZqe4Ko706EV9CxPYcFL%3DA11mR7A%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CA%2BqatLgnXX7Rp7D%2Bmn5XNOfZqe4Ko706EV9CxPYcFL%3DA11mR7A%40mail.gmail.com?utm_medium=email&utm_source=footer) > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > Danishka Navin > [http://danishkanavin.blogspot.com](http://danishkanavin.blogspot.com) > [http://twitter.com/danishkanavin](http://twitter.com/danishkanavin) > [http://www.flickr.com/photos/danishkanavin/](http://www.flickr.com/photos/danishkanavin/) > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAKjxiWRAWCRVa0Y2OMN4vcscUzDqMvuH5daWhvnc9BMudcs5XA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKjxiWRAWCRVa0Y2OMN4vcscUzDqMvuH5daWhvnc9BMudcs5XA%40mail.gmail.com) > [https://groups.google.com/d/msgid/elasticsearch/CAKjxiWRAWCRVa0Y2OMN4vcscUzDqMvuH5daWhvnc9BMudcs5XA%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKjxiWRAWCRVa0Y2OMN4vcscUzDqMvuH5daWhvnc9BMudcs5XA%40mail.gmail.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X8wNdZSya2ynEm\_R6N4gdggz1RWpbBirTfhaoVsi-j2iA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X8wNdZSya2ynEm_R6N4gdggz1RWpbBirTfhaoVsi-j2iA%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Danishka\_Navin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danishka_navin/32/1013_2.png) [@Danishka\_Navin](https://discuss.elastic.co/u/Danishka_Navin)\ **Post date:** [January 3, 2015, 6:11pm UTC](https://discuss.elastic.co/t/high-cpu-usage-of-elasticsearch/21450/6 "2015-01-03T18:11:47Z")
Hi Mark, I have removed the indices.memory.index\_buffer\_size entry and also update elasticsearch to 1.4.2 Moreover I have deleted lot of indexes. I can't see any difference of CPU usage by elasticsearch. PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 27719 elastics 20 0 152g 16g 4.5g S 395.1 55.1 29:06.51 /usr/bin/java -Xms15g -Xmx15g -Xss256k -Djava.awt.headles 27634 logstash 39 19 3459m 670m 13m S 4.0 2.2 2:56.79 /usr/bin/java -Djava.io.tmpdir=/var/lib/logstash -Xmx1g -X here is the jstat information for both elasticsearch and logstash > **[ur1.ca](http://ur1.ca/jbeja)** > > This domain may be for sale! Btw, Do I need to update Kibana to 3.1.2 ? On Sat, Jan 3, 2015 at 6:34 AM, Mark Walkom [markwalkom@gmail.com](mailto:markwalkom@gmail.com) wrote: > You're just hitting limits of your node. Drop some data, add more nodes or > more heap are pretty much the options you have. Upgrade to 1.4.2 while you > are at it. > > Setting indices.memory.index\_buffer\_size so high probably isn't a good > idea unless you know what it does, if you have such a high index rate then > look at adding more nodes to spread the load. > > On 3 January 2015 at 08:18, Danishka Navin [danishka@gmail.com](mailto:danishka@gmail.com) wrote: > > > Hi, > > > > Here is stats of my single cluster > > [http://fpaste.org/165093/20232576/](http://fpaste.org/165093/20232576/) > > > > Elasticsearch configuration > > [http://fpaste.org/165092/14202325/](http://fpaste.org/165092/14202325/) > > > > I don't use Redis. > > There were 20+ million events per 10minutes > > > > (Attached screen-dump of events over time) > > > > On Sat, Jan 3, 2015 at 2:26 AM, Christopher Rimondi \< > > [chris.rimondi@gmail.com](mailto:chris.rimondi@gmail.com)\> wrote: > > > > > I agree with what Mark said. Nice will be just masking a deeper issue. > > > Have you tried looking at hot threads? > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-nodes-hot-threads.html) > > > Also, if you are seeing CPUs sustained at 100% CPU that seems like old GCs > > > that just are never finishing. So check the GC logs. > > > > > > Do you have any idea on the number of events per second you are trying > > > to index and size of the events? If you are using logstash and Redis is the > > > queue backing up because it can't index? > > > > > > On Thu, Jan 1, 2015 at 4:49 PM, Mark Walkom [markwalkom@gmail.com](mailto:markwalkom@gmail.com) > > > wrote: > > > > > > > How much data do you have on the node? How many indexes? Have you > > > > checked the logs for GC issues? > > > > > > > > You can use nice on the OS level to manage CPU use, but it's not a good > > > > idea, and instead you should figure out why ES is using that CPU. > > > > > > > > On 2 January 2015 at 05:09, Danishka Navin [danishka@gmail.com](mailto:danishka@gmail.com) wrote: > > > > > > > > > Hi, > > > > > > > > > > I am new to elasticsearch and logstash. > > > > > using elasticsearch-1.1.1 and logstash-1.4.2-1 with Kibana. > > > > > Its a single node with 4 vCPU and 30GB of physical memory. > > > > > > > > > > Currently logstash (single node) receive logs from 40 jboss servers. > > > > > > > > > > Most of the time elasticsearch use almost all the CPU resource. > > > > > Is there any way I can limit the CPU consumption by tuning? > > > > > > > > > > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > > > > > 12216 elastics 20 0 514g 19g 3.6g S 389.8 63.5 3224:32 > > > > > /usr/bin/java -Xms15g -Xmx15g -Xss256k -Djava.awt.headles > > > > > 11722 logstash 39 19 3443m 1.2g 6496 S 8.6 3.9 2037:27 > > > > > /usr/bin/java -Djava.io.tmpdir=/var/lib/logstash -Xmx1g -X > > > > > > > > > > Any recommendations? > > > > > > > > > > I already followed following article. > > > > > [ElasticSearch and Logstash Tuning – Vaidas Jablonskis](http://jablonskis.org/2013/elasticsearch-and-logstash-tuning/index.html) > > > > > > > > > > Appreciate your help. > > > > > > > > > > Thanks, > > > > > Danishka > > > > > > > > > > -- > > > > > You received this message because you are subscribed to the Google > > > > > Groups "elasticsearch" group. > > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > To view this discussion on the web visit > > > > > [https://groups.google.com/d/msgid/elasticsearch/e623e1a9-2a59-4b2f-bce0-11c8d2006c54%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/e623e1a9-2a59-4b2f-bce0-11c8d2006c54%40googlegroups.com) > > > > > [https://groups.google.com/d/msgid/elasticsearch/e623e1a9-2a59-4b2f-bce0-11c8d2006c54%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/e623e1a9-2a59-4b2f-bce0-11c8d2006c54%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > > . > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > To view this discussion on the web visit > > > > [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X\_9VizVn4m7C61Xo8GBakRfa00iQV5WiAKhdGHRWAic1g%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_9VizVn4m7C61Xo8GBakRfa00iQV5WiAKhdGHRWAic1g%40mail.gmail.com) > > > > [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X\_9VizVn4m7C61Xo8GBakRfa00iQV5WiAKhdGHRWAic1g%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_9VizVn4m7C61Xo8GBakRfa00iQV5WiAKhdGHRWAic1g%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > > . > > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > -- > > > Chris Rimondi | [http://twitter.com/crimondi](http://twitter.com/crimondi) | [securitygrit.com](http://securitygrit.com) > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/CA%2BqatLgnXX7Rp7D%2Bmn5XNOfZqe4Ko706EV9CxPYcFL%3DA11mR7A%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CA%2BqatLgnXX7Rp7D%2Bmn5XNOfZqe4Ko706EV9CxPYcFL%3DA11mR7A%40mail.gmail.com) > > > [https://groups.google.com/d/msgid/elasticsearch/CA%2BqatLgnXX7Rp7D%2Bmn5XNOfZqe4Ko706EV9CxPYcFL%3DA11mR7A%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CA%2BqatLgnXX7Rp7D%2Bmn5XNOfZqe4Ko706EV9CxPYcFL%3DA11mR7A%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > . > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > Danishka Navin > > [http://danishkanavin.blogspot.com](http://danishkanavin.blogspot.com) > > [http://twitter.com/danishkanavin](http://twitter.com/danishkanavin) > > [http://www.flickr.com/photos/danishkanavin/](http://www.flickr.com/photos/danishkanavin/) > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/CAKjxiWRAWCRVa0Y2OMN4vcscUzDqMvuH5daWhvnc9BMudcs5XA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKjxiWRAWCRVa0Y2OMN4vcscUzDqMvuH5daWhvnc9BMudcs5XA%40mail.gmail.com) > > [https://groups.google.com/d/msgid/elasticsearch/CAKjxiWRAWCRVa0Y2OMN4vcscUzDqMvuH5daWhvnc9BMudcs5XA%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKjxiWRAWCRVa0Y2OMN4vcscUzDqMvuH5daWhvnc9BMudcs5XA%40mail.gmail.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X8wNdZSya2ynEm\_R6N4gdggz1RWpbBirTfhaoVsi-j2iA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X8wNdZSya2ynEm_R6N4gdggz1RWpbBirTfhaoVsi-j2iA%40mail.gmail.com) > [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X8wNdZSya2ynEm\_R6N4gdggz1RWpbBirTfhaoVsi-j2iA%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X8wNdZSya2ynEm_R6N4gdggz1RWpbBirTfhaoVsi-j2iA%40mail.gmail.com?utm_medium=email&utm_source=footer) > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- Danishka Navin > **[Danishka's Diary](http://danishkanavin.blogspot.com)** [http://twitter.com/danishkanavin](http://twitter.com/danishkanavin) [![Imgur](https://us1.discourse-cdn.com/elastic/original/3X/4/e/4e1d7fd95311b3a0453cd3c7e3a83bfd359ec27d.jpeg "Danishka Navin") ](https://www.flickr.com/photos/danishkanavin/) -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKjxiWT4\_3G%2Bp%3DR-W6\_9xRFMDLPec3eNofndiQ%3DyxoRemQ1mNA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKjxiWT4_3G%2Bp%3DR-W6_9xRFMDLPec3eNofndiQ%3DyxoRemQ1mNA%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 12:41am UTC](https://discuss.elastic.co/t/high-cpu-usage-of-elasticsearch/21450/7 "2017-07-06T00:41:02Z")