# Elasticsearch CPU utilization problem with Logstash **URL:** \ **Category:** Elasticsearch\ **Created:** [October 28, 2021, 10:39am UTC](https://discuss.elastic.co/t/elasticsearch-cpu-utilization-problem-with-logstash/287903 "2021-10-28T10:39:26Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![Hur](https://avatars.discourse-cdn.com/v4/letter/h/e9c0ed/32.png) [@Hur](https://discuss.elastic.co/u/Hur)\ **Post date:** [October 28, 2021, 10:39am UTC](https://discuss.elastic.co/t/elasticsearch-cpu-utilization-problem-with-logstash/287903/1 "2021-10-28T10:39:26Z")
I have trouble with using Elasticsearch with Logstash. When inserting data to ES through Logstash (not using Filebeat. Elasticsearch & Logstash only), CPU utilization of ES is over 90%. Also, I expected if I use multiple Logstash, data inserting time is same or decreased compared to using 1 Logstash. However, my testing result is not like that. Please advise how to use ES and multiple Logstash efficiently. Below is how I tested. **[TEST ENVIRONMENT]** - O/S : CentOS 7.9 - cpu 32 vCore - mem 64GB - ELK version : OSS 6.8.17 - Run 1 ES & 2 Logstash (different port) on same server **[TEST#1 - using 1 Logstash]** - Send 100mb of data to Logstash - takes about 2 minutes to insert data to ES - ES CPU utilization: 90 ~ 100% - (only 100mb data, but CPU utilization is too high. It seems ES does not use all CPU cores) - CPU Idle of server: 90 ~ 99% **[TEST#2 - using 2 Logstash]** - Send 100mb of data to each Logstash (total of 200mb) - takes about 4 minutes to insert data to ES - (each Logstash handles same size of data as TEST#1, but takes twice longer than TEST#1) - ES CPU utilization: 80 ~ 100% - CPU Idle of server: 90 ~ 99% **[thread\_pool status during testing]** - most of numbers are 0. > curl [http://localhost:9200/\_cat/thread\_pool?v](http://localhost:9200/_cat/thread_pool?v) ```auto node_name name active queue rejected node-01 analyze 0 0 0 node-01 fetch_shard_started 0 0 0 node-01 fetch_shard_store 0 0 0 node-01 flush 0 0 0 node-01 force_merge 0 0 0 node-01 generic 0 0 0 node-01 get 0 0 0 node-01 index 0 0 0 node-01 listener 0 0 0 node-01 management 1 0 0 node-01 refresh 0 0 0 node-01 search 0 0 0 node-01 search_throttled 0 0 0 node-01 snapshot 0 0 0 node-01 warmer 0 0 0 node-01 write 0 0 0 ``` **[ES SETTING - elasticsearch.yml]** ```auto network.host: ["_local_", "_site_"] http.port: 9200 transport.port: 9300 bootstrap.memory_lock: true bootstrap.system_call_filter: false thread_pool.search.max_queue_size: 10000 thread_pool.bulk.size: 16 thread_pool.bulk.queue_size: 10000 thread_pool.write.size: 32 thread_pool.write.queue_size: 10000 http.max_content_length: 100mb network.tcp.no_delay: true network.tcp.keep_alive : true network.tcp.reuse_address: true network.tcp.send_buffer_size : 1024mb network.tcp.receive_buffer_size : 1024mb ``` ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [October 28, 2021, 11:24am UTC](https://discuss.elastic.co/t/elasticsearch-cpu-utilization-problem-with-logstash/287903/2 "2021-10-28T11:24:27Z")
> [@Hur](#): > > ```auto > thread_pool.search.max_queue_size: 10000 > thread_pool.bulk.size: 16 > thread_pool.bulk.queue_size: 10000 > thread_pool.write.size: 32 > thread_pool.write.queue_size: 10000 > > http.max_content_length: 100mb > network.tcp.no_delay: true > network.tcp.keep_alive : true > network.tcp.reuse_address: true > network.tcp.send_buffer_size : 1024mb > network.tcp.receive_buffer_size : 1024mb > > ``` Why have you overridden these parameters? These are expert settings and incorrect settings can have adverse effects. I would recommend restoring these to the deafults. > [@Hur](#): > > - O/S : CentOS 7.9 > - cpu 32 vCore > - mem 64GB > - ELK version : OSS 6.8.17 Elasticsearch indexing is often very I/O intensive so it is important to verify that your storage is not the bottleneck here. What type of storage are you using? Local SSD? What does disk utilisation and iowait look like during indexing? ---
**Author:** ![Hur](https://avatars.discourse-cdn.com/v4/letter/h/e9c0ed/32.png) [@Hur](https://discuss.elastic.co/u/Hur)\ **Post date:** [October 29, 2021, 2:34am UTC](https://discuss.elastic.co/t/elasticsearch-cpu-utilization-problem-with-logstash/287903/3 "2021-10-29T02:34:19Z")
As your reply, I reset the parameters and test again, but have same result. I'm using local SSD. Here is I/O status - I/O status during ES test and stress test. Based on the below disk stress test result, I guess there is no bottleneck on disk I/O during my ES test. **[I/O Status - ES TEST using 2 Logstash]** - Test condition: Send 100mb of data to each Logstash (total of 200mb) - Average kB\_wrtn/s value of sdb: 4000 ~ 8000 - (Sometimes it reaches to 10096.00, 20076.00, 33344.00) ```auto -------------------------------------------------------------------------- # iostat 1 avg-cpu: %user %nice %system %iowait %steal %idle 3.92 0.00 0.63 3.14 0.00 92.32 Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn sda 0.00 0.00 0.00 0 0 sdb 194.00 0.00 6596.00 0 6596 -------------------------------------------------------------------------- ``` **[I/O Status - Stress Test]** > dd if=/dev/sdb of=test.file bs=64M count=10000 oflag=dsync ```auto -------------------------------------------------------------------------- # iostat 1 avg-cpu: %user %nice %system %iowait %steal %idle 0.03 0.00 0.97 2.22 0.00 96.78 Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn sda 0.00 0.00 0.00 0 0 sdb 1705.00 196612.00 165972.00 196612 165972 ... -------------------------------------------------------------------------- ``` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 26, 2021, 2:34am UTC](https://discuss.elastic.co/t/elasticsearch-cpu-utilization-problem-with-logstash/287903/4 "2021-11-26T02:34:56Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.