# Tuning the ES performance **URL:** \ **Category:** Elasticsearch\ **Created:** [October 31, 2017, 3:41am UTC](https://discuss.elastic.co/t/tuning-the-es-performance/105832 "2017-10-31T03:41:56Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![f26227279](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/f26227279/32/21296_2.png) [@f26227279](https://discuss.elastic.co/u/f26227279)\ **Post date:** [October 31, 2017, 3:41am UTC](https://discuss.elastic.co/t/tuning-the-es-performance/105832/1 "2017-10-31T03:41:56Z")
I am using ES to ingest flow data from other application, but I found that the speed isn't high enough, so sometimes it would drop flow. How can I change the setting of ES performance ? I have seen the elasticsearch.yml, jvm.options and log4j2.properties, but there seems no setting about ES performance. Thanks a lot. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [October 31, 2017, 3:57am UTC](https://discuss.elastic.co/t/tuning-the-es-performance/105832/2 "2017-10-31T03:57:36Z")
The speed of which parts exactly? ---
**Author:** ![f26227279](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/f26227279/32/21296_2.png) [@f26227279](https://discuss.elastic.co/u/f26227279)\ **Post date:** [October 31, 2017, 3:59am UTC](https://discuss.elastic.co/t/tuning-the-es-performance/105832/3 "2017-10-31T03:59:28Z")
the speed of ES receive data from other application. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [October 31, 2017, 4:00am UTC](https://discuss.elastic.co/t/tuning-the-es-performance/105832/4 "2017-10-31T04:00:58Z")
How are you indexing into Elasticsearch? What version are you on? What hardware? Are you using bulk? ---
**Author:** ![f26227279](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/f26227279/32/21296_2.png) [@f26227279](https://discuss.elastic.co/u/f26227279)\ **Post date:** [October 31, 2017, 4:09am UTC](https://discuss.elastic.co/t/tuning-the-es-performance/105832/5 "2017-10-31T04:09:25Z")
I connect ES with nProbe to collect flow data. version 5.6 2 core CPU, 8G RAM ![image](https://us1.discourse-cdn.com/elastic/original/3X/3/b/3b24b96644d968c4b28428b21aae6a109f13d391.png) receive speed about : [362.6 Flows/s] could I speed it up? what is bulk \>\< ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [October 31, 2017, 4:16am UTC](https://discuss.elastic.co/t/tuning-the-es-performance/105832/6 "2017-10-31T04:16:10Z")
If nProbe is sending to Elasticsearch directly you will likely need to ask them. Is Elasticsearch overloaded? Are you using the Monitoring functionality to see what is happening? ---
**Author:** ![f26227279](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/f26227279/32/21296_2.png) [@f26227279](https://discuss.elastic.co/u/f26227279)\ **Post date:** [October 31, 2017, 4:21am UTC](https://discuss.elastic.co/t/tuning-the-es-performance/105832/7 "2017-10-31T04:21:34Z")
now seems like ES overload. No, how to monitor it ? ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [October 31, 2017, 4:22am UTC](https://discuss.elastic.co/t/tuning-the-es-performance/105832/8 "2017-10-31T04:22:04Z")
[https://www.elastic.co/guide/en/x-pack/5.6/xpack-monitoring.html](https://www.elastic.co/guide/en/x-pack/5.6/xpack-monitoring.html) 🙂 ---
**Author:** ![f26227279](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/f26227279/32/21296_2.png) [@f26227279](https://discuss.elastic.co/u/f26227279)\ **Post date:** [October 31, 2017, 4:32am UTC](https://discuss.elastic.co/t/tuning-the-es-performance/105832/9 "2017-10-31T04:32:38Z")
When I install x-pack in ES it show the error: Could not find any executable java binary. Please install java in your PATH or set JAVA\_HOME but I have install the java jdk to the environment variable. ---
**Author:** ![f26227279](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/f26227279/32/21296_2.png) [@f26227279](https://discuss.elastic.co/u/f26227279)\ **Post date:** [October 31, 2017, 5:18am UTC](https://discuss.elastic.co/t/tuning-the-es-performance/105832/10 "2017-10-31T05:18:06Z")
![image](https://us1.discourse-cdn.com/elastic/original/3X/b/7/b7b143877085296864469c0a60b386d61f6424fb.png) ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [November 6, 2017, 7:27am UTC](https://discuss.elastic.co/t/tuning-the-es-performance/105832/11 "2017-11-06T07:27:24Z")
Does you collection pipeline use bulk requests to index into Elasticsearch? What does disk I/O and iowait look like on the Elasticsearch host? Continuing the discussion from this parallel thread: [How to tune elasticsearch performance](https://discuss.elastic.co/t/how-to-tune-elasticsearch-performance/106472/7) ---
**Author:** ![f26227279](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/f26227279/32/21296_2.png) [@f26227279](https://discuss.elastic.co/u/f26227279)\ **Post date:** [November 6, 2017, 7:32am UTC](https://discuss.elastic.co/t/tuning-the-es-performance/105832/12 "2017-11-06T07:32:28Z")
sorry, How could I know whether I use bulk requests to index in ES. disk I/O and IO wait do you mean this : ![image](https://us1.discourse-cdn.com/elastic/original/3X/1/3/136fbbd16cfb8c2c54d7084677de154b31dc0e9d.png) ---
**Author:** ![f26227279](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/f26227279/32/21296_2.png) [@f26227279](https://discuss.elastic.co/u/f26227279)\ **Post date:** [November 6, 2017, 7:36am UTC](https://discuss.elastic.co/t/tuning-the-es-performance/105832/13 "2017-11-06T07:36:10Z")
![image](https://us1.discourse-cdn.com/elastic/original/3X/4/1/410ea88b40388d3e97831035d63e063039f5f4c5.png) ---
**Author:** ![f26227279](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/f26227279/32/21296_2.png) [@f26227279](https://discuss.elastic.co/u/f26227279)\ **Post date:** [November 6, 2017, 8:12am UTC](https://discuss.elastic.co/t/tuning-the-es-performance/105832/14 "2017-11-06T08:12:56Z")
yes I use /\_bulk ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [November 6, 2017, 8:25am UTC](https://discuss.elastic.co/t/tuning-the-es-performance/105832/15 "2017-11-06T08:25:51Z")
What is the size of your bulk requests? How large are your documents? ---
**Author:** ![f26227279](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/f26227279/32/21296_2.png) [@f26227279](https://discuss.elastic.co/u/f26227279)\ **Post date:** [November 6, 2017, 8:44am UTC](https://discuss.elastic.co/t/tuning-the-es-performance/105832/16 "2017-11-06T08:44:34Z")
sorry, where can I find the information. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [November 6, 2017, 8:47am UTC](https://discuss.elastic.co/t/tuning-the-es-performance/105832/17 "2017-11-06T08:47:57Z")
That will be decided by the application ingesting data into Elasticsearch, so you will need to look there. You may also be able to find out by looking at the network traffic. ---
**Author:** ![f26227279](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/f26227279/32/21296_2.png) [@f26227279](https://discuss.elastic.co/u/f26227279)\ **Post date:** [November 6, 2017, 8:58am UTC](https://discuss.elastic.co/t/tuning-the-es-performance/105832/18 "2017-11-06T08:58:44Z")
Does the ES can tune the performance or the only way is to improve my device (such as CPU or RAM) ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [November 6, 2017, 9:08am UTC](https://discuss.elastic.co/t/tuning-the-es-performance/105832/19 "2017-11-06T09:08:07Z")
Indexing individual documents generally results in dramatically lower indexing throughput compared to using bulk requests, so I would recommend ensuring that you are using bulk requests of an appropriate size before starting to try and tune Elasticsearch. ---
**Author:** ![f26227279](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/f26227279/32/21296_2.png) [@f26227279](https://discuss.elastic.co/u/f26227279)\ **Post date:** [November 6, 2017, 9:14am UTC](https://discuss.elastic.co/t/tuning-the-es-performance/105832/20 "2017-11-06T09:14:19Z")
yes, I thought I am using bulk requests this is the Application command to pass the flow to ES: ntopng /c -F "es;ntopng;ntopng-%Y.%m.%d;[http://192.168.0.157:9200/\_bulk;](http://192.168.0.157:9200/_bulk;)" [Next page](https://discuss.elastic.co/t/tuning-the-es-performance/105832.md?page=2)