# Slow Indexing rate **URL:** \ **Category:** Elasticsearch\ **Created:** [August 11, 2015, 4:41pm UTC](https://discuss.elastic.co/t/slow-indexing-rate/27206 "2015-08-11T16:41:20Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![Ramky](https://avatars.discourse-cdn.com/v4/letter/r/b2d939/32.png) [@Ramky](https://discuss.elastic.co/u/Ramky)\ **Post date:** [August 11, 2015, 4:41pm UTC](https://discuss.elastic.co/t/slow-indexing-rate/27206/1 "2015-08-11T16:41:20Z")
I am trying to index json events with 16 string fields each and maximum EPS that i am able to index is 5k. These json events are fed to elasticsearch by traversing list of strings which are loaded from a file . Elastic Search cluster consists of 3 nodes with 32 GB RAM and 8 core CPU. We are performing bulk indexing with following settings index.number\_of\_shards=5 index.number\_of\_replicas=1 index.translog.flush\_threshold\_ops=50000 index.refresh\_interval=30 indices.memory.index\_buffer\_size=50% indices.fielddata.cache.size=20% indices.fielddata.cache.expire=1h indices.cache.filter.size=20% indices.cache.filter.expire=1h bulkprocessor.BulkActions=25000 bulkprocessor.BulkSize=15 bulkprocessor.FlushInterval=15 bulkprocessor.ConcurrentRequests=10 Please help to increase indexing speed Regards Rama Krishna P ---
**Author:** ![jpountz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jpountz/32/45836_2.png) [@jpountz](https://discuss.elastic.co/u/jpountz)\ **Post date:** [August 11, 2015, 5:49pm UTC](https://discuss.elastic.co/t/slow-indexing-rate/27206/2 "2015-08-11T17:49:42Z")
When the bulk test is running, is Elasticsearch maxing out CPU and/or I/O? If not then maybe you just need to send data from more threads? ---
**Author:** ![Ramky](https://avatars.discourse-cdn.com/v4/letter/r/b2d939/32.png) [@Ramky](https://discuss.elastic.co/u/Ramky)\ **Post date:** [August 12, 2015, 10:04am UTC](https://discuss.elastic.co/t/slow-indexing-rate/27206/3 "2015-08-12T10:04:02Z")
Thanks for reply. Indexing 1 million events with single thread, EPS (events per second) is around 7k, but when tried using 4 threads EPS is around 1.7k. Memory Consumption is 28% (nearly 9GB) CPU is idle 96%, Maximum CPU usage is 10%. Out of 1 million events, indexing speed is around 40k EPS till 700k events, but overall processing rate is around 7k EPS. Below is I/O taken from one machine **Before:** avg-cpu: %user %nice %system %iowait %steal %idle 0.54 0.00 0.13 1.70 0.00 97.63 Device: tps Blk\_read/s Blk\_wrtn/s Blk\_read Blk\_wrtn sda 11.45 458.74 199.38 722502 314016 dm-0 25.07 434.87 105.50 684914 166160 dm-1 0.20 1.64 0.00 2576 0 dm-2 9.32 18.01 72.04 28370 113456 **During indexing:** avg-cpu: %user %nice %system %iowait %steal %idle 1.27 0.00 0.14 3.81 0.00 94.78 Device: tps Blk\_read/s Blk\_wrtn/s Blk\_read Blk\_wrtn sda 10.81 358.03 1294.19 722886 2613096 dm-0 19.67 339.36 83.17 685194 167928 dm-1 0.16 1.28 0.00 2576 0 dm-2 149.50 14.09 1193.99 28458 2410768 ---
**Author:** ![jpountz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jpountz/32/45836_2.png) [@jpountz](https://discuss.elastic.co/u/jpountz)\ **Post date:** [August 12, 2015, 4:45pm UTC](https://discuss.elastic.co/t/slow-indexing-rate/27206/4 "2015-08-12T16:45:17Z")
So your system looks essentially idle. There must be an issue with the benchmark somehow. Maybe the bottleneck is to read data from your file? Also for the record, setting time expiry on the fielddata and filter caches is almost always wrong as fielddata entries are very expensive to regenerate and filter entries can't go outdated as they are cached per segment. ---
**Author:** ![Ramky](https://avatars.discourse-cdn.com/v4/letter/r/b2d939/32.png) [@Ramky](https://discuss.elastic.co/u/Ramky)\ **Post date:** [August 13, 2015, 8:29am UTC](https://discuss.elastic.co/t/slow-indexing-rate/27206/5 "2015-08-13T08:29:36Z")
Thanks for reply. The file is loaded only once and maintained in-memory and stats are collected after loading the file. So, i don't think reading data from file is bottleneck. Even though disabled the expiry settings on field data and filter cache, there isn't any change in indexing rate. ---
**Author:** ![jpountz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jpountz/32/45836_2.png) [@jpountz](https://discuss.elastic.co/u/jpountz)\ **Post date:** [August 13, 2015, 10:32am UTC](https://discuss.elastic.co/t/slow-indexing-rate/27206/6 "2015-08-13T10:32:50Z")
> [@Ramky](#): > > The file is loaded only once and maintained in-memory and stats are collected after loading the file. So, i don't think reading data from file is bottleneck. Then I'm very confused why you can't max out either I/O or CPU on your server. There must be something wrong somewhere... Are you using bulk or separate indexing requests to load data? Do you have good network connectivity between your server and the machine that runs the benchmark? > [@Ramky](#): > > Even though disabled the expiry settings on field data and filter cache, there isn't any change in indexing rate. Right I didn't expect it to change indexing rate, this was more a side note. ---
**Author:** ![Ramky](https://avatars.discourse-cdn.com/v4/letter/r/b2d939/32.png) [@Ramky](https://discuss.elastic.co/u/Ramky)\ **Post date:** [August 13, 2015, 11:38am UTC](https://discuss.elastic.co/t/slow-indexing-rate/27206/7 "2015-08-13T11:38:42Z")
I created one transport client object for 3 node cluster and send JSON events from java util list loaded from file to cluster using bulk index API. Following are parameters used for bulk indexing API : bulkprocessor.BulkActions=10000 bulkprocessor.BulkSize=10 bulkprocessor.FlushInterval=20 bulkprocessor.ConcurrentRequests=10 I tried multiple ways but unable to increase CPU or Memory utilization and also indexing rate. Moreover all machines are in 1 GBPS local LAN ---
**Author:** ![jpountz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jpountz/32/45836_2.png) [@jpountz](https://discuss.elastic.co/u/jpountz)\ **Post date:** [August 14, 2015, 2:58pm UTC](https://discuss.elastic.co/t/slow-indexing-rate/27206/8 "2015-08-14T14:58:43Z")
Sorry I don't have more ideas about what is wrong, but there certainly is something... ---
**Author:** ![Ramky](https://avatars.discourse-cdn.com/v4/letter/r/b2d939/32.png) [@Ramky](https://discuss.elastic.co/u/Ramky)\ **Post date:** [August 17, 2015, 4:53am UTC](https://discuss.elastic.co/t/slow-indexing-rate/27206/9 "2015-08-17T04:53:53Z")
Thanks for your help in trying to address the issue. Based on few trail and error methods able to get 50k EPS. ---
**Author:** ![Chang\_Oh\_Heo](https://avatars.discourse-cdn.com/v4/letter/c/d9b06d/32.png) [@Chang\_Oh\_Heo](https://discuss.elastic.co/u/Chang_Oh_Heo)\ **Post date:** [September 12, 2016, 4:52pm UTC](https://discuss.elastic.co/t/slow-indexing-rate/27206/10 "2016-09-12T16:52:00Z")
Hi, I got a similar problem. Could you let me know how to solve the problem? ---
**Author:** ![Chen\_Jian](https://avatars.discourse-cdn.com/v4/letter/c/c89c15/32.png) [@Chen\_Jian](https://discuss.elastic.co/u/Chen_Jian)\ **Post date:** [March 8, 2017, 5:31am UTC](https://discuss.elastic.co/t/slow-indexing-rate/27206/11 "2017-03-08T05:31:12Z")
Got similar issue, ES data node CPU is idle and index rate is extremely slow, any suggestions? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [March 8, 2017, 1:30pm UTC](https://discuss.elastic.co/t/slow-indexing-rate/27206/12 "2017-03-08T13:30:52Z")
Please open a new thread for your question and provide more details around your setup and achieved performance. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 10:02pm UTC](https://discuss.elastic.co/t/slow-indexing-rate/27206/13 "2017-07-05T22:02:28Z")