# Please help me understand why indexing is so slow on my 2 node cluster **URL:** \ **Category:** Elasticsearch\ **Created:** [November 5, 2013, 4:32pm UTC](https://discuss.elastic.co/t/please-help-me-understand-why-indexing-is-so-slow-on-my-2-node-cluster/14271 "2013-11-05T16:32:40Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Melanie\_Zamora](https://avatars.discourse-cdn.com/v4/letter/m/45deac/32.png) [@Melanie\_Zamora](https://discuss.elastic.co/u/Melanie_Zamora)\ **Post date:** [November 5, 2013, 4:32pm UTC](https://discuss.elastic.co/t/please-help-me-understand-why-indexing-is-so-slow-on-my-2-node-cluster/14271/1 "2013-11-05T16:32:40Z")
Hello, I have a 2 node ES cluster. They are both linux VMs with 2vCPUs and 4GB RAM. I've kept the default of 5 shards and 1 replica. I've allocated 2GB of RAM for heap on each box, set memory lock to avoid swapping, and changed the index refresh interval from 1s to 15s; all in hope of getting ES to index faster. I also have a mapping template in place so its not constantly doing an update\_mapping. I've downloaded the plugin, Paramedic, and this is what I'm seeing: - indices.indexing.index\_current is very high on just one node (up to 18,000), and almost 0 on the other node. - cpu and heap usage is about the same on box servers - - _My questions is: Why is index\_current, which I'm assuming is time to index in milliseconds, so high on just one node and not the other?_ [https://lh5.googleusercontent.com/-it2alkCbSP4/UnkaRxtMkrI/AAAAAAAAAOQ/kysBHJWR3Yc/s1600/esclusterqa.jpg](https://lh5.googleusercontent.com/-it2alkCbSP4/UnkaRxtMkrI/AAAAAAAAAOQ/kysBHJWR3Yc/s1600/esclusterqa.jpg) -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [November 5, 2013, 8:47pm UTC](https://discuss.elastic.co/t/please-help-me-understand-why-indexing-is-so-slow-on-my-2-node-cluster/14271/2 "2013-11-05T20:47:12Z")
index\_current is the number of index operations currently in progress on this node. There is no need to worry, it just shows indexing activity. Just as a side note, if you are on a VM, there is not much use in setting memory lock to avoid swap, because the host OS decides when to swap, not the guest OS. If you want maximum indexing speed out of ES, choosing VM is limiting this a bit. VMs use virtualized store which is not helping in gaining speedy I/O. You will be better off with non-virtualized servers. Jörg -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Melanie\_Zamora](https://avatars.discourse-cdn.com/v4/letter/m/45deac/32.png) [@Melanie\_Zamora](https://discuss.elastic.co/u/Melanie_Zamora)\ **Post date:** [November 5, 2013, 8:50pm UTC](https://discuss.elastic.co/t/please-help-me-understand-why-indexing-is-so-slow-on-my-2-node-cluster/14271/3 "2013-11-05T20:50:57Z")
Hi Jorg. Thanks for the reply. Good to know that index\_current is the # of operations and not how long it's taking to index. However, do you know why only one node is doing all the indexing, while the other node isn't doing much indexing at all? Shouldn't we expect the indexing throughput to be evenly distributed across the 2 nodes? On Tuesday, November 5, 2013 2:47:12 PM UTC-6, Jörg Prante wrote: > index\_current is the number of index operations currently in progress on > this node. There is no need to worry, it just shows indexing activity. > > Just as a side note, if you are on a VM, there is not much use in setting > memory lock to avoid swap, because the host OS decides when to swap, not > the guest OS. > > If you want maximum indexing speed out of ES, choosing VM is limiting this > a bit. VMs use virtualized store which is not helping in gaining speedy > I/O. You will be better off with non-virtualized servers. > > Jörg -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [November 5, 2013, 10:00pm UTC](https://discuss.elastic.co/t/please-help-me-understand-why-indexing-is-so-slow-on-my-2-node-cluster/14271/4 "2013-11-05T22:00:25Z")
index\_current should be 0 if the index is quiet. I'm not sure what you are doing with the index, what shards the nodes hold, and how the output of the \_stats command looks like over time, so it's not easy to comment. In general, you are correct, the load should be distributed across the nodes. To measure the indexing throughput, best thing is to measure numbers in the client when it sends data over the API, or use monitoring tools. Also note the client should connect to both nodes, if you want a fair comparison between the load of the nodes. Jörg -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Melanie\_Zamora](https://avatars.discourse-cdn.com/v4/letter/m/45deac/32.png) [@Melanie\_Zamora](https://discuss.elastic.co/u/Melanie_Zamora)\ **Post date:** [November 6, 2013, 5:22pm UTC](https://discuss.elastic.co/t/please-help-me-understand-why-indexing-is-so-slow-on-my-2-node-cluster/14271/5 "2013-11-06T17:22:43Z")
Thanks Jorg. In my ES setup, I have the client sending logs over to just one of the nodes (using its IP address). Instead of doing this should I have the client sending over to both nodes? What I mean is should I put a load balancer in front of the 2 nodes so that load can be round robined between the 2 nodes? I was under the impression that I can just tell the client to send over to one node, and ES will take care of the load balancing. On Tuesday, November 5, 2013 4:00:25 PM UTC-6, Jörg Prante wrote: > index\_current should be 0 if the index is quiet. I'm not sure what you are > doing with the index, what shards the nodes hold, and how the output of the > \_stats command looks like over time, so it's not easy to comment. In > general, you are correct, the load should be distributed across the nodes. > To measure the indexing throughput, best thing is to measure numbers in the > client when it sends data over the API, or use monitoring tools. Also note > the client should connect to both nodes, if you want a fair comparison > between the load of the nodes. > > Jörg -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [November 6, 2013, 6:17pm UTC](https://discuss.elastic.co/t/please-help-me-understand-why-indexing-is-so-slow-on-my-2-node-cluster/14271/6 "2013-11-06T18:17:21Z")
Yes, you can study where the shards are located. ES will care about distributing the data to the shards. What I am concerned - it's maybe a non-issue - that processing the logs takes extra load on one node. Not sure what you mean by a load balancer. ES clients can usually connect to one than more IP for failover, but I do not know what client you use. Jörg -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:08am UTC](https://discuss.elastic.co/t/please-help-me-understand-why-indexing-is-so-slow-on-my-2-node-cluster/14271/7 "2017-07-06T02:08:29Z")