# NRT indexing speed **URL:** \ **Category:** Elasticsearch\ **Created:** [May 21, 2012, 4:07pm UTC](https://discuss.elastic.co/t/nrt-indexing-speed/7788 "2012-05-21T16:07:01Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Crwe](https://avatars.discourse-cdn.com/v4/letter/c/898d66/32.png) [@Crwe](https://discuss.elastic.co/u/Crwe)\ **Post date:** [May 21, 2012, 4:07pm UTC](https://discuss.elastic.co/t/nrt-indexing-speed/7788/1 "2012-05-21T16:07:01Z")
I plan to use the NRT feature heavily, for near-real-time indexing of documents, let's say adding 1,000 documents at a time via bulk index. I'd like to know: Does the bulk indexing performance depend on the size of the index (no. documents already indexed)? If so, linearly or sub-linearly? Does the indexing performance depend on the number of shards (nodes)? Linearly? Does the indexing performance depend on the number of replicas? Linearly? Should I expect significant spikes in indexing latency (because of some internal ES re-allocations, merges, etc.)? What is a "normal" standard deviation of the time between "start bulk index" and "can query the new docs in search"? Sorry for my basic questions, I do not know Lucene well. Thank you. ---
**Author:** ![Crwe](https://avatars.discourse-cdn.com/v4/letter/c/898d66/32.png) [@Crwe](https://discuss.elastic.co/u/Crwe)\ **Post date:** [May 23, 2012, 4:56pm UTC](https://discuss.elastic.co/t/nrt-indexing-speed/7788/2 "2012-05-23T16:56:43Z")
Bump. Any insights on scalability of NRT indexing in Elasticsearch are most welcome. On May 21, 6:07 pm, Crwe [tester.teste...@gmail.com](mailto:tester.teste...@gmail.com) wrote: > I plan to use the NRT feature heavily, for near-real-time indexing of > documents, let's say adding 1,000 documents at a time via bulk index. > I'd like to know: > > Does the bulk indexing performance depend on the size of the index > (no. documents already indexed)? If so, linearly or sub-linearly? > > Does the indexing performance depend on the number of shards (nodes)? > Linearly? > > Does the indexing performance depend on the number of replicas? > Linearly? > > Should I expect significant spikes in indexing latency (because of > some internal ES re-allocations, merges, etc.)? What is a "normal" > standard deviation of the time between "start bulk index" and "can > query the new docs in search"? > > Sorry for my basic questions, I do not know Lucene well. Thank you. ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [May 23, 2012, 10:53pm UTC](https://discuss.elastic.co/t/nrt-indexing-speed/7788/3 "2012-05-23T22:53:58Z")
Kindda hard to answer this general questions. You are asking question beyond NRT, so I will first answer NRT, what happens is that periodically, the index of a shard is refreshed and makes recent changes available for search (by default, its 1 second). Other question might be answered by watching the following video as a starter to understand how ES works: > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. . On Mon, May 21, 2012 at 6:07 PM, Crwe [tester.testerus@gmail.com](mailto:tester.testerus@gmail.com) wrote: > I plan to use the NRT feature heavily, for near-real-time indexing of > documents, let's say adding 1,000 documents at a time via bulk index. > I'd like to know: > > Does the bulk indexing performance depend on the size of the index > (no. documents already indexed)? If so, linearly or sub-linearly? > > Does the indexing performance depend on the number of shards (nodes)? > Linearly? > > Does the indexing performance depend on the number of replicas? > Linearly? > > Should I expect significant spikes in indexing latency (because of > some internal ES re-allocations, merges, etc.)? What is a "normal" > standard deviation of the time between "start bulk index" and "can > query the new docs in search"? > > Sorry for my basic questions, I do not know Lucene well. Thank you. ---
**Author:** ![Crwe](https://avatars.discourse-cdn.com/v4/letter/c/898d66/32.png) [@Crwe](https://discuss.elastic.co/u/Crwe)\ **Post date:** [May 24, 2012, 7:26pm UTC](https://discuss.elastic.co/t/nrt-indexing-speed/7788/4 "2012-05-24T19:26:07Z")
Thanks Shay, watched it. In case someone comes across this thread later, here are the answers I gathered from that video: > > Does the bulk indexing performance depend on the size of the index > > (no. documents already indexed)? If so, linearly or sub-linearly? No. > > Does the indexing performance depend on the number of shards (nodes)? > > Linearly? No. The performance is determined by indexing on the slowest shard, but otherwise constant in the number of shards. > > Does the indexing performance depend on the number of replicas? > > Linearly? Same as above. > > Should I expect significant spikes in indexing latency (because of > > some internal ES re-allocations, merges, etc.)? What is a "normal" > > standard deviation of the time between "start bulk index" and "can > > query the new docs in search"? Yes. Lucene merges segments from time to time, there are commits and stuff, plus there's Java's GC. I should expect large stddev of indexing times and occasional lags. Hopefully I didn't get it too wrong 🙂 ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:27am UTC](https://discuss.elastic.co/t/nrt-indexing-speed/7788/5 "2017-07-06T03:27:12Z")