# NRT search with elastic search **URL:** \ **Category:** Elasticsearch\ **Created:** [August 11, 2011, 2:44am UTC](https://discuss.elastic.co/t/nrt-search-with-elastic-search/5122 "2011-08-11T02:44:48Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![rcch](https://avatars.discourse-cdn.com/v4/letter/r/4af34b/32.png) [@rcch](https://discuss.elastic.co/u/rcch)\ **Post date:** [August 11, 2011, 2:44am UTC](https://discuss.elastic.co/t/nrt-search-with-elastic-search/5122/1 "2011-08-11T02:44:48Z")
Hi all, 1. I'd like to get some feedback/hear experiences from people who've used elastic search for near-real-time search applications. 2. How does elastic search compare to using near-real time search in lucene ? 3. I believe there's a 1 second delay before documents can be made searchable (due to how the index gets updated asynchronously). Is this a parameter that can be tuned ? Our application requires an ingestion of content of ~1000s of documents/second, but they need to be searchable right-away.. Thanks for your input. Cheers, Vijay ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [August 11, 2011, 8:25am UTC](https://discuss.elastic.co/t/nrt-search-with-elastic-search/5122/2 "2011-08-11T08:25:18Z")
On Thu, Aug 11, 2011 at 5:44 AM, rcch [vijay.cmu@gmail.com](mailto:vijay.cmu@gmail.com) wrote: > Hi all, > > 1. I'd like to get some feedback/hear experiences from people who've > used Elasticsearch for near-real-time search applications. > > 2. How does Elasticsearch compare to using near-real time search in > lucene ? elasticsearch uses Lucene NRT. > 1. I believe there's a 1 second delay before documents can be made > searchable (due to how the index gets updated asynchronously). Is this > a parameter that can be tuned ? The index is not updated asynchronously, when you index data its applied to the shard and the replicas in a sync manner including writing to a transaction log. A "fresh" view of the content to be searchable is opened every 1 second by default (thats NRT). > Our application requires an ingestion of content of ~1000s of > documents/second, but they need to be searchable right-away.. > > Thanks for your input. > > Cheers, > > Vijay ---
**Author:** ![Pavel\_Penchev](https://avatars.discourse-cdn.com/v4/letter/p/ec9cab/32.png) [@Pavel\_Penchev](https://discuss.elastic.co/u/Pavel_Penchev)\ **Post date:** [August 11, 2011, 3:13pm UTC](https://discuss.elastic.co/t/nrt-search-with-elastic-search/5122/3 "2011-08-11T15:13:58Z")
Hi On 11.08.2011 11:25, Shay Banon wrote: > On Thu, Aug 11, 2011 at 5:44 AM, rcch \<[vijay.cmu@gmail.com](mailto:vijay.cmu@gmail.com) > [mailto:vijay.cmu@gmail.com](mailto:vijay.cmu@gmail.com)\> wrote: > > ``` > Hi all, > > 1) I'd like to get some feedback/hear experiences from people who've > used elastic search for near-real-time search applications. > > 2) How does elastic search compare to using near-real time search in > lucene ? > > ``` > > elasticsearch uses Lucene NRT. > > ``` > 3) I believe there's a 1 second delay before documents can be made > searchable (due to how the index gets updated asynchronously). Is this > a parameter that can be tuned ? > > ``` > > The index is not updated asynchronously, when you index data its > applied to the shard and the replicas in a sync manner including > writing to a transaction log. A "fresh" view of the content to be > searchable is opened every 1 second by default (thats NRT). > > ``` > Our application requires an ingestion of content of ~1000s of > documents/second, but they need to be searchable right-away.. > > ``` In case you only need them "get-able" right-away - ES has this since 0.17.0, see [Realtime GET · Issue #1060 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/1060) Regards, Pavel > ``` > Thanks for your input. > > Cheers, > > Vijay > > ``` ---
**Author:** ![rcch](https://avatars.discourse-cdn.com/v4/letter/r/4af34b/32.png) [@rcch](https://discuss.elastic.co/u/rcch)\ **Post date:** [August 12, 2011, 7:02pm UTC](https://discuss.elastic.co/t/nrt-search-with-elastic-search/5122/4 "2011-08-12T19:02:26Z")
Thanks for your replies, Shay, Pavel. Can you please tell me 1. Is it true that there will always be a 1 second delay before content can be searchable? 2. How does ES compare to Zoie (from Linkedin) for real-time-search applications ? 3. What kind of ingestion rates can ES support comfortably ? Is 10s/ second ? 100s/second ? or 1000s/second ? Can you suggest where you think the breakdown is? Thanks, Cheers, Vijay On Aug 11, 8:13 am, Pavel Penchev [pavel.penc...@gmail.com](mailto:pavel.penc...@gmail.com) wrote: > Hi > > On 11.08.2011 11:25, Shay Banon wrote: > > > On Thu, Aug 11, 2011 at 5:44 AM, rcch \<[vijay....@gmail.com](mailto:vijay....@gmail.com) > > [mailto:vijay....@gmail.com](mailto:vijay....@gmail.com)\> wrote: > > > ``` > > Hi all, > > > > ``` > > > ``` > > 1) I'd like to get some feedback/hear experiences from people who've > > used elastic search for near-real-time search applications. > > > > ``` > > > ``` > > 2) How does elastic search compare to using near-real time search in > > lucene ? > > > > ``` > > > elasticsearch uses Lucene NRT. > > > ``` > > 3) I believe there's a 1 second delay before documents can be made > > searchable (due to how the index gets updated asynchronously). Is this > > a parameter that can be tuned ? > > > > ``` > > > The index is not updated asynchronously, when you index data its > > applied to the shard and the replicas in a sync manner including > > writing to a transaction log. A "fresh" view of the content to be > > searchable is opened every 1 second by default (thats NRT). > > > ``` > > Our application requires an ingestion of content of ~1000s of > > documents/second, but they need to be searchable right-away.. > > > > ``` > > In case you only need them "get-able" right-away - ES has this since > 0.17.0, seehttps://github.com/elasticsearch/elasticsearch/issues/1060 > > Regards, > Pavel > > > ``` > > Thanks for your input. > > > > ``` > > > ``` > > Cheers, > > > > ``` > > > ``` > > Vijay > > > > ``` ---
**Author:** ![Berkay\_Mollamustafao](https://avatars.discourse-cdn.com/v4/letter/b/22d042/32.png) [@Berkay\_Mollamustafao](https://discuss.elastic.co/u/Berkay_Mollamustafao)\ **Post date:** [August 12, 2011, 7:44pm UTC](https://discuss.elastic.co/t/nrt-search-with-elastic-search/5122/5 "2011-08-12T19:44:52Z")
On Fri, Aug 12, 2011 at 3:02 PM, rcch [vijay.cmu@gmail.com](mailto:vijay.cmu@gmail.com) wrote: > Thanks for your replies, Shay, Pavel. > > Can you please tell me > > 1. Is it true that there will always be a 1 second delay before > content can be searchable? By default, refresh is called every 1 sec, so there would be delay up to 1 sec. However, refresh interval is configurable and can also be called programmatically. Technically you can call it after indexing every document but this would have significant performance impact. > 1. How does ES compare to Zoie (from Linkedin) for real-time-search > applications ? > 1. What kind of ingestion rates can ES support comfortably ? Is 10s/ > second ? 100s/second ? or 1000s/second ? Can you suggest where you > think the breakdown is? Hard to tell. What is the document size? How many servers (CPU, Memory, Disks) in the cluster? ES is horizontally scalable so event 1000s/sec is achievable with multiple servers. No better way to find it out is to test with your own docs. > Thanks, > > Cheers, > > Vijay > > On Aug 11, 8:13 am, Pavel Penchev [pavel.penc...@gmail.com](mailto:pavel.penc...@gmail.com) wrote: > > > Hi > > > > On 11.08.2011 11:25, Shay Banon wrote: > > > > > On Thu, Aug 11, 2011 at 5:44 AM, rcch \<[vijay....@gmail.com](mailto:vijay....@gmail.com) > > > [mailto:vijay....@gmail.com](mailto:vijay....@gmail.com)\> wrote: > > > > > ``` > > > Hi all, > > > > > > ``` > > > > > ``` > > > 1) I'd like to get some feedback/hear experiences from people > > > > > > ``` > > who've > > > > ``` > > > used elastic search for near-real-time search applications. > > > > > > ``` > > > > > ``` > > > 2) How does elastic search compare to using near-real time search > > > > > > ``` > > in > > > > ``` > > > lucene ? > > > > > > ``` > > > > > elasticsearch uses Lucene NRT. > > > > > ``` > > > 3) I believe there's a 1 second delay before documents can be made > > > searchable (due to how the index gets updated asynchronously). Is > > > > > > ``` > > this > > > > ``` > > > a parameter that can be tuned ? > > > > > > ``` > > > > > The index is not updated asynchronously, when you index data its > > > applied to the shard and the replicas in a sync manner including > > > writing to a transaction log. A "fresh" view of the content to be > > > searchable is opened every 1 second by default (thats NRT). > > > > > ``` > > > Our application requires an ingestion of content of ~1000s of > > > documents/second, but they need to be searchable right-away.. > > > > > > ``` > > > > In case you only need them "get-able" right-away - ES has this since > > 0.17.0, seehttps://github.com/elasticsearch/elasticsearch/issues/1060 > > > > Regards, > > Pavel > > > > > ``` > > > Thanks for your input. > > > > > > ``` > > > > > ``` > > > Cheers, > > > > > > ``` > > > > > ``` > > > Vijay > > > > > > ``` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:57am UTC](https://discuss.elastic.co/t/nrt-search-with-elastic-search/5122/6 "2017-07-06T03:57:17Z")