I'd like to get some feedback/hear experiences from people who've
used elastic search for near-real-time search applications.
How does elastic search compare to using near-real time search in
lucene ?
I believe there's a 1 second delay before documents can be made
searchable (due to how the index gets updated asynchronously). Is this
a parameter that can be tuned ?
Our application requires an ingestion of content of ~1000s of
documents/second, but they need to be searchable right-away..
I'd like to get some feedback/hear experiences from people who've
used Elasticsearch for near-real-time search applications.
How does Elasticsearch compare to using near-real time search in
lucene ?
elasticsearch uses Lucene NRT.
I believe there's a 1 second delay before documents can be made
searchable (due to how the index gets updated asynchronously). Is this
a parameter that can be tuned ?
The index is not updated asynchronously, when you index data its applied to
the shard and the replicas in a sync manner including writing to a
transaction log. A "fresh" view of the content to be searchable is opened
every 1 second by default (thats NRT).
Our application requires an ingestion of content of ~1000s of
documents/second, but they need to be searchable right-away..
Hi all,
1) I'd like to get some feedback/hear experiences from people who've
used elastic search for near-real-time search applications.
2) How does elastic search compare to using near-real time search in
lucene ?
elasticsearch uses Lucene NRT.
3) I believe there's a 1 second delay before documents can be made
searchable (due to how the index gets updated asynchronously). Is this
a parameter that can be tuned ?
The index is not updated asynchronously, when you index data its
applied to the shard and the replicas in a sync manner including
writing to a transaction log. A "fresh" view of the content to be
searchable is opened every 1 second by default (thats NRT).
Our application requires an ingestion of content of ~1000s of
documents/second, but they need to be searchable right-away..
Is it true that there will always be a 1 second delay before
content can be searchable?
How does ES compare to Zoie (from Linkedin) for real-time-search
applications ?
What kind of ingestion rates can ES support comfortably ? Is 10s/
second ? 100s/second ? or 1000s/second ? Can you suggest where you
think the breakdown is?
1) I'd like to get some feedback/hear experiences from people who've
used elastic search for near-real-time search applications.
2) How does elastic search compare to using near-real time search in
lucene ?
elasticsearch uses Lucene NRT.
3) I believe there's a 1 second delay before documents can be made
searchable (due to how the index gets updated asynchronously). Is this
a parameter that can be tuned ?
The index is not updated asynchronously, when you index data its
applied to the shard and the replicas in a sync manner including
writing to a transaction log. A "fresh" view of the content to be
searchable is opened every 1 second by default (thats NRT).
Our application requires an ingestion of content of ~1000s of
documents/second, but they need to be searchable right-away..
In case you only need them "get-able" right-away - ES has this since
0.17.0, seehttps://github.com/elasticsearch/elasticsearch/issues/1060
Is it true that there will always be a 1 second delay before
content can be searchable?
By default, refresh is called every 1 sec, so there would be delay up to 1
sec. However, refresh interval is configurable and can also be called
programmatically. Technically you can call it after indexing every document
but this would have significant performance impact.
How does ES compare to Zoie (from Linkedin) for real-time-search
applications ?
What kind of ingestion rates can ES support comfortably ? Is 10s/
second ? 100s/second ? or 1000s/second ? Can you suggest where you
think the breakdown is?
Hard to tell. What is the document size? How many servers (CPU, Memory,
Disks) in the cluster? ES is horizontally scalable so event 1000s/sec is
achievable with multiple servers. No better way to find it out is to test
with your own docs.
1) I'd like to get some feedback/hear experiences from people
who've
used elastic search for near-real-time search applications.
2) How does elastic search compare to using near-real time search
in
lucene ?
elasticsearch uses Lucene NRT.
3) I believe there's a 1 second delay before documents can be made
searchable (due to how the index gets updated asynchronously). Is
this
a parameter that can be tuned ?
The index is not updated asynchronously, when you index data its
applied to the shard and the replicas in a sync manner including
writing to a transaction log. A "fresh" view of the content to be
searchable is opened every 1 second by default (thats NRT).
Our application requires an ingestion of content of ~1000s of
documents/second, but they need to be searchable right-away..
In case you only need them "get-able" right-away - ES has this since
0.17.0, seehttps://github.com/elasticsearch/elasticsearch/issues/1060
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.