We are running one application which has very large amount of data updating
very frequently. And we are using SQL for any command like
update/delete/add records. And for querying part we are using
ElasticSearch. So is there any way which syncs my data very frequently say
in 5 to 10 seconds. Is there any push mechanism available with
ElasticSearch which pushes data from SQL to ES? And more what is the best
option for synching? Please reply urgently with the best option :).
We are running one application which has very large amount of data updating very frequently. And we are using SQL for any command like update/delete/add records. And for querying part we are using ElasticSearch. So is there any way which syncs my data very frequently say in 5 to 10 seconds. Is there any push mechanism available with ElasticSearch which pushes data from SQL to ES? And more what is the best option for synching? Please reply urgently with the best option :).
Thanks David for your prompt response. Actually we are using MSSQL not
MySQL. So this solution will not work for us.
On Thursday, March 20, 2014 1:04:28 PM UTC+5:30, David Pilato wrote:
Best: when you push in your application to MySQL, push as well to
elasticsearch.
--
David
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
Le 20 mars 2014 à 08:29, Komal Parekh <komald...@gmail.com <javascript:>>
a écrit :
Hello,
We are running one application which has very large amount of data
updating very frequently. And we are using SQL for any command like
update/delete/add records. And for querying part we are using
Elasticsearch. So is there any way which syncs my data very frequently say
in 5 to 10 seconds. Is there any push mechanism available with
Elasticsearch which pushes data from SQL to ES? And more what is the best
option for synching? Please reply urgently with the best option :).
Thanks David for your prompt response. But we want some automatic Push or
poll mechanism for this.
On Thursday, March 20, 2014 1:04:28 PM UTC+5:30, David Pilato wrote:
Best: when you push in your application to MySQL, push as well to
elasticsearch.
--
David
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
Le 20 mars 2014 à 08:29, Komal Parekh <komald...@gmail.com <javascript:>>
a écrit :
Hello,
We are running one application which has very large amount of data
updating very frequently. And we are using SQL for any command like
update/delete/add records. And for querying part we are using
Elasticsearch. So is there any way which syncs my data very frequently say
in 5 to 10 seconds. Is there any push mechanism available with
Elasticsearch which pushes data from SQL to ES? And more what is the best
option for synching? Please reply urgently with the best option :).
Yes we have done some implementation on river but it does not give real
time data. And also it is not as much mature. So for real time data we need
to use some push mechanism which can help up to have realtime data.
On Thursday, March 20, 2014 3:13:13 PM UTC+5:30, Jörg Prante wrote:
As David said, for push, you must modify your middleware that performs the
insert/update/delete - there is nothing ES can do for you. You must add an
ES client that can execute the respective operations on your data against
an ES cluster.
Poll method does not scale, push does scale.
I do not understand JDBC river is not mature, I happily accept bug reports
and patches. Maybe you discovered that river instances do not scale and are
kind of limited.
Yes we have done some implementation on river but it does not give real
time data. And also it is not as much mature. So for real time data we need
to use some push mechanism which can help up to have realtime data.
On Thursday, March 20, 2014 3:13:13 PM UTC+5:30, Jörg Prante wrote:
This is a complex issue with no single, simple, answer. Performing
synchronization between two databases is a non-trivial undertaking for any
technology. For an overview of strategies and techniques please see my
article on the topic Keeping Elasticsearch in Synchttps://www.found.no/foundation/keeping-elasticsearch-in-sync/
.
Andrew Cholakian
Developer, Found Hosted Elasticsearch
www.found.no, @foundsays
On Thursday, March 20, 2014 2:29:30 AM UTC-5, Komal Parekh wrote:
Hello,
We are running one application which has very large amount of data
updating very frequently. And we are using SQL for any command like
update/delete/add records. And for querying part we are using
Elasticsearch. So is there any way which syncs my data very frequently say
in 5 to 10 seconds. Is there any push mechanism available with
Elasticsearch which pushes data from SQL to ES? And more what is the best
option for synching? Please reply urgently with the best option :).
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
