# Keeping jdbc river in sync with DB **URL:** \ **Category:** Elasticsearch\ **Created:** [January 21, 2015, 3:56pm UTC](https://discuss.elastic.co/t/keeping-jdbc-river-in-sync-with-db/21751 "2015-01-21T15:56:00Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Abid\_Hussain](https://avatars.discourse-cdn.com/v4/letter/a/dfb087/32.png) [@Abid\_Hussain](https://discuss.elastic.co/u/Abid_Hussain)\ **Post date:** [January 21, 2015, 3:56pm UTC](https://discuss.elastic.co/t/keeping-jdbc-river-in-sync-with-db/21751/1 "2015-01-21T15:56:00Z")
Hi all, I'm trying to figure out how to configure jdbc river in production environment. We want to index a DB with appr. 10% write queries and 90% read queries and need to keep ES index in sync with DB. So I wonder how two configure river to keep Index in near-realtime Sync with DB. I'm thinking about pulling the DB changes every minute. Is jdbc-river capable of updating it's already indexed data and is this a good strategy? Regards, Abid -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/99857c16-1ba3-4267-94c8-7538971f366b%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/99857c16-1ba3-4267-94c8-7538971f366b%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [January 21, 2015, 4:02pm UTC](https://discuss.elastic.co/t/keeping-jdbc-river-in-sync-with-db/21751/2 "2015-01-21T16:02:20Z")
Yes, you can set up the JDBC plugin with a 'schedule' parameter so that it is started every minute. See the doc at > **[GitHub - jprante/elasticsearch-jdbc: JDBC importer for Elasticsearch](https://github.com/jprante/elasticsearch-jdbc)** > > JDBC importer for Elasticsearch. Contribute to jprante/elasticsearch-jdbc development by creating an account on GitHub. Note, if you delete rows in DB, this is not per se detected by JDBC plugin. There is no synchronization between DB rows and JSON docs. You can only work with documents from the rows that your SQL statement discovers. JSON doc deletions require some additional effort. Jörg On Wed, Jan 21, 2015 at 4:56 PM, Abid Hussain [hussain@novacom.mygbiz.com](mailto:hussain@novacom.mygbiz.com) wrote: > Hi all, > > I'm trying to figure out how to configure jdbc river in production > environment. > > We want to index a DB with appr. 10% write queries and 90% read queries > and need to keep ES index in sync with DB. > > So I wonder how two configure river to keep Index in near-realtime Sync > with DB. > > I'm thinking about pulling the DB changes every minute. Is jdbc-river > capable of updating it's already indexed data and is this a good strategy? > > Regards, > > Abid > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/99857c16-1ba3-4267-94c8-7538971f366b%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/99857c16-1ba3-4267-94c8-7538971f366b%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/99857c16-1ba3-4267-94c8-7538971f366b%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/99857c16-1ba3-4267-94c8-7538971f366b%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGXpv-28GuNLR86YW79rC%2BUjqx%3DwYsCGEpEbn8ZOgAFsA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGXpv-28GuNLR86YW79rC%2BUjqx%3DwYsCGEpEbn8ZOgAFsA%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Abid\_Hussain](https://avatars.discourse-cdn.com/v4/letter/a/dfb087/32.png) [@Abid\_Hussain](https://discuss.elastic.co/u/Abid_Hussain)\ **Post date:** [January 22, 2015, 5:44pm UTC](https://discuss.elastic.co/t/keeping-jdbc-river-in-sync-with-db/21751/3 "2015-01-22T17:44:44Z")
Thanks for you quick reply. You say _There is no synchronization between DB rows and JSON docs._ So how is the index being updated? I guess that new entries are being added to index. _Does the river also recognize differences between single columns and their corresponding field in index and merge them into index?_ Regards, Abid Am Mittwoch, 21. Januar 2015 17:02:27 UTC+1 schrieb Jörg Prante: > Yes, you can set up the JDBC plugin with a 'schedule' parameter so that it > is started every minute. > > See the doc at > > [GitHub - jprante/elasticsearch-jdbc: JDBC importer for Elasticsearch](https://github.com/jprante/elasticsearch-river-jdbc) > > Note, if you delete rows in DB, this is not per se detected by JDBC > plugin. There is no synchronization between DB rows and JSON docs. You can > only work with documents from the rows that your SQL statement discovers. > JSON doc deletions require some additional effort. > > Jörg > > On Wed, Jan 21, 2015 at 4:56 PM, Abid Hussain \<[hus...@novacom.mygbiz.com](mailto:hus...@novacom.mygbiz.com) > \\> wrote: > > > Hi all, > > > > I'm trying to figure out how to configure jdbc river in production > > environment. > > > > We want to index a DB with appr. 10% write queries and 90% read queries > > and need to keep ES index in sync with DB. > > > > So I wonder how two configure river to keep Index in near-realtime Sync > > with DB. > > > > I'm thinking about pulling the DB changes every minute. Is jdbc-river > > capable of updating it's already indexed data and is this a good strategy? > > > > Regards, > > > > Abid > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/99857c16-1ba3-4267-94c8-7538971f366b%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/99857c16-1ba3-4267-94c8-7538971f366b%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/99857c16-1ba3-4267-94c8-7538971f366b%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/99857c16-1ba3-4267-94c8-7538971f366b%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/227dada3-0ea0-442a-80c5-e8a59b0462fa%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/227dada3-0ea0-442a-80c5-e8a59b0462fa%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [January 22, 2015, 6:05pm UTC](https://discuss.elastic.co/t/keeping-jdbc-river-in-sync-with-db/21751/4 "2015-01-22T18:05:36Z")
All you can do is setting up an SQL command, the result rows are fetched, and converted to JSON docs, which are indexed. Jörg On Thu, Jan 22, 2015 at 6:44 PM, Abid Hussain [hussain@novacom.mygbiz.com](mailto:hussain@novacom.mygbiz.com) wrote: > Thanks for you quick reply. > > You say _There is no synchronization between DB rows and JSON docs._ > > So how is the index being updated? I guess that new entries are being > added to index. _Does the river also recognize differences between single > columns and their corresponding field in index and merge them into index?_ > > Regards, > > Abid > > Am Mittwoch, 21. Januar 2015 17:02:27 UTC+1 schrieb Jörg Prante: > > > Yes, you can set up the JDBC plugin with a 'schedule' parameter so that > > it is started every minute. > > > > See the doc at > > > > [GitHub - jprante/elasticsearch-jdbc: JDBC importer for Elasticsearch](https://github.com/jprante/elasticsearch-river-jdbc) > > > > Note, if you delete rows in DB, this is not per se detected by JDBC > > plugin. There is no synchronization between DB rows and JSON docs. You can > > only work with documents from the rows that your SQL statement discovers. > > JSON doc deletions require some additional effort. > > > > Jörg > > > > On Wed, Jan 21, 2015 at 4:56 PM, Abid Hussain [hus...@novacom.mygbiz.com](mailto:hus...@novacom.mygbiz.com) > > wrote: > > > > > Hi all, > > > > > > I'm trying to figure out how to configure jdbc river in production > > > environment. > > > > > > We want to index a DB with appr. 10% write queries and 90% read queries > > > and need to keep ES index in sync with DB. > > > > > > So I wonder how two configure river to keep Index in near-realtime Sync > > > with DB. > > > > > > I'm thinking about pulling the DB changes every minute. Is jdbc-river > > > capable of updating it's already indexed data and is this a good strategy? > > > > > > Regards, > > > > > > Abid > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > msgid/elasticsearch/99857c16-1ba3-4267-94c8-7538971f366b% > > > [40googlegroups.com](http://40googlegroups.com) > > > [https://groups.google.com/d/msgid/elasticsearch/99857c16-1ba3-4267-94c8-7538971f366b%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/99857c16-1ba3-4267-94c8-7538971f366b%40googlegroups.com?utm_medium=email&utm_source=footer) > > > . > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/227dada3-0ea0-442a-80c5-e8a59b0462fa%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/227dada3-0ea0-442a-80c5-e8a59b0462fa%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/227dada3-0ea0-442a-80c5-e8a59b0462fa%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/227dada3-0ea0-442a-80c5-e8a59b0462fa%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFWp2RXAzM3HGCt88wGAhFfEKnogv%3D8oFjr3YNMCKWLww%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFWp2RXAzM3HGCt88wGAhFfEKnogv%3D8oFjr3YNMCKWLww%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![PascalR](https://avatars.discourse-cdn.com/v4/letter/p/258eb7/32.png) [@PascalR](https://discuss.elastic.co/u/PascalR)\ **Post date:** [March 15, 2016, 9:52am UTC](https://discuss.elastic.co/t/keeping-jdbc-river-in-sync-with-db/21751/5 "2016-03-15T09:52:48Z")
Is there also a way to update ES through a commit event from the SQL database? Something like an Active MQ trigger. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 11:08pm UTC](https://discuss.elastic.co/t/keeping-jdbc-river-in-sync-with-db/21751/6 "2017-07-05T23:08:22Z")