Hi all,
I am working on mysql to ES replication using
elasticsearch-river-jdbc.my configuration is,
curl -XPUT '10.10.10.53:9200/_river/test/_meta' -d '{
"type" : "jdbc",
"jdbc" : {
"driver" : "com.mysql.jdbc.Driver",
"url" : "jdbc:mysql://localhost:3306/Mahesh",
"user" : "root",
"password" : "root",
"sql" : "select id as _id,name,age from main" ,
"schedule": "* * * * * ?" ,
"autocommit" : true,
"strategy" : "simple"
} }'
It replicates data every time when mysql table got updated.mysql table has
nine number of rows.when i see the elasticsearch logs, every time the
number of rows get doubled.
2015-02-13 10:24:28,139][INFO ][river.jdbc.RiverMetrics ] pipeline
org.xbib.elasticsearch.plugin.jdbc.RiverPipeline@204426cd complete: river
jdbc/test metrics: 9 rows, 3.0049217412872427 mean, (0.0 0.0 0.0), ingest
metrics: elapsed 2 seconds, 643.0 bytes bytes, 64.0 bytes avg, 0 MB/s
[2015-02-13 10:24:28,160][INFO ][index.shard.service ] [Volla]
[jdbc][0] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:28,160][INFO ][index.shard.service ] [Volla]
[jdbc][3] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:28,160][INFO ][index.shard.service ] [Volla]
[jdbc][1] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:28,160][INFO ][index.shard.service ] [Volla]
[jdbc][2] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:28,161][INFO ][index.shard.service ] [Volla]
[jdbc][4] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:28,257][INFO ][index.shard.service ] [Volla]
[jdbc][0] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:28,258][INFO ][index.shard.service ] [Volla]
[jdbc][3] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:28,258][INFO ][index.shard.service ] [Volla]
[jdbc][1] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:28,258][INFO ][index.shard.service ] [Volla]
[jdbc][2] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:28,258][INFO ][index.shard.service ] [Volla]
[jdbc][4] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:28,387][INFO ][river.jdbc.RiverMetrics ] pipeline
org.xbib.elasticsearch.plugin.jdbc.RiverPipeline@7dcf6e34 complete: river
jdbc/test metrics: 18 rows, 5.544464111788077 mean, (0.0 0.0 0.0), ingest
metrics: elapsed 0 seconds, 643.0 bytes bytes, 33.0 bytes avg, 0.003 MB/s
[2015-02-13 10:24:28,410][INFO ][index.shard.service ] [Volla]
[jdbc][0] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:28,417][INFO ][index.shard.service ] [Volla]
[jdbc][3] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:28,417][INFO ][index.shard.service ] [Volla]
[jdbc][1] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:28,417][INFO ][index.shard.service ] [Volla]
[jdbc][2] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:28,417][INFO ][index.shard.service ] [Volla]
[jdbc][4] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:28,509][INFO ][index.shard.service ] [Volla]
[jdbc][0] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:28,509][INFO ][index.shard.service ] [Volla]
[jdbc][3] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:28,509][INFO ][index.shard.service ] [Volla]
[jdbc][1] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:28,510][INFO ][index.shard.service ] [Volla]
[jdbc][2] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:28,510][INFO ][index.shard.service ] [Volla]
[jdbc][4] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:28,619][INFO ][river.jdbc.RiverMetrics ] pipeline
org.xbib.elasticsearch.plugin.jdbc.RiverPipeline@18630651 complete: river
jdbc/test metrics: 27 rows, 7.762557850074249 mean, (0.0 0.0 0.0), ingest
metrics: elapsed 0 seconds, 643.0 bytes bytes, 22.0 bytes avg, 0.003 MB/s
[2015-02-13 10:24:29,011][INFO ][index.shard.service ] [Volla]
[jdbc][0] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:29,011][INFO ][index.shard.service ] [Volla]
[jdbc][3] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:29,011][INFO ][index.shard.service ] [Volla]
[jdbc][1] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:29,011][INFO ][index.shard.service ] [Volla]
[jdbc][2] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:29,012][INFO ][index.shard.service ] [Volla]
[jdbc][4] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:29,083][INFO ][index.shard.service ] [Volla]
[jdbc][0] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:29,083][INFO ][index.shard.service ] [Volla]
[jdbc][3] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:29,083][INFO ][index.shard.service ] [Volla]
[jdbc][1] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:29,084][INFO ][index.shard.service ] [Volla]
[jdbc][2] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:29,084][INFO ][index.shard.service ] [Volla]
[jdbc][4] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:29,142][INFO ][river.jdbc.RiverMetrics ] pipeline
org.xbib.elasticsearch.plugin.jdbc.RiverPipeline@3a9922cd complete: river
jdbc/test metrics: 36 rows, 8.995315015057283 mean, (0.0 0.0 0.0), ingest
metrics: elapsed 0 seconds, 643.0 bytes bytes, 17.0 bytes avg, 0.004 MB/s
[2015-02-13 10:24:30,013][INFO ][index.shard.service ] [Volla]
[jdbc][0] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:30,014][INFO ][index.shard.service ] [Volla]
[jdbc][3] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:30,014][INFO ][index.shard.service ] [Volla]
[jdbc][1] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:30,014][INFO ][index.shard.service ] [Volla]
[jdbc][2] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:30,014][INFO ][index.shard.service ] [Volla]
[jdbc][4] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:30,076][INFO ][index.shard.service ] [Volla]
[jdbc][0] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:30,076][INFO ][index.shard.service ] [Volla]
[jdbc][3] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:30,076][INFO ][index.shard.service ] [Volla]
[jdbc][1] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:30,076][INFO ][index.shard.service ] [Volla]
[jdbc][2] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:30,077][INFO ][index.shard.service ] [Volla]
[jdbc][4] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:30,146][INFO ][river.jdbc.RiverMetrics ] pipeline
org.xbib.elasticsearch.plugin.jdbc.RiverPipeline@53a3cfef complete: river
jdbc/test metrics: 45 rows, 8.990752399785658 mean, (0.0 0.0 0.0), ingest
metrics: elapsed 0 seconds, 643.0 bytes bytes, 13.0 bytes avg, 0.004 MB/s
[2015-02-13 10:24:31,011][INFO ][index.shard.service ] [Volla]
[jdbc][0] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:31,011][INFO ][index.shard.service ] [Volla]
[jdbc][3] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:31,011][INFO ][index.shard.service ] [Volla]
[jdbc][1] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:31,011][INFO ][index.shard.service ] [Volla]
[jdbc][2] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:31,011][INFO ][index.shard.service ] [Volla]
[jdbc][4] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:31,073][INFO ][index.shard.service ] [Volla]
[jdbc][0] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:31,073][INFO ][index.shard.service ] [Volla]
[jdbc][3] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:31,073][INFO ][index.shard.service ] [Volla]
[jdbc][1] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:31,073][INFO ][index.shard.service ] [Volla]
[jdbc][2] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:31,073][INFO ][index.shard.service ] [Volla]
[jdbc][4] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:31,146][INFO ][river.jdbc.RiverMetrics ] pipeline
org.xbib.elasticsearch.plugin.jdbc.RiverPipeline@523ad6a0 complete: river
jdbc/test metrics: 54 rows, 8.990958072186503 mean, (0.0 0.0 0.0), ingest
metrics: elapsed 0 seconds, 643.0 bytes bytes, 11.0 bytes avg, 0.004 MB/s
[2015-02-13 10:24:32,010][INFO ][index.shard.service ] [Volla]
[jdbc][0] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:32,010][INFO ][index.shard.service ] [Volla]
[jdbc][3] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:32,011][INFO ][index.shard.service ] [Volla]
[jdbc][1] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:32,011][INFO ][index.shard.service ] [Volla]
[jdbc][2] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:32,011][INFO ][index.shard.service ] [Volla]
[jdbc][4] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:32,074][INFO ][index.shard.service ] [Volla]
[jdbc][0] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:32,075][INFO ][index.shard.service ] [Volla]
[jdbc][3] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:32,075][INFO ][index.shard.service ] [Volla]
[jdbc][1] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:32,075][INFO ][index.shard.service ] [Volla]
[jdbc][2] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:32,075][INFO ][index.shard.service ] [Volla]
[jdbc][4] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:32,141][INFO ][river.jdbc.RiverMetrics ] pipeline
org.xbib.elasticsearch.plugin.jdbc.RiverPipeline@39154de7 complete: river
jdbc/test metrics: 63 rows, 8.998767886129954 mean, (0.0 0.0 0.0), ingest
metrics: elapsed 0 seconds, 643.0 bytes bytes, 10.0 bytes avg, 0.004 MB/s
[2015-02-13 10:24:33,010][INFO ][index.shard.service ] [Volla]
[jdbc][0] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:33,010][INFO ][index.shard.service ] [Volla]
[jdbc][3] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:33,010][INFO ][index.shard.service ] [Volla]
[jdbc][1] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:33,011][INFO ][index.shard.service ] [Volla]
[jdbc][2] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:33,011][INFO ][index.shard.service ] [Volla]
[jdbc][4] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:33,087][INFO ][index.shard.service ] [Volla]
[jdbc][0] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:33,087][INFO ][index.shard.service ] [Volla]
[jdbc][3] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:33,087][INFO ][index.shard.service ] [Volla]
[jdbc][1] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:33,087][INFO ][index.shard.service ] [Volla]
[jdbc][2] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:33,088][INFO ][index.shard.service ] [Volla]
[jdbc][4] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:33,159][INFO ][river.jdbc.RiverMetrics ] pipeline
org.xbib.elasticsearch.plugin.jdbc.RiverPipeline@3e91cae complete: river
jdbc/test metrics: 72 rows, 8.979638876501681 mean, (0.0 0.0 0.0), ingest
metrics: elapsed 0 seconds, 643.0 bytes bytes, 8.0 bytes avg, 0.004 MB/s
[2015-02-13 10:24:34,010][INFO ][index.shard.service ] [Volla]
[jdbc][0] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:34,010][INFO ][index.shard.service ] [Volla]
[jdbc][3] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:34,010][INFO ][index.shard.service ] [Volla]
[jdbc][1] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:34,010][INFO ][index.shard.service ] [Volla]
[jdbc][2] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:34,010][INFO ][index.shard.service ] [Volla]
[jdbc][4] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:34,070][INFO ][index.shard.service ] [Volla]
[jdbc][0] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:34,071][INFO ][index.shard.service ] [Volla]
[jdbc][3] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:34,071][INFO ][index.shard.service ] [Volla]
[jdbc][1] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:34,071][INFO ][index.shard.service ] [Volla]
[jdbc][2] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:34,071][INFO ][index.shard.service ] [Volla]
[jdbc][4] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:34,133][INFO ][river.jdbc.RiverMetrics ] pipeline
org.xbib.elasticsearch.plugin.jdbc.RiverPipeline@3723a913 complete: river
jdbc/test metrics: 81 rows, 9.007423179575905 mean, (0.0 0.0 0.0), ingest
metrics: elapsed 0 seconds, 643.0 bytes bytes, 7.0 bytes avg, 0.005 MB/s
[2015-02-13 10:24:35,013][INFO ][index.shard.service ] [Volla]
[jdbc][0] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:35,013][INFO ][index.shard.service ] [Volla]
[jdbc][3] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:35,013][INFO ][index.shard.service ] [Volla]
[jdbc][1] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:35,014][INFO ][index.shard.service ] [Volla]
[jdbc][2] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:35,014][INFO ][index.shard.service ] [Volla]
[jdbc][4] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:35,075][INFO ][index.shard.service ] [Volla]
[jdbc][0] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:35,079][INFO ][index.shard.service ] [Volla]
[jdbc][3] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:35,081][INFO ][index.shard.service ] [Volla]
[jdbc][1] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:35,081][INFO ][index.shard.service ] [Volla]
[jdbc][2] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:35,082][INFO ][index.shard.service ] [Volla]
[jdbc][4] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:35,142][INFO ][river.jdbc.RiverMetrics ] pipeline
org.xbib.elasticsearch.plugin.jdbc.RiverPipeline@4a90682e complete: river
jdbc/test metrics: 90 rows, 8.998411807311246 mean, (0.0 0.0 0.0), ingest
metrics: elapsed 0 seconds, 643.0 bytes bytes, 7.0 bytes avg, 0.004 MB/s
[2015-02-13 10:24:36,011][INFO ][index.shard.service ] [Volla]
[jdbc][0] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:36,011][INFO ][index.shard.service ] [Volla]
[jdbc][3] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:36,011][INFO ][index.shard.service ] [Volla]
[jdbc][1] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:36,011][INFO ][index.shard.service ] [Volla]
[jdbc][2] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:36,011][INFO ][index.shard.service ] [Volla]
[jdbc][4] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:36,217][INFO ][index.shard.service ] [Volla]
[jdbc][0] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:36,217][INFO ][index.shard.service ] [Volla]
[jdbc][3] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:36,217][INFO ][index.shard.service ] [Volla]
[jdbc][1] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:36,217][INFO ][index.shard.service ] [Volla]
[jdbc][2] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:36,217][INFO ][index.shard.service ] [Volla]
[jdbc][4] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:36,286][INFO ][river.jdbc.RiverMetrics ] pipeline
org.xbib.elasticsearch.plugin.jdbc.RiverPipeline@3a298e12 complete: river
jdbc/test metrics: 99 rows, 8.882277483775141 mean, (0.0 0.0 0.0), ingest
metrics: elapsed 0 seconds, 643.0 bytes bytes, 6.0 bytes avg, 0.002 MB/s
[2015-02-13 10:24:37,010][INFO ][index.shard.service ] [Volla]
[jdbc][0] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:37,011][INFO ][index.shard.service ] [Volla]
[jdbc][3] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:37,011][INFO ][index.shard.service ] [Volla]
[jdbc][1] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:37,011][INFO ][index.shard.service ] [Volla]
[jdbc][2] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:37,011][INFO ][index.shard.service ] [Volla]
[jdbc][4] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:37,077][INFO ][index.shard.service ] [Volla]
[jdbc][0] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:37,077][INFO ][index.shard.service ] [Volla]
[jdbc][3] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:37,077][INFO ][index.shard.service ] [Volla]
[jdbc][1] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:37,077][INFO ][index.shard.service ] [Volla]
[jdbc][2] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:37,078][INFO ][index.shard.service ] [Volla]
[jdbc][4] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:37,138][INFO ][river.jdbc.RiverMetrics ] pipeline
org.xbib.elasticsearch.plugin.jdbc.RiverPipeline@1565e146 complete: river
jdbc/test metrics: 108 rows, 9.00217201705776 mean, (0.0 0.0 0.0), ingest
metrics: elapsed 0 seconds, 643.0 bytes bytes, 5.0 bytes avg, 0.005 MB/s
[2015-02-13 10:24:38,013][INFO ][index.shard.service ] [Volla]
[jdbc][0] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:38,013][INFO ][index.shard.service ] [Volla]
[jdbc][3] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:38,013][INFO ][index.shard.service ] [Volla]
[jdbc][1] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:38,013][INFO ][index.shard.service ] [Volla]
[jdbc][2] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:38,013][INFO ][index.shard.service ] [Volla]
[jdbc][4] updating refresh_interval from [1s] to [-1]
[2015-02-13 10:24:38,103][INFO ][index.shard.service ] [Volla]
[jdbc][0] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:38,103][INFO ][index.shard.service ] [Volla]
[jdbc][3] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:38,104][INFO ][index.shard.service ] [Volla]
[jdbc][1] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:38,104][INFO ][index.shard.service ] [Volla]
[jdbc][2] updating refresh_interval from [-1] to [1s]
[2015-02-13 10:24:38,104][INFO ][index.shard.service ] [Volla]
[jdbc][4] updating refresh_interval from [-1] to [1s]
Its looks like elasticsearch index gets deleted and adds mysql table data
again and again every time.Kindly give me the solution for this issue.
Thanks
Mahesh.S
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/9a08f6eb-14e9-485a-bcec-9048c61aedd3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.