Sorry. I can't. I've never played with jdbc river.
Probably Jörg could help you here.
Good luck.
--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr
Le 17 décembre 2013 at 08:56:31, Frank Zhou (zhouxuesong@gmail.com) a écrit:
Hi, David
I am not sure, seems like now there are 4 sessions in mysql , run my 4 queries,
| 14989 | abc | devcassandra1.dfengg.com:34544 | ng30 | Query | 2818 | Writing to net | select ID as _id,BusinessId,MessageSent,MessageTo,Type,MessageFrom,CallBack,Protocol,CampaignId,Cust |
| 16726 | abc | 172.16.12.73:46379 | ng19 | Query | 88 | Writing to net | select ID+ 541009000 as _id,BusinessId,MessageSent,MessageTo,Type,MessageFrom,CallBack,Protocol,Camp |
| 16757 | abc | devcassandra2.dfengg.com:50452 | ng6 | Query | 63 | Writing to net | select ID+94502600 as _id,BusinessId,MessageSent,MessageTo,Type,MessageFrom,CallBack,Protocol,Campai |
| 16768 | abc | devcassandra3.dfengg.com:50952 | ng35 | Query | 49 | Writing to net | select ID+1597134900 as _id,BusinessId,MessageSent,MessageTo,Type,MessageFrom,CallBack,Protocol,Camp |
I am using "oneshot" now, and hope those 4 queries inject to /test2/SMSMT type. But until now, I just saw a /jdbc index got created there, and size continue increase, nothing added into /test2/SMSMT type. It is different than previous test which I see data in /test/ng30-SMSMT immediately after issue simple strategy query.
Could you help me review the jdbc river commands I am using this time?
Thanks
Frank
The commands I am using this time
- From ng30 load into SMSMT
curl -XPUT 'localhost:9200/_river/SMSMT30/_meta' -d '
{
"type" : "jdbc",
"jdbc" : {
"strategy" : "oneshot",
"poll" : "1m",
"max_retries" : 5,
"fetchsize" : 100,
"driver" : "com.mysql.jdbc.Driver",
"url" : "jdbc:mysql://172.16.2.95:3306/ng30",
"user" : "d3",
"autocommit" : true,
"password" : "ondemand",
"sql" : "select ID as _id,BusinessId,MessageSent,MessageTo,Type,MessageFrom,CallBack,Protocol,CampaignId,CustomerId,Industry,Subject,MessageText,FormattedMessageText,AppointmentId,ScheduledDeliveryTime,ActualDeliveryTime,ReadTime,ConfirmedTime,LatestDeliveryTime,MessageId,MessageStatusDescription,MessageStatus,Status,CreatedDate,CreatedUserID,LastModifiedDate,LastModifiedUserID,ScheduledTime,VisitId,BouncedTime,AssetId,DeliveryTime,ScompTime,ScompText,BouncedText,StructuredMessageText,ReferralID,AssetScheduleId,LastVisitTime,AppointmentHotlistId,EntityID from SMSMT"
},
"index" : {
"index" : "test2",
"type" : "SMSMT",
"fetchsize" : 30000,
"bulk_size": 30000,
"max_bulk_requests": 40
}
}'
- From ng19 load into SMSMT
mysql> select max(id),count(1) from ng30.SMSMT;
+-----------+----------+
| max(id) | count(1) |
+-----------+----------+
| 541008901 | 22317315 |
+-----------+----------+
1 row in set (1 min 9.71 sec)
curl -XPUT 'localhost:9200/_river/SMSMT19/_meta' -d '
{
"type" : "jdbc",
"jdbc" : {
"strategy" : "oneshot",
"poll" : "1m",
"max_retries" : 5,
"fetchsize" : 100,
"driver" : "com.mysql.jdbc.Driver",
"url" : "jdbc:mysql://172.16.2.95:3306/ng19",
"user" : "d3",
"autocommit" : true,
"password" : "ondemand",
"sql" : "select ID+ 541009000 as _id,BusinessId,MessageSent,MessageTo,Type,MessageFrom,CallBack,Protocol,CampaignId,CustomerId,Industry,Subject,MessageText,FormattedMessageText,AppointmentId,ScheduledDeliveryTime,ActualDeliveryTime,ReadTime,ConfirmedTime,LatestDeliveryTime,MessageId,MessageStatusDescription,MessageStatus,Status,CreatedDate,CreatedUserID,LastModifiedDate,LastModifiedUserID,ScheduledTime,VisitId,BouncedTime,AssetId,DeliveryTime,ScompTime,ScompText,BouncedText,StructuredMessageText,ReferralID,AssetScheduleId,LastVisitTime,AppointmentHotlistId,EntityID from SMSMT"
},
"index" : {
"index" : "test2",
"type" : "SMSMT",
"fetchsize" : 30000,
"bulk_size": 30000,
"max_bulk_requests": 40
}
}'
- From ng6 load into SMSMT
mysql> select max(id)+541009000,count(1) from ng19.SMSMT;
+-------------------+----------+
| max(id)+541009000 | count(1) |
+-------------------+----------+
| 945025141 | 7771470 |
+-------------------+----------+
curl -XPUT 'localhost:9200/_river/SMSMT6/_meta' -d '
{
"type" : "jdbc",
"jdbc" : {
"strategy" : "oneshot",
"poll" : "1m",
"max_retries" : 5,
"fetchsize" : 100,
"driver" : "com.mysql.jdbc.Driver",
"url" : "jdbc:mysql://172.16.2.95:3306/ng6",
"user" : "d3",
"autocommit" : true,
"password" : "ondemand",
"sql" : "select ID+94502600 as _id,BusinessId,MessageSent,MessageTo,Type,MessageFrom,CallBack,Protocol,CampaignId,CustomerId,Industry,Subject,MessageText,FormattedMessageText,AppointmentId,ScheduledDeliveryTime,ActualDeliveryTime,ReadTime,ConfirmedTime,LatestDeliveryTime,MessageId,MessageStatusDescription,MessageStatus,Status,CreatedDate,CreatedUserID,LastModifiedDate,LastModifiedUserID,ScheduledTime,VisitId,BouncedTime,AssetId,DeliveryTime,ScompTime,ScompText,BouncedText,StructuredMessageText,ReferralID,AssetScheduleId,LastVisitTime,AppointmentHotlistId,EntityID from SMSMT"
},
"index" : {
"index" : "test2",
"type" : "SMSMT",
"fetchsize" : 30000,
"bulk_size": 30000,
"max_bulk_requests": 40
}
}'
- From ng35 load into SMSMT
mysql> select max(id)+94502600,count(1) from ng6.SMSMT;
+------------------+----------+
| max(id)+94502600 | count(1) |
+------------------+----------+
| 1597134881 | 10162439 |
+------------------+----------+
1 row in set (1 min 37.13 sec)
curl -XPUT 'localhost:9200/_river/SMSMT35/_meta' -d '
{
"type" : "jdbc",
"jdbc" : {
"strategy" : "oneshot",
"poll" : "1m",
"max_retries" : 5,
"fetchsize" : 100,
"driver" : "com.mysql.jdbc.Driver",
"url" : "jdbc:mysql://172.16.2.95:3306/ng35",
"user" : "d3",
"autocommit" : true,
"password" : "ondemand",
"sql" : "select ID+1597134900 as _id,BusinessId,MessageSent,MessageTo,Type,MessageFrom,CallBack,Protocol,CampaignId,CustomerId,Industry,Subject,MessageText,FormattedMessageText,AppointmentId,ScheduledDeliveryTime,ActualDeliveryTime,ReadTime,ConfirmedTime,LatestDeliveryTime,MessageId,MessageStatusDescription,MessageStatus,Status,CreatedDate,CreatedUserID,LastModifiedDate,LastModifiedUserID,ScheduledTime,VisitId,BouncedTime,AssetId,DeliveryTime,ScompTime,ScompText,BouncedText,StructuredMessageText,ReferralID,AssetScheduleId,LastVisitTime,AppointmentHotlistId,EntityID from SMSMT"
},
"index" : {
"index" : "test2",
"type" : "SMSMT",
"fetchsize" : 30000,
"bulk_size": 30000,
"max_bulk_requests": 40
}
}'
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/2a3dc22b-b7d5-4ee4-98a4-f2075ff41c54%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.52b009ec.7724c67e.6956%40MacBook-Air-de-David.local.
For more options, visit https://groups.google.com/groups/opt_out.
