No indexing with JDBC River plugin


(Eric Greene) #1

Hi I am getting started with elasticsearch and the jdbc river plugin. I
want to sync to a mysql database. I appear to have everything set up
correctly... ES starts with no trouble, I have installed the plugin and
copied the mysql connector. However when I begin with a simple test
command:

curl -XPUT 'localhost:9200/my_index/videos/_meta' -d '{
"type" : "jdbc",
"jdbc" : {
"url" : "jdbc:mysql://localhost:3306/my_db",
"user" : "eric",
"password" : "my_password",
"sql" : "select title, description, created, active from video",
"index" : "my_index",
"type" : "videos"
}
}'

Nothing from the mysql db is indexed. Instead it just indexes the
information above instead of the mysql data.

So if I follow the above with
curl -XGET 'localhost:9200/my_index/_search?pretty&q=*

I get the following for hits:

"hits" : {
"total" : 1,
"max_score" : 1.0,
"hits" : [ {
"_index" : "my_index",
"_type" : "videos",
"_id" : "_meta",
"_score" : 1.0, "_source" : {
"type" : "jdbc",
"jdbc" : {
"url" : "jdbc:mysql://localhost:3306/my_db",
"user" : "eric",
"password" : "my_password",
"sql" : "select title, description, created, active from video",
"index" : "my_index",
"type" : "videos"
}

Logs show that the index was created. But it isn't an index from mysql.

I have also double and tripled checked the mysql credentials are working
and that the sql statement works. I have also verified there is in fact
data in the video table.

It just seems like the river is being ignored entirely.

The following is from the log file:

[2014-03-03 10:09:28,565][INFO ][cluster.metadata ] [es_node]
[my_index] creating index, cause [auto(index api)], shards [5]/[1],
mappings []
[2014-03-03 10:09:28,800][INFO ][cluster.metadata ] [es_node]
[my_index] update_mapping [videos] (dynamic)

What could I be missing here?

I am using the following versions:

ES version 1.0.1
JDBC River version 1.0.0.1
mySQL-connector-java version 5.1.28

on Ubuntu 12.04

Thanks for any help.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/71a431cb-58a6-4fd6-87fb-6ec08ee37bc4%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Eric Greene) #2

I have figured it out. CHMOD of the mySQL-connector-river jar file to 755
fixed my issue. - Eric

On Monday, March 3, 2014 10:17:28 AM UTC-8, Eric Greene wrote:

Hi I am getting started with elasticsearch and the jdbc river plugin. I
want to sync to a mysql database. I appear to have everything set up
correctly... ES starts with no trouble, I have installed the plugin and
copied the mysql connector. However when I begin with a simple test
command:

curl -XPUT 'localhost:9200/my_index/videos/_meta' -d '{
"type" : "jdbc",
"jdbc" : {
"url" : "jdbc:mysql://localhost:3306/my_db",
"user" : "eric",
"password" : "my_password",
"sql" : "select title, description, created, active from video",
"index" : "my_index",
"type" : "videos"
}
}'

Nothing from the mysql db is indexed. Instead it just indexes the
information above instead of the mysql data.

So if I follow the above with
curl -XGET 'localhost:9200/my_index/_search?pretty&q=*

I get the following for hits:

"hits" : {
"total" : 1,
"max_score" : 1.0,
"hits" : [ {
"_index" : "my_index",
"_type" : "videos",
"_id" : "_meta",
"_score" : 1.0, "_source" : {
"type" : "jdbc",
"jdbc" : {
"url" : "jdbc:mysql://localhost:3306/my_db",
"user" : "eric",
"password" : "my_password",
"sql" : "select title, description, created, active from video",
"index" : "my_index",
"type" : "videos"
}

Logs show that the index was created. But it isn't an index from mysql.

I have also double and tripled checked the mysql credentials are working
and that the sql statement works. I have also verified there is in fact
data in the video table.

It just seems like the river is being ignored entirely.

The following is from the log file:

[2014-03-03 10:09:28,565][INFO ][cluster.metadata ] [es_node]
[my_index] creating index, cause [auto(index api)], shards [5]/[1],
mappings []
[2014-03-03 10:09:28,800][INFO ][cluster.metadata ] [es_node]
[my_index] update_mapping [videos] (dynamic)

What could I be missing here?

I am using the following versions:

ES version 1.0.1
JDBC River version 1.0.0.1
mySQL-connector-java version 5.1.28

on Ubuntu 12.04

Thanks for any help.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ce42222e-1ef2-46a8-9bea-c9a7371abbaa%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Jörg Prante) #3

All you ned are read permissions on jars in the classpath.

Jörg

On Mon, Mar 3, 2014 at 8:06 PM, Eric Greene ericdgreene@gmail.com wrote:

I have figured it out. CHMOD of the mySQL-connector-river jar file to 755
fixed my issue. - Eric

On Monday, March 3, 2014 10:17:28 AM UTC-8, Eric Greene wrote:

Hi I am getting started with elasticsearch and the jdbc river plugin. I
want to sync to a mysql database. I appear to have everything set up
correctly... ES starts with no trouble, I have installed the plugin and
copied the mysql connector. However when I begin with a simple test
command:

curl -XPUT 'localhost:9200/my_index/videos/_meta' -d '{
"type" : "jdbc",
"jdbc" : {
"url" : "jdbc:mysql://localhost:3306/my_db",
"user" : "eric",
"password" : "my_password",
"sql" : "select title, description, created, active from video",
"index" : "my_index",
"type" : "videos"
}
}'

Nothing from the mysql db is indexed. Instead it just indexes the
information above instead of the mysql data.

So if I follow the above with
curl -XGET 'localhost:9200/my_index/_search?pretty&q=*

I get the following for hits:

"hits" : {
"total" : 1,
"max_score" : 1.0,
"hits" : [ {
"_index" : "my_index",
"_type" : "videos",
"_id" : "_meta",
"_score" : 1.0, "_source" : {
"type" : "jdbc",
"jdbc" : {
"url" : "jdbc:mysql://localhost:3306/my_db",
"user" : "eric",
"password" : "my_password",
"sql" : "select title, description, created, active from video",
"index" : "my_index",
"type" : "videos"
}

Logs show that the index was created. But it isn't an index from mysql.

I have also double and tripled checked the mysql credentials are working
and that the sql statement works. I have also verified there is in fact
data in the video table.

It just seems like the river is being ignored entirely.

The following is from the log file:

[2014-03-03 10:09:28,565][INFO ][cluster.metadata ] [es_node]
[my_index] creating index, cause [auto(index api)], shards [5]/[1],
mappings []
[2014-03-03 10:09:28,800][INFO ][cluster.metadata ] [es_node]
[my_index] update_mapping [videos] (dynamic)

What could I be missing here?

I am using the following versions:

ES version 1.0.1
JDBC River version 1.0.0.1
mySQL-connector-java version 5.1.28

on Ubuntu 12.04

Thanks for any help.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/ce42222e-1ef2-46a8-9bea-c9a7371abbaa%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/ce42222e-1ef2-46a8-9bea-c9a7371abbaa%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHFn5MeU0CzxoMQUceMixaAWmMNg_ch1K9GA%3DYkA_M8SA%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(system) #4