Cassandra river plugin installation issue


(shamsul haque) #1

I have downloaded river from: https://github.com/eBay/cassandra-river

change the settings in file: CassandraRiver.java as per my Cassandra
setting:

if (riverSettings.settings().containsKey("cassandra")) {
@SuppressWarnings("unchecked")
Map<String, Object> couchSettings = (Map<String, Object>)
settings.settings().get("cassandra");
this.clusterName =
XContentMapValues.nodeStringValue(couchSettings.get("cluster_name"), "Test
Cluster");
this.keyspace =
XContentMapValues.nodeStringValue(couchSettings.get("keyspace"),
"topic_space");
this.columnFamily =
XContentMapValues.nodeStringValue(couchSettings.get("column_family"),
"users");
this.batchSize =
XContentMapValues.nodeIntegerValue(couchSettings.get("batch_size"), 1000);
this.hosts =
XContentMapValues.nodeStringValue(couchSettings.get("hosts"),
"localhost:9160");
this.username =
XContentMapValues.nodeStringValue(couchSettings.get("username"),
"USERNAME");
this.password =
XContentMapValues.nodeStringValue(couchSettings.get("password"), "P$$WD");
} else {
/*
* Set default values
*/
this.clusterName = "Test Cluster";
this.keyspace = "topic_space";
this.columnFamily = "users";
this.batchSize = 1000;
this.hosts = "localhost:9160";
this.username = "USERNAME";
this.password = "P$$WD";
}

when i build maven using given command, mvn clean package in TEST mvn log
it shows:


T E S T S

Running org.elasticsearch.river.cassandra.CassandraRiverIntegrationTest
Configuring TestNG with:
org.apache.maven.surefire.testng.conf.TestNG652Configurator@67eaf25d
Exception in thread "Queue-Indexer-thread-0" java.lang.NullPointerException
at
org.elasticsearch.river.cassandra.CassandraRiver$Indexer.run(CassandraRiver.java:149)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Exception in thread "Queue-Indexer-thread-2" java.lang.NullPointerException
at
org.elasticsearch.river.cassandra.CassandraRiver$Indexer.run(CassandraRiver.java:149)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Exception in thread "Queue-Indexer-thread-5" java.lang.NullPointerException
at
org.elasticsearch.river.cassandra.CassandraRiver$Indexer.run(CassandraRiver.java:149)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Exception in thread "Queue-Indexer-thread-4" java.lang.NullPointerException

i tried to do same after installing plugin in ES, it shows same error
continuously.
Anybody have any idea, whats going wrong with my setup??

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ef16f8fa-3145-43be-87ce-e8f53060938f%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(David Pilato) #2

Why do you build your own version of the Cassandra river?
You don't need to do that to use a river, right?

That said, what is line CassandraRiver.java:149

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr

Le 8 janvier 2014 at 11:06:42, shamsul haque (shams.hq@gmail.com) a écrit:

I have downloaded river from: https://github.com/eBay/cassandra-river

change the settings in file: CassandraRiver.java as per my Cassandra setting:

if (riverSettings.settings().containsKey("cassandra")) {
@SuppressWarnings("unchecked")
Map<String, Object> couchSettings = (Map<String, Object>) settings.settings().get("cassandra");
this.clusterName = XContentMapValues.nodeStringValue(couchSettings.get("cluster_name"), "Test Cluster");
this.keyspace = XContentMapValues.nodeStringValue(couchSettings.get("keyspace"), "topic_space");
this.columnFamily = XContentMapValues.nodeStringValue(couchSettings.get("column_family"), "users");
this.batchSize = XContentMapValues.nodeIntegerValue(couchSettings.get("batch_size"), 1000);
this.hosts = XContentMapValues.nodeStringValue(couchSettings.get("hosts"), "localhost:9160");
this.username = XContentMapValues.nodeStringValue(couchSettings.get("username"), "USERNAME");
this.password = XContentMapValues.nodeStringValue(couchSettings.get("password"), "P$$WD");
} else {
/*
* Set default values
*/
this.clusterName = "Test Cluster";
this.keyspace = "topic_space";
this.columnFamily = "users";
this.batchSize = 1000;
this.hosts = "localhost:9160";
this.username = "USERNAME";
this.password = "P$$WD";
}

when i build maven using given command, mvn clean package in TEST mvn log it shows:


T E S T S

Running org.elasticsearch.river.cassandra.CassandraRiverIntegrationTest
Configuring TestNG with: org.apache.maven.surefire.testng.conf.TestNG652Configurator@67eaf25d
Exception in thread "Queue-Indexer-thread-0" java.lang.NullPointerException
at org.elasticsearch.river.cassandra.CassandraRiver$Indexer.run(CassandraRiver.java:149)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Exception in thread "Queue-Indexer-thread-2" java.lang.NullPointerException
at org.elasticsearch.river.cassandra.CassandraRiver$Indexer.run(CassandraRiver.java:149)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Exception in thread "Queue-Indexer-thread-5" java.lang.NullPointerException
at org.elasticsearch.river.cassandra.CassandraRiver$Indexer.run(CassandraRiver.java:149)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Exception in thread "Queue-Indexer-thread-4" java.lang.NullPointerException

i tried to do same after installing plugin in ES, it shows same error continuously.
Anybody have any idea, whats going wrong with my setup??

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ef16f8fa-3145-43be-87ce-e8f53060938f%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.52cd23e2.12200854.1449b%40MacBook-Air-de-David.local.
For more options, visit https://groups.google.com/groups/opt_out.


(shamsul haque) #3

CassandraRiver.java:149 contains:
logger.info("Starting thread with {} keys", this.keys.rowColumnMap.size());
where rowColumnMap is a map, and may be empty thats why this error comes

And at first i build that river module normally and install it as a plugin
in ES.
But when i ran script:
curl -XPUT 'localhost:9200/_river/userinfo/_meta' -d '{
"type" : "cassandra",
"cassandra" : {
"cluster_name" : "Test Cluster",
"keyspace" : "topic_space",
"column_family" : "users",
"batch_size" : 100,
"hosts" : "localhost:9160"
},
"index" : {
"index" : "userinfo",
"type" : "users"
}
}'

same error comes in ES console, same as i have copied from maven console.
and it also not fetching data from cassandra to ES.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/0bc5a111-0562-46ff-a640-9b9241638887%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(shamsul haque) #4

1 change which i have made in that cassandra-river project is to change the
casandra jar version from 1.3 to 2.0.3 in pox.xml as i am using Cassandra
2.0.4
Any idea whats going wrong?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/3f8867fd-5f92-47a5-bf6b-5f4b2f5306ee%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(David Pilato) #5

So probably

CassandraCFData cassandraData = db.getCFData(columnFamily, start, 1000);

did not get any data from Cassandra?

Never played with this plugin either Cassandra so I'm afraid I can't help more here!

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr

Le 8 janvier 2014 at 11:21:26, shamsul haque (shams.hq@gmail.com) a écrit:

CassandraRiver.java:149 contains:
logger.info("Starting thread with {} keys", this.keys.rowColumnMap.size());
where rowColumnMap is a map, and may be empty thats why this error comes

And at first i build that river module normally and install it as a plugin in ES.
But when i ran script:
curl -XPUT 'localhost:9200/_river/userinfo/_meta' -d '{
"type" : "cassandra",
"cassandra" : {
"cluster_name" : "Test Cluster",
"keyspace" : "topic_space",
"column_family" : "users",
"batch_size" : 100,
"hosts" : "localhost:9160"
},
"index" : {
"index" : "userinfo",
"type" : "users"
}
}'

same error comes in ES console, same as i have copied from maven console.
and it also not fetching data from cassandra to ES.

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/0bc5a111-0562-46ff-a640-9b9241638887%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.52cd2806.109cf92e.1449b%40MacBook-Air-de-David.local.
For more options, visit https://groups.google.com/groups/opt_out.


(shamsul haque) #6

ok, Thanks for pointing this.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/69213f6c-0e23-4a8d-bbf8-f9423d7200b3%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(shamsul haque) #7

issue solved:

in river code, when fetching data from Casandra it
uses HFactory.createRangeSlicesQuery(keyspace, STR, STR, STR); to get data
and the table which i was using to get data contain Primary Key as int id,
after changing that to text it starts pulling data from Cassandra to ES.

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/050a535b-990b-4dd7-9af9-6caea0e6d4a5%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(AKhan) #8

Shamsul Haque, I am also getting the same error on Elastic Search console
even though the type of id is string not int in my case. Any clue ?

On Wednesday, January 8, 2014 11:06:36 AM UTC+1, shamsul haque wrote:

I have downloaded river from: https://github.com/eBay/cassandra-river

change the settings in file: CassandraRiver.java as per my Cassandra
setting:

if (riverSettings.settings().containsKey("cassandra")) {
@SuppressWarnings("unchecked")
Map<String, Object> couchSettings = (Map<String, Object>)
settings.settings().get("cassandra");
this.clusterName =
XContentMapValues.nodeStringValue(couchSettings.get("cluster_name"), "Test
Cluster");
this.keyspace =
XContentMapValues.nodeStringValue(couchSettings.get("keyspace"),
"topic_space");
this.columnFamily =
XContentMapValues.nodeStringValue(couchSettings.get("column_family"),
"users");
this.batchSize =
XContentMapValues.nodeIntegerValue(couchSettings.get("batch_size"), 1000);
this.hosts =
XContentMapValues.nodeStringValue(couchSettings.get("hosts"),
"localhost:9160");
this.username =
XContentMapValues.nodeStringValue(couchSettings.get("username"),
"USERNAME");
this.password =
XContentMapValues.nodeStringValue(couchSettings.get("password"), "P$$WD");
} else {
/*
* Set default values
*/
this.clusterName = "Test Cluster";
this.keyspace = "topic_space";
this.columnFamily = "users";
this.batchSize = 1000;
this.hosts = "localhost:9160";
this.username = "USERNAME";
this.password = "P$$WD";
}

when i build maven using given command, mvn clean package in TEST mvn log
it shows:


T E S T S

Running org.elasticsearch.river.cassandra.CassandraRiverIntegrationTest
Configuring TestNG with:
org.apache.maven.surefire.testng.conf.TestNG652Configurator@67eaf25d
Exception in thread "Queue-Indexer-thread-0" java.lang.NullPointerException
at
org.elasticsearch.river.cassandra.CassandraRiver$Indexer.run(CassandraRiver.java:149)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Exception in thread "Queue-Indexer-thread-2" java.lang.NullPointerException
at
org.elasticsearch.river.cassandra.CassandraRiver$Indexer.run(CassandraRiver.java:149)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Exception in thread "Queue-Indexer-thread-5" java.lang.NullPointerException
at
org.elasticsearch.river.cassandra.CassandraRiver$Indexer.run(CassandraRiver.java:149)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Exception in thread "Queue-Indexer-thread-4" java.lang.NullPointerException

i tried to do same after installing plugin in ES, it shows same error
continuously.
Anybody have any idea, whats going wrong with my setup??

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/281251a1-7226-477b-8c75-1d5934336d8e%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(shamsul haque) #9

The issue is with Hector API, i have posted an issue in Hector API issue

link https://github.com/hector-client/hector/issues/649 . I have
changed river implementation for getting data from Cassandra by Hector API
to Cassandra Java Driver.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/4a0426b1-b735-4185-a9d4-050ba63f02eb%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Abhishek Mukherjee) #10

Is the Hector API issue fixed? I have setup the ebay/cassandra river and it
shows 2 documents inside. However I cannot search them. I get an timed out
exception. I suspect this is again from Hector API because of the H before
timedoutexception.

:{"error":"HTimedOutException[TimedOutException()]; nested:
TimedOutException;
","node":{"id":"11vByNw4QPKKLrtUx-vkjw","name":"Krang","transport_address":"inet[/192.168.1.103:9300]"}}

On Wednesday, February 5, 2014 11:35:05 AM UTC+5:30, Shams Haque wrote:

The issue is with Hector API, i have posted an issue in Hector API issue

link https://github.com/hector-client/hector/issues/649 . I have
changed river implementation for getting data from Cassandra by Hector API
to Cassandra Java Driver.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/6eb46de3-09c6-4e3c-af78-a2ba805339bb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(shamsul haque) #11

Hi,

The issue was not with Hector API, issue has been fixed by using "WITH
COMPACT STORAGE" when creating column families in Cassandra.
Here i have posted it:

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/9de84b4d-0d99-4483-bd1e-5f9471c0b97d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(system) #12