# How to continuously pull from Cassandra into Logstash

**URL:** <https://discuss.elastic.co/t/how-to-continuously-pull-from-cassandra-into-logstash/44674>\
**Category:** Logstash\
**Created:** [March 17, 2016, 8:48am UTC](https://discuss.elastic.co/t/how-to-continuously-pull-from-cassandra-into-logstash/44674 "2016-03-17T08:48:56Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![nkanov](https://avatars.discourse-cdn.com/v4/letter/n/97f17d/32.png) [@nkanov](https://discuss.elastic.co/u/nkanov)\
**Post date:** [March 17, 2016, 8:48am UTC](https://discuss.elastic.co/t/how-to-continuously-pull-from-cassandra-into-logstash/44674/1 "2016-03-17T08:48:56Z")

</div>

Hi all,

I'm trying to setup Logstash to continuously pull information from Cassandra, in the same fashion it works with file inputs. I'm using the [Cassandra JDBC Driver](http://www.dbschema.com/cassandra-jdbc-driver.html) and I can successfully pull the contents, but once I get the result of my SELECT statement, Logstash closes down. The next time I start it, it reads the entire table again.  
Is there a way to configure it to run as a daemon and pull any new entries matching my statement?

Here is my current config:

input {  
jdbc {  
jdbc\_connection\_string =\> "jdbc:cassandra://hostname:9160"  
jdbc\_user =\> "cassandra"  
jdbc\_password =\> "cassandra"  
jdbc\_driver\_library =\> "$PATH/cassandra\_driver.jar"  
jdbc\_driver\_class =\> "org.apache.cassandra.cql.jdbc.CassandraDriver"  
statement =\> "SELECT \* FROM keyspace.table"  
}  
}

output {  
stdout {  
codec =\> json\_lines  
}  
}

Here's what happens when I start it:

root@012456ce9188:/# /opt/logstash/bin/logstash -f /test\_cassandra.conf  
Settings: Default pipeline workers: 8  
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".  
SLF4J: Defaulting to no-operation (NOP) logger implementation  
SLF4J: See [http://www.slf4j.org/codes.html#StaticLoggerBinder](http://www.slf4j.org/codes.html#StaticLoggerBinder) for further details.  
Logstash startup completed  
.....  
_entries returned here, as expected_  
......  
Logstash shutdown completed

Any help would be appreciated, I'm very new to Logstash.

---

<div class="post-metadata">

**Author:** ![ebuildy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ebuildy/32/6070_2.png) [@ebuildy](https://discuss.elastic.co/u/ebuildy)\
**Post date:** [March 17, 2016, 9:22am UTC](https://discuss.elastic.co/t/how-to-continuously-pull-from-cassandra-into-logstash/44674/2 "2016-03-17T09:22:48Z")

</div>

I think you forgot to set schedule option:

> **[Jdbc input plugin | Logstash Reference \[8.11\] | Elastic](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-jdbc.html#plugins-inputs-jdbc-schedule)**

> There is no schedule by default. If no schedule is given, then the statement is run exactly once.

---

<div class="post-metadata">

**Author:** ![nkanov](https://avatars.discourse-cdn.com/v4/letter/n/97f17d/32.png) [@nkanov](https://discuss.elastic.co/u/nkanov)\
**Post date:** [March 17, 2016, 11:24am UTC](https://discuss.elastic.co/t/how-to-continuously-pull-from-cassandra-into-logstash/44674/3 "2016-03-17T11:24:41Z")

</div>

Hi Thomas,

You are right, I had no schedule at all. Thanks for pointing me to the jdbc plugin documentation, it was something I had not found.

The solution to my problem has two parts:

1. Adding a schedule, like you said.
2. Using [predefined parameters](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-jdbc.html#_predefined_parameters), so that I only select the entries which have been added since the last time the query was run.

Thanks again!

---

<div class="post-metadata">

**Author:** ![Srivastavark](https://avatars.discourse-cdn.com/v4/letter/s/cc9497/32.png) [@Srivastavark](https://discuss.elastic.co/u/Srivastavark)\
**Post date:** [July 1, 2016, 1:33pm UTC](https://discuss.elastic.co/t/how-to-continuously-pull-from-cassandra-into-logstash/44674/4 "2016-07-01T13:33:34Z")

</div>

> [@nkanov](#):
>
> org.apache.cassandra.cql.jdbc.CassandraDriver

Hi Nkanov,

Can you please help me to use JDBC input plugin in logstash for cassandra. Here is my configuration,  
I am using same JDBC driver as you mentioned.

input {  
jdbc {  
jdbc\_connection\_string =\> "jdbc:cassandra://localhost:9160/mydata"  
jdbc\_user =\> "cassandra"  
jdbc\_driver\_library =\> "D:\driver\cassandrajdbc1.1"  
jdbc\_driver\_class =\> "org.apache.cassandra.cql.jdbc.CassandraDriver"  
statement =\> "SELECT \* from sampledata"  
}  
}  
output {

```
	stdout { codec => json }

```

}

error  
D:\ELK\logstash-2.3.3\logstash-2.3.3\bin\>logstash -f d:\cass.conf  
io/console not supported; tty will not be manipulated  
Settings: Default pipeline workers: 4  
←[31mPipeline aborted due to error {:exception=\>#\<LogStash::ConfigurationError:  
org.apache.cassandra.cql.jdbc.CassandraDriver not loaded. Are you sure you've in  
cluded the correct jdbc driver in :jdbc\_driver\_library?\>,

:backtrace=\>["D:/ELK/logstash-2.3.3/logstash-2.3.3/vendor/bundle/jruby/1.9/gems/logstash-input-jdbc-3.0.2/lib/logstash/plugin\_mixins/jdbc.rb:156:in  
`prepare_jdbc_connection'", "D:/ELK/logstash-2.3.3/logstash-2.3.3/vendor/bundle/jruby/1.9/gems/logstash-input-jdbc-3.0.2/lib/logstash/inputs/jdbc.rb:167:in `register'", "D:/ELK/logstash-2.3.3/logstash-2.3.3/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.3-java/lib/logstash/pipeline.rb:330:in  
`start_inputs'", "org/jruby/RubyArray.java:1613:in `each'", "D:/ELK/logstash-2.3.3/logstash-2.3.3/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.3-java/lib/logstash/pipeline.rb:329:in  
`start_inputs'", "D:/ELK/logstash-2.3.3/logstash-2.3.3/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.3-java/lib/logstash/pipeline.rb:180:in `start\_workers'", "D:/ELK/logstash-2.3.3/logstash-2.3.3/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.3-java/lib/logstash/pipeline.rb:136:in  
`run'", "D:/ELK/logstash-2.3.3/logstash-2.3.3/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.3-java/lib/logstash/agent.rb:473:in `start\_pipeline'"], :level=\>:error}←[0m  
stopping pipeline {:id=\>"main"}  
The signal HUP is in use by the JVM and will not work correctly on this platform

Thanks

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:49am UTC](https://discuss.elastic.co/t/how-to-continuously-pull-from-cassandra-into-logstash/44674/5 "2017-07-06T04:49:54Z")

</div>


