Elasticsearch taking input from cassandra and visualizing through kibana

[jafferwilson]

Hello everyone,
Please let me know whether it is possible to connect the cassandra data with elasticsearch. Can I fetch the data from cassandra into elasticsearch and use kibana to visualize it. if yes then how?
if no then why not?

Kindly, help me. I do not know whether to use elasticsearch or not for this purpose.

[dadoonet (David Pilato)]

Best option IMHO is to send data from your application to both systems at the same time.

[jafferwilson]

Please can you explain to me what exactly you want to say. I am new to elasticsearch. Hence do not know how I can make elasticsearch interact with cassandra. Currently I am just experimenting whether elasticsearch is the best solution for my search or not. So I am using the cassandra and elasticsearch on localhost or standalone system.
If you can explain with an example, it will be awesome and very helpful.

[dadoonet (David Pilato)]

I meant that you probably have an application which is writing data to Cassandra.

In this application, while you send data to Cassandra, send also data to elasticsearch.

I explained something similar but for RDBMS at https://david.pilato.fr/blog/2015-05-09-advanced-search-for-your-legacy-application/.

To me you can apply the same concept to any other datastore and Elasticsearch.

I'm not sure if there are projects to synchronize from Cassandra to elasticsearch.

You can look at using Logstash JDBC input with a Cassandra JDBC driver as someone explained at:

How to continuously pull from Cassandra into Logstash

Hi all, I'm trying to setup Logstash to continuously pull information from Cassandra, in the same fashion it works with file inputs. I'm using the Cassandra JDBC Driver and I can successfully pull the contents, but once I get the result of my SELECT statement, Logstash closes down. The next time I start it, it reads the entire table again. Is there a way to configure it to run as a daemon and pull any new entries matching my statement? Here is my current config: input { jdbc { jdbc_connect…

But it's more a batch which run every X minutes so not really real time. Might fit your needs though.

[eperry (Ed)]

If you want to just visualize data in Casandra you can look at this project
https://zeppelin.apache.org/

If you want historical and other data of your database
Logstash JDBC Connector would work.

I agree with DaDoonNet, that it is good to have the application send the data to PostGres and ELK for more real-time data but that is an architectural and requirements needs
https://www.confluent.io/blog/bottled-water-real-time-integration-of-postgresql-and-kafka/

once in kafka you can use logstash to consume and insert to Elastic