# Realtime data streaming from cassandra to elasticsearch

**URL:** <https://discuss.elastic.co/t/realtime-data-streaming-from-cassandra-to-elasticsearch/170141>\
**Category:** Elasticsearch\
**Created:** [February 27, 2019, 10:34am UTC](https://discuss.elastic.co/t/realtime-data-streaming-from-cassandra-to-elasticsearch/170141 "2019-02-27T10:34:02Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![yasin\_mohammed](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yasin_mohammed/32/64406_2.png) [@yasin\_mohammed](https://discuss.elastic.co/u/yasin_mohammed)\
**Post date:** [February 27, 2019, 10:34am UTC](https://discuss.elastic.co/t/realtime-data-streaming-from-cassandra-to-elasticsearch/170141/1 "2019-02-27T10:34:03Z")

</div>

Hi...

What are the alternatives to extract data from cassandra to elasticsearch. CQL has many limitations with queries so we are not able to do it. Also apart from logstash is there any other way.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [February 27, 2019, 11:50am UTC](https://discuss.elastic.co/t/realtime-data-streaming-from-cassandra-to-elasticsearch/170141/2 "2019-02-27T11:50:41Z")

</div>

I'd recommend writing to elasticsearch directly from the application which is writing data in Cassandra. That's IMHO the best way, whatever the storage database is, to be as most as possible real time.

---

<div class="post-metadata">

**Author:** ![yasin\_mohammed](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yasin_mohammed/32/64406_2.png) [@yasin\_mohammed](https://discuss.elastic.co/u/yasin_mohammed)\
**Post date:** [February 27, 2019, 11:53am UTC](https://discuss.elastic.co/t/realtime-data-streaming-from-cassandra-to-elasticsearch/170141/3 "2019-02-27T11:53:46Z")

</div>

We are getting the data from nagios agents which translates the data before sending it to cassandra

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [February 27, 2019, 12:12pm UTC](https://discuss.elastic.co/t/realtime-data-streaming-from-cassandra-to-elasticsearch/170141/4 "2019-02-27T12:12:10Z")

</div>

I'm not familiar with nagios but can't you do the same with elasticsearch?  
Which leads me to another question: what do you need Cassandra for?

---

<div class="post-metadata">

**Author:** ![yasin\_mohammed](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yasin_mohammed/32/64406_2.png) [@yasin\_mohammed](https://discuss.elastic.co/u/yasin_mohammed)\
**Post date:** [February 27, 2019, 12:29pm UTC](https://discuss.elastic.co/t/realtime-data-streaming-from-cassandra-to-elasticsearch/170141/5 "2019-02-27T12:29:35Z")

</div>

The actual data lake is cassandra all the data is stored to cassandra. We are consuming this data to elastic building a ML on it and visualizing it to Kibana. I've always got the feel this is over engineering. If you could share your thoughts on this.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [February 27, 2019, 12:45pm UTC](https://discuss.elastic.co/t/realtime-data-streaming-from-cassandra-to-elasticsearch/170141/6 "2019-02-27T12:45:24Z")

</div>

Well. I think (but I don't know all the details so take it with caution), that data such as logs, monitoring... should not be stored in many systems unless there's a real need for that.

Probably I'd go for elasticsearch only and it's built in anomaly detection feature. But I'm biased as I'm working at elastic.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 27, 2019, 12:45pm UTC](https://discuss.elastic.co/t/realtime-data-streaming-from-cassandra-to-elasticsearch/170141/7 "2019-03-27T12:45:35Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
