SQL and elasticsearch

[kanthimathi]

Hi all,

Planning to find out the possibilities of implementing elastic search to connect with databases and use kibana as a visualization tool. As we already use elastic search for logging I am aware of the visualizations available and the deployment options available. For this purpose, we used filebeat and metricbeat.

Our requirement is to create dashboards from databases such as SQL, MongoDB, S3, REST API and stripe. I did check the input plugin available in the logstash and made sure the options available. Now I would like to know few things.

1. When we use logstash to ship SQL into ES and use kibana for visualization, will it work same as what we did for logging?

2. how will the data be indexed? i.e. When there are tables in SQL how will it appear in ES? I went through few post and found that there is no JOIN option available if that is the case how can we merge them?

3. Is there any option to connect mongoDB with elasticsearch?

Thanks,
Kanthi.

[magnusbaeck (Magnus Bäck)]

When we use logstash to ship SQL into ES and use kibana for visualization, will it work same as what we did for logging?

Probably yes, but I'm not sure what you're getting at.

how will the data be indexed? i.e. When there are tables in SQL how will it appear in ES? I went through few post and found that there is no JOIN option available if that is the case how can we merge them?

It depends. There's no single true way of transferring normalized relational data to denormalized documents in Elasticsearch.

[kanthimathi]

Thanks for your reply @magnusbaeck

We would like to try kibana instead of other BI tool we currently use. Would like to know how it works and the possibilities of that