Concept MS-SQL and Elasticsearch

Elastic Stack Elasticsearch
1

Hi @ all,

First of all I thank you for the support you gave me. I have learned a lot and all was for private purpose.

For my school I have to write something about: Dynamic analysis and I have to find and compare different solutions. One of my solution is elasticsearch. In our company we develop software which is using SQL.

In the end I want the date which is alreasdy in our SQL database also in the Elasticsearch DB. Yes for that we have different options as dadoonet mentioned:

@dadoonet
I ready your website as well and several posts concerning this topic.

  1. Solution:
    Our developers are programming an output which is in .json and I send that to elasticsearch? <- How I'm going to that if they could give me a json which already contains everyhting. Do I use beats to send that to Elasticsearch without the use of Logstash?

  2. Solution:
    I use this JDBC Plugin. How do I use that? Connect to MS SQL database and then put the date into elasticsearch. What happens with primary keys?
    Jdbc input plugin | Logstash Reference [8.11] | Elastic

  3. Any other Ideas?

On google I have looked for companys which are giving me the possibility to connect to SQL and have a simple user interface to define later the charts. I have found a lot of them.
Does anyone know on which platform they are based? Most of them look similar to each other and it looks like they have often ELK in the background and added something like a custom theme on Kibana or I don't know ;=). For me it looks like they all use ELK and have added something which can handle primary keys from SQL.
Are there any ressources about this problem or any thoughts are welcome.

Thank you for your information and time.

2

As I wrote in my blog post, I prefer a direct connection to elasticsearch than having to read again the data from the database later.

Do I use beats to send that to Elasticsearch without the use of Logstash?

Ideally they just send directly to elasticsearch or they send the JSON document to a message queue system like Kafka or RabbitMQ and read the queue with Logstash then.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.