Elasticsearch and sql tables


(Garry Robertson) #1

I have data stored in a sql table that i would like to load into elasticsearch.
What would be the best way to do this.
The table is updated frequently and i need elasticsearch to reflect this.

Garry


(Mark Walkom) #2

Logstash has a JDBC input you could use to pull data from the table periodically.

Otherwise you should consider writing to both the DB and Elasticsearch at the same time, if you want more real time updates.


(Garry Robertson) #3

What about using filebeats as it seems to be more lightweight?
I could then export the db data to a flat file on a regular basis.


(Mark Walkom) #4

That could work.


(Garry Robertson) #5

In order to stop data duplication would i need to make the document_id a primary key to replicate the sql table pk?


(Mark Walkom) #6

Yep.


(Garry Robertson) #7

Thanks


(system) #8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.