How to keep Database and Elasticsearch consistent using Logstash

(Patrick) #1

I have a Relational database, I have used Logstash to migrate my Relational Postgres data to elasticsearch.

Now how can I keep my Database and Elasticsearch Server in sync?

(David Pilato) #2

I shared most of my thoughts there:

Basically, I'd recommend modifying the application layer if possible and send data to elasticsearch in the same "transaction" as you are sending your data to the database.

(Patrick) #3

@dadoonet Is there any other way of doing it ?

(David Pilato) #4

Did you read my blog? The start of it at least?

(Patrick) #5

@dadoonet Yes I had gone through it earlier. The blog is very helpful. It is great help

I was able to find the solution I used 'record_last_run' to record the last run the logstash and used the scheduler to check for changes in the Database

(David Pilato) #6