Sync Postgresql and Elasticsearch using Filebeat

Elastic Stack Elasticsearch
1

Hi,

I have a considerable amount of information in a Postgresql database. Registers are inserted on this database on demand. We are currently interested in syncronize this database and Elasticsearch in order to have the same information in both the Posgresql data base and an Elasticsearch index, also considering that the Elasticsearch index is updated when new information is inserted into the database (I've seen this process called streaming of information).

I've searched for some solutions online, most of which proposes using Logstash or external tools. For another indexing task we, have an already configured filebeat process, so I'd like to know if there's a way of configuring filebeat to perform this syncronisation. I've read some tutorials regarding the indexing of postgresql log information using filebeat; however, we are interested in the information sync.

Thanks in advance,
Best regards,

2

Hi @tmslara.a

Filebeat is not the right tool to sync PostGresql Content with Elasticsearch. Filebeat is meant to ship logs, not sync contents of other Sources / RDBMs etc.

Logstash or another ETL Tool would be a better fit for sure.

3

I've seen some solutions that use Logstash to perform the sync Postgres-ES, but something is not completely clear for me. After configuring Logtash, when I change (or include) some register into the Postgres database, will this also update the information in ES?

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.