Using 2 different data sources to create/update index

Elastic Stack Elasticsearch
1

We are trying to build an index using 2 data sources. Data source A is the actual database where all records are updated regularly. Data source B is a referential database. It contains ids of some elements of data source A and other information associated with it.

So what we need to achieve is create documents using data source A and then monitor data source A for any updates. Update(add fields) using data source B and monitor those fields using data source B.

Does any body has any idea how this can be achieved using logstash or do we need to write code(java) specifically for that purpose without logstash.

2

I think this could be easily achieved by writing a service in any of the ES supported languages - java, python, etc. It would be an always-on service, of course. I'm not sure about logstash (it's been a while, for me) - but I can't imagine that being a use case for it.

Depending on the type of database(s) you use, you might be able to create sql triggers/procedures to do this from within the database itself - that might be something worth looking into as well.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.