We are trying to build an index using 2 data sources. Data source A is the actual database where all records are updated regularly. Data source B is a referential database. It contains ids of some elements of data source A and other information associated with it.
So what we need to achieve is create documents using data source A and then monitor data source A for any updates. Update(add fields) using data source B and monitor those fields using data source B.
Does any body has any idea how this can be achieved using logstash or do we need to write code(java) specifically for that purpose without logstash.
I think this could be easily achieved by writing a service in any of the ES supported languages - java, python, etc. It would be an always-on service, of course. I'm not sure about logstash (it's been a while, for me) - but I can't imagine that being a use case for it.
Depending on the type of database(s) you use, you might be able to create sql triggers/procedures to do this from within the database itself - that might be something worth looking into as well.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.