Elasticsearch Data Streams: Update Strategies, Concerns, and Alternatives

[jainesh_singh]

Hey Christian,
Thanks for your help. I feel the solution you suggested would work for me. Using your suggested approach, I am facing certain challenges in Logstash implementation.
Could you share some light on :

Updating Elasticsearch Indices conditionally when referring to 2 database table

Description: We have two SQL tables: FileDetail for storing file details and FileUserActivity for file activities. Using Logstash, we're indexing data into Elasticsearch with a flat index approach, combining file details and activities in a single document. However, when file details change in the FileDetail table (e.g., file owner modification), we need a solution to update all previous Elasticsearch indices related to that file. Database Tables: FileDetail Table: Columns: fileid, ..., o…