Can we join multiple index in Logstash after reading for elasticsearch and then create a new index

[Stephy_Jacob]

Hi Team,
I have my data of three different views ingested in Elasticsearch as three different indices. Now I have a requirement to create a Dashboard in Kibana with the combination of data from three views. So now I need to ingest the data again after joining and then create a new index in elasticsearch as elasticsearch does not support join in 6.6.
Can we use Logstash in this case? As in can we read all three elasticsearch indices to Logstash, do the join transformation in logstash and ingest it back to elasticsearch as a new index and then build visualizations on that index.

[Hendrik_Muhs]

Hi,

combining indices can be done with Transform, however you need at least 7.3.

Can you provide more information, best with some data examples (If you do not want to leak information, rename the fields and put in some generic values in the example)?

With an index pattern that has all 3 indices in it, you might not even need to re-ingest but can directly query the 3 indices with 1 search request. But again, that's highly data dependent and depends on whether joining data has to be done on document level before visualization.