Connecting documents from different kibana indexes

Hello Elastic-Community,
I am wondering if there is a way to connect two datasets from two different indexes with each other to create a visualization.

For example:

From Index_1 I receive general informations of a device, that looks like this:

"DeviceName": "machine_1",
"ID": 001,

And from Index_2 I get temperature-values from this device, but i don't have the "DeviceName"-field:

"ID": 001,
"temperature": "20", 
"timestamp": "2021-04-15T07:34:33.079Z",

Now i want to create a visualization, where i can filter or search for the "DeviceName" such as "machine_1" and get the current temperature and a chart with the temperatur over time.

Thanks for any kind of help!

@timroes can we have more inputs here? Thank you.


this is unfortunately not really able that way. There is a join field type, but that's unfortunately not supported in Kibana. In general when working with search engines like Elasticsearch (in contrast to relational databases) it's recommended to denormalize data (instead of normalization what you might be used from a relational database), i.e. in this case including the device name into the second document as well. Given how Elasticsearch (and most search engines) are build technically, this is the preferred version for gaining the relevant speed and still be able to execute the desired queries you want.


I agree to @timroes

There are also inbuilt possibilities to do so.
You can use transforms to do the merge or you use the enrich processor on index time.
I would use the second alternative in your case.

Can you please provide some further information to your suggestion. Where can I find these functionalities and how do I use them correctly?

You can find it in Kibana -> Management - > Ingest Pipelines
There you configure a new ingest Pipeline for your data using the enrich processor.

Finally you add this new pipeline as default pipeline to the Meeting index template of your main index.