Help with visualizations, multiple indexes, and a common field

icsy7867
Mon, 14 Aug 2023 12:40:27 +0000
Kibana

I am having some confusion on how to properly do this. Most of my "database" experience comes from relational databases. So my mind immediately goes to inner/outer joins, which I know is not applicable here. I have read through many posts here, and have found lots of different types of information...

HOWEVER, I have several indexes, mainly to do with information on computers in specific systems. I have made an API class using the elastic PHP agent, and I have configured the ability to add a common field. So each index has what I call a "CommonField" to capture another field. So each of my 5 indexes have this field with the same data (And all lowercase).

So now, I am trying to create a table that uses the multiple indexes that shows two fields (in this case timestamps) of each "CommonField" (Which is a computername). For the most part, in each of the indexes each "CommonField" or computer name has exactly 1 row.

So I discovered runtime lookup fields. This seemed like it solved the issue, but then I found out that this does not really work for visualizations if I read that correctly.

Then I found transforms. So I am reading into this, and it might be a good solution. However I am considering just joining all of the datasets together. However my supervisor is against "Duplicating data" and thinks we should be able to use the existing datasets (Which makes sense to me, but perhaps isnt ideal in this situation).

I am new to EKS, and it's such a powerful tool, and I have been down so many rabbits holes that I decided to reach out some folks that may no better. Does anyone have any guidance on the proper path forward here? For my current data we are talking around 1000 rows, which is not too many, but for larger datasets does this recommendation change? Thank you!

8 posts - 4 participants