Datastructure relational data

Elastic Stack Elasticsearch
1

I have the following data structure: I have a hierarchical structure where every element can link to n-further elements. (Similar to a tree, but an object can have multiple roots. A link is only downwards possible). The data is distributed on 4-5 hierarchical levels.

I want to store this data in elasticsearch and query the dependencies in elasticsearch and create visualizations in Kibana. Whats the best way to store this data? If I would denormalize the data it would take x20 the disk space, as there are many relations between the objects. Joins in the query are not possible?

Are there application side ways to join the data in Kibana?

2

No, joins are not possible.

Not that I am aare of. Kibana works best with flattened data.

Is this an estimate or the result of a test?

3

Is this an estimate or the result of a test?

The disk space is an estimation.

Which data structure would you recommend?

4

I would recommend trying to fully denormalize your data and index a good amount of it too see exactly how much extra space it takes up on disk as this is the type of data Kibana works best with. It could be that your estimate is accurate but it could also be less.

5

Thanks a lot, I'll try that and tell you the results.
You think I should fully denormalize the data? Why not use nested objects?

6

Kibana does not support nested documents well and having very large deeply nested structures can be quite inefficient.

7

Ok, so how should I denormalize this data structure. I for example want to query the status of all elements related to obj1:

( obj1: status)  -----
                       (obj3: status)
(obj2: status) -------


(obj4: status) -------- (obj5: status)

So my approach would be the following: (obj1: status, status2, status3) and (obj4: status, status5). Is this the right way? Hand how would you name the status fields of all connected elemnts?

8

I'm wondering what's the best way to name the fields. It isn't possible to have the same name, but if you have different ones, it's hard to search.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.