Need recommendations for related data querying

Elastic Stack Elasticsearch
1

I'm new to elastic search and are now struggling with modeling the data structure.
I have many devices like
device: {
identifier,
IP,
MAC,
name,
provider,
last_active_time,
etc..
}

and devices keep reporting their status(or property) continuously,
property is like:
property {
device_identifier,
property_name,
property_value,
modified_time,
etc...
}
device_identifier + property_name is unique and property is updated(or upserted) each time it got reported.
Since there are many devices and each device have one or multiple property. And I need to filter out
devices which match query condition based on both device and property document.
should I use nested document or children document ?
I tried application-side join, but there are too many devices and properties(millions), and is slow.

2

I'd start with something like:

PUT property/_doc/1
{
  device: {
    identifier: "",
    IP: "",
    MAC: "",
    name: "",
    provider: "",
    last_active_time: "",
  },
  property_name: "",
  property_value: "",
  modified_time: ""
}

And see where this goes.

3

Thanks for reply.
Are you suggesting that we should use one document ?
Each device may have multiple different properties and I need to put it in a list in this case.
I'm not sure if I can do queries as I expected because the document says:

Arrays of objects
Arrays of objects do not work as you would expect: you cannot query each object independently of the other objects in the array. If you need to be able to do this then you should use the nested datatype instead of the object datatype.

This is explained in more detail in Nested datatype.

or should I use nested document ?
document says that any updates to nested types would reindex the source document, I don't want to
do so because other queries for device, like query devices by their names, may be blocked or slowed down.

4

Yes.

And may be both:

  • properties
  • devices

The later to be able to search for devices only whatever the properties.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.