Visualize two time series with nested objects

Elastic Stack Kibana
1

Hello everyone,

I am trying to visualize two time series in Kibana reading the data from Elasticsearch.
The structure of my document is the following:

{
  "sensorName": "Sensor1",
  "values": [
    {
      "feature": {
        "key": "feature1",
        "value": 67
      }
    },
    {
      "feature": {
        "key": "feature2",
        "value": 15
      }
    }
  ],
  "timestamp": 1631519919904
}

I want to visualize one time series for the values in the first element of the array (the one with “key”=“feature1”) and another time series for the second element of the array (“key”=“feature2”).

I tried the following query in KQL:

values.feature:{key:"feature1"}

But I got a visualization error:

[esaggs] > values.feature.key is not a nested field but is in nested group "values.feature" in the KQL expression.

How can I fix the visualization?

Thank you for your help.

2

Hello @maccn

I think this might be relevant - Kibana Query Language | Kibana Guide [7.15] | Elastic

Whats the mapping for this index?

3

Thank you for your answer. I tried to change my query in:

values:{feature.key: "feature1" }

But I got a similar error:

[esaggs] > values.feature.key is not a nested field but is in nested group "values" in the KQL expression.

The mapping for the index is:

{
  "topic-sensor" : {
    "mappings" : {
      "properties" : {
        "timestamp" : {
          "type" : "date"
        },
        "values" : {
          "properties" : {
            "feature" : {
              "properties" : {
                "key" : {
                  "type" : "text",
                  "fields" : {
                    "keyword" : {
                      "type" : "keyword",
                      "ignore_above" : 256
                    }
                  }
                },
                "value" : {
                  "type" : "float"
                }
              }
            }
          }
        },
        "sensorName" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        }
      }
    }
  }
}
4

If possible, this would likely be easier if you could denormalize your data. Instead of having both array elements in the same doc, put them in different docs.

You mention using nested fields but thats not shown in the mapping - Nested field type | Elasticsearch Guide [7.14] | Elastic

It looks like you data is indexed as object types - Object field type | Elasticsearch Guide [7.14] | Elastic

5

Unfortunately, I can't denormalize the data. Is there another solution to obtain what I want?

Moreover, I thought that "values" is the nested field and the element of the array are the objects, am I wrong?

6

It might be worthwhile to create a new index to test queries against.

Moreover, I thought that "values" is the nested field and the element of the array are the objects, am I wrong?

This seems like a reasonable expectation but its not confirmed by the mapping. I'd like to see values be marked as type nested.

7

I tried to create a new index on a different instance of Elasticsearch with the following mapping:

PUT /topic-sensor
{
  "mappings" : {
    "properties" : {
      "timestamp" : {
        "type" : "date"
      },
      "values" : {
        "type": "nested", 
        "properties" : {
          "feature" : {
            "properties" : {
              "key" : {
                "type" : "text"
              },
              "value" : {
                "type" : "float"
              }
            }
          }
        }
      },
      "sensorName" : {
        "type" : "text"
      }
    }
  }
}

Now, when I try to create a visualization, I can't choose any field in the Aggregation if I select the Average operation. The error is the following:

The index pattern topic-sensor does not contain any of the following compatible field types: number or histogram

But, in the search bar, Kibana suggests me the correct field (like in the example on the documentation):

values:{ feature.key : "feature1"}

It seems that the query will work but the field values.feature.value is not visible for the aggregation operation.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.