[Kibana]: Nested child fields can not be visualized in discovery and be used like a filter aggregation

Elastic Stack Kibana
1

Hello,

Could you please give me an advice on how to be able to visualize and count field which is in the child doc and it is type: nested.
I have seen several questions since '15 but no workarounds for this specific issue.

However, in the Kibana console i managed to get the proper count but when i put it like a raw query int the discovery section it gives a different and not accurate count.

Following query:

GET my_index/_count
{

"query" : {
    "nested": {
        "path": "field1",
        "query": {
            "bool": {
                "must": [
                    {
                        "exists": {
                            "field": "field1.field2"
                        }
                    }
                ]
            }
        }
    }
}

}

2

The KQL query language has recently added support for nested fields: https://www.elastic.co/guide/en/kibana/7.6/kuery-query.html#kuery-query-nested-field

Those should be easier to use than raw queries

3

Thank you for your answer, but since my field with type nested is in the child doc its seems the KQL queries don't work either. This is a portion of my predefined mapping

   "field1" : {
				"type" : "nested",
				"properties" : {
					"field2" : {"type" : "keyword"},
					"field3" : {"type" : "keyword"},
					"field4" : {
						"type": "nested",
						"properties" : {
							"field5" : {"type" : "keyword"},
							"timestamp" : {"type" : "date"}
						}
					}
				}
			}

I want to check how many docs have the field: "field1.field2"(ex. exists:field1.field2). Can you suggest me a solution query, or maybe I need to put the include_in_parent/root parameter in the mapping just below the type:nested parameter so i can perform exists or KQL queries?

4

If it's possible for you, then the easiest way now is to copy the fields into a non-nested type via copy_to so Kibana can work with it. Only KQL supports some situations around nested objects at the moment, so "unnesting" is required to create e.g. visualizations.

5

My doc mapping, in regards to "field1", must be nested because one particular document can have several "field1" objects, so "unnesting" does not really work for me.
So if there is not Kibana support for querying and visualizing nested child fields, can i make scripted field or my own custom filter in order to visualize (or in extend to filter with Python and than somehow visualize in Kibana)? This particular visualization is of great importance for me.
Thank you for taking the time answering my questions! I really appreciate it!

6

copy_to will also work if there are multiple values - a field in Elasticsearch can always also contain an array of its type. The things you loose is the association between the individual fields of the nested objects -

{ nested: [ { field1: 'a', field2:'b' }, { field1: 'c', field2:'d' } ] }

would become

{ nested.field1: ['a', 'c'], nested.field2: ['b', 'd']}

So Kibana wouldn't know anymore whether 'a' occured together with 'b' or 'd'. But for simple exist queries and also quite some visualizations this wouldn't matter.

If this is important for your visualization, another approach is to split up the documents and create a separate top level document for each nested object.

{ nested: [ { field1: 'a', field2:'b' }, { field1: 'c', field2:'d' } ], field3: 1234 }

would become these two documents:

{ nested: { field1: 'a', field2:'b' }, field3: 1234 }
{ nested: { field1: 'c', field2:'d' }, field3: 1234 }
1 Like
7

Thank you very much this worked for me! I also tried, just to include "include_in_parent:true" parameter beneath the nested child field, and I was able to apply exists aggregation on it :slight_smile:

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.