Kibana visualizations over nested objects


(Hayden Marchant) #1

We are experimenting with Kibana for creating dashboards over our index,
and for some use cases, we need to do aggregations over fields in nested
objects of our document.

We are having problems with getting visualizations to work for fields over
nested objects

So in our case, our document represents a user who can have zero or more
accounts. Each account has a product and account id. The account object
needs to be defined as a nested object since we want to keep the context of
each account in order to support queries such as prod='ABC' and
accid='1234'. Pretty much same reason as the example
here: http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/nested-objects.html

THe relevant part of the mapping mapping is here:

"profiles": {
"_id": {
"index": "not_analyzed"
},
"properties": {
"account": {
"properties": {
"product": {
"index": "not_analyzed",
"type": "string"
},
"id_int": {
"fielddata": {
"loading": "lazy",
"format": "doc_values"
},
"doc_values": true,
"type": "long"
}
},
"type": "nested"
},
"home_region": {
"index": "not_analyzed",
"type": "string"
},
"home_postal_code": {
"index": "not_analyzed",
"type": "string"
},
"home_city": {
"index": "not_analyzed",
"type": "string"
}

}

}

Our simplest use case is to have a pie chart that is segmented by the
product field which is part of the nested object.

We would normally submit a query such as :

{
"size" : 0,
"query" : {
"filtered" : {
"filter" : {
"match_all" : { }
}
}
},
"aggregations" : {
"product" : {
"nested" : {
"path" : "account"
},
"aggregations" : {
"product" : {
"terms" : {
"field" : "product",
"size" : 5,
"order" : {
"_count" : "desc"
}
}
}
}
}
}
}

From sniffing the network, seems that kibana is doing the following:

"query":{"query_string":{"query":""}},"aggs":{"2":{"terms":{"field":"account.product","size":5,"order":{"_count":"desc"}}}},"fields":["","_source"],"script_fields":{}

This works great for non-nested objects, but not for nested objects.

Is there a way we can customize the visualization to do this type of
aggregation? Looks like visState and
kibanaSavedObjectsMeta.searchSourceJSON in the Edit Visualization Object
screen might be a good start, though I have not yet been able to follow the
logic that generates the query string that is submitted to Elasticsearch.

Many thanks in advance,
Hayden

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/1ff55288-18a1-4ab2-864b-d8d24517921e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(pavan.bkv) #2

@Hayden_Marchant, did you find a solution on how to visualize in kibana for nested fields?


(Hayden Marchant) #3

If I recall correctly, we modeled the nested object twice - once as a
nested object and once as a flat object. Kibana knew to visualize the flat
object. Our API needed the nested for complex contextual queries so we kept
that.

Hope that helps,
Hayden
pavan_bkv http://discuss.elastic.co/users/pavan_bkv pavan.bkv
http://discuss.elastic.co/users/pavan_bkv
August 6

@Hayden_Marchant http://discuss.elastic.co/users/hayden_marchant, did you
find a solution on how to visualize in kibana for nested fields?

To respond, reply to this email or visit


in your browser.


(pavan.bkv) #4

Wow that's a bummer!


(Pierre Padovani) #5

I have a fork that implements this support, see the discussion on this github issue:

Fork can be found here:
https://github.com/homeaway/kibana/tree/fullNestedSupport

README for the changes:
https://github.com/homeaway/kibana/blob/fullNestedSupport/NESTED_README.md


(system) #6