Visualizing Nested Data Type in Kibana

Elastic Stack Kibana
1

Hi All,

I am new to Kibana. I have been trying to create dashboards to visualize nested datatype but no luck, even simple stuff like adding field from nested datatype as column in Data Table is not working. Here are my questions -

  • I read few old threads and seems like it's not possible, is that still true with Kibana 7.6.1?
  • What are the alternatives/best practice for such usecase?

Here is an example of what I am trying to achieve:
Example Index pattern mapping in ES:
{
"mappings": {
"properties": {
"Employee": {
"type": "nested",
"properties" : {
"first" : { "type" : "text" },
"last" : { "type" : "text" },
"salary" : { "type" : "double" }
}}}
}}

On such mapping I want to lets say create Data Table in Kibana that shows

|FirstName | LastName |Salary |

but instead in Kibana I only see nested field as JSON, so I only see Employee as column and first, last and salary are shown only as JSON. I even tried playing around with "include_in_root" & "include_in_parent" fields in mapping but no luck. Once I get past this the next step would be to get some pie charts etc working on nested fields for aggregation etc kind of operations.

Is the best strategy to deal with nested fields to store data in separate index in ES after transforming to allow easy Visualization?

Will really appreciate help and guidance!

Thanks!

2

That is still true, there is no way to use nested datatypes in Kibana. The recommendation is to flatten the data for now.

1 Like
3

Thanks for quick response. These may be more of ES question

  1. but isn't include_in_root suppose to flatten and put nested type under parent document and once fields are in parent doc then shouldn't kibana be able to see those flatten fields?
  2. What's the best way to flatten declare it as Object which results in incorrect result as relationship will be lost in above e.g all first Names will be stored Employee.first: [], all last Names will be Employee.last: [] then that could impact search accuracy for certain cases where relationship is important and with flattening that relationship is lost? Is there any better approach where you don't lose search accuracy while still be able to support Kibana usecases?
    Is it possible I can store both nested data type and flatten type both maybe by writing ingest processor to create flatten data during index? If it's possible then is it the right approach and is ingest processor the optimal way for this operation? If that's the right way then Can you give an example of ingest processor to achieve the same?
  3. Another question is related to performance assuming scale of incoming data is huge, and you want to create some visualization charts for aggregation, sum etc across network counters then is it better to calculate them and store in separate index in ES or query from Kibana and then display? Considering both performance and user experience

Thanks

2 Likes
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.