Visualization of multi-value field with aggregation

Elastic Stack Elasticsearch
1

Newbie to the ELK stack and semi-technical. Our use case is to use the stack to run some analytics for pricing and product data broken down by different multi-value fields. I will start with source data structure and what has been done, followed by questions.

Note: this past post is very similar and unfortunately a solution has not been posted/described for us to try (past post)

  1. Data - we receive product sales and quantity based on different regions and stores. The regions and stores are dynamic and differ by product. Here's sample data with following fields in order - ProductName, Sales (), Regions (with sales) and Stores (with quantity)
    Data (there is timestamp field as well):
    ProductA, $100,000, ["Region 1": $25,000, "Region 2": $30,000, "Region 3": $45,000], ["Store 101": 100, "Store 215": 355, "Store 189": 417, "Store 39?": 55]
    Product B ...similar data
    Product A...similar data

  2. We attempted to solve this using Nested data type to store the values as described here (https://www.elastic.co/guide/en/elasticsearch/reference/current/nested.html). We are able to see the index parsing these multifields, however Kibana is not able to recognize these values as independent fields to aggregate. Our requirement is to create aggregations by Region, Store and Product together and separately. Here's sample mapping we used to test this out

"Regions": {
"type": "nested" ,
"properties": {
"Region": {
"type": "text"
},
"sales": {
"type": "float"
}
}
}
}

  1. Is this possible? or is there a better design or approach? We are unsure if we have to create separate fields while ingestion to hold this data which defeats the purpose

As indicated, I am not that technical so please ask questions if the above is unclear.
Thanks

2

Why not flatten the data model instead of using nested documents, which is not well supported by Kibana at all. What I mean is basically turning the following:

Into documents something like this:

{"product": "ProductA", "region": "Region 1", "sales": 25000}
{"product": "ProductA", "region": "Region 2", "sales": 30000}
{"product": "ProductA", "region": "Region 3", "sales": 45000}
{"product": "ProductA", "store": "Store 101", "quantity": 100}
{"product": "ProductA", "store": "Store 215", "quantity": 355}
{"product": "ProductA", "store": "Store 189", "quantity": 417}
{"product": "ProductA", "store": "Store 39?", "quantity": 55}
3

Thanks. This sounds interesting.
Pardon my ignorance.

Are you suggesting to break one document into multiple documents by breaking them based on the multiple value field?

4

Yes, that is what I was suggesting. I did not see a link between regions and stores so put them in different documents, but I might be reading your data wrong.

5

Thanks. You are onto something here. Let me work through this suggestion and get back. Thanks a ton.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.