How to remove duplication when aggregating data for visualization?

Elastic Stack Kibana
1

Hi!

Based on answers from other posts, I learned that I need to upload "normalized data" to Kibana, which may come at the cost of sending duplicated data:

{ file: foo, project: foo, count: 123, id: 1 } 
{ file: foo, project: bar, count: 123, id: 1 }    

{ file: foo, project: foo, count: 321, id: 2 } 
{ file: foo, project: bar, count: 321, id: 2 } 

{ file: bar, project: foo, count: 111, id: 1 } 
{ file: bar, project: bar, count: 111, id: 1 } 

{ file: bar, project: foo, count: 222, id: 2 } 
{ file: bar, project: bar, count: 222, id: 2 }

Let's say I want a Table summing the count for all given ids, disregarding the duplicated entry due to project (which is needed somewhere else). I.e. the table should produce:

File                        Sum of count across id
foo                        123 + 321 = 444
bar                        111 + 222 = 333

However if I use a naive Sum aggregation, it will double-count the entries for different projects, producing e.g. 888 and 666 respectively.

Is there a good way to achieve what I want? Thanks!

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.