# How to remove duplication when aggregating data for visualization?

**URL:** <https://discuss.elastic.co/t/how-to-remove-duplication-when-aggregating-data-for-visualization/324407>\
**Category:** Kibana\
**Tags:** visualisation\
**Created:** [February 1, 2023, 10:10am UTC](https://discuss.elastic.co/t/how-to-remove-duplication-when-aggregating-data-for-visualization/324407 "2023-02-01T10:10:12Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![tinrik](https://avatars.discourse-cdn.com/v4/letter/t/d78d45/32.png) [@tinrik](https://discuss.elastic.co/u/tinrik)\
**Post date:** [February 1, 2023, 10:10am UTC](https://discuss.elastic.co/t/how-to-remove-duplication-when-aggregating-data-for-visualization/324407/1 "2023-02-01T10:10:12Z")

</div>

Hi!

Based on answers from other posts, I learned that I need to upload "normalized data" to Kibana, which may come at the cost of sending duplicated data:

```auto
{ file: foo, project: foo, count: 123, id: 1 } 
{ file: foo, project: bar, count: 123, id: 1 }    

{ file: foo, project: foo, count: 321, id: 2 } 
{ file: foo, project: bar, count: 321, id: 2 } 

{ file: bar, project: foo, count: 111, id: 1 } 
{ file: bar, project: bar, count: 111, id: 1 } 

{ file: bar, project: foo, count: 222, id: 2 } 
{ file: bar, project: bar, count: 222, id: 2 } 

```

Let's say I want a Table summing the `count` for all given `id`s, disregarding the duplicated entry due to `project` (which is needed somewhere else). I.e. the table should produce:

```auto
File Sum of count across id
foo 123 + 321 = 444
bar 111 + 222 = 333

```

However if I use a naive `Sum` aggregation, it will double-count the entries for different projects, producing e.g. 888 and 666 respectively.

Is there a good way to achieve what I want? Thanks!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 1, 2023, 10:11am UTC](https://discuss.elastic.co/t/how-to-remove-duplication-when-aggregating-data-for-visualization/324407/2 "2023-03-01T10:11:00Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.