# Excluding Duplicate Values in Kibana Filter

**URL:** <https://discuss.elastic.co/t/excluding-duplicate-values-in-kibana-filter/195108>\
**Category:** Kibana\
**Created:** [August 13, 2019, 9:34pm UTC](https://discuss.elastic.co/t/excluding-duplicate-values-in-kibana-filter/195108 "2019-08-13T21:34:09Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![misiakj](https://avatars.discourse-cdn.com/v4/letter/m/bcef8e/32.png) [@misiakj](https://discuss.elastic.co/u/misiakj)\
**Post date:** [August 13, 2019, 9:34pm UTC](https://discuss.elastic.co/t/excluding-duplicate-values-in-kibana-filter/195108/1 "2019-08-13T21:34:09Z")

</div>

I have an index that looks similar to the below (it really has a number of additional fields that aren't relevant for this post)

case\_id: int  
group: string  
unique\_id: case\_id + group  
value: int

for any one case\_id value I might have several documents, each with a different group. Each of these documents should have the same value in the "Value" field. Different case\_ids might also go through different sets of groups

For Example

case\_id: 1  
group: A  
value: 3

case\_id: 1  
group: B  
value: 3

case\_id: 1  
group: C  
value: 3

case\_id: 2  
group: A  
value: 1

case\_id: 2  
group: B  
value: 1

case\_id: 3  
group: D  
value: 5

I would like to Average the Value of each case\_id. But I would only like to calculate the value of each case\_id once. So for example with the above data set I want  
case\_id 1 = 3  
case\_id 2 = 1  
case\_id 3 = 5

(3 + 1 + 5) / 3 = 3. So I should get 3 as a result.

The problem is I can't just average all case\_id values of all the documents or I would get something like  
(3 + 3 + 3 + 1 + 1 + 5) ~= 2.6667

How can I first subset a group to just one document per case\_id (doesn't matter which document) and then average the value fields?

Thanks for any help!

---

<div class="post-metadata">

**Author:** ![lukas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lukas/32/6812_2.png) [@lukas](https://discuss.elastic.co/u/lukas)\
**Post date:** [August 13, 2019, 10:52pm UTC](https://discuss.elastic.co/t/excluding-duplicate-values-in-kibana-filter/195108/2 "2019-08-13T22:52:57Z")

</div>

Hmm, okay I think I can get you close to what you want.

First you'll need to create a data table visualization, and go into the options and turn on "Show total", and use the average.

Then, you'll select "top hit" as your metric, and select "value" as the field.

Then, for the buckets, you'll want to use a terms aggregation over "case\_id".

You'll have one row for each case\_id and then a summary (average) at the bottom of the table.

Does that help?

---

<div class="post-metadata">

**Author:** ![misiakj](https://avatars.discourse-cdn.com/v4/letter/m/bcef8e/32.png) [@misiakj](https://discuss.elastic.co/u/misiakj)\
**Post date:** [August 14, 2019, 4:58pm UTC](https://discuss.elastic.co/t/excluding-duplicate-values-in-kibana-filter/195108/3 "2019-08-14T16:58:09Z")

</div>

Awesome! I created what I need doing what you said. (Example of my values in the images)  
The only issue is the actual number of case\_ids that I have is ~ 30k and growing. So the data table takes a very long time to load.  
Is there a way to only really load the average rather than loading every case\_id and then averaging the results?  
Additionally, instead of using Average can I use 80th Percentile somehow?

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/e/1/e1855784f8c7ec31f42404797c031256c62f9d07.png)

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/5/9/59bf610cd288630f065dac1f2ff21f31022349dd.png)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 11, 2019, 4:58pm UTC](https://discuss.elastic.co/t/excluding-duplicate-values-in-kibana-filter/195108/4 "2019-09-11T16:58:22Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.