# HowTo Filter documents contain duplicates in array with PainLess

**URL:** <https://discuss.elastic.co/t/howto-filter-documents-contain-duplicates-in-array-with-painless/256829>\
**Category:** Elasticsearch\
**Tags:** painless\
**Created:** [November 27, 2020, 12:27am UTC](https://discuss.elastic.co/t/howto-filter-documents-contain-duplicates-in-array-with-painless/256829 "2020-11-27T00:27:31Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![vsmelov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vsmelov/32/79783_2.png) [@vsmelov](https://discuss.elastic.co/u/vsmelov)\
**Post date:** [November 27, 2020, 12:27am UTC](https://discuss.elastic.co/t/howto-filter-documents-contain-duplicates-in-array-with-painless/256829/1 "2020-11-27T00:27:31Z")

</div>

I have documents like

```auto
{"tokens" [
    {"city": "RU-Moscow", ...}, 
    {"city": "RU-Piter", ...}, 
    {"city": "RU-Moscow", ...}, 
    {"city": "GE-Berlin", ...},
    {"city": "NL-Amsterdam", ...}
    {"city": "NL-Amsterdam", ...}
    {"city": "NL-Amsterdam", ...}
], ...}

```

I want to find documents which have the same Russian city in `tokens` at least twice.

So I wrote such painless script:

```auto
String[] cities = new String[] {};
for (def token in doc['tokens']){
    String city = token['city'].toUpperCase();
    if (city.indexOf('RU') == 0) {
        if (cities.contains(city)) {
             return true;
        } else {
            cities.add(city);
        }
    }
}
return false;

```

but it fails with a runtime error  
I even feel that I have no access to array `tokens` at all fron the filter context!!!

I also found similar problem here:

> <https://stackoverflow.com/questions/54022283/elasticsearch-search-query-why-params-source-nested-field-size-is-not-workin>

  
In this link he said that I have no access to params.\_source from filter context.

Is it really true?  
How can I solve my problem?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 25, 2020, 12:28am UTC](https://discuss.elastic.co/t/howto-filter-documents-contain-duplicates-in-array-with-painless/256829/2 "2020-12-25T00:28:00Z")

</div>

Closed on December 25, 2020, 12:28am UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.