# How to filter and narrow the data before aggregation

**URL:** <https://discuss.elastic.co/t/how-to-filter-and-narrow-the-data-before-aggregation/259147>\
**Category:** Elasticsearch\
**Created:** [December 19, 2020, 5:03am UTC](https://discuss.elastic.co/t/how-to-filter-and-narrow-the-data-before-aggregation/259147 "2020-12-19T05:03:13Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![jun\_miao](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jun_miao/32/81141_2.png) [@jun\_miao](https://discuss.elastic.co/u/jun_miao)\
**Post date:** [December 19, 2020, 5:03am UTC](https://discuss.elastic.co/t/how-to-filter-and-narrow-the-data-before-aggregation/259147/1 "2020-12-19T05:03:13Z")

</div>

Hello, all

I have a query which group by `groupId`.

```auto
{
   "aggs":{
      "result":{
          "terms":{
              "field":"groupId",
              "size": 100
         }
     }
  },
  "size": 0
}

```

Because the index contains more than 100 million documents, and `groupId` is a high-cardinality field, it is very slow.  
But I added a filter in the query, like:

```auto
{
   "aggs":{
      "result":{
          "terms":{
              "field":"groupId",
              "size": 100
         }
     }
  },
  "query":{
      "terms":{
         "groupId":["aaaaaaaaa","bbbbbbbbb","ccccccccc"]
     }
  },
  "size": 0
}

```

or

```auto
{
   "aggs":{
     "filter_by_group":{
         "filter":{
              "terms":{
                 "groupId":["aaaaaaaaa","bbbbbbbbb","ccccccccc"]
              }
         },
        "aggs":{
            "result":{
                "terms":{
                    "field":"groupId",
                    "size": 100
               }
           }
      }
  },
  "size": 0
}

```

It seems to have no effect.

1. How to achieve by query DSL？
2. How to define the data scope of terms aggregation？  
Please give some suggestions to improve performance.

Thanks.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [December 19, 2020, 8:10am UTC](https://discuss.elastic.co/t/how-to-filter-and-narrow-the-data-before-aggregation/259147/2 "2020-12-19T08:10:29Z")

</div>

What does the data in the groupId field look like? What is the mapping for this field?

---

<div class="post-metadata">

**Author:** ![jun\_miao](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jun_miao/32/81141_2.png) [@jun\_miao](https://discuss.elastic.co/u/jun_miao)\
**Post date:** [December 21, 2020, 1:44am UTC](https://discuss.elastic.co/t/how-to-filter-and-narrow-the-data-before-aggregation/259147/3 "2020-12-21T01:44:37Z")

</div>

The doc looks like this:

```auto
{
          "id" : "c07d9542a307e27b",
          "timestamp" : 1608187994071,
          "service" : "monitor",
          "system": "APM",
          "groupId": "8d5b716ebac05c4c",
          "parentId": "7681adf0a845d277",
          ......
}

```

The mapping is:

```auto
{
      "properties" : {
          "dynamic" : "true",
            "id" : {
              "type" : "keyword"
            },
            "service" : {
              "type" : "keyword",
              "normalizer" : "lowercase_normalizer"
            },
            "system" : {
              "type" : "keyword",
              "normalizer" : "lowercase_normalizer"
            },
            "groupId" : {
              "type" : "keyword"
            },
            "parentId" : {
              "type" : "keyword"
            },
            "timestamp" : {
              "type" : "date",
              "format" : "epoch_millis"
            }
       }
}

```

I want to achieve a similar function `select distinct groupId where condition`

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 18, 2021, 1:44am UTC](https://discuss.elastic.co/t/how-to-filter-and-narrow-the-data-before-aggregation/259147/4 "2021-01-18T01:44:39Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.