# After the group, the results were inconsistent

**URL:** https://discuss.elastic.co/t/after-the-group-the-results-were-inconsistent/341897
**Category:** Elastic Search
**Tags:** elastic-app-search
**Created:** [August 29, 2023, 3:01pm UTC](https://discuss.elastic.co/t/after-the-group-the-results-were-inconsistent/341897 "2023-08-29T15:01:09Z")
**Posts on this page:** 9
**Page:** 1

<div class="post-metadata">

### Author: ![ZE\_Share](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ze_share/32/99298_2.png) [@ZE\_Share](https://discuss.elastic.co/u/ZE_Share)
#### Post date: [August 29, 2023, 3:01pm UTC](https://discuss.elastic.co/t/after-the-group-the-results-were-inconsistent/341897/1 "2023-08-29T15:01:09Z")

</div>

In low precision, I want to remove the fuzzy matching, so I will use `search_explain` and `elasticsearch/_search` to complete the `group` operation.

I found a problem:

Set params:

```auto
query: 'tshirt',
group: { 'field': 'companyId', 'size': 5 },
filter: {'all': ['companyId': ['68036', '67900', '67758', '67735', '68639', '68530', '40834', '67665', '68501', '68479', '68910', '68333', '67757', '67840', '68920', '68460', '68832', '68534', '68615', '68361', '67868', '68387', '67819', '68351', '68538', '68256', '67745', '68317', '68293', '68292', '68343', '68518', '67718', '68694', '68162', '68816']]}
page: {'size': 5}

```

got results:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/f/7/f7e0abb789677fd37ada1232483789c35b5315e6.png)

then, I change

```auto
.....
page: {'size': 36}

```

got results:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/1/b/1bb7917a847b0dd7aaa2acd51b49c3eb768e3869.png)

Finally, when the size is different, the two results are inconsistent, Why is this?

---

<div class="post-metadata">

### Author: ![Sean\_Story](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sean_story/32/69987_2.png) [@Sean\_Story](https://discuss.elastic.co/u/Sean_Story)
#### Post date: [August 29, 2023, 3:04pm UTC](https://discuss.elastic.co/t/after-the-group-the-results-were-inconsistent/341897/2 "2023-08-29T15:04:51Z")

</div>

What were the results of using the [search\_explain](https://www.elastic.co/guide/en/app-search/current/search-explain.html) API for both queries?

---

<div class="post-metadata">

### Author: ![ZE\_Share](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ze_share/32/99298_2.png) [@ZE\_Share](https://discuss.elastic.co/u/ZE_Share)
#### Post date: [August 29, 2023, 3:12pm UTC](https://discuss.elastic.co/t/after-the-group-the-results-were-inconsistent/341897/3 "2023-08-29T15:12:21Z")

</div>

@Sean_Story

```auto
{
  "query": {
    "bool": {
      "must": {
        "function_score": {
          "boost_mode": "sum",
          "score_mode": "sum",
          "query": {
            "bool": {
              "must": [
                {
                  "bool": {
                    "should": [
                      {
                        "multi_match": {
                          "query": "tshirt",
                          "minimum_should_match": "1<-1 3<49%",
                          "type": "cross_fields",
                          "fields": [
                            ......
                          ]
                        }
                      }
                    ]
                  }
                }
              ]
            }
          },
          "functions": [
            {
              "script_score": {
                "script": {
                  "source": "Math.max(_score * ((0.2 * Math.max(0.0001, Math.log(Math.max(0.0001, (doc.containsKey(\"rank.float\") && !doc[\"rank.float\"].empty ? doc[\"rank.float\"].value + 1 : 1)))))) - _score, 0)"
                }
              }
            }
          ]
        }
      },
      "filter": {
        "bool": {
          "must": [
            {
              "terms": {
                "company_id.enum": [
                  "68036",
                  "67900",
                  "67758",
                  "67735",
                  "68639",
                  "68530",
                  "40834",
                  "67665",
                  "68501",
                  "68479",
                  "68910",
                  "68333",
                  "67757",
                  "67840",
                  "68920",
                  "68460",
                  "68832",
                  "68534",
                  "68615",
                  "68361",
                  "67868",
                  "68387",
                  "67819",
                  "68351",
                  "68538",
                  "68256",
                  "67745",
                  "68317",
                  "68293",
                  "68292",
                  "68343",
                  "68518",
                  "67718",
                  "68694",
                  "68162",
                  "68816"
                ]
              }
            }
          ]
        }
      }
    }
  },
  "sort": [
    {
      "_score": "desc"
    },
    {
      "_doc": "desc"
    }
  ],
  "highlight": {
    "fragment_size": 300,
    "type": "plain",
    "number_of_fragments": 1,
    "order": "score",
    "encoder": "html",
    "require_field_match": false,
    "fields": {}
  },
  "size": 0,
  "from": 0,
  "timeout": "30000ms",
  "_source": [
    "id"
  ],
  "aggs": {
    "group": {
      "terms": {
        "field": "company_id.enum",
        "size": 5,
        "order": {
          "top_hit": "desc"
        }
      },
      "aggs": {
        "group_hits": {
          "top_hits": {
            "size": 5,
            "sort": [
              {
                "_score": "desc"
              }
            ],
            "highlight": {
              "fragment_size": 300,
              "type": "plain",
              "number_of_fragments": 1,
              "order": "score",
              "encoder": "html",
              "require_field_match": false,
              "fields": {}
            }
          }
        },
        "top_hit": {
          "max": {
            "script": {
              "source": "_score"
            }
          }
        }
      }
    },
    "estimated_total_groups": {
      "cardinality": {
        "field": "company_id.enum"
      }
    }
  }
}

```

and

```auto
{
  "query": {
    "bool": {
      "must": {
        "function_score": {
          "boost_mode": "sum",
          "score_mode": "sum",
          "query": {
            "bool": {
              "must": [
                {
                  "bool": {
                    "should": [
                      {
                        "multi_match": {
                          "query": "tshirt",
                          "minimum_should_match": "1<-1 3<49%",
                          "type": "cross_fields",
                          "fields": [
                            ......
                          ]
                        }
                      }
                    ]
                  }
                }
              ]
            }
          },
          "functions": [
            {
              "script_score": {
                "script": {
                  "source": "Math.max(_score * ((0.2 * Math.max(0.0001, Math.log(Math.max(0.0001, (doc.containsKey(\"rank.float\") && !doc[\"rank.float\"].empty ? doc[\"rank.float\"].value + 1 : 1)))))) - _score, 0)"
                }
              }
            }
          ]
        }
      },
      "filter": {
        "bool": {
          "must": [
            {
              "terms": {
                "company_id.enum": [
                  "68036",
                  "67900",
                  "67758",
                  "67735",
                  "68639",
                  "68530",
                  "40834",
                  "67665",
                  "68501",
                  "68479",
                  "68910",
                  "68333",
                  "67757",
                  "67840",
                  "68920",
                  "68460",
                  "68832",
                  "68534",
                  "68615",
                  "68361",
                  "67868",
                  "68387",
                  "67819",
                  "68351",
                  "68538",
                  "68256",
                  "67745",
                  "68317",
                  "68293",
                  "68292",
                  "68343",
                  "68518",
                  "67718",
                  "68694",
                  "68162",
                  "68816"
                ]
              }
            }
          ]
        }
      }
    }
  },
  "sort": [
    {
      "_score": "desc"
    },
    {
      "_doc": "desc"
    }
  ],
  "highlight": {
    "fragment_size": 300,
    "type": "plain",
    "number_of_fragments": 1,
    "order": "score",
    "encoder": "html",
    "require_field_match": false,
    "fields": {}
  },
  "size": 0,
  "from": 0,
  "timeout": "30000ms",
  "_source": [
    "id"
  ],
  "aggs": {
    "group": {
      "terms": {
        "field": "company_id.enum",
        "size": 36,
        "order": {
          "top_hit": "desc"
        }
      },
      "aggs": {
        "group_hits": {
          "top_hits": {
            "size": 5,
            "sort": [
              {
                "_score": "desc"
              }
            ],
            "highlight": {
              "fragment_size": 300,
              "type": "plain",
              "number_of_fragments": 1,
              "order": "score",
              "encoder": "html",
              "require_field_match": false,
              "fields": {}
            }
          }
        },
        "top_hit": {
          "max": {
            "script": {
              "source": "_score"
            }
          }
        }
      }
    },
    "estimated_total_groups": {
      "cardinality": {
        "field": "company_id.enum"
      }
    }
  }
}

```

---

<div class="post-metadata">

### Author: ![Kathleen\_DeRusso](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kathleen_derusso/32/132039_2.png) [@Kathleen\_DeRusso](https://discuss.elastic.co/u/Kathleen_DeRusso)
#### Post date: [August 29, 2023, 6:24pm UTC](https://discuss.elastic.co/t/after-the-group-the-results-were-inconsistent/341897/4 "2023-08-29T18:24:14Z")

</div>

Hey there @ZE_Share thanks for reaching out!

I think that this is working correctly based on the grouping configuration. As you can see in the `search_explain` API that @Sean_Story suggested, grouping in app search is doing a `terms` aggregation of size 36 with a nested `top_hits` subaggregation of size 5. This means that for each of the 36 terms we allow up to 5 top hits.

Are you trying to paginate these grouped results? If that's the case I think you want to add `"collapse": true` to your [group](https://www.elastic.co/guide/en/app-search/current/grouping.html). Note that this flag is experimental, and will only work if the grouped fields don't have multiple values - but that's the way to accomplish pagination using groups at this time with app search.

Hope that helps!

---

<div class="post-metadata">

### Author: ![ZE\_Share](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ze_share/32/99298_2.png) [@ZE\_Share](https://discuss.elastic.co/u/ZE_Share)
#### Post date: [August 30, 2023, 12:02am UTC](https://discuss.elastic.co/t/after-the-group-the-results-were-inconsistent/341897/5 "2023-08-30T00:02:36Z")

</div>

@Kathleen_DeRusso Thank you for your reply. In fact, I am more curious about the different settings of `page.size` on the outer layer. Why does it affect nested `top_hits`?

---

<div class="post-metadata">

### Author: ![Kathleen\_DeRusso](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kathleen_derusso/32/132039_2.png) [@Kathleen\_DeRusso](https://discuss.elastic.co/u/Kathleen_DeRusso)
#### Post date: [August 30, 2023, 6:57pm UTC](https://discuss.elastic.co/t/after-the-group-the-results-were-inconsistent/341897/6 "2023-08-30T18:57:02Z")

</div>

It should affect the number of results that are returned, not the number of results returned in an individual top hits result. I'm not really sure what you're seeing that's anomalous here, I think we'd need to see actual queries and results to be more helpful.

---

<div class="post-metadata">

### Author: ![ZE\_Share](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ze_share/32/99298_2.png) [@ZE\_Share](https://discuss.elastic.co/u/ZE_Share)
#### Post date: [August 31, 2023, 4:27am UTC](https://discuss.elastic.co/t/after-the-group-the-results-were-inconsistent/341897/7 "2023-08-31T04:27:36Z")

</div>

> [@Kathleen\_DeRusso](#):
>
> It should affect the number of results that are returned, not the number of results returned in an individual top hits result. I'm not really sure what you're seeing that's anomalous here, I think we'd need to see actual queries and results to be more helpful.

Look at the question I sent again, the `items` in the red box are the result of top-hits (I converted the format);

The same query conditions, but `page.size` is different, but the results of top-hits are inconsistent; using 68036 as an example, when size=36, the number of top\_hits is returned as 4, but when size=5, the number of top\_hits is 2.

---

<div class="post-metadata">

### Author: ![Kathleen\_DeRusso](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kathleen_derusso/32/132039_2.png) [@Kathleen\_DeRusso](https://discuss.elastic.co/u/Kathleen_DeRusso)
#### Post date: [August 31, 2023, 8:45pm UTC](https://discuss.elastic.co/t/after-the-group-the-results-were-inconsistent/341897/8 "2023-08-31T20:45:09Z")

</div>

Right, I'm not sure why you're seeing that, so if you could trim your example down to something that we could reproduce, it would be easier to help answer you. Thank you!

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [September 28, 2023, 8:45pm UTC](https://discuss.elastic.co/t/after-the-group-the-results-were-inconsistent/341897/9 "2023-09-28T20:45:28Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
