# Composite aggregation returns old version documents

**URL:** <https://discuss.elastic.co/t/composite-aggregation-returns-old-version-documents/252353>\
**Category:** Elasticsearch\
**Created:** [October 16, 2020, 1:37pm UTC](https://discuss.elastic.co/t/composite-aggregation-returns-old-version-documents/252353 "2020-10-16T13:37:00Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![And390](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/and390/32/77332_2.png) [@And390](https://discuss.elastic.co/u/And390)\
**Post date:** [October 16, 2020, 1:37pm UTC](https://discuss.elastic.co/t/composite-aggregation-returns-old-version-documents/252353/1 "2020-10-16T13:37:01Z")

</div>

Hello. I found that in some cases composite aggregation returns old documents (the initial indexed version instead of updated). Steps to reproduce:

```auto
curl -X PUT -H 'Content-Type: application/json' localhost:9200/test -d '{
      "settings": {
        "index": {
          "sort.field": "id"
        }
      },
      "mappings": {
        "properties": {
          "id": {
            "type": "keyword"
          },
          "name": {
            "type": "text"
          }
        }
      }
    }'
curl -X PUT -H 'Content-Type: application/json' localhost:9200/test/_doc/1 -d '{
  "id": 1,
  "value": "Old Value"
}'
curl -X POST -H 'Content-Type: application/json' localhost:9200/test/_update/1 -d '{
  "doc": {
    "value": "New Value"
  }
}'
curl -X GET -H 'Content-Type: application/json' localhost:9200/test/_search?pretty -d '{
  "aggs": {
    "composite": {
      "composite": {
        "size": 1,
        "sources": 
          { "id": { "terms": { "field": "id" } } }
        ,"after": {"id": "0"}
      },
      "aggs": {
        "top_hits": {
          "top_hits": {
            "size": 1
          }
        }
      }
    }
  }
}'

```

Response:

```auto
{
  "took" : 102,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 1,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "test",
        "_type" : "_doc",
        "_id" : "1",
        "_score" : 1.0,
        "_source" : {
          "id" : 1,
          "value" : "New Value"
        }
      }
    ]
  },
  "aggregations" : {
    "composite" : {
      "after_key" : {
        "id" : "1"
      },
      "buckets" : [
        {
          "key" : {
            "id" : "1"
          },
          "doc_count" : 2,
          "top_hits" : {
            "hits" : {
              "total" : {
                "value" : 2,
                "relation" : "eq"
              },
              "max_score" : 1.0,
              "hits" : [
                {
                  "_index" : "test",
                  "_type" : "_doc",
                  "_id" : "1",
                  "_score" : 1.0,
                  "_source" : {
                    "id" : 1,
                    "value" : "Old Value"
                  }
                }
              ]
            }
          }
        }
      ]
    }
  }
}

```

As you can see the top level hits contain the actual values but the composite aggregation buckets contain old values.  
This problem disappears when I remove the index sort setting from the mapping (which I think is important for performance if I want to use pagination) or the "after" property from the query.  
Is there something I do wrong or is it a bug in Elasticcearch?

Elasticsearch version: 7.9.2

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [October 16, 2020, 2:00pm UTC](https://discuss.elastic.co/t/composite-aggregation-returns-old-version-documents/252353/2 "2020-10-16T14:00:22Z")

</div>

What if you issue an explicit refresh after the update?

---

<div class="post-metadata">

**Author:** ![And390](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/and390/32/77332_2.png) [@And390](https://discuss.elastic.co/u/And390)\
**Post date:** [October 16, 2020, 2:56pm UTC](https://discuss.elastic.co/t/composite-aggregation-returns-old-version-documents/252353/3 "2020-10-16T14:56:00Z")

</div>

Explicit refresh doesn't help

---

<div class="post-metadata">

**Author:** ![And390](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/and390/32/77332_2.png) [@And390](https://discuss.elastic.co/u/And390)\
**Post date:** [October 16, 2020, 3:16pm UTC](https://discuss.elastic.co/t/composite-aggregation-returns-old-version-documents/252353/4 "2020-10-16T15:16:38Z")

</div>

But looks like it works after some time, about several minutes (without explicit refresh). Is this the correct behavior or is there something wrong?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [October 16, 2020, 5:30pm UTC](https://discuss.elastic.co/t/composite-aggregation-returns-old-version-documents/252353/5 "2020-10-16T17:30:02Z")

</div>

How are you doing the refresh? Are you waiting until it has completed?

---

<div class="post-metadata">

**Author:** ![And390](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/and390/32/77332_2.png) [@And390](https://discuss.elastic.co/u/And390)\
**Post date:** [October 16, 2020, 7:57pm UTC](https://discuss.elastic.co/t/composite-aggregation-returns-old-version-documents/252353/6 "2020-10-16T19:57:57Z")

</div>

```
 curl -X POST -H 'Content-Type: application/json' localhost:9200/test/_update/1?refresh=wait_for -d ...

```

or

```
curl -X POST localhost:9200/test/_refresh

```

Both don't help and both wait for completion if I understand it right. But as I mentioned before after several minutes I can see the updated value even without these explicit refreshes.  
My Elasticsearch have default settings (just downloaded and installed the latest version).

---

<div class="post-metadata">

**Author:** ![jimczi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jimczi/32/47985_2.png) [@jimczi](https://discuss.elastic.co/u/jimczi)\
**Post date:** [October 19, 2020, 7:40am UTC](https://discuss.elastic.co/t/composite-aggregation-returns-old-version-documents/252353/7 "2020-10-19T07:40:47Z")

</div>

Thanks for reporting @And390, this is indeed a bug. I opened [https://github.com/elastic/elasticsearch/pull/63864](https://github.com/elastic/elasticsearch/pull/63864) for the fix.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 16, 2020, 7:40am UTC](https://discuss.elastic.co/t/composite-aggregation-returns-old-version-documents/252353/8 "2020-11-16T07:40:49Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.