# Nested sorting differs between ES7 and ES8?

**URL:** <https://discuss.elastic.co/t/nested-sorting-differs-between-es7-and-es8/337904>\
**Category:** Elasticsearch\
**Created:** [July 7, 2023, 3:56pm UTC](https://discuss.elastic.co/t/nested-sorting-differs-between-es7-and-es8/337904 "2023-07-07T15:56:43Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![MarynaCherniavska](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marynacherniavska/32/119071_2.png) [@MarynaCherniavska](https://discuss.elastic.co/u/MarynaCherniavska)\
**Post date:** [July 7, 2023, 3:56pm UTC](https://discuss.elastic.co/t/nested-sorting-differs-between-es7-and-es8/337904/1 "2023-07-07T15:56:43Z")

</div>

Dear team, we're moving from 7.17 to 8.7 (I know, big jump) and as part of the move, we have been running the e2e tests of the application on the new cluster, before actually switching the application to it. Some tests, however, started failing. We traced these to requests where a sorting was happening on a nested field.

Specifically, the error we're getting is as follows:

```json
{
        "type": "query_shard_exception",
        "reason": "it is mandatory to set the [nested] context on the nested sort field: [trace.origami.timestamp].",
        "index_uuid": "_xvEa8gNSFyCDm0aFXqYhg",
        "index": "article_ad1"
      }

```

This seems to refer to the sort clause that we have:

```auto
"sort": [
      {
        "trace.origami.timestamp": {
          "order": "desc",
          "missing": "_last",
          "unmapped_type": "float",
          "mode": "min"
        }
      }
    ]

```

The [page about sorting on a nested field for ES 8.8 (current)](https://www.elastic.co/guide/en/elasticsearch/reference/8.8/sort-search-results.html#nested-sorting) says that there should be a path specified in a "nested.path" clause of the sort. However, the [same page for ES 7.17](https://www.elastic.co/guide/en/elasticsearch/reference/7.17/sort-search-results.html#nested-sorting) states exactly the same, but the query still runs fine without that clause! I know how to fix the query to make it work. What I don't understand is, what changed between the versions so that it started erroring out in ES8, whereas in ES7 it was working fine, despite stating in the doc that the parameter is optional?

I've prepared a small test case to reproduce as close as possible to what we're having.

```shell
ES_PATH=http://localhost:9200
TEST_INDEX=test-index

curl -X PUT "$ES_PATH/$TEST_INDEX"

curl -X POST --url "$ES_PATH/$TEST_INDEX/_mapping" -H "Content-Type:application/json" -d '{
  "properties": {
    "name": {
      "type": "text"
    },
    "trace": {
      "type": "nested",
      "properties": {
        "origami": {
          "properties": {
            "timestamp": {
              "type": "long"
            }
          }
        }
      }
    }
  }
}'

curl -X POST --url "$ES_PATH/$TEST_INDEX/_bulk" -H "Content-Type:application/json" -d '
{ "index" : { "_id" : "1" } }
{ "trace" : { "origami": { "timestamp": 1688732801 } } }
{ "index" : { "_id" : "2" } }
{ "trace" : { "origami": { "timestamp": 1688732701 } } }
{ "index" : { "_id" : "3" } }
{ "trace" : { "origami": { "timestamp": 1688732601 } } }
'

# this succeeds in both ES7 and ES8

curl -X POST --url "$ES_PATH/$TEST_INDEX/_search" -H "Content-Type:application/json" -d '
{
  "query": {
    "match_all": {}
  },
  "sort": [
      {
        "trace.origami.timestamp": {
          "order": "asc",
          "missing": "_last",
          "unmapped_type": "float",
          "mode": "min",
          "nested": {
            "path": "trace"
          }
        }
      }
    ]
}' | jq

# this fails in ES8 but works in ES7

curl -X POST --url "$ES_PATH/$TEST_INDEX/_search" -H "Content-Type:application/json" -d '
{
  "query": {
    "match_all": {}
  },
  "sort": [
      {
        "trace.origami.timestamp": {
          "order": "asc",
          "missing": "_last",
          "unmapped_type": "float",
          "mode": "min"
        }
      }
    ]
}' | jq

```

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [July 25, 2023, 12:17pm UTC](https://discuss.elastic.co/t/nested-sorting-differs-between-es7-and-es8/337904/2 "2023-07-25T12:17:59Z")

</div>

Thanks for the detailed script. I check it and you're right, it's ok in 7.17.12 but fails in 8.x.

I believe that in 7.17 there was may be a bug which has been fixed in 8?

BTW, I updated and simplified your script. In case someone else wants to test/comment it:

```auto
DELETE /test-index
PUT /test-index
{
  "mappings": {
    "properties": {
      "name": {
        "type": "text"
      },
      "trace": {
        "type": "nested",
        "properties": {
          "origami": {
            "properties": {
              "timestamp": {
                "type": "long"
              }
            }
          }
        }
      }
    }
  }
}

POST /test-index/_bulk
{ "index" : { "_id" : "1" } }
{ "trace" : { "origami": { "timestamp": 1688732801 } } }
{ "index" : { "_id" : "2" } }
{ "trace" : { "origami": { "timestamp": 1688732701 } } }
{ "index" : { "_id" : "3" } }
{ "trace" : { "origami": { "timestamp": 1688732601 } } }

GET /test-index/_search
{
  "sort": [
    {
      "trace.origami.timestamp": {
        "nested": {
          "path": "trace"
        }
      }
    }
  ]
}

# this fails in ES8 but works in ES7

GET /test-index/_search
{
  "sort": [
    {
      "trace.origami.timestamp": {
        "nested": {
          "path": "trace"
        }
      }
    }
  ]
}

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 22, 2023, 12:18pm UTC](https://discuss.elastic.co/t/nested-sorting-differs-between-es7-and-es8/337904/3 "2023-08-22T12:18:36Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.