Index sorting causes illegal_state_exception

Heavily inspired by the Introducing Index Sorting in Elasticsearch 6.0 | Elastic Blog article I went to try index sorting out for faster AND conjunctions.

This is my index structure:

{
  "index_name": {
    "mappings": {
      "dynamic": "false",
      "properties": {
        "brand_id": { "type": "keyword" },
        "category_id": { "type": "keyword" },
        "status_id": { "type": "keyword" },
        "user_id": { "type": "keyword" },
        "price": { "type": "scaled_float", "scaling_factor": 100.0 },
        "created_at": { "type": "date" }
      }
    },
    "settings": {
      "index": {
        "sort": {
          "field": ["category_id", "brand_id", "status_id", "created_at"],
          "order": ["asc", "asc", "asc", "desc"]
        }
      }
    }
  }
}

And this is the actual query:

{
  "_source": false,
  "size": 24,
  "query": {
    "bool": {
      "filter": [
        {
          "term": {
            "category_id": 197
          }
        },
        {
          "term": {
            "brand_id": 51
          }
        },
        {
          "term": {
            "status_id": 2
          }
        },
        {
          "range": {
            "price": {
              "gte": 2.0,
              "lte": 5.0
            }
          }
        }
      ],
      "must_not": {
        "terms": {
          "user_id": [12345]
        }
      }
    }
  },
  "sort": {
    "created_at": "desc"
  }
}

The results have been mixed.

Some performed great and their performance has skyrocketed, however, some became completely horrible and started throwing the following exceptions:

{
  "error": {
    "root_cause": [
      {
        "type": "illegal_state_exception",
        "reason": "topDocs already consumed"
      }
    ],
    "type": "search_phase_execution_exception",
    "reason": "all shards failed",
    "phase": "query",
    "grouped": true,
    "failed_shards": [
      {
        "shard": 0,
        "index": "index_name",
        "node": "lx9S3vwES4y2n2xa-PFyUw",
        "reason": {
          "type": "illegal_state_exception",
          "reason": "topDocs already consumed"
        }
      }
    ],
    "caused_by": {
      "type": "illegal_state_exception",
      "reason": "topDocs already consumed",
      "caused_by": {
        "type": "illegal_state_exception",
        "reason": "topDocs already consumed"
      }
    }
  },
  "status": 500
}

This surely must be related to index sorting but I couldn't find anything indicating why this could be raised.

  • The index contains 100M documents and weights 10G
  • Elasticsearch 7.12.0
  • Using built-in JVM
  • Running on CentOS servers

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.