"sort" on multiple indices with a missing field

Elastic Stack Elasticsearch
1

Hi,

I see queries on multiple indices with "sort" with a missing field causes an exception (query and exception are attached). And,

  • the same queries on individual index (either with or w/o a missing field) are fine
  • tried a couple of scripts but can't make it work (scripts and logs are attached)
  • Elasticsearch 5.3

Any insight, solution? Thanks in advance,

GET event-2017.09.1*/_search
{
  "sort":
     {
        "incidentLastSeen": {
           "order": "desc",
           "unmapped_type": "long"
        }
     }
}

{
   "error": {
      "root_cause": [],
      "type": "reduce_search_phase_exception",
      "reason": "[reduce] ",
      "phase": "query",
      "grouped": true,
      "failed_shards": [],
      "caused_by": {
         "type": "class_cast_exception",
         "reason": "java.lang.Long cannot be cast to java.lang.Float"
      }
   },
   "status": 503
}

== using script - using 'empty' ==
- index with a missing field: script_exception
- index w/o a missing field: OK

GET event-2017.09.10/_search
{
  "sort" : {
    "_script": {
        "script":"doc['incidentLastSeen'].empty ? 0 : doc['incidentLastSeen'].value",
        "type": "number",
        "order": "desc"
    }
  }
}

{
   "error": {
      "root_cause": [
         {
            "type": "script_exception",
            "reason": "runtime error",
            "script_stack": [
               "org.elasticsearch.search.lookup.LeafDocLookup.get(LeafDocLookup.java:80)",
               "doc['incidentLastSeen'].empty ? 0 : doc['incidentLastSeen'].value",
               "    ^---- HERE"
            ],
            "script": "doc['incidentLastSeen'].empty ? 0 : doc['incidentLastSeen'].value",
            "lang": "painless"
         }
      ],
      "type": "search_phase_execution_exception",
      "reason": "all shards failed",
      "phase": "query",
      "grouped": true,
      "failed_shards": [
         {
            "shard": 0,
            "index": "event-2017.09.10",
            "node": "wa6IdgVdSP2KuG6yZeU0xQ",
            "reason": {
               "type": "script_exception",
               "reason": "runtime error",
               "script_stack": [
                  "org.elasticsearch.search.lookup.LeafDocLookup.get(LeafDocLookup.java:80)",
                  "doc['incidentLastSeen'].empty ? 0 : doc['incidentLastSeen'].value",
                  "    ^---- HERE"
               ],
               "script": "doc['incidentLastSeen'].empty ? 0 : doc['incidentLastSeen'].value",
               "lang": "painless",
               "caused_by": {
                  "type": "illegal_argument_exception",
                  "reason": "No field found for [incidentLastSeen] in mapping with types []"
               }
            }
         }
      ],
      "caused_by": {
         "type": "script_exception",
         "reason": "runtime error",
         "script_stack": [
            "org.elasticsearch.search.lookup.LeafDocLookup.get(LeafDocLookup.java:80)",
            "doc['incidentLastSeen'].empty ? 0 : doc['incidentLastSeen'].value",
            "    ^---- HERE"
         ],
         "script": "doc['incidentLastSeen'].empty ? 0 : doc['incidentLastSeen'].value",
         "lang": "painless",
         "caused_by": {
            "type": "illegal_argument_exception",
            "reason": "No field found for [incidentLastSeen] in mapping with types []"
         }
      }
   },
   "status": 500
}

== using script - using 'containsKey' ==
- script_exception on either with or w/o a missing field

GET event-2017.09.11/_search
{
  "sort" : {
    "_script": {
        "script":"doc.containsKey['incidentLastSeen'] ? doc['incidentLastSeen'].value : 0",
        "type": "number",
        "order": "desc"
    }
  }
}

{
   "error": {
      "root_cause": [
         {
            "type": "script_exception",
            "reason": "runtime error",
            "script_stack": [
               "org.elasticsearch.search.lookup.LeafDocLookup.get(LeafDocLookup.java:80)",
               "doc.containsKey['incidentLastSeen'] ? doc['incidentLastSeen'].value : 0",
               "   ^---- HERE"
            ],
            "script": "doc.containsKey['incidentLastSeen'] ? doc['incidentLastSeen'].value : 0",
            "lang": "painless"
         }
      ],
      "type": "search_phase_execution_exception",
      "reason": "all shards failed",
      "phase": "query",
      "grouped": true,
      "failed_shards": [
         {
            "shard": 0,
            "index": "event-2017.09.11",
            "node": "wa6IdgVdSP2KuG6yZeU0xQ",
            "reason": {
               "type": "script_exception",
               "reason": "runtime error",
               "script_stack": [
                  "org.elasticsearch.search.lookup.LeafDocLookup.get(LeafDocLookup.java:80)",
                  "doc.containsKey['incidentLastSeen'] ? doc['incidentLastSeen'].value : 0",
                  "   ^---- HERE"
               ],
               "script": "doc.containsKey['incidentLastSeen'] ? doc['incidentLastSeen'].value : 0",
               "lang": "painless",
               "caused_by": {
                  "type": "illegal_argument_exception",
                  "reason": "No field found for [containsKey] in mapping with types []"
               }
            }
         }
      ],
      "caused_by": {
         "type": "script_exception",
         "reason": "runtime error",
         "script_stack": [
            "org.elasticsearch.search.lookup.LeafDocLookup.get(LeafDocLookup.java:80)",
            "doc.containsKey['incidentLastSeen'] ? doc['incidentLastSeen'].value : 0",
            "   ^---- HERE"
         ],
         "script": "doc.containsKey['incidentLastSeen'] ? doc['incidentLastSeen'].value : 0",
         "lang": "painless",
         "caused_by": {
            "type": "illegal_argument_exception",
            "reason": "No field found for [containsKey] in mapping with types []"
         }
      }
   },
   "status": 500
}
2

Found the reason - in one of indices, the field was mapped to a different type (float). :sweat:

3

Still, I can't figure why 1505172945000, 1111111543123 were mapped as 'float' (using dynamic mapping).

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.