ElasticSearch “Can't get text on a VALUE_NULL at 1:60”

Elastic Stack Elasticsearch
1

When running queries on one of my indexes I get an error on Elastic.

{
  "error" : {
    "root_cause" : [ {
      "type" : "illegal_state_exception",
      "reason" : "Can't get text on a VALUE_NULL at 1:60"
    } ],
    "type" : "search_phase_execution_exception",
    "reason" : "all shards failed",
    "phase" : "query",
    "grouped" : true,
    "failed_shards" : [ {
      "shard" : 0,
      "index" : "someIndexName",
      "node" : "sfsfd5sdf",
      "reason" : {
        "type" : "illegal_state_exception",
        "reason" : "Can't get text on a VALUE_NULL at 1:60"
      }
    } ]
  },
  "status" : 500
}

My configuration:

{
  "cluster_name" : "clusterName",
  "status" : "green",
  "number_of_nodes" : 2,
  "number_of_data_nodes" : 2,
  "active_primary_shards" : 16,
  "active_shards" : 32,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 100.0
}


"version" : {
    "number" : "2.3.1",
    "build_hash" : "bd980929010aef404e7cb0843e61d0665269fc39",
    "build_timestamp" : "2016-04-04T12:25:05Z",
    "build_snapshot" : false,
    "lucene_version" : "5.5.0"
},

Offending query example:

{
   "fields": ["id", "coordsField",null],
   "query": {
   },
   "size": 0,
   "aggregations": {
       "zoom1": {
           "geohash_grid": {
               "field": "coordsField",
               "size": 5000,
               "precision": 7
           },
           "aggs": {
               "geohash": {
                   "top_hits": {
                       "sort": {
                           "id": {
                               "order": "desc",
                               "ignore_unmapped": true
                           }
                       },
                       "_source": false,
                       "fielddata_fields": ["id", "coordsField", null],
                       "size": 1
                   }
               }
           }
       }
   }

}

Googled but didn't find anything related. Where should I start searching to get an ideea for the cause ?

other data:

response for dev ( ES 2.2.1 + ES 2.3.0 )

{
  "took" : 4,
  "timed_out" : false,
  "_shards" : {
    "total" : 4,
    "successful" : 2,
    "failed" : 2,
    "failures" : [ {
      "shard" : 0,
      "index" : "locations",
      "node" : "IyfjTQqeQEuZVWDTX_p4qA",
      "reason" : {
        "type" : "illegal_state_exception",
        "reason" : "Can't get text on a VALUE_NULL at 3:35"
      }
    } ]
  },
  "hits" : {
    "total" : 0,
    "max_score" : 0.0,
    "hits" : [ ]
  },
  "aggregations" : {
    "zoom1" : {
      "buckets" : [ ]
    }
  }
}

response for ES 2.3.1

{
  "error" : {
    "root_cause" : [ {
      "type" : "illegal_state_exception",
      "reason" : "Can't get text on a VALUE_NULL at 3:35"
    } ],
    "type" : "search_phase_execution_exception",
    "reason" : "all shards failed",
    "phase" : "query",
    "grouped" : true,
    "failed_shards" : [ {
      "shard" : 0,
      "index" : "locations",
      "node" : "Dxztr9L1TiiNi5gU02MYGQ",
      "reason" : {
        "type" : "illegal_state_exception",
        "reason" : "Can't get text on a VALUE_NULL at 3:35"
      }
    } ]
  },
  "status" : 500
}
2

Can you post the offending query?

3

done, the queries works fine on my dev box, but on the staging cluster they fail.
I'm suspecting either the data/indexing to be different or the Elastic version (have 2.2.1 on my 2 node dev cluster)

4

I suggest removing clauses from the example query until you can distill it down to the smallest example that reproduces the error.

5

good idea, will come back with the results

6

The problem was the null in the fields, in the old (2.2.1) version it didn't care about it I guess.

7

Do you have the final simplified query?
It might be a behaviour change that justifies a bugfix so it would be good to get a reproducible example.

8

updated and added the responses. In the 2.2.1 version it returned the aggregation results.
Also seems thing are more complicated on the dev side as one of the nodes had 2.3.0 :slight_smile:

9

Many thanks for that.
I hadn't appreciated you were passing "null" as a field name (minus the quotes).
That's not a scenario I think we have anticipated but an error seems appropriate - possibly with a better error message thrown at query-parse-time rather than query-execute-time