Query_string filter fails on _exists_:fieldname query in Elasticsearch 5.5.0

Elastic Stack Elasticsearch
1

In years past (with ES 1.x and 2.x, and Kibana 3.x and 4.x), we would filter for records containing a field using the _exists_:fieldname query in the Kibana search bar. The format for these was something along the lines of:

{ "size": 0, "query": { "query_string":{ "query": "_exists_:req_size" } } }

In Elasticsearch 5.5.0 and Kibana 5.5.0, there's a button under the search bar for "Add a filter", which lets you add an exists query. It uses a different query, simplified to:

{ "size": 0, "query": { "exists": { "field": "req_size" } } }

They both work in Elasticsearch 5.5.0, as long as the index being queried has documents in it. If you issue the query against an empty index, you get a NullPointerException and a large count of "Courier Error: N of M shards failed" in Kibana.

The raw error returned is:

{
  "error" : {
    "root_cause" : [
      {
        "type" : "query_shard_exception",
        "reason" : "failed to create query: {\n  \"query_string\" : {\n    \"query\" : \"_exists_:req_size\",\n    \"fields\" : [ ],\n    \"use_dis_max\" : true,\n    \"tie_breaker\" : 0.0,\n    \"default_operator\" : \"or\",\n    \"auto_generate_phrase_queries\" : false,\n    \"max_determinized_states\" : 10000,\n    \"enable_position_increments\" : true,\n    \"fuzziness\" : \"AUTO\",\n    \"fuzzy_prefix_length\" : 0,\n    \"fuzzy_max_expansions\" : 50,\n    \"phrase_slop\" : 0,\n    \"escape\" : false,\n    \"split_on_whitespace\" : true,\n    \"boost\" : 1.0\n  }\n}",
        "index_uuid" : "<index-uuid-redacted>",
        "index" : "<index-name-redacted>"
      }
    ],
    "type" : "search_phase_execution_exception",
    "reason" : "all shards failed",
    "phase" : "query",
    "grouped" : true,
    "failed_shards" : [
      {
        "shard" : 0,
        "index" : "<index-name-redacted>",
        "node" : "<node-id-redacted>",
        "reason" : {
          "type" : "query_shard_exception",
          "reason" : "failed to create query: {\n  \"query_string\" : {\n    \"query\" : \"_exists_:req_size\",\n    \"fields\" : [ ],\n    \"use_dis_max\" : true,\n    \"tie_breaker\" : 0.0,\n    \"default_operator\" : \"or\",\n    \"auto_generate_phrase_queries\" : false,\n    \"max_determinized_states\" : 10000,\n    \"enable_position_increments\" : true,\n    \"fuzziness\" : \"AUTO\",\n    \"fuzzy_prefix_length\" : 0,\n    \"fuzzy_max_expansions\" : 50,\n    \"phrase_slop\" : 0,\n    \"escape\" : false,\n    \"split_on_whitespace\" : true,\n    \"boost\" : 1.0\n  }\n}",
          "index_uuid" : "<index-uuid-redacted>",
          "index" : "<index-name-redacted>",
          "caused_by" : {
            "type" : "null_pointer_exception",
            "reason" : null
          }
        }
      }
    ]
  },
  "status" : 400
}

I suspect I'm doing something wrong. Can someone help me understand? Our users have grown accustomed to typing _exists_:foo in the search bar, and they are noticing the Courier errors.

2

Hi,

The only way I'm able to reproduce this is where the index is completely empty with no mapping. For example:

PUT xyz1

GET xyz1/_search
{
  "query": {
    "query_string": {
      "query": "_exists_:req_size"
    }
  }
}

Results in the same null_pointer_exception. However if you do:

PUT xyz2
{
  "mappings": {
    "type": {
      "properties": {
        "name": {
          "type": "text"
        }
      }
    }
  }
}

Then you'll get a response with an empty hits. I filed an issue here as I couldn't find an existing issue:

But as a workaround you may want to add a template that matches the index pattern, so that a mapping is automatically added.

3

Thanks for filing that! I wasn't sure if it was worthy of an issue or not. We're seeing this bug on indexes that are created with very large _default_ mappings (I've trimmed out the ~700 field mappings after the action field, and most of the dynamic_templates section for brevity), but no actual mappings in place yet:

{
	"mappings": {
		"_default_": {
			"_all": {
				"enabled": true,
				"norms": false
			},
			"dynamic_templates": [{
				"string_fields": {
					"match": "*",
					"match_mapping_type": "string",
					"mapping": {
						"doc_values": true,
						"ignore_above": 1024,
						"index": "not_analyzed",
						"type": "string"
					}
				}
			}],
			"properties": {
				"@timestamp": {
					"type": "date",
					"format": "dateOptionalTime"
				},
				"@version": {
					"type": "keyword"
				},
				"actconn": {
					"type": "long"
				},
				"action": {
					"type": "keyword",
					"ignore_above": 1024
				}
			}
		}
	}
}

I've tested this in a VM, and I can confirm that having only set of _default_ mappings has the same NullPointerException.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.