Null_value not working

[NoZomIBK]

I have an integer field which may be null and should have the null_value 0. for testing i prepared an index:

curl -H "Content-Type: application/json" -XPUT -d "{"mappings":{"_doc":{"properties":{"name":{"type":"text"},"documentCounter":{"type":"integer","null_value":0}}}}}" localhost:9200/nulltest

and added one doc:

curl -H "Content-Type: application/json" -XPUT -d "{"name":"Some name"}" localhost:9200/nulltest/_doc/1

it can be queried without problems:

curl localhost:9200/nulltest/_doc/1

{"_index":"nulltest","_type":"_doc","_id":"1","_version":1,"found":true,"_source":{"name":"Some name"}}

i would expect it to be found by this search:

curl -H "Content-Type: application/json" -XPOST -d "{"query":{"bool":{"must":[{"term":{"documentCounter":0}}]}}}" localhost:9200/nulltest/_search

but the response has no hits

{"took":5,"timed_out":false,"_shards":{"total":5,"successful":5,"skipped":0,"failed":0},"hits":{"total":0,"max_score":null,"hits":}}

and on top of this there is a bigger problem for me. When using the documentCounter field in a function it happens to fail depending on the client (which seems strange).

curl -H "Content-Type: application/json" -XPOST -d "{"from":0,"size":8192,"query":{"bool":{"must":[{"function_score":{"query":{"bool":{"should":[{"match":{"name":{"query":"Somename","operator":"AND","fuzziness":"0","prefix_length":0,"max_expansions":50,"fuzzy_transpositions":true,"lenient":false,"zero_terms_query":"NONE","cutoff_frequency":0.35,"auto_generate_synonyms_phrase_query":true,"boost":2}}}],"adjust_pure_negative":true,"boost":1}},"functions":[{"filter":{"match_all":{"boost":1}},"field_value_factor":{"field":"documentCounter","factor":1,"modifier":"log1p"}}],"score_mode":"sum","max_boost":3.4028235e+38,"boost":1}}],"adjust_pure_negative":true,"boost":1}},"version":true,"sort":[{"_score":{"order":"desc"}}]}" localhost:9200/nulltest/_search

the response with curl is:

{"took":4,"timed_out":false,"_shards":{"total":5,"successful":5,"skipped":0,"failed":0},"hits":{"total":0,"max_score":null,"hits":}}

when i make the exact same query with the chrome addon "elasticsearch head" one shard failes:

{"took": 5,"timed_out": false,"_shards": {"total": 5,"successful": 4,"skipped": 0,"failed": 1,"failures": [{"shard": 3,"index": "nulltest","node": "6vOdvUnpQ6u9BciVk7LwFw","reason": {"type": "exception","reason": "Missing value for field [documentCounter]"}}]},"hits": {"total": 0,"max_score": null,"hits": }}

when i make this request with the elasticsearch java-client 6.5.3 the whole call fails with all shards failed:

{"error":{"root_cause":[{"type":"exception","reason":"Missing value for field [documentCounter]"}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"nulltest","node":"6vOdvUnpQ6u9BciVk7LwFw","reason":{"type":"exception","reason":"Missing value for field [documentCounter]"}}]},"status":500}

i ran the searches with the three clients multiple times in all orders without any change in the results.

My server runs on 6.4.2

the expected behaviour would be:

1. search for documentcounter:0 to return the document
2. no failures of the search with the filter and same behaviour for same queries in different clients

[NoZomIBK]

Server updated to 6.6.0 did not resolve the problem either

[NoZomIBK]

Ok, the issue was, the documentCounter was not set explicit to null. i would preferr for it working on missing values, too, or at least having an option for this.

But the documentation (null_value | Elasticsearch Guide [6.6] | Elastic) is a bit wrong. null, empty array and array with only null values should be set to the null_value.

When a field is set to null , (or an empty array or an array of null values) it is treated as though that field has no values.

But in the example the empty array does not trigger the null_value to be inserted

An empty array does not contain an explicit null , and so won’t be replaced with the null_value .