NumberFormatException - how can I filter queries around it?

Elastic Stack Elasticsearch
1

I have a rather large (or at least non-small) logs use-case, with an index per day with many types. Many of those types have a "status" field, some in nested objects and some right at _source level. I have struggled to rename all the non-integer versions of the fields named "status" to other things (such as "delivery_status" for mail logs).

One user wants to sort on response.status but it keeps erroring out with the well-known "Number Format Exception [Invalid shift value....]". I have dug through the day's index and found some "status":null entries.

Can anyone tell me whether these are the likely cause of the NumberFormatException, and whether there's any way to do a filtered query around them?
I have done: curl 'localhost:9200/logs-2015.06.12/sometype/_search?pretty=1&size=10' -d '{"query":{"filtered":{"filter":{"exists":{"field":"response.status"}}}}' and if I add a sort clause on that field it still fails in that way.

I am adding "default_value":0 clauses to all the various "status" fields in the template, for future indices, but short of re-indexing past days (and they tend to be in the 1G doc range), I'm hoping a query can be made to handle them.

Thanks for whatever you might be able to tell me.

2

maybe try missing values? https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-sort.html#_missing_values check out unmapped_type too.

hth

3

Thank you for that suggestion!

I put in both missing_value and unmapped_type; still getting the error. This is most vexing.

Any clues on how to find what value in what record is fouling the field?

Thanks...