Terms aggregation with “missing”

Elastic Stack Elasticsearch
1

I am observing these behaviors.

For a query like under works.

"size" :0,
  "aggregations" : {
    “keywordTypeField” : {
      "terms" : {
        "field" : “keywordTypeField”,
        "missing": “N/A”,
        "min_doc_count" : 1
      }
   }

But if I use non-keyword type fields, such as long or ip, it gives exceptions.

 "size" :0,
   "aggregations" : {
     “longTypeField” : {
       "terms" : {
         "field" : “longTypeField”,
         "missing": “N/A”,
         "min_doc_count" : 1
       }
   }

Exceptions like:
(for ip type) unknown_host_exception, addr is of illegal length
(for long type) number_format_exception, empty String

However, once I remove the “missing” line, the query works.
Is this expected, or something in my env?
I am using Elasticsearch 5.6.4.

Thanks in advance.

1 Like
2

The missing is the value that Elasticsearch will assign to documents that do not have a value for the field that you are aggregating upon (keywordTypeField and longTypeField in your examples).

It works for keywordTypeField because the string "N/A" is a valid value for a field of type keyword. However, "N/A" is not a valid long nor a valid ip). Maybe you can change missing to a value like -1 for long type aggregations, and 192.0.2.0 for ip type aggregations.

1 Like
3

Ahh ... Thanks, great!

FYI, for ip, still get the exception - using tools like "Elasticsearch head" or curl.
However, with Transport API it works. When print out it's json, it's like (nothing special),

        "missing" : "0.0.0.0",

But if I use tools with copy & paste the line, get the exception.
If you can shed light on, that will be greater ^^. Regardless, thanks again.

4

It looks like you have to provide the IP as an array of the 4 digits of the IP. The following seems to work:

"missing": "\u0000\u0000\u0000\u0000"

5

Nice ... :grinning:
All working now. Transport API case also using that format seems more reliable.
Thanks Abdon,

6

I have raised a GitHub issue about the fact that missing does not accept IPs: https://github.com/elastic/elasticsearch/issues/27788

7

Thanks. Good to see it will be better and also good Elasticsearch inside insight as well. ^^

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.