Returning bucket with 0 count in terms aggregation

Elastic Stack Elasticsearch
1

I want to get bucket for all my required values along with their count. I am doing that via terms aggregations. But somehow, the query only returns buckets that match some documents even though I set min_doc_count to 0. See the query below:

POST myindex/_search
{
    "size": 0,
    "aggs":{
      "matching_values_field1": {
         "filter": { 
            "terms" : { "myfield" : ["1", "2", "3","100"]}
         },
         "aggs": {
            "myfield" : {
                "terms" : { 
                    "field" : "myfield",
                    "min_doc_count": 0
                }                
            }
         }
      }
   }    
}

Following is the response, where there is no bucket for value 100. Any ideas?

"aggregations": {
      "matching_values_1": {
         "doc_count": 11,
         "myfield": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
               {
                  "key": 2,
                  "doc_count": 6
               },
               {
                  "key": 1,
                  "doc_count": 3
               },
               {
                  "key": 3,
                  "doc_count": 2
               },
               {
                  "key": 0,
                  "doc_count": 0
               },
               {
                  "key": 0.5,
                  "doc_count": 0
               },
               {
                  "key": 4,
                  "doc_count": 0
               },
               {
                  "key": 5,
                  "doc_count": 0
               },
               {
                  "key": 6,
                  "doc_count": 0
               },
               {
                  "key": 14,
                  "doc_count": 0
               }
            ]
         }
      }
   }
2

This means that 100 does not exist in your index. A value of 0 for min_doc_count means that all terms that are present in the index (the dictionary of the field) will be considered but it doesn't return terms that are not in any document.

3

Thank you for your reply. Is there a way for enforce ES to return doc count for the key that is not in the dictionary? The key could be specified via terms aggregation (like show in my example) or something else.

4

You can use the include option of the terms aggregation:
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-terms-aggregation.html#_filtering_values_with_exact_values

5

doesn't seem to work for me. What am I missing?

See query:

POST myindex/_search
{
    "size": 0,
    "aggs": {
            "myfield" : {
                "terms" : { 
                    "field" : "myfield",
                    "include": ["1", "2", "3", "100"],
                    "min_doc_count": 0,    
                    "shard_min_doc_count": 0,
                    "size": 20
                }                
            }
         }   
}

RESPONSE:

"aggregations": {
      "QID11_4": {
         "doc_count_error_upper_bound": 0,
         "sum_other_doc_count": 0,
         "buckets": [
            {
               "key": 2,
               "doc_count": 6
            },
            {
               "key": 1,
               "doc_count": 3
            },
            {
               "key": 3,
               "doc_count": 2
            }
         ]
      }
   }
6

Yes sorry the includes will only filter documents but cannot create the bucket if the term does not exist in the index.
Though the fact that this term is not in the response indicates that there is no document with this this value.
If you really want the bucket to be returned, you'll have to use the filters aggregation:

{
	"size": 0,
	"aggs" : {
    "matching_values_field1" : {
      "filters" : {
        "filters" : {
          "1" :   { "term" : { "my_field" : 100   }},
          "100" : { "term" : { "my_field" : 1 }}}
      }
    }
  }
}
1 Like
7

Thanks. That seems to be working.

Is there any way to simply add the term to the index while creating the index or dynamically? For example, if we already are aware of the possible domains (values) for a field in the index, can we add it while creating it or dynamically?

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.