Field Type: token_count - major query speed differences on range vs. term query

[spinscale (Alexander Reelsen)]

Hey,

I've been running a query that is doing a term query against a field mapped as token_count. This query has very different speeds depending if I use a term query like this:

{ "term" : { "token_count_field": 1 }

vs

{ "range" : { "token_count_field": { "lte": 1, "gte": 1} }

The latter one is much faster. A sample query that runs around 30ms now is down to 15ms by just using the range query.

The profiler shows, that the is a PointRangeQuery on the token count field, where as the fast query is a IndexOrDocValuesQuery.

Am I missing something or should that be the default for a term query for the token count filter?

This is on 7.17.18.

--Alex

[spinscale (Alexander Reelsen)]

Here is a sample to reproduce, happens also on 8.11.4

DELETE test 

PUT test/
{
  "mappings": {
    "properties": {
      "token_count" : {
        "type": "token_count",
        "analyzer": "standard"
      }
    }
  }
}

PUT test/_doc/21refresh
{
  "token_count" : "foo bar baz"
}

GET test/_search
{
  "profile": true, 
  "query": {
    "term": {
      "token_count": {
        "value": "3"
      }
    }
  }
}

GET test/_search
{
  "profile": true, 
  "query": {
    "range": {
      "token_count": {
        "gte": "3", "lte" : 3
      }
    }
  }
}