Case insensitive search and doc_values

Elastic Stack Elasticsearch
1

I'm storing lots of keyword like strings in my ES 2.3.3. I treat them like terms meaning that I don't want to analyze them.

However I do want to perform case-insensitive term queries and aggregations. The only way I've found so far is to make these fields analyzed and use keyword analyzer with lowercase filter. This works fine, however I can not utilize doc_values any more. Is there a way to achieve case-insensitive search without loosing doc_values feature?

2

We call those keywords after the analyzer.

In 5.0 you'll be able to do it with the keyword field type. Before that you'll need to use fields inside the field like this:

"title": {
    "type": "string",
    "analyzer": "keyword_lowercase",
    "fields": {
        "raw":   { "type": "string", "index": "not_analyzed" }
    }
}

Then you can use doc values on the raw field and search on the regular field. This means that doc values keep the original case but that might be ok for your use case. With the keyword field the doc values would all be lower case. If you used a lowercasing keyword analyzer.

3

Yeah, I've thought about adding a raw field and use it for aggregation, and the lowercased field for search/filtering. This has the usual question of trade-off between RAM saved by using doc values and disk space "wasted" by storing the data twice. And it's hard to simulate, since I don't know what will be used more - searches or aggregations.

It's great we'll have the best of both worlds in ES 5.0. Thanks for good news!

Zaar