# Keyword case insensitive search with non-ascii **URL:** \ **Category:** Elasticsearch\ **Created:** [August 23, 2023, 10:17am UTC](https://discuss.elastic.co/t/keyword-case-insensitive-search-with-non-ascii/341460 "2023-08-23T10:17:48Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Volodymyr\_Kovtun](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/volodymyr_kovtun/32/123392_2.png) [@Volodymyr\_Kovtun](https://discuss.elastic.co/u/Volodymyr_Kovtun)\ **Post date:** [August 23, 2023, 10:17am UTC](https://discuss.elastic.co/t/keyword-case-insensitive-search-with-non-ascii/341460/1 "2023-08-23T10:17:48Z")
Hi Could you please clarify if it is possible to have case-insensitive **term.keyword** search with non-ascii symbols? It seems not to work. Here is the example: Indexing 4 documents (2 ascii and 2 non-ascii) ```auto >>> es.index(index='testi', document={'text': 'ċ'}) # non-ascii >>> es.index(index='testi', document={'text': 'Ċ'}) # non-ascii >>> es.index(index='testi', document={'text': 'a'}) # ascii >>> es.index(index='testi', document={'text': 'A'}) # ascii ``` It works well with ascii symbols ```auto >>> pprint(es.search(index='testi', query={"bool":{"must":[{"term": {'text.keyword': {'value': 'a', 'case_insensitive': 'true' } }}]}})['hits']) {'hits': [{'_id': 'QV3ZIYoBpIJf8GfZm98T', '_index': 'testi', '_score': 1.0, '_source': {'text': 'a'}}, {'_id': 'Ql3ZIYoBpIJf8GfZpd8n', '_index': 'testi', '_score': 1.0, '_source': {'text': 'A'}}], 'max_score': 1.0, 'total': {'relation': 'eq', 'value': 2}} ``` and seems to now work with non-ascii ```auto >>> pprint(es.search(index='testi', query={"bool":{"must":[{"term": {'text.keyword': {'value': 'ċ', 'case_insensitive': 'true' } }}]}})['hits']) {'hits': [{'_id': 'P13YIYoBpIJf8GfZbd8D', '_index': 'testi', '_score': 1.0, '_source': {'text': 'ċ'}}], 'max_score': 1.0, 'total': {'relation': 'eq', 'value': 1}} ``` ```auto >>> pprint(es.search(index='testi', query={"bool":{"must":[{"term": {'text.keyword': {'value': 'Ċ', 'case_insensitive': 'true' } }}]}})['hits']) {'hits': [{'_id': 'QF3ZIYoBpIJf8GfZgt_s', '_index': 'testi', '_score': 1.0, '_source': {'text': 'Ċ'}}], 'max_score': 1.0, 'total': {'relation': 'eq', 'value': 1}} ``` All settings/mappings are default. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [August 23, 2023, 10:24am UTC](https://discuss.elastic.co/t/keyword-case-insensitive-search-with-non-ascii/341460/2 "2023-08-23T10:24:31Z")
I suspect you may need to define a [suitable (possibly custom) normalizer](https://www.elastic.co/guide/en/elasticsearch/reference/8.9/analysis-normalizers.html) in your mappings. ---
**Author:** ![Volodymyr\_Kovtun](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/volodymyr_kovtun/32/123392_2.png) [@Volodymyr\_Kovtun](https://discuss.elastic.co/u/Volodymyr_Kovtun)\ **Post date:** [August 23, 2023, 10:38am UTC](https://discuss.elastic.co/t/keyword-case-insensitive-search-with-non-ascii/341460/3 "2023-08-23T10:38:56Z")
```auto >>> es.indices.analyze(text='Ċ', analyzer='default') ObjectApiResponse({'tokens': [{'token': 'ċ', 'start_offset': 0, 'end_offset': 1, 'type': '', 'position': 0}]}) >>> es.indices.analyze(text='ċ', analyzer='default') ObjectApiResponse({'tokens': [{'token': 'ċ', 'start_offset': 0, 'end_offset': 1, 'type': '', 'position': 0}]}) ``` Does not this mean that text is analyzed as expected ? **match** seems to work as expected too ```auto >>> pprint(es.search(index='testi', query={"bool":{"must":[{"match": {'text': 'ċ' }}]}})['hits']) {'hits': [{'_id': 'P13YIYoBpIJf8GfZbd8D', '_index': 'testi', '_score': 0.6931471, '_source': {'text': 'ċ'}}, {'_id': 'QF3ZIYoBpIJf8GfZgt_s', '_index': 'testi', '_score': 0.6931471, '_source': {'text': 'Ċ'}}], 'max_score': 0.6931471, 'total': {'relation': 'eq', 'value': 2}} ``` ---
**Author:** ![Volodymyr\_Kovtun](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/volodymyr_kovtun/32/123392_2.png) [@Volodymyr\_Kovtun](https://discuss.elastic.co/u/Volodymyr_Kovtun)\ **Post date:** [August 25, 2023, 12:17pm UTC](https://discuss.elastic.co/t/keyword-case-insensitive-search-with-non-ascii/341460/4 "2023-08-25T12:17:11Z")
Am I wrong regarding analyzers ? Can someone explain how adding custom normalizer will resolve the issue ? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [August 25, 2023, 12:34pm UTC](https://discuss.elastic.co/t/keyword-case-insensitive-search-with-non-ascii/341460/5 "2023-08-25T12:34:42Z")
The example you are showing here uses the default analyzer, which seems to provide the expected result with respect to lower casing. The case insensitive option within the term query you initially showed, does as far as I know not use the default analyser, so the behaviour may be different. Based on your initial post it would seem like it only is able to handle lower-casing of ascii characters, which is why I suggested creating a separate lowercased multifield in your mappings that use a char filter to ensure you get the behaviour you are expecting. It could be that this is a bug or maybe a known limitation. ---
**Author:** ![Mark\_Harwood1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood1/32/101255_2.png) [@Mark\_Harwood1](https://discuss.elastic.co/u/Mark_Harwood1)\ **Post date:** [August 26, 2023, 7:49am UTC](https://discuss.elastic.co/t/keyword-case-insensitive-search-with-non-ascii/341460/6 "2023-08-26T07:49:13Z")
The [case insensitive setting on the term query](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-term-query.html#term-field-params) only works for ASCII text and is documented as such. It was introduced mainly as a simple tool to help with searching keyword fields where no attempts to process the content had been made by the administrator (tokenising, stemming, normalising etc). It’s not intended to be an alternative to normalizers or analyzers which are the tools required to deal with a variety of languages and content ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [September 23, 2023, 7:50am UTC](https://discuss.elastic.co/t/keyword-case-insensitive-search-with-non-ascii/341460/7 "2023-09-23T07:50:12Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.