# Question about asciifolding filter

**URL:** <https://discuss.elastic.co/t/question-about-asciifolding-filter/3347>\
**Category:** Elasticsearch\
**Created:** [September 19, 2010, 8:38pm UTC](https://discuss.elastic.co/t/question-about-asciifolding-filter/3347 "2010-09-19T20:38:04Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Andrei](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrei/32/2856_2.png) [@Andrei](https://discuss.elastic.co/u/Andrei)\
**Post date:** [September 19, 2010, 8:38pm UTC](https://discuss.elastic.co/t/question-about-asciifolding-filter/3347/1 "2010-09-19T20:38:04Z")

</div>

I've added a custom analyzer that uses asciifolding filter as follows:

index:  
analysis:  
analyzer:  
eulang:  
type: custom  
tokenizer: standard  
filter: [standard, lowercase, asciifolding, stop]

I then created a new index with the following mapping (so that title,  
notes, and tags fields use the eulang analyzer):

{  
"place" : {  
"\_all" : {enabled: false},  
"properties" : {  
"user\_id" : {"type" : "integer", "index" :  
"not\_analyzed"},  
"title" : {"type" : "string", "boost" : 1.5, "analyzer" :  
"eulang"},  
"notes" : {"type" : "string", "analyzer" : "eulang"},  
"tags" : {"type" : "string", "index\_name" : "tag",  
"boost" : 1.5, "analyzer" : "eulang"},  
"created\_on" : {"type" : "date", "format" : "YYYY-MM-DD  
HH:mm:ss"}  
}  
}  
}'

After inserting a few documents and querying I see that the ASCII  
folding works at index time, but not at query time for some reason:

$ curl '[http://localhost:9200/places\_2010091901/\_search](http://localhost:9200/places_2010091901/_search)?  
q=notes:café' {"\_shards":{"total":1,"successful":1,"failed":0},"hits":  
{"total":0,"max\_score":null,"hits":[]}}

$ curl '[http://localhost:9200/places\_2010091901/\_search?q=notes:cafe](http://localhost:9200/places_2010091901/_search?q=notes:cafe)'  
{"\_shards":{"total":1,"successful":1,"failed":0},"hits":{"total":  
1,"max\_score":0.5,"hits":  
[{"\_index":"places\_2010091901","\_type":"place","\_id":"1","\_score":0.5,  
"\_source" : {"notes": "Café and event space. À côté.", "title":  
"Watershed", "user\_id": 11, "created\_on": "2010-09-19 11:01:19",  
"tags": ["coffee", "wifi", "view"]}}]}}

I was under the impression that the asciifolding filter would be  
applied to the query string as well..

---

<div class="post-metadata">

**Author:** ![Andrei](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrei/32/2856_2.png) [@Andrei](https://discuss.elastic.co/u/Andrei)\
**Post date:** [September 22, 2010, 10:22pm UTC](https://discuss.elastic.co/t/question-about-asciifolding-filter/3347/2 "2010-09-22T22:22:51Z")

</div>

Seems that it works fine for the query\_string query, not the term one  
I was doing.

-Andrei

On Sep 19, 4:38 pm, Andrei [and...@zmievski.org](mailto:and...@zmievski.org) wrote:

> I've added a custom analyzer that uses asciifolding filter as follows:
> 
> index:  
> analysis:  
> analyzer:  
> eulang:  
> type: custom  
> tokenizer: standard  
> filter: [standard, lowercase, asciifolding, stop]
> 
> I then created a new index with the following mapping (so that title,  
> notes, and tags fields use the eulang analyzer):
> 
> {  
> "place" : {  
> "\_all" : {enabled: false},  
> "properties" : {  
> "user\_id" : {"type" : "integer", "index" :  
> "not\_analyzed"},  
> "title" : {"type" : "string", "boost" : 1.5, "analyzer" :  
> "eulang"},  
> "notes" : {"type" : "string", "analyzer" : "eulang"},  
> "tags" : {"type" : "string", "index\_name" : "tag",  
> "boost" : 1.5, "analyzer" : "eulang"},  
> "created\_on" : {"type" : "date", "format" : "YYYY-MM-DD  
> HH:mm:ss"}  
> }  
> }
> 
> }'
> 
> After inserting a few documents and querying I see that the ASCII  
> folding works at index time, but not at query time for some reason:
> 
> $ curl '[http://localhost:9200/places\_2010091901/\_search](http://localhost:9200/places_2010091901/_search)?  
> q=notes:café' {"\_shards":{"total":1,"successful":1,"failed":0},"hits":  
> {"total":0,"max\_score":null,"hits":}}
> 
> $ curl '[http://localhost:9200/places\_2010091901/\_search?q=notes:cafe](http://localhost:9200/places_2010091901/_search?q=notes:cafe)'  
> {"\_shards":{"total":1,"successful":1,"failed":0},"hits":{"total":  
> 1,"max\_score":0.5,"hits":  
> [{"\_index":"places\_2010091901","\_type":"place","\_id":"1","\_score":0.5,  
> "\_source" : {"notes": "Café and event space. À côté.", "title":  
> "Watershed", "user\_id": 11, "created\_on": "2010-09-19 11:01:19",  
> "tags": ["coffee", "wifi", "view"]}}]}}
> 
> I was under the impression that the asciifolding filter would be  
> applied to the query string as well..

---

<div class="post-metadata">

**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\
**Post date:** [September 23, 2010, 4:59pm UTC](https://discuss.elastic.co/t/question-about-asciifolding-filter/3347/3 "2010-09-23T16:59:23Z")

</div>

Hey,

When you say term query, is that a query DSL using the term query, or the  
first example with the \_search?q=... ? The latter gets translated into a  
query\_string query. When using the term query itself, then no analysis  
happens on the provided term value, when using the query\_string or field  
queries, then analysis does happen based on the analyzer associated with the  
queries field.

-shay.banon

On Thu, Sep 23, 2010 at 12:22 AM, Andrei [andrei@zmievski.org](mailto:andrei@zmievski.org) wrote:

> Seems that it works fine for the query\_string query, not the term one  
> I was doing.
> 
> -Andrei
> 
> On Sep 19, 4:38 pm, Andrei [and...@zmievski.org](mailto:and...@zmievski.org) wrote:
> 
> > I've added a custom analyzer that uses asciifolding filter as follows:
> > 
> > index:  
> > analysis:  
> > analyzer:  
> > eulang:  
> > type: custom  
> > tokenizer: standard  
> > filter: [standard, lowercase, asciifolding, stop]
> > 
> > I then created a new index with the following mapping (so that title,  
> > notes, and tags fields use the eulang analyzer):
> > 
> > {  
> > "place" : {  
> > "\_all" : {enabled: false},  
> > "properties" : {  
> > "user\_id" : {"type" : "integer", "index" :  
> > "not\_analyzed"},  
> > "title" : {"type" : "string", "boost" : 1.5, "analyzer" :  
> > "eulang"},  
> > "notes" : {"type" : "string", "analyzer" : "eulang"},  
> > "tags" : {"type" : "string", "index\_name" : "tag",  
> > "boost" : 1.5, "analyzer" : "eulang"},  
> > "created\_on" : {"type" : "date", "format" : "YYYY-MM-DD  
> > HH:mm:ss"}  
> > }  
> > }
> > 
> > }'
> > 
> > After inserting a few documents and querying I see that the ASCII  
> > folding works at index time, but not at query time for some reason:
> > 
> > $ curl '[http://localhost:9200/places\_2010091901/\_search](http://localhost:9200/places_2010091901/_search)?  
> > q=notes:café' {"\_shards":{"total":1,"successful":1,"failed":0},"hits":  
> > {"total":0,"max\_score":null,"hits":}}
> > 
> > $ curl '[http://localhost:9200/places\_2010091901/\_search?q=notes:cafe](http://localhost:9200/places_2010091901/_search?q=notes:cafe)'  
> > {"\_shards":{"total":1,"successful":1,"failed":0},"hits":{"total":  
> > 1,"max\_score":0.5,"hits":  
> > [{"\_index":"places\_2010091901","\_type":"place","\_id":"1","\_score":0.5,  
> > "\_source" : {"notes": "Café and event space. À côté.", "title":  
> > "Watershed", "user\_id": 11, "created\_on": "2010-09-19 11:01:19",  
> > "tags": ["coffee", "wifi", "view"]}}]}}
> > 
> > I was under the impression that the asciifolding filter would be  
> > applied to the query string as well..

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:19am UTC](https://discuss.elastic.co/t/question-about-asciifolding-filter/3347/4 "2017-07-06T04:19:00Z")

</div>