# Asciifolding character filter **URL:** \ **Category:** Elasticsearch\ **Created:** [January 19, 2015, 6:18pm UTC](https://discuss.elastic.co/t/asciifolding-character-filter/21716 "2015-01-19T18:18:47Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Mathijs\_Biesmans](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mathijs_biesmans/32/987_2.png) [@Mathijs\_Biesmans](https://discuss.elastic.co/u/Mathijs_Biesmans)\ **Post date:** [January 19, 2015, 6:18pm UTC](https://discuss.elastic.co/t/asciifolding-character-filter/21716/1 "2015-01-19T18:18:47Z")
I'm curious whether there exists an asciifolding _character_ filter, I know there is a asciifolding _token_ filter and that the analysis chain works as follows: input text \> char\_filter \> tokenizer \> token filter \> output tokens. The text on [http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/asciifolding-token-filter.html](http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/asciifolding-token-filter.html) mentions: [...]With Western languages, this can be done with the asciifolding _character_ filter.[...], though the url says _asciifolding-token-filter_. An error in the docs? I also checked the icu-plugin: the _icu\_normalizer_ can be used both as a character filter and a token filter. But the _icu\_folding_ filter is only available as a token filter (that actually incorporates the icu\_normalizer). I'm generating ngrams and shingles, so it seems more logical to aplpy ascii/icu folding as a character filter. But I can't find one? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/1fe8fcec-7d9b-4b92-ad29-d4a7289de8dc%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/1fe8fcec-7d9b-4b92-ad29-d4a7289de8dc%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [January 19, 2015, 7:39pm UTC](https://discuss.elastic.co/t/asciifolding-character-filter/21716/2 "2015-01-19T19:39:44Z")
Hey, cool idea. That's fairly easy to implement. I've just added a char folding char filter into my version of ICU plugin > Jörg On Mon, Jan 19, 2015 at 7:18 PM, Mathijs Biesmans \< [mathijs.biesmans@gmail.com](mailto:mathijs.biesmans@gmail.com)\> wrote: > I'm curious whether there exists an asciifolding _character_ filter, I > know there is a asciifolding _token_ filter and that the analysis chain > works as follows: input text \> char\_filter \> tokenizer \> token filter \> > output tokens. > > The text on > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/asciifolding-token-filter.html) > mentions: [...]With Western languages, this can be done with the > asciifolding _character_ filter.[...], though the url says > _asciifolding-token-filter_. An error in the docs? > > I also checked the icu-plugin: the _icu\_normalizer_ can be used both as a > character filter and a token filter. But the _icu\_folding_ filter is only > available as a token filter (that actually incorporates the icu\_normalizer). > > I'm generating ngrams and shingles, so it seems more logical to aplpy > ascii/icu folding as a character filter. But I can't find one? > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/1fe8fcec-7d9b-4b92-ad29-d4a7289de8dc%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/1fe8fcec-7d9b-4b92-ad29-d4a7289de8dc%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/1fe8fcec-7d9b-4b92-ad29-d4a7289de8dc%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/1fe8fcec-7d9b-4b92-ad29-d4a7289de8dc%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoF8BYg%2BqzHfodFyp913Cf-NhbvwqHFwRwV34RFJafbW9g%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoF8BYg%2BqzHfodFyp913Cf-NhbvwqHFwRwV34RFJafbW9g%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Mathijs\_Biesmans](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mathijs_biesmans/32/987_2.png) [@Mathijs\_Biesmans](https://discuss.elastic.co/u/Mathijs_Biesmans)\ **Post date:** [January 20, 2015, 9:18am UTC](https://discuss.elastic.co/t/asciifolding-character-filter/21716/3 "2015-01-20T09:18:49Z")
Thanks Jörg. So I guess the character filter didn't existed. Will this be pushed to official releases? Currently I'm using a hosted ES-cluster, and I don't want to install custom plugins... Op maandag 19 januari 2015 20:39:50 UTC+1 schreef Jörg Prante: > Hey, cool idea. That's fairly easy to implement. I've just added a char > folding char filter into my version of ICU plugin > > [adding ICU folding char filter · jprante/elasticsearch-plugin-bundle@e4294cc · GitHub](https://github.com/jprante/elasticsearch-plugin-bundle/commit/e4294cc0f4d45dabf50d840713820f8eb57152b6) > > Jörg > > On Mon, Jan 19, 2015 at 7:18 PM, Mathijs Biesmans \<[mathijs....@gmail.com](mailto:mathijs....@gmail.com) > \\> wrote: > > > I'm curious whether there exists an asciifolding _character_ filter, I > > know there is a asciifolding _token_ filter and that the analysis chain > > works as follows: input text \> char\_filter \> tokenizer \> token filter \> > > output tokens. > > > > The text on > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/asciifolding-token-filter.html) > > mentions: [...]With Western languages, this can be done with the > > asciifolding _character_ filter.[...], though the url says > > _asciifolding-token-filter_. An error in the docs? > > > > I also checked the icu-plugin: the _icu\_normalizer_ can be used both as > > a character filter and a token filter. But the _icu\_folding_ filter is > > only available as a token filter (that actually incorporates the > > icu\_normalizer). > > > > I'm generating ngrams and shingles, so it seems more logical to aplpy > > ascii/icu folding as a character filter. But I can't find one? > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/1fe8fcec-7d9b-4b92-ad29-d4a7289de8dc%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/1fe8fcec-7d9b-4b92-ad29-d4a7289de8dc%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/1fe8fcec-7d9b-4b92-ad29-d4a7289de8dc%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/1fe8fcec-7d9b-4b92-ad29-d4a7289de8dc%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/5abd890b-1caf-46a8-afd7-c36a0b79bb05%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/5abd890b-1caf-46a8-afd7-c36a0b79bb05%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [January 20, 2015, 11:38pm UTC](https://discuss.elastic.co/t/asciifolding-character-filter/21716/4 "2015-01-20T23:38:47Z")
I can't tell. Official elasticsearch ICU plugin is lagging behind Lucene 5.0, ICUCollationKeyAnalyzer / collation key field type, API deprecation updates etc. so I hope it soon will take up pace again to get ready for new features. Jörg On Tue, Jan 20, 2015 at 10:18 AM, Mathijs Biesmans \< [mathijs.biesmans@gmail.com](mailto:mathijs.biesmans@gmail.com)\> wrote: > Thanks Jörg. So I guess the character filter didn't existed. > > Will this be pushed to official releases? Currently I'm using a hosted > ES-cluster, and I don't want to install custom plugins... > > Op maandag 19 januari 2015 20:39:50 UTC+1 schreef Jörg Prante: > > > Hey, cool idea. That's fairly easy to implement. I've just added a char > > folding char filter into my version of ICU plugin > > > > [Update README.adoc · jprante/elasticsearch-plugin-bundle@164387f · GitHub](https://github.com/jprante/elasticsearch-plugin-bundle/commit/) > > e4294cc0f4d45dabf50d840713820f8eb57152b6 > > > > Jörg > > > > On Mon, Jan 19, 2015 at 7:18 PM, Mathijs Biesmans [mathijs....@gmail.com](mailto:mathijs....@gmail.com) > > wrote: > > > > > I'm curious whether there exists an asciifolding _character_ filter, I > > > know there is a asciifolding _token_ filter and that the analysis chain > > > works as follows: input text \> char\_filter \> tokenizer \> token filter \> > > > output tokens. > > > > > > The text on [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/guide/) > > > current/asciifolding-token-filter.html mentions: [...]With Western > > > languages, this can be done with the asciifolding _character_ filter.[...], > > > though the url says _asciifolding-token-filter_. An error in the docs? > > > > > > I also checked the icu-plugin: the _icu\_normalizer_ can be used both as > > > a character filter and a token filter. But the _icu\_folding_ filter is > > > only available as a token filter (that actually incorporates the > > > icu\_normalizer). > > > > > > I'm generating ngrams and shingles, so it seems more logical to aplpy > > > ascii/icu folding as a character filter. But I can't find one? > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > msgid/elasticsearch/1fe8fcec-7d9b-4b92-ad29-d4a7289de8dc% > > > [40googlegroups.com](http://40googlegroups.com) > > > [https://groups.google.com/d/msgid/elasticsearch/1fe8fcec-7d9b-4b92-ad29-d4a7289de8dc%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/1fe8fcec-7d9b-4b92-ad29-d4a7289de8dc%40googlegroups.com?utm_medium=email&utm_source=footer) > > > . > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/5abd890b-1caf-46a8-afd7-c36a0b79bb05%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/5abd890b-1caf-46a8-afd7-c36a0b79bb05%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/5abd890b-1caf-46a8-afd7-c36a0b79bb05%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/5abd890b-1caf-46a8-afd7-c36a0b79bb05%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHs1vgPoBgpB7KrfGu4AqikyC\_XNWYuu8G77J4vrB%3D\_FQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHs1vgPoBgpB7KrfGu4AqikyC_XNWYuu8G77J4vrB%3D_FQ%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 12:37am UTC](https://discuss.elastic.co/t/asciifolding-character-filter/21716/5 "2017-07-06T00:37:38Z")