# How to sort Norwegian special characters with the ICU plugin? **URL:** \ **Category:** Elasticsearch\ **Created:** [July 23, 2013, 1:36pm UTC](https://discuss.elastic.co/t/how-to-sort-norwegian-special-characters-with-the-icu-plugin/12901 "2013-07-23T13:36:24Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Olav\_Gronas\_Gjerde](https://avatars.discourse-cdn.com/v4/letter/o/e19b73/32.png) [@Olav\_Gronas\_Gjerde](https://discuss.elastic.co/u/Olav_Gronas_Gjerde)\ **Post date:** [July 23, 2013, 1:36pm UTC](https://discuss.elastic.co/t/how-to-sort-norwegian-special-characters-with-the-icu-plugin/12901/1 "2013-07-23T13:36:24Z")
I have a problem sorting my searches correctly by using norwegian collation. I've installed the [https://github.com/elasticsearch/elasticsearch-analysis-icu](https://github.com/elasticsearch/elasticsearch-analysis-icu) plugin, and I've created my index with the following properties. Java: .startObject("analysis") .startObject("analyzer") .startObject("collation") .field("tokenizer", "keyword") .field("filter", "norwegianCollator") .endObject() .endObject() .startObject("filter") .startObject("norwegianCollator") .field("type", "icu\_collation") .field("language", "norwegian") .endObject() .endObject() .endObject() JSON: { "index": { "analysis": { "analyzer": { "collation": { "tokenizer": "keyword", "filter": "norwegianCollator" } }, "filter": { "norwegianCollator": { "type": "icu\_collation", "language": "nb" } } } } } Elasticsearch Head show this as: settings: { - index.analysis.analyzer.default.filter: norwegianCollator - index.analysis.filter.norwegianCollator.type: icu\_collation - index.analysis.analyzer.collation.tokenizer: keyword - index.analysis.filter.norwegianCollator.language: norwegian } Is this correctly configured? For search I'm doing this: ``` QueryBuilder qb = QueryBuilders.matchQuery( "customer.partner", "1" ); SearchResponse response = this.esClient.prepareSearch(this.client.getName()) .setTypes("customer") .setSearchType(SearchType.DFS_QUERY_THEN_FETCH) .setQuery(qb) .setFrom(0).setSize(size) .addSort("name", SortOrder.DESC) .execute() .actionGet(); ``` But the result I get is: DEBUG 15:21:00 search.ESSearch - {enabled=true, name=ølen fiskelag, partner=1} DEBUG 15:21:00 search.ESSearch - {enabled=true, name=æ vil ha dæ, partner=1} DEBUG 15:21:00 search.ESSearch - {enabled=true, name=åges plateselskap as, partner=1} DEBUG 15:21:00 search.ESSearch - {enabled=true, name=yara international, partner=1} When it should be: DEBUG 15:21:00 search.ESSearch - {enabled=true, name=åges plateselskap as, partner=1} DEBUG 15:21:00 search.ESSearch - {enabled=true, name=ølen fiskelag, partner=1} DEBUG 15:21:00 search.ESSearch - {enabled=true, name=æ vil ha dæ, partner=1} DEBUG 15:21:00 search.ESSearch - {enabled=true, name=yara international, partner=1} What am I doing wrong? What could be missing in my configuration? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Lukas\_Vlcek1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lukas_vlcek1/32/819_2.png) [@Lukas\_Vlcek1](https://discuss.elastic.co/u/Lukas_Vlcek1)\ **Post date:** [July 23, 2013, 1:57pm UTC](https://discuss.elastic.co/t/how-to-sort-norwegian-special-characters-with-the-icu-plugin/12901/2 "2013-07-23T13:57:06Z")
Hi, just quickly looked at your code and I am not sure you are providing a valid language code. As far as I understand you need to provide code according to [ISO 639-2 Language Code List - Codes for the representation of names of languages (Library of Congress)](http://www.loc.gov/standards/iso639-2/php/code_list.php) (for norwegian there are couple of options: no, nb, nn. I dunno what is he difference.) Can you try it with "no" for example? Also note that it might be more helpful if you can provide examples using curl (see [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/help/) for details). Regards, Lukas On Tue, Jul 23, 2013 at 3:36 PM, Olav Grønås Gjerde [olavgg@gmail.com](mailto:olavgg@gmail.com)wrote: > I have a problem sorting my searches correctly by using norwegian > collation. > > I've installed the > [GitHub - elastic/elasticsearch-analysis-icu: ICU Analysis plugin for Elasticsearch](https://github.com/elasticsearch/elasticsearch-analysis-icu) plugin, and > I've created my index with the following properties. > > Java: > .startObject("analysis") > .startObject("analyzer") > .startObject("collation") > .field("tokenizer", "keyword") > .field("filter", "norwegianCollator") > .endObject() > .endObject() > .startObject("filter") > .startObject("norwegianCollator") > .field("type", "icu\_collation") > .field("language", "norwegian") > .endObject() > .endObject() > .endObject() > > JSON: > { > "index": { > "analysis": { > "analyzer": { > "collation": { > "tokenizer": "keyword", > "filter": "norwegianCollator" > } > }, > "filter": { > "norwegianCollator": { > "type": "icu\_collation", > "language": "nb" > } > } > } > } > } > > Elasticsearch Head show this as: > > settings: { > > - index.analysis.analyzer.default.filter: norwegianCollator > - index.analysis.filter.norwegianCollator.type: icu\_collation > - index.analysis.analyzer.collation.tokenizer: keyword > - index.analysis.filter.norwegianCollator.language: norwegian > > } > > Is this correctly configured? > > For search I'm doing this: > > ``` > QueryBuilder qb = QueryBuilders.matchQuery( > "customer.partner", > "1" > ); > SearchResponse response = > this.esClient.prepareSearch(this.client.getName()) > .setTypes("customer") > .setSearchType(SearchType.DFS_QUERY_THEN_FETCH) > .setQuery(qb) > .setFrom(0).setSize(size) > .addSort("name", SortOrder.DESC) > .execute() > .actionGet(); > > ``` > > But the result I get is: > > DEBUG 15:21:00 search.ESSearch - {enabled=true, name=ølen fiskelag, > partner=1} > DEBUG 15:21:00 search.ESSearch - {enabled=true, name=æ vil ha dæ, > partner=1} > DEBUG 15:21:00 search.ESSearch - {enabled=true, name=åges plateselskap as, > partner=1} > DEBUG 15:21:00 search.ESSearch - {enabled=true, name=yara international, > partner=1} > > When it should be: > > DEBUG 15:21:00 search.ESSearch - {enabled=true, name=åges plateselskap as, > partner=1} > DEBUG 15:21:00 search.ESSearch - {enabled=true, name=ølen fiskelag, > partner=1} > DEBUG 15:21:00 search.ESSearch - {enabled=true, name=æ vil ha dæ, > partner=1} > DEBUG 15:21:00 search.ESSearch - {enabled=true, name=yara international, > partner=1} > > What am I doing wrong? What could be missing in my configuration? > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [July 23, 2013, 4:25pm UTC](https://discuss.elastic.co/t/how-to-sort-norwegian-special-characters-with-the-icu-plugin/12901/3 "2013-07-23T16:25:10Z")
They are valid, 'nb' is Bokmål, and 'nn' is Nynorsk, both are official norwegian languages. You should try language code "nb" instead of "norwegian" in the Java example. Jörg On Tue, Jul 23, 2013 at 3:57 PM, Lukáš Vlček [lukas.vlcek@gmail.com](mailto:lukas.vlcek@gmail.com) wrote: > Hi, > > just quickly looked at your code and I am not sure you are providing a > valid language code. As far as I understand you need to provide code > according to [ISO 639-2 Language Code List - Codes for the representation of names of languages (Library of Congress)](http://www.loc.gov/standards/iso639-2/php/code_list.php) (for norwegian > there are couple of options: no, nb, nn. I dunno what is he difference.) > Can you try it with "no" for example? > > Also note that it might be more helpful if you can provide examples using > curl (see [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/help/) for details). > > Regards, > Lukas > > On Tue, Jul 23, 2013 at 3:36 PM, Olav Grønås Gjerde [olavgg@gmail.com](mailto:olavgg@gmail.com)wrote: > > > I have a problem sorting my searches correctly by using norwegian > > collation. > > > > I've installed the > > [GitHub - elastic/elasticsearch-analysis-icu: ICU Analysis plugin for Elasticsearch](https://github.com/elasticsearch/elasticsearch-analysis-icu) plugin, and > > I've created my index with the following properties. > > > > Java: > > .startObject("analysis") > > .startObject("analyzer") > > .startObject("collation") > > .field("tokenizer", "keyword") > > .field("filter", "norwegianCollator") > > .endObject() > > .endObject() > > .startObject("filter") > > .startObject("norwegianCollator") > > .field("type", "icu\_collation") > > .field("language", "norwegian") > > .endObject() > > .endObject() > > .endObject() > > > > JSON: > > { > > "index": { > > "analysis": { > > "analyzer": { > > "collation": { > > "tokenizer": "keyword", > > "filter": "norwegianCollator" > > } > > }, > > "filter": { > > "norwegianCollator": { > > "type": "icu\_collation", > > "language": "nb" > > } > > } > > } > > } > > } > > > > Elasticsearch Head show this as: > > > > settings: { > > > > - index.analysis.analyzer.default.filter: norwegianCollator > > - index.analysis.filter.norwegianCollator.type: icu\_collation > > - index.analysis.analyzer.collation.tokenizer: keyword > > - index.analysis.filter.norwegianCollator.language: norwegian > > > > } > > > > Is this correctly configured? > > > > For search I'm doing this: > > > > ``` > > QueryBuilder qb = QueryBuilders.matchQuery( > > "customer.partner", > > "1" > > ); > > SearchResponse response = > > this.esClient.prepareSearch(this.client.getName()) > > .setTypes("customer") > > .setSearchType(SearchType.DFS_QUERY_THEN_FETCH) > > .setQuery(qb) > > .setFrom(0).setSize(size) > > .addSort("name", SortOrder.DESC) > > .execute() > > .actionGet(); > > > > ``` > > > > But the result I get is: > > > > DEBUG 15:21:00 search.ESSearch - {enabled=true, name=ølen fiskelag, > > partner=1} > > DEBUG 15:21:00 search.ESSearch - {enabled=true, name=æ vil ha dæ, > > partner=1} > > DEBUG 15:21:00 search.ESSearch - {enabled=true, name=åges plateselskap > > as, partner=1} > > DEBUG 15:21:00 search.ESSearch - {enabled=true, name=yara international, > > partner=1} > > > > When it should be: > > > > DEBUG 15:21:00 search.ESSearch - {enabled=true, name=åges plateselskap > > as, partner=1} > > DEBUG 15:21:00 search.ESSearch - {enabled=true, name=ølen fiskelag, > > partner=1} > > DEBUG 15:21:00 search.ESSearch - {enabled=true, name=æ vil ha dæ, > > partner=1} > > DEBUG 15:21:00 search.ESSearch - {enabled=true, name=yara international, > > partner=1} > > > > What am I doing wrong? What could be missing in my configuration? > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Olav\_Gronas\_Gjerde](https://avatars.discourse-cdn.com/v4/letter/o/e19b73/32.png) [@Olav\_Gronas\_Gjerde](https://discuss.elastic.co/u/Olav_Gronas_Gjerde)\ **Post date:** [July 23, 2013, 8:29pm UTC](https://discuss.elastic.co/t/how-to-sort-norwegian-special-characters-with-the-icu-plugin/12901/4 "2013-07-23T20:29:10Z")
I've tried nb, no and nn, sorting of the æ, ø ,å letters are still in wrong order. kl. 18:25:10 UTC+2 tirsdag 23. juli 2013 skrev Jörg Prante følgende: > They are valid, 'nb' is Bokmål, and 'nn' is Nynorsk, both are official > norwegian languages. > > You should try language code "nb" instead of "norwegian" in the Java > example. > > Jörg > > On Tue, Jul 23, 2013 at 3:57 PM, Lukáš Vlček \<[lukas...@gmail.com](mailto:lukas...@gmail.com)\ > > > wrote: > > > Hi, > > > > just quickly looked at your code and I am not sure you are providing a > > valid language code. As far as I understand you need to provide code > > according to [ISO 639-2 Language Code List - Codes for the representation of names of languages (Library of Congress)](http://www.loc.gov/standards/iso639-2/php/code_list.php)(for norwegian > > there are couple of options: no, nb, nn. I dunno what is he difference.) > > Can you try it with "no" for example? > > > > Also note that it might be more helpful if you can provide examples using > > curl (see [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/help/) for details). > > > > Regards, > > Lukas > > > > On Tue, Jul 23, 2013 at 3:36 PM, Olav Grønås Gjerde \<[ola...@gmail.com](mailto:ola...@gmail.com)\ > > > > > wrote: > > > > > I have a problem sorting my searches correctly by using norwegian > > > collation. > > > > > > I've installed the > > > [GitHub - elastic/elasticsearch-analysis-icu: ICU Analysis plugin for Elasticsearch](https://github.com/elasticsearch/elasticsearch-analysis-icu) plugin, and > > > I've created my index with the following properties. > > > > > > Java: > > > .startObject("analysis") > > > .startObject("analyzer") > > > .startObject("collation") > > > .field("tokenizer", "keyword") > > > .field("filter", "norwegianCollator") > > > .endObject() > > > .endObject() > > > .startObject("filter") > > > .startObject("norwegianCollator") > > > .field("type", "icu\_collation") > > > .field("language", "norwegian") > > > .endObject() > > > .endObject() > > > .endObject() > > > > > > JSON: > > > { > > > "index": { > > > "analysis": { > > > "analyzer": { > > > "collation": { > > > "tokenizer": "keyword", > > > "filter": "norwegianCollator" > > > } > > > }, > > > "filter": { > > > "norwegianCollator": { > > > "type": "icu\_collation", > > > "language": "nb" > > > } > > > } > > > } > > > } > > > } > > > > > > Elasticsearch Head show this as: > > > > > > settings: { > > > > > > - index.analysis.analyzer.default.filter: norwegianCollator > > > - index.analysis.filter.norwegianCollator.type: icu\_collation > > > - index.analysis.analyzer.collation.tokenizer: keyword > > > - index.analysis.filter.norwegianCollator.language: norwegian > > > > > > } > > > > > > Is this correctly configured? > > > > > > For search I'm doing this: > > > > > > ``` > > > QueryBuilder qb = QueryBuilders.matchQuery( > > > "customer.partner", > > > "1" > > > ); > > > SearchResponse response = > > > this.esClient.prepareSearch(this.client.getName()) > > > .setTypes("customer") > > > .setSearchType(SearchType.DFS_QUERY_THEN_FETCH) > > > .setQuery(qb) > > > .setFrom(0).setSize(size) > > > .addSort("name", SortOrder.DESC) > > > .execute() > > > .actionGet(); > > > > > > ``` > > > > > > But the result I get is: > > > > > > DEBUG 15:21:00 search.ESSearch - {enabled=true, name=ølen fiskelag, > > > partner=1} > > > DEBUG 15:21:00 search.ESSearch - {enabled=true, name=æ vil ha dæ, > > > partner=1} > > > DEBUG 15:21:00 search.ESSearch - {enabled=true, name=åges plateselskap > > > as, partner=1} > > > DEBUG 15:21:00 search.ESSearch - {enabled=true, name=yara international, > > > partner=1} > > > > > > When it should be: > > > > > > DEBUG 15:21:00 search.ESSearch - {enabled=true, name=åges plateselskap > > > as, partner=1} > > > DEBUG 15:21:00 search.ESSearch - {enabled=true, name=ølen fiskelag, > > > partner=1} > > > DEBUG 15:21:00 search.ESSearch - {enabled=true, name=æ vil ha dæ, > > > partner=1} > > > DEBUG 15:21:00 search.ESSearch - {enabled=true, name=yara international, > > > partner=1} > > > > > > ## What am I doing wrong? What could be missing in my configuration? > > > > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [July 24, 2013, 7:23am UTC](https://discuss.elastic.co/t/how-to-sort-norwegian-special-characters-with-the-icu-plugin/12901/5 "2013-07-24T07:23:49Z")
It works here, see this curl example [Norwegian Bokmål sort with Elasticsearch · GitHub](https://gist.github.com/jprante/6068637) Is the issue your Java code? Jörg On Tue, Jul 23, 2013 at 10:29 PM, Olav Grønås Gjerde [olavgg@gmail.com](mailto:olavgg@gmail.com)wrote: > I've tried nb, no and nn, sorting of the æ, ø ,å letters are still in > wrong order. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Olav\_Gronas\_Gjerde](https://avatars.discourse-cdn.com/v4/letter/o/e19b73/32.png) [@Olav\_Gronas\_Gjerde](https://discuss.elastic.co/u/Olav_Gronas_Gjerde)\ **Post date:** [July 24, 2013, 8:46am UTC](https://discuss.elastic.co/t/how-to-sort-norwegian-special-characters-with-the-icu-plugin/12901/6 "2013-07-24T08:46:48Z")
Thank you! That full example solved it! I had not added the "analyzer" : "bokmalAnalyzer" part to my index property mapping. However that wasn't the only thing, I had to remove the .field("index", "not\_analyzed") property setting 😉 But now it works exactly as it should! Again, thank you for your help! 🙂 kl. 09:23:49 UTC+2 onsdag 24. juli 2013 skrev Jörg Prante følgende: > It works here, see this curl example > [Norwegian Bokmål sort with Elasticsearch · GitHub](https://gist.github.com/jprante/6068637) > > Is the issue your Java code? > > Jörg > > On Tue, Jul 23, 2013 at 10:29 PM, Olav Grønås Gjerde \<[ola...@gmail.com](mailto:ola...@gmail.com)\ > > > wrote: > > > I've tried nb, no and nn, sorting of the æ, ø ,å letters are still in > > wrong order. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:24am UTC](https://discuss.elastic.co/t/how-to-sort-norwegian-special-characters-with-the-icu-plugin/12901/7 "2017-07-06T02:24:49Z")