# Dealing with languages **URL:** \ **Category:** Elasticsearch\ **Created:** [October 21, 2010, 1:08pm UTC](https://discuss.elastic.co/t/dealing-with-languages/3464 "2010-10-21T13:08:52Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![James\_Cook](https://avatars.discourse-cdn.com/v4/letter/j/898d66/32.png) [@James\_Cook](https://discuss.elastic.co/u/James_Cook)\ **Post date:** [October 21, 2010, 1:08pm UTC](https://discuss.elastic.co/t/dealing-with-languages/3464/1 "2010-10-21T13:08:52Z")
I am indexing documents where some are English and others are in Arabic. There is a locale property on each document to indicate its language. What strategies exist to choose the correct analyzer to use on a field in this document? Can I somehow apply a language specific analyzer to say the description field based on the value of the locale field? Or, is there some other approach? ---
**Author:** ![rmuir](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rmuir/32/44949_2.png) [@rmuir](https://discuss.elastic.co/u/rmuir)\ **Post date:** [October 21, 2010, 1:15pm UTC](https://discuss.elastic.co/t/dealing-with-languages/3464/2 "2010-10-21T13:15:43Z")
On Thu, Oct 21, 2010 at 9:08 AM, James Cook [jcook@tracermedia.com](mailto:jcook@tracermedia.com) wrote: > I am indexing documents where some are English and others are in Arabic. > There is a locale property on each document to indicate its language. What > strategies exist to choose the correct analyzer to use on a field in this > document? Can I somehow apply a language specific analyzer to say the > description field based on the value of the locale field? Or, is there some > other approach? If you have mixed english/arabic, why not just make a combined analyzer (e.g. one that uses both PorterStemFilter and ArabicStemFilter/ArabicNormalizationFilter). Because they don't share any characters in unicode, the ArabicStemFilter will only work on arabic text, and won't mess with the english, and the PorterStemFilter will only work on latin text, and won't mess with your arabic. Same with the stopwords, etc. So i don't think you need to use your locale field at all in this case... ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [October 21, 2010, 1:21pm UTC](https://discuss.elastic.co/t/dealing-with-languages/3464/3 "2010-10-21T13:21:59Z")
On Thu, 2010-10-21 at 09:08 -0400, James Cook wrote: > I am indexing documents where some are English and others are in > Arabic. There is a locale property on each document to indicate its > language. What strategies exist to choose the correct analyzer to use > on a field in this document? Can I somehow apply a language specific > analyzer to say the description field based on the value of the locale > field? Or, is there some other approach? I don't think you can. Don't forget that when you search for those documents, your query string is analysed with the same analyzer - so which should it choose? English or Arabic? Don't know if this is the best solution, but you could either have two different types eg 'english\_doc' or 'arabic\_doc', or two different fields eg 'english\_content', 'arabic\_content' Then when you search, you can search across both types|fields and the correct analyser would be applied. (Not sure how the \_all field works in this case, but if it is clever enough, you should just be able to search against that, and your query string would be analysed differently for each component field) clint ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:17am UTC](https://discuss.elastic.co/t/dealing-with-languages/3464/4 "2017-07-06T04:17:43Z")