# Best practices for indexing documents with alternate names in many languages

**URL:** <https://discuss.elastic.co/t/best-practices-for-indexing-documents-with-alternate-names-in-many-languages/5338>\
**Category:** Elasticsearch\
**Created:** [September 9, 2011, 2:43pm UTC](https://discuss.elastic.co/t/best-practices-for-indexing-documents-with-alternate-names-in-many-languages/5338 "2011-09-09T14:43:28Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![AEvar\_Arnfjord\_Bjarm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aevar_arnfjord_bjarm/32/2746_2.png) [@AEvar\_Arnfjord\_Bjarm](https://discuss.elastic.co/u/AEvar_Arnfjord_Bjarm)\
**Post date:** [September 9, 2011, 2:43pm UTC](https://discuss.elastic.co/t/best-practices-for-indexing-documents-with-alternate-names-in-many-languages/5338/1 "2011-09-09T14:43:28Z")

</div>

I have a dataset that includes a lot of records that I'm currently  
re-indexing for each language. E.g. each of these will be separate  
documents:

```
{
    "name": "London",
    "lang": "en",
    *data*
}

{
    "name": "Londra",
    "lang": "it",
    *same data*
}

```

I'd like to eliminate this redundancy and have something like:

```
{
    "names": [
        {
            "name": "London",
            "lang": "en",
        },
        {
            "name": "Londra",
            "lang": "it",
        },
    ],
    *data*
}

```

However as I understand it that'll be flattened to this internally:

```
{
    "names": [
        {
            "name": "London Londra",
            "lang": "en it",
        },
    ],
    *data*
}

```

One alternative schema would be:

```
{
    "name_en": "London",
    "name_it": "Londra",
    *same data*
}

```

Or using the the initial suggestion but with the nested type  
([http://www.elasticsearch.org/guide/reference/mapping/nested-type.html](http://www.elasticsearch.org/guide/reference/mapping/nested-type.html)).

What I'm actually trying to accomplish here is:

- Reduce noise in my search queries: Currently if I search for  
e.g. "London" I'll end up getting documents back for all the  
"London"'s that happen to be called that in their native language  
(which is quite a lot).

- Not to have a bias towards entries that happen to have more / fewer  
translations. E.g. if I have 4 translations of London I might end  
up with a string like:

- I'd like to use the language-specific indexing features of Lucene  
to e.g. apply German stemming to the German version of the names,  
and perhaps have a different tokenizer for asian languages.

Maybe there's a much better way to do this that I've missed. But  
hopefully I've given enough information for the list to either  
recommend one of the above, or tell me that I'm being completely stupid  
and recommend something else.

Thanks in advance for any help you can give.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 3:54am UTC](https://discuss.elastic.co/t/best-practices-for-indexing-documents-with-alternate-names-in-many-languages/5338/2 "2017-07-06T03:54:57Z")

</div>