We are building our search platform which will support search service (indexing and searching) for multiple sites (might be 100+ sites) with multi-language (about 50 languages) support.
Some basic requirements:
- All sites will share some common index fields, but allow some site specific fields.
- We may show mixed search results from multiple sites
- Anytime when a new site added, the index of existing sites should not be updated.
Here some some options so far we are considering.
Build different index for different site
- Hard to control the relevance of cross sites search results, since different sites are in different index.
- hard to maintain, 100+ sites will have 100+ indices, seems too many
- Too many indices might have performance issue
Build one big index shared by all sites, but different site has different type under the index
1) Anytime when a site need update its site specific mapping, NO efficient way to only re-index docs for that type, In ES 1.x, it is possible to delete the old type and add a new type, but in ES 2.x, it is not able to delete the type anymore, so if one type need update the mapping, seems the only way is to re-create the whole index and re-index.
Both 1 & 2 need build language specific multi-fields for each language for a fields need language analyzer, for example, for "title" fields, we need define "title_en", "title_de", "title_fr".... etc.
None of above two options is perfect.
Does anyone has better solutions?