One index for each type, or force all documents to fit one type in one index?

[Karolinebryn (Karoline Brynildsen)]

Hi!

I have an index in ES 1.7 and I am working on moving it to ES 5.x. This index has 8 types with very different mappings. Just a few fields are shared across the different types and some of these have different schema on one type then the other. This is no longer allowed in ES 2.x (I'm upgrading to 2.x before moving to 5.x), and taking the issue with sparsity into consideration I understand that each index should only have one type (or several types that has the same mapping).

In our case we have very little data, with all these types chucked into one index the index only contains ca 6000 documents. Splitting this into one index for each type (8 indexes) will cause some of the indexes to have a few hundred documents, and the largest one will be at around 3000 documents. This seems like it will have an impact on document scoring and cutoff_frequency. How great will this impact be? Is this still the "best practice" solution?

What about designing a "general type" and have all the different documents fit into this "general type" before indexing them? I understand that there might still be issues with sparsity, but as stated before; we do not have large amounts of data.

I'm hoping that someone has been through this issue and can provide me with some guidence and suggestions as to how they (would have) solved this problem

[warkolm (Mark Walkom)]

@Karolinebryn

This seems like it will have an impact on document scoring and cutoff_frequency. How great will this impact be? Is this still the "best practice" solution?

Yes. Depends you need to test. Depends on what you need.
If you need to be doing the scoring across all of these docs then you need to find a way to normalise them enough to put them into a single index/shard.