One index or many indecies for documents in diffrent groups?

I think this question is best described by using an example.
Say I have one index called "myindex". The fields in this index have the following structure:

  • Each document in this index contains a field called "group" which has a value between 1 and 100.
  • Each document in the index has 20 fields which have the same name and datatype across all documents.
  • Each document in group x has 5-20 fields which are only used by group x.

Should all these documents be stored in one index or should I make one index per group?

Some of the searches and visualisations I use in kibana will be across many groups. This means that if I split the data into one index per group there will be searches and aggregations done over many elasticsearch-indecies. Will this impact performance?

If I keep everything in one index, it will contain hundreds of fields, but only 25-40 fields will be "used" in each document. How will this impact performance?

Are there any other ways of structuring this, other than the two options I have considered in this post?

Thanks.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.