I'm writing an analytics system based on elasticsearch 2.3 and I'm trying to decide which approach to use: one index for all documents OR multi indexes (index per month).
Since I'm doing aggregations on date range (max 6 months back) - I'm afraid It might have performance issues with cross indexes aggregations... I couldn't find an answer on the internet, so I decided to ask here.
I'm expecting to have about 3M documents per month at the beginning.
and I assume that most aggregations will have a date range of 2 months (will have to access 2 indexes in case of multi-index solution).
So, what would you do?
- One index of ~90M documents + add deletion plugin for cleanups
- Creating 12 indexes (easy monthly documents cleanup)
Thanks in advance