Hi We are using ES as our search engine for our Cassandra DB. We are dumping our business model data to both Cassandra and Elastic search. I want to design an index management strategy for this scenario. How would I do it ?
For example: Lets say we User Model data, which persists all the user related (first, last, address, phone number etc)
- Should I actually just create just one index for all the users or Create weekly/monthly indices based on when user gets created in our system ?
- How many shards do I need to allocate if the user data is like 1G.
- Lets say the scenario completely changes and we decide to all more data into ES. And then the data might exponentially grow to 200GB or so. If so, then whats the criterion for allocating the more shards to ES. How do I calculate the shards etc.
4.Since I would not know how my system grows ahead of my time, lets say i make mistake in allocation shards (either too little or too many) Then is there a way to dyanimically shrink or expand them as and when more data is dumped into ES.
Thank you very much for your help..