Hello All,
Which is a better approach: single index or should it to be split into multiple index. Please can you point out the pros and cons of both approaches.
Rgds,
-Akhil
That generally depends on the use-case, so you will need to provide some additional information if you want anything but very general advice.
Say, I have environments- dev, staging and production. Currently all the indexes are going to one index pattern logstash-* on Kibana. It looks that the searches are quite slow. Instead if 3 different index patterns are created like logstashdev-* ,logstashstagging-* and logstashprod-* will it improve the search performance?
Is there any tradeoff if so?
Creating separate indices for different types of environments may be a good idea as they could very well have different retention requirements, which often is managed at the index level. Whether it will help with performance will depend on the reason performance is bad. Do you have slow storage? Do you have too much data for the hardware you have? Is the cluster under heavy load? Do you have a lot of small shards and indices?
There are around 30M documents per day. We have a 7 node cluster on azure- 1 Client, 3 data and 3 master each of 16GB RAM and 2CPUs per node. 3 Data nodes are connected with Azure premium SSD disks. Current retention period is 2 weeks.
Currently all daily micro-service log are going to the same index logstash-. Which means only on index per day. Default settings is 5 Shards and 1 replica.
If the hardware is not sufficient could you please help on how to do a sizing in here.
You probably have too many shards per node.
May I suggest you look at the following resources about sizing:
https://www.elastic.co/elasticon/conf/2016/sf/quantitative-cluster-sizing
And https://www.elastic.co/webinars/using-rally-to-get-your-elasticsearch-cluster-size-right
Thanks 
I will check on these and make the necessary changes.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.