Hi Team, i am having 1TB of data over the span of 1 month where we create daily indices and currently we are creating around 20 daily indices with total data of 30-35 GB per day . Field-stats api is taking 50 secs to get the 1000 shards for the period of 1 month and our backend store is Network Storage . Can you please let us know if field stats will take 50 secs in this scenario from Kibana.
Our Elasticsearch version is 2.1 and kibana is 4.3.0
Sounds like you have too many shards, that won't be helping.
Hi we are creating index per app and we have multiple apps. So instead of creating 20 primary shards per day , do you suggest us to consolidate the shards and use shard routing to route the small apps logs to 1 shard?
Just create daily/weekly indices with a lot less shards
Also if 1 of my app has a need of 1 TB data per day , in that case i will have to create more number of shards, In that case i would not have solution to reduce number of shards . Please suggest.
I missed that you have network storage, that will be slow.
You're best option is to remove the network storage and add more nodes.
Thanks for providing inputs . Can we achieve some sort of improvement by moving from NAS to SAN . Currently network storage is only option.
Maybe, you'd have to test.
We don't usually recommend network based storage.