# Fine Tuned Cluster - Consultation **URL:** \ **Category:** Elasticsearch\ **Created:** [June 23, 2017, 12:40pm UTC](https://discuss.elastic.co/t/fine-tuned-cluster-consultation/90638 "2017-06-23T12:40:16Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![panda2004](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/panda2004/32/19473_2.png) [@panda2004](https://discuss.elastic.co/u/panda2004)\ **Post date:** [June 23, 2017, 12:40pm UTC](https://discuss.elastic.co/t/fine-tuned-cluster-consultation/90638/1 "2017-06-23T12:40:16Z")
Hello, Nowadays we're currently building our cluster and play with the different components for best performance. - Current Cluster Nodes: 3 data nodes, each with 16 GB RAM, 6 CPU cores. 3 master nodes, each with 4 GB RAM, 4 CPU cores. - Data: At the beginning we wanted to just make a big single index - but understood quickly it's bad for performance and maintenance. We found a way to make it time based, so we have a weekly index of about 100 GB. It's important to notice that our search granularity is usually 1 month - meaning that the search request involves 4 indices in practice. Refresh interval: 30 seconds. - Throughput: We have a low write throughput (about 50-100 documents per second) written in bulks. We don't really have many search requests per second. Let's say there are 10 users which make 6 concurrent searches once and a while. The search requests are 6 serious aggregation queries which runs in parallel to ElasticSearch. Each query consists of a bool query and a chained nested aggregation, filter aggregation and a terms aggregation sorted by reverse nested aggregation. Right now we don't use multi search API. This is the best practice rules we know: - Shard size should be lower than 50 GB. - In 1 index: #Shards = #Nodes \* #Cores in each node. By the way, the #Shards includes the replica shards or only primary shards? * * * We obviously know we need to strengthen the cluster. We would like to hear about any general recommendation. ---
**Author:** ![timost](https://avatars.discourse-cdn.com/v4/letter/t/e95f7d/32.png) [@timost](https://discuss.elastic.co/u/timost)\ **Post date:** [June 25, 2017, 5:49am UTC](https://discuss.elastic.co/t/fine-tuned-cluster-consultation/90638/2 "2017-06-25T05:49:53Z")
Hi ! There a few things that I used when trying to improve search performance: - read [Tune for search speed](https://www.elastic.co/guide/en/elasticsearch/reference/master/tune-for-search-speed.html) - While running the queries use system monitoring tools such as _htop_, _iotop_ and _nethogs_ to determine what is the main bottleneck. - Use [x-pack's profiler in kibana](https://www.elastic.co/guide/en/x-pack/current/xpack-profiler.html) if you can. - Replicas are used when running the search query so I guess you can count them in your `#Shards = #Nodes * #Cores` rule Hope this helps. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 23, 2017, 5:50am UTC](https://discuss.elastic.co/t/fine-tuned-cluster-consultation/90638/3 "2017-07-23T05:50:12Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.