# Performance searching single index vs multiple indices **URL:** \ **Category:** Elasticsearch\ **Created:** [June 29, 2018, 9:38am UTC](https://discuss.elastic.co/t/performance-searching-single-index-vs-multiple-indices/137926 "2018-06-29T09:38:39Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![tjliu](https://avatars.discourse-cdn.com/v4/letter/t/d9b06d/32.png) [@tjliu](https://discuss.elastic.co/u/tjliu)\ **Post date:** [June 29, 2018, 9:38am UTC](https://discuss.elastic.co/t/performance-searching-single-index-vs-multiple-indices/137926/1 "2018-06-29T09:38:39Z")
Hi All, How to understand this tip? From [The Definitive Guide](https://www.elastic.co/guide/en/elasticsearch/guide/master/multiple-indices.html) > Searching 1 index of 50 shards is exactly equivalent to searching 50 indices with 1 shard each: both search requests hit 50 shards. I have 2048G of Data and 8 machines. From this tip, I can create **1 index with 70 shards** or **10 indexes with 7 shards each index**. Both are 70 shards. If I have 2 search implementation. 1) For 1 index, I use one thread to search. 2) For 10 indexes, I use 10 threads, each thread search one index. Which implementation has better performance? or they're exactly same? Thanks. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [June 29, 2018, 9:44am UTC](https://discuss.elastic.co/t/performance-searching-single-index-vs-multiple-indices/137926/2 "2018-06-29T09:44:52Z")
In both cases you are hitting 70 shards, which will be processed in parallel on different threads, so they should basically be the same. ---
**Author:** ![tjliu](https://avatars.discourse-cdn.com/v4/letter/t/d9b06d/32.png) [@tjliu](https://discuss.elastic.co/u/tjliu)\ **Post date:** [June 29, 2018, 10:23am UTC](https://discuss.elastic.co/t/performance-searching-single-index-vs-multiple-indices/137926/3 "2018-06-29T10:23:26Z")
Thanks for your reply. Another question: I have 8 machines for 2048G data. Which design is better? **1) 8 shards with each 256G**, so each machine has one shard; **2) 70 shards**, so each machine has about 8 shards. I know there are 2 best practices: keep shard at 30G and one machine has one shard. In my case, I need the best performance. Which one is better? Thanks. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [June 29, 2018, 10:27am UTC](https://discuss.elastic.co/t/performance-searching-single-index-vs-multiple-indices/137926/4 "2018-06-29T10:27:47Z")
Are you optimising for query latency or query throughput or a combination of the two? The optimal shard count will depend on your use case, hardware, data and queries, so I would recommend you run a benchmark to find out. ---
**Author:** ![tjliu](https://avatars.discourse-cdn.com/v4/letter/t/d9b06d/32.png) [@tjliu](https://discuss.elastic.co/u/tjliu)\ **Post date:** [June 29, 2018, 10:31am UTC](https://discuss.elastic.co/t/performance-searching-single-index-vs-multiple-indices/137926/5 "2018-06-29T10:31:34Z")
I'm optimizing for query latency. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [June 29, 2018, 10:38am UTC](https://discuss.elastic.co/t/performance-searching-single-index-vs-multiple-indices/137926/6 "2018-06-29T10:38:53Z")
Do you have a specific number of concurrent queries you are targeting? ---
**Author:** ![tjliu](https://avatars.discourse-cdn.com/v4/letter/t/d9b06d/32.png) [@tjliu](https://discuss.elastic.co/u/tjliu)\ **Post date:** [June 29, 2018, 10:42am UTC](https://discuss.elastic.co/t/performance-searching-single-index-vs-multiple-indices/137926/7 "2018-06-29T10:42:29Z")
Concurrent queries might be 100. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [June 29, 2018, 10:50am UTC](https://discuss.elastic.co/t/performance-searching-single-index-vs-multiple-indices/137926/8 "2018-06-29T10:50:47Z")
It is hard to reason about, so I am still recommending that you benchmarking it. Have a look at these talks for some guidance: [https://www.elastic.co/elasticon/conf/2016/sf/quantitative-cluster-sizing](https://www.elastic.co/elasticon/conf/2016/sf/quantitative-cluster-sizing) [https://www.elastic.co/webinars/using-rally-to-get-your-elasticsearch-cluster-size-right](https://www.elastic.co/webinars/using-rally-to-get-your-elasticsearch-cluster-size-right) [https://www.elastic.co/elasticon/conf/2018/sf/the-seven-deadly-sins-of-elasticsearch-benchmarking](https://www.elastic.co/elasticon/conf/2018/sf/the-seven-deadly-sins-of-elasticsearch-benchmarking) ---
**Author:** ![tjliu](https://avatars.discourse-cdn.com/v4/letter/t/d9b06d/32.png) [@tjliu](https://discuss.elastic.co/u/tjliu)\ **Post date:** [June 29, 2018, 10:53am UTC](https://discuss.elastic.co/t/performance-searching-single-index-vs-multiple-indices/137926/9 "2018-06-29T10:53:32Z")
OK. Thanks very much for your reply @Christian_Dahlqvist ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 27, 2018, 10:53am UTC](https://discuss.elastic.co/t/performance-searching-single-index-vs-multiple-indices/137926/10 "2018-07-27T10:53:43Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.