# Windows Elasticsearch cluster performance tuning **URL:** \ **Category:** Elasticsearch\ **Created:** [March 23, 2014, 5:08pm UTC](https://discuss.elastic.co/t/windows-elasticsearch-cluster-performance-tuning/16550 "2014-03-23T17:08:46Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Eric\_Brandes](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eric_brandes/32/1480_2.png) [@Eric\_Brandes](https://discuss.elastic.co/u/Eric_Brandes)\ **Post date:** [March 23, 2014, 5:08pm UTC](https://discuss.elastic.co/t/windows-elasticsearch-cluster-performance-tuning/16550/1 "2014-03-23T17:08:46Z")
Hey all, I have a 3 node Elasticsearch 1.0.1 cluster running on Windows Server 2012 (in Azure). There's about 20 million documents that take up a total of 40GB (including replicas). There's about 400 indexes in total, with some having millions of documents and some having just a few. Each index is set to have 3 shards and 1 replica. The main cluster is running on three 4 core machines with 7GB of ram. The min/max JVM heap size is set to 4GB. The primary use case for this cluster is faceting/aggregations over the documents. There's almost no full text searching, so everything is pretty much based on exact values (which are stored but not analyzed at index time) When doing some term facets on a few of these indexes (the biggest one contains about 8 million documents) I'm seeing really long response times (\> 5 sec). There are potentially thousands of distinct values for the term I'm faceting on, but I would have still expected faster performance. So my goal is to speed up these queries to get the responses sub second if possible. To that end I had some questions: 1. Would switching to Linux give me better performance in general? 2. I could collapse almost all of these 400 indexes in to a single big index and use aliases + filters instead. Would this be advisable? 3. Would mucking with the field data cache yield any better results? If I can add any more data to this discussion please let me know! Thanks! Eric -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/eb5fb6bf-be2c-4d5f-b73a-edc1ef5813f1%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/eb5fb6bf-be2c-4d5f-b73a-edc1ef5813f1%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [March 23, 2014, 6:54pm UTC](https://discuss.elastic.co/t/windows-elasticsearch-cluster-performance-tuning/16550/2 "2014-03-23T18:54:49Z")
IMHO 800 shards per node is far too much. And with only 4gb of memory... I guess you have lot of GC or you forget to disable SWAP. My 2 cents. -- David 😉 Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > Le 23 mars 2014 à 18:08, Eric Brandes [eric.brandes@gmail.com](mailto:eric.brandes@gmail.com) a écrit : > > Hey all, I have a 3 node Elasticsearch 1.0.1 cluster running on Windows Server 2012 (in Azure). There's about 20 million documents that take up a total of 40GB (including replicas). There's about 400 indexes in total, with some having millions of documents and some having just a few. Each index is set to have 3 shards and 1 replica. The main cluster is running on three 4 core machines with 7GB of ram. The min/max JVM heap size is set to 4GB. > > The primary use case for this cluster is faceting/aggregations over the documents. There's almost no full text searching, so everything is pretty much based on exact values (which are stored but not analyzed at index time) > > When doing some term facets on a few of these indexes (the biggest one contains about 8 million documents) I'm seeing really long response times (\> 5 sec). There are potentially thousands of distinct values for the term I'm faceting on, but I would have still expected faster performance. > > So my goal is to speed up these queries to get the responses sub second if possible. To that end I had some questions: > > 1. Would switching to Linux give me better performance in general? > 2. I could collapse almost all of these 400 indexes in to a single big index and use aliases + filters instead. Would this be advisable? > 3. Would mucking with the field data cache yield any better results? > > If I can add any more data to this discussion please let me know! > Thanks! > Eric > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/eb5fb6bf-be2c-4d5f-b73a-edc1ef5813f1%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/eb5fb6bf-be2c-4d5f-b73a-edc1ef5813f1%40googlegroups.com). > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/C6157A06-390B-45C0-8425-3723F37D3766%40pilato.fr](https://groups.google.com/d/msgid/elasticsearch/C6157A06-390B-45C0-8425-3723F37D3766%40pilato.fr). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [March 23, 2014, 7:00pm UTC](https://discuss.elastic.co/t/windows-elasticsearch-cluster-performance-tuning/16550/3 "2014-03-23T19:00:18Z")
Forget to say that you should extra large instances and not large. With larges, you could suffer from noisy neighbors. -- David 😉 Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > Le 23 mars 2014 à 19:54, David Pilato [david@pilato.fr](mailto:david@pilato.fr) a écrit : > > IMHO 800 shards per node is far too much. And with only 4gb of memory... > > I guess you have lot of GC or you forget to disable SWAP. > > My 2 cents. > > -- > David 😉 > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > > Le 23 mars 2014 à 18:08, Eric Brandes [eric.brandes@gmail.com](mailto:eric.brandes@gmail.com) a écrit : > > > > Hey all, I have a 3 node Elasticsearch 1.0.1 cluster running on Windows Server 2012 (in Azure). There's about 20 million documents that take up a total of 40GB (including replicas). There's about 400 indexes in total, with some having millions of documents and some having just a few. Each index is set to have 3 shards and 1 replica. The main cluster is running on three 4 core machines with 7GB of ram. The min/max JVM heap size is set to 4GB. > > > > The primary use case for this cluster is faceting/aggregations over the documents. There's almost no full text searching, so everything is pretty much based on exact values (which are stored but not analyzed at index time) > > > > When doing some term facets on a few of these indexes (the biggest one contains about 8 million documents) I'm seeing really long response times (\> 5 sec). There are potentially thousands of distinct values for the term I'm faceting on, but I would have still expected faster performance. > > > > So my goal is to speed up these queries to get the responses sub second if possible. To that end I had some questions: > > > > 1. Would switching to Linux give me better performance in general? > > 2. I could collapse almost all of these 400 indexes in to a single big index and use aliases + filters instead. Would this be advisable? > > 3. Would mucking with the field data cache yield any better results? > > > > If I can add any more data to this discussion please let me know! > > Thanks! > > Eric > > > > -- > > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/eb5fb6bf-be2c-4d5f-b73a-edc1ef5813f1%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/eb5fb6bf-be2c-4d5f-b73a-edc1ef5813f1%40googlegroups.com). > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/C6157A06-390B-45C0-8425-3723F37D3766%40pilato.fr](https://groups.google.com/d/msgid/elasticsearch/C6157A06-390B-45C0-8425-3723F37D3766%40pilato.fr). > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/0AB6FD4E-0C22-4256-8E29-2F40C06E793E%40pilato.fr](https://groups.google.com/d/msgid/elasticsearch/0AB6FD4E-0C22-4256-8E29-2F40C06E793E%40pilato.fr). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Eric\_Brandes](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eric_brandes/32/1480_2.png) [@Eric\_Brandes](https://discuss.elastic.co/u/Eric_Brandes)\ **Post date:** [March 23, 2014, 7:11pm UTC](https://discuss.elastic.co/t/windows-elasticsearch-cluster-performance-tuning/16550/4 "2014-03-23T19:11:36Z")
Interesting - so in general would you recommend consolidating all 400 indexes in to a single index and using aliases/filters to address them? (they're currently broken out by user, and all operations are scoped to a specific user) If I were to consolidate to a single index, how many shards would be recommended? On Sunday, March 23, 2014 2:00:18 PM UTC-5, David Pilato wrote: > Forget to say that you should extra large instances and not large. > With larges, you could suffer from noisy neighbors. > > -- > David 😉 > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > Le 23 mars 2014 à 19:54, David Pilato \<[da...@pilato.fr](mailto:da...@pilato.fr) \\> a > écrit : > > IMHO 800 shards per node is far too much. And with only 4gb of memory... > > I guess you have lot of GC or you forget to disable SWAP. > > My 2 cents. > > -- > David 😉 > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > Le 23 mars 2014 à 18:08, Eric Brandes \<[eric.b...@gmail.com](mailto:eric.b...@gmail.com) \\> > a écrit : > > Hey all, I have a 3 node Elasticsearch 1.0.1 cluster running on Windows > Server 2012 (in Azure). There's about 20 million documents that take up a > total of 40GB (including replicas). There's about 400 indexes in total, > with some having millions of documents and some having just a few. Each > index is set to have 3 shards and 1 replica. The main cluster is running > on three 4 core machines with 7GB of ram. The min/max JVM heap size is > set to 4GB. > > The primary use case for this cluster is faceting/aggregations over the > documents. There's almost no full text searching, so everything is pretty > much based on exact values (which are stored but not analyzed at index time) > > When doing some term facets on a few of these indexes (the biggest one > contains about 8 million documents) I'm seeing really long response times > (\> 5 sec). There are potentially thousands of distinct values for the term > I'm faceting on, but I would have still expected faster performance. > > So my goal is to speed up these queries to get the responses sub second if > possible. To that end I had some questions: > > 1. Would switching to Linux give me better performance in general? > 2. I could collapse almost all of these 400 indexes in to a single big > index and use aliases + filters instead. Would this be advisable? > 3. Would mucking with the field data cache yield any better results? > > If I can add any more data to this discussion please let me know! > Thanks! > Eric > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/eb5fb6bf-be2c-4d5f-b73a-edc1ef5813f1%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/eb5fb6bf-be2c-4d5f-b73a-edc1ef5813f1%40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/eb5fb6bf-be2c-4d5f-b73a-edc1ef5813f1%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/eb5fb6bf-be2c-4d5f-b73a-edc1ef5813f1%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/C6157A06-390B-45C0-8425-3723F37D3766%40pilato.fr](https://groups.google.com/d/msgid/elasticsearch/C6157A06-390B-45C0-8425-3723F37D3766%40pilato.fr)[https://groups.google.com/d/msgid/elasticsearch/C6157A06-390B-45C0-8425-3723F37D3766%40pilato.fr?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/C6157A06-390B-45C0-8425-3723F37D3766%40pilato.fr?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/04ea8acb-a62f-4232-a483-bdde916c48c2%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/04ea8acb-a62f-4232-a483-bdde916c48c2%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [March 23, 2014, 7:13pm UTC](https://discuss.elastic.co/t/windows-elasticsearch-cluster-performance-tuning/16550/5 "2014-03-23T19:13:49Z")
Yes. Hard to say. You need to test how much docs a single shard could hold. It depends on your use case. -- David 😉 Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > Le 23 mars 2014 à 20:11, Eric Brandes [eric.brandes@gmail.com](mailto:eric.brandes@gmail.com) a écrit : > > Interesting - so in general would you recommend consolidating all 400 indexes in to a single index and using aliases/filters to address them? (they're currently broken out by user, and all operations are scoped to a specific user) > > If I were to consolidate to a single index, how many shards would be recommended? > > > On Sunday, March 23, 2014 2:00:18 PM UTC-5, David Pilato wrote: > > Forget to say that you should extra large instances and not large. > > With larges, you could suffer from noisy neighbors. > > > > -- > > David 😉 > > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > > > > Le 23 mars 2014 à 19:54, David Pilato [da...@pilato.fr](mailto:da...@pilato.fr) a écrit : > > > > > > IMHO 800 shards per node is far too much. And with only 4gb of memory... > > > > > > I guess you have lot of GC or you forget to disable SWAP. > > > > > > My 2 cents. > > > > > > -- > > > David 😉 > > > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > > > > > > Le 23 mars 2014 à 18:08, Eric Brandes [eric.b...@gmail.com](mailto:eric.b...@gmail.com) a écrit : > > > > > > > > Hey all, I have a 3 node Elasticsearch 1.0.1 cluster running on Windows Server 2012 (in Azure). There's about 20 million documents that take up a total of 40GB (including replicas). There's about 400 indexes in total, with some having millions of documents and some having just a few. Each index is set to have 3 shards and 1 replica. The main cluster is running on three 4 core machines with 7GB of ram. The min/max JVM heap size is set to 4GB. > > > > > > > > The primary use case for this cluster is faceting/aggregations over the documents. There's almost no full text searching, so everything is pretty much based on exact values (which are stored but not analyzed at index time) > > > > > > > > When doing some term facets on a few of these indexes (the biggest one contains about 8 million documents) I'm seeing really long response times (\> 5 sec). There are potentially thousands of distinct values for the term I'm faceting on, but I would have still expected faster performance. > > > > > > > > So my goal is to speed up these queries to get the responses sub second if possible. To that end I had some questions: > > > > > > > > 1. Would switching to Linux give me better performance in general? > > > > 2. I could collapse almost all of these 400 indexes in to a single big index and use aliases + filters instead. Would this be advisable? > > > > 3. Would mucking with the field data cache yield any better results? > > > > > > > > If I can add any more data to this discussion please let me know! > > > > Thanks! > > > > Eric > > > > > > > > -- > > > > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/eb5fb6bf-be2c-4d5f-b73a-edc1ef5813f1%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/eb5fb6bf-be2c-4d5f-b73a-edc1ef5813f1%40googlegroups.com). > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > > > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/C6157A06-390B-45C0-8425-3723F37D3766%40pilato.fr](https://groups.google.com/d/msgid/elasticsearch/C6157A06-390B-45C0-8425-3723F37D3766%40pilato.fr). > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/04ea8acb-a62f-4232-a483-bdde916c48c2%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/04ea8acb-a62f-4232-a483-bdde916c48c2%40googlegroups.com). > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/B736C9EB-1617-483C-8900-A5F7CDDAD8DA%40pilato.fr](https://groups.google.com/d/msgid/elasticsearch/B736C9EB-1617-483C-8900-A5F7CDDAD8DA%40pilato.fr). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:41am UTC](https://discuss.elastic.co/t/windows-elasticsearch-cluster-performance-tuning/16550/6 "2017-07-06T01:41:17Z")