# Possible ES bottleneck in redis-logstash-elasticsearch system **URL:** \ **Category:** Elasticsearch\ **Created:** [August 8, 2013, 3:36pm UTC](https://discuss.elastic.co/t/possible-es-bottleneck-in-redis-logstash-elasticsearch-system/13131 "2013-08-08T15:36:03Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![tom\_rkba](https://avatars.discourse-cdn.com/v4/letter/t/e79b87/32.png) [@tom\_rkba](https://discuss.elastic.co/u/tom_rkba)\ **Post date:** [August 8, 2013, 3:36pm UTC](https://discuss.elastic.co/t/possible-es-bottleneck-in-redis-logstash-elasticsearch-system/13131/1 "2013-08-08T15:36:03Z")
Hi all, I believe ES is bottlenecking. I researched this in logstash-users. I have a redis-logstash-ES cluster with logs pre-indexed by the client. The error I am receiving in logstash is: {:timestamp=\>"2013-08-08T03:43:20.705000+0000", :message=\>"Too many active ES requests, blocking now.", :inflight\_requests=\>50, :max\_inflight\_requests=\>50, :level=\>:info} The thread that referred me to a possible ES bottleneck is: [https://groups.google.com/forum/?fromgroups=#!searchin/logstash-users/too$20many$20es$20requests/logstash-users/zNCSD90auxY/UL0NGm3xX4cJ](https://groups.google.com/forum/?fromgroups=#!searchin/logstash-users/too%2420many%2420es%2420requests/logstash-users/zNCSD90auxY/UL0NGm3xX4cJ) The ES hosts have 128 gb RAM, quad CPUs with 8 cores each and 1.9 TB disk available in RAID 10. ES has 67 GB RAM dedicated to it. Logstash gets 2-10 GB RAM. Disk has 93% available. ES Setup: cluster.name: "vault\_elasticsearch" node.name: "host-02" node.max\_local\_storage\_nodes: 1 index.number\_of\_shards: 3 index.number\_of\_replicas: 2 bootstrap.mlockall: true gateway.recover\_after\_nodes: 2 gateway.recover\_after\_time: 10s gateway.expected\_nodes: 3 discovery.zen.minimum\_master\_nodes: 2 discovery.zen.ping.multicast.enabled: false discovery.zen.ping.unicast.hosts: ["host-01:9300", "host-02:9300", "host-03:9300"] indices.memory.index\_buffer\_size: 50% index.translog.flush\_threshold\_ops: 50000 index.refresh\_interval: 60s index.merge.policy.use\_compound\_file: false I searched around various ES resources to come up with this configuration. I am just learning about ES operations. What are your recommendations to avoid this problem? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![tom\_rkba](https://avatars.discourse-cdn.com/v4/letter/t/e79b87/32.png) [@tom\_rkba](https://discuss.elastic.co/u/tom_rkba)\ **Post date:** [August 8, 2013, 3:41pm UTC](https://discuss.elastic.co/t/possible-es-bottleneck-in-redis-logstash-elasticsearch-system/13131/2 "2013-08-08T15:41:48Z")
ES version is 90.1 On Thursday, August 8, 2013 11:36:03 AM UTC-4, tom rkba wrote: > Hi all, > > I believe ES is bottlenecking. I researched this in logstash-users. I > have a redis-logstash-ES cluster with logs pre-indexed by the client. The > error I am receiving in logstash is: > > {:timestamp=\>"2013-08-08T03:43:20.705000+0000", :message=\>"Too many active > ES requests, blocking now.", :inflight\_requests=\>50, > :max\_inflight\_requests=\>50, :level=\>:info} > > The thread that referred me to a possible ES bottleneck is: > [Redirecting to Google Groups](https://groups.google.com/forum/?fromgroups=#!searchin/logstash-users/too$20many$20es$20requests/logstash-users/zNCSD90auxY/UL0NGm3xX4cJ) > > The ES hosts have 128 gb RAM, quad CPUs with 8 cores each and 1.9 TB disk > available in RAID 10. ES has 67 GB RAM dedicated to it. Logstash gets 2-10 > GB RAM. Disk has 93% available. > > ES Setup: > > cluster.name: "vault\_elasticsearch" > node.name: "host-02" > node.max\_local\_storage\_nodes: 1 > index.number\_of\_shards: 3 > index.number\_of\_replicas: 2 > bootstrap.mlockall: true > gateway.recover\_after\_nodes: 2 > gateway.recover\_after\_time: 10s > gateway.expected\_nodes: 3 > discovery.zen.minimum\_master\_nodes: 2 > discovery.zen.ping.multicast.enabled: false > discovery.zen.ping.unicast.hosts: ["host-01:9300", "host-02:9300", > "host-03:9300"] > indices.memory.index\_buffer\_size: 50% > index.translog.flush\_threshold\_ops: 50000 > index.refresh\_interval: 60s > index.merge.policy.use\_compound\_file: false > > I searched around various ES resources to come up with this > configuration. I am just learning about ES operations. What are your > recommendations to avoid this problem? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:22am UTC](https://discuss.elastic.co/t/possible-es-bottleneck-in-redis-logstash-elasticsearch-system/13131/3 "2017-07-06T02:22:06Z")