# Logstash consuming too much bandwidth
**URL:** https://discuss.elastic.co/t/logstash-consuming-too-much-bandwidth/190593
**Category:** Logstash
**Created:** [July 15, 2019, 5:58pm UTC](https://discuss.elastic.co/t/logstash-consuming-too-much-bandwidth/190593 "2019-07-15T17:58:30Z")
**Posts on this page:** 13
**Page:** 1
### Author:  [@Jitendra\_Kumhar](https://discuss.elastic.co/u/Jitendra_Kumhar)
#### Post date: [July 15, 2019, 5:58pm UTC](https://discuss.elastic.co/t/logstash-consuming-too-much-bandwidth/190593/1 "2019-07-15T17:58:30Z")
Hello,
I am sending logs from filebeat to Logstash and then from logstash to ElasticSearch.
The problem is Logstash is consuming too much bandwidth.I have attached network usage graph of logstash server.
(green is incoming from filebeat)
In the starting I am just writing to file on Logstash server instead of sending to ElasticSearch.You can see the incoming rate.Then when I turn on ElasticSearch output.The incoming rate decreases and also outgoing is 10 times more data.
Can anyone help please.What configurations changes or hardware improvements should I make?
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [July 15, 2019, 6:11pm UTC](https://discuss.elastic.co/t/logstash-consuming-too-much-bandwidth/190593/2 "2019-07-15T18:11:07Z")
You may want to enable [http\_compression](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html) for the Elasticsearch output if you have not already.
---
### Author:  [@Jitendra\_Kumhar](https://discuss.elastic.co/u/Jitendra_Kumhar)
#### Post date: [July 15, 2019, 6:36pm UTC](https://discuss.elastic.co/t/logstash-consuming-too-much-bandwidth/190593/3 "2019-07-15T18:36:29Z")
Thank You for the reply
But why is my incoming data rate reduced when elasticsearch output is enabled?
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [July 15, 2019, 6:47pm UTC](https://discuss.elastic.co/t/logstash-consuming-too-much-bandwidth/190593/4 "2019-07-15T18:47:43Z")
Maybe Elasticsearch is not able to keep up, which results in back pressure being applied?
---
### Author:  [@Jitendra\_Kumhar](https://discuss.elastic.co/u/Jitendra_Kumhar)
#### Post date: [July 16, 2019, 7:12pm UTC](https://discuss.elastic.co/t/logstash-consuming-too-much-bandwidth/190593/5 "2019-07-16T19:12:17Z")
Currently I have 1 ingest node,1 master node and 2 data nodes.
I added persistent queues in logstash of 30 gb.They gets filled up quickly.
So the reason is mostly back pressure by ElasticSearch.
Do you have any suggestion how do I handle this?
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [July 16, 2019, 7:14pm UTC](https://discuss.elastic.co/t/logstash-consuming-too-much-bandwidth/190593/6 "2019-07-16T19:14:04Z")
I would recommend adding resources to the Elasticsearch cluster. What is the current hardware specification?
---
### Author:  [@Jitendra\_Kumhar](https://discuss.elastic.co/u/Jitendra_Kumhar)
#### Post date: [July 16, 2019, 7:41pm UTC](https://discuss.elastic.co/t/logstash-consuming-too-much-bandwidth/190593/7 "2019-07-16T19:41:23Z")
**Ingest node:**
> Model name: Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz
> Stepping: 3
> CPU MHz: 3000.000
> BogoMIPS: 6000.00
> Hypervisor vendor: KVM
> Virtualization type: full
> L1d cache: 32K
> L1i cache: 32K
> L2 cache: 1024K
> L3 cache: 25344K
`Ram : 3.549 GB`
`Total disk : 31 GB`
**Data Node(2 x):**
`Ram : 31.548 GB`
`Total disk : 2.3 TB`
**Master Node**
`Ram : 3.549 GB`
`Total disk : 31 GB`
---
### Author:  [@Jitendra\_Kumhar](https://discuss.elastic.co/u/Jitendra_Kumhar)
#### Post date: [July 22, 2019, 6:15pm UTC](https://discuss.elastic.co/t/logstash-consuming-too-much-bandwidth/190593/8 "2019-07-22T18:15:42Z")
Any suggestion?
---
### Author:  [@rugenl](https://discuss.elastic.co/u/rugenl)
#### Post date: [July 22, 2019, 6:20pm UTC](https://discuss.elastic.co/t/logstash-consuming-too-much-bandwidth/190593/9 "2019-07-22T18:20:15Z")
Look at the logs @18:04, when the lines crossed. How many events per second were ingesting before that time?
I think you need to determine if you have an ingest error condition before assuming it's a capacity issue.
---
### Author:  [@Jitendra\_Kumhar](https://discuss.elastic.co/u/Jitendra_Kumhar)
#### Post date: [July 22, 2019, 6:55pm UTC](https://discuss.elastic.co/t/logstash-consuming-too-much-bandwidth/190593/10 "2019-07-22T18:55:28Z")
How do I check how many events ingested at that time ?
---
### Author:  [@rugenl](https://discuss.elastic.co/u/rugenl)
#### Post date: [July 24, 2019, 3:55pm UTC](https://discuss.elastic.co/t/logstash-consuming-too-much-bandwidth/190593/11 "2019-07-24T15:55:08Z")
Kibana monitoring, if you had it enabled.
---
### Author:  [@Jitendra\_Kumhar](https://discuss.elastic.co/u/Jitendra_Kumhar)
#### Post date: [July 29, 2019, 5:17pm UTC](https://discuss.elastic.co/t/logstash-consuming-too-much-bandwidth/190593/12 "2019-07-29T17:17:08Z")
Hey,
I enabled http\_compression on logstash.
Now that machines network monitoring looks like this
The graph is MiB per minute
Green is receiving and yellow is transmitting.I don't understand why it is exactly asymmetric.I have done speed test on this machine and it can give speed upto 500 Mb/s download and 500 Mb/s upload speed.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 26, 2019, 5:17pm UTC](https://discuss.elastic.co/t/logstash-consuming-too-much-bandwidth/190593/13 "2019-08-26T17:17:16Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.