# How to improve the speed of Logstash output to Elasticsearch?
**URL:** https://discuss.elastic.co/t/how-to-improve-the-speed-of-logstash-output-to-elasticsearch/206254
**Category:** Logstash
**Created:** [November 2, 2019, 7:30pm UTC](https://discuss.elastic.co/t/how-to-improve-the-speed-of-logstash-output-to-elasticsearch/206254 "2019-11-02T19:30:48Z")
**Posts on this page:** 9
**Page:** 1
### Author:  [@StruggleYang](https://discuss.elastic.co/u/StruggleYang)
#### Post date: [November 2, 2019, 7:30pm UTC](https://discuss.elastic.co/t/how-to-improve-the-speed-of-logstash-output-to-elasticsearch/206254/1 "2019-11-02T19:30:48Z")
I have a lot of application logs to collect, and data conversion is like, filebeat =\> kafka =\> logstash =\> elasticsearch ,Everything is smooth, But sometimes the application log will suddenly increase, such as when the user volume becomes very large.Kafka produces much faster than logstash,I want to change that but I don't know where to start.
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [November 3, 2019, 8:38am UTC](https://discuss.elastic.co/t/how-to-improve-the-speed-of-logstash-output-to-elasticsearch/206254/2 "2019-11-03T08:38:25Z")
Logstash can only send as fast as Elasticsearch can accept the data. How have you determined that Logstash is the bottleneck and not Elasticsearch?
---
### Author:  [@StruggleYang](https://discuss.elastic.co/u/StruggleYang)
#### Post date: [November 3, 2019, 9:20am UTC](https://discuss.elastic.co/t/how-to-improve-the-speed-of-logstash-output-to-elasticsearch/206254/3 "2019-11-03T09:20:18Z")
I tried to do a stress test on Elasticsearch (use "esrally"), and the result is that the write speed will be twice or more than the logstash write.So I began to wonder if the bottleneck at Logstash affected the write.
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [November 3, 2019, 9:23am UTC](https://discuss.elastic.co/t/how-to-improve-the-speed-of-logstash-output-to-elasticsearch/206254/4 "2019-11-03T09:23:30Z")
Did you test with the type of data you are indexing or with one of the standard tracks? How do you measure indexing throughput?
---
### Author:  [@StruggleYang](https://discuss.elastic.co/u/StruggleYang)
#### Post date: [November 3, 2019, 9:33am UTC](https://discuss.elastic.co/t/how-to-improve-the-speed-of-logstash-output-to-elasticsearch/206254/5 "2019-11-03T09:33:35Z")
1. I think I may not be testing enough, I used a simpler structure, the field will be less than the real, and some of the word-breakout fields are not taken into account. I feel like I need to test it with data close to the real and give you an answer.
2. "esrally" usually gives post-test reports, and kibana's monitoring can be seen
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [November 3, 2019, 9:46am UTC](https://discuss.elastic.co/t/how-to-improve-the-speed-of-logstash-output-to-elasticsearch/206254/6 "2019-11-03T09:46:22Z")
If you use documents per second as a measurement this will vary quite a lot depending on the document size as Elasticsearch need to do more work for larger documents and there is more disk I/O for larger documents too.
---
### Author:  [@StruggleYang](https://discuss.elastic.co/u/StruggleYang)
#### Post date: [November 3, 2019, 9:56am UTC](https://discuss.elastic.co/t/how-to-improve-the-speed-of-logstash-output-to-elasticsearch/206254/7 "2019-11-03T09:56:34Z")
I see what you mean, maybe as you said, my document is sometimes larger, such as the nginx log or java program's log stack, which may affect the larger
---
### Author:  [@StruggleYang](https://discuss.elastic.co/u/StruggleYang)
#### Post date: [November 3, 2019, 10:10am UTC](https://discuss.elastic.co/t/how-to-improve-the-speed-of-logstash-output-to-elasticsearch/206254/8 "2019-11-03T10:10:46Z")
I think I should try to try to test again with near-real data and observe the performance of elasticsearch, and before that, thank you very much for your answer, I may ask you again after the test:100:
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 1, 2019, 10:10am UTC](https://discuss.elastic.co/t/how-to-improve-the-speed-of-logstash-output-to-elasticsearch/206254/9 "2019-12-01T10:10:51Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.