# How can I debug what influence the write-in speed from logstash:input:file to es?
**URL:** https://discuss.elastic.co/t/how-can-i-debug-what-influence-the-write-in-speed-from-logstash-input-file-to-es/67173
**Category:** Logstash
**Created:** [November 25, 2016, 10:18am UTC](https://discuss.elastic.co/t/how-can-i-debug-what-influence-the-write-in-speed-from-logstash-input-file-to-es/67173 "2016-11-25T10:18:50Z")
**Posts on this page:** 9
**Page:** 1
### Author:  [@Morriaty](https://discuss.elastic.co/u/Morriaty)
#### Post date: [November 25, 2016, 10:18am UTC](https://discuss.elastic.co/t/how-can-i-debug-what-influence-the-write-in-speed-from-logstash-input-file-to-es/67173/1 "2016-11-25T10:18:50Z")
Hi
I found it very slow when writing to es.
And I've made a test
```auto
input {
file {
path => "access.log"
}
}
output {
elasticsearch {
index => "ur_test_%{+YYYY.MM.dd}"
document_type => "k_gif"
flush_size => 1000
idle_flush_time => 3
hosts => ["10.10.22.13:9200"]
}
}
# my server memory is limited, so I set the workers to 4
./logstash -f test.conf -w 4
```
The raw log write-in speed is near 110 lines per second. But the es index increase about only 50 docs per 3 seconds.
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [November 25, 2016, 10:35am UTC](https://discuss.elastic.co/t/how-can-i-debug-what-influence-the-write-in-speed-from-logstash-input-file-to-es/67173/2 "2016-11-25T10:35:21Z")
What versions of Logstash and Elasticsearch are you using? What is the specification of the hosts running Logstash and Elasticsearch?
---
### Author:  [@Morriaty](https://discuss.elastic.co/u/Morriaty)
#### Post date: [November 25, 2016, 10:54am UTC](https://discuss.elastic.co/t/how-can-i-debug-what-influence-the-write-in-speed-from-logstash-input-file-to-es/67173/3 "2016-11-25T10:54:10Z")
Hi @Christian_Dahlqvist
I am using logstash-2.3.1 and es-2.3.2. The hosts are in the same datacenter, but different vlans, from 10.10.20.0/255.255.255.0 to 10.10.22.0/255.255.255.0.
I suppose it's not network problem.
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [November 25, 2016, 10:55am UTC](https://discuss.elastic.co/t/how-can-i-debug-what-influence-the-write-in-speed-from-logstash-input-file-to-es/67173/4 "2016-11-25T10:55:58Z")
How much CPU and RAM do the hosts have? What type of storage are you using? How are you measuring indexing throughput?
---
### Author:  [@Morriaty](https://discuss.elastic.co/u/Morriaty)
#### Post date: [November 25, 2016, 11:05am UTC](https://discuss.elastic.co/t/how-can-i-debug-what-influence-the-write-in-speed-from-logstash-input-file-to-es/67173/5 "2016-11-25T11:05:51Z")
The Logstash host have 16G memory and 16 cores AMD 6128, but the free memory is about only 3G.
I'm not clear with the es cluster's hardware. But there are many other logs writing to es, it is the cluster operation engineer told me that my log write-in speed was much slower than others.
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [November 25, 2016, 11:15am UTC](https://discuss.elastic.co/t/how-can-i-debug-what-influence-the-write-in-speed-from-logstash-input-file-to-es/67173/6 "2016-11-25T11:15:12Z")
I would recommend upgrading to Logstash 2.4 and try increasing the number of [elasticsearch output workers](https://www.elastic.co/guide/en/logstash/2.4/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-workers) to 4 (the same number as your number of worker threads).
If you do not have access to the Elasticsearch cluster to measure throughput, you can try introducing the [metrics filter](https://www.elastic.co/guide/en/logstash/2.4/plugins-filters-metrics.html) and write this to e.g. a file using [conditionals](https://www.elastic.co/guide/en/logstash/2.4/event-dependent-configuration.html).
---
### Author:  [@Morriaty](https://discuss.elastic.co/u/Morriaty)
#### Post date: [November 28, 2016, 10:46am UTC](https://discuss.elastic.co/t/how-can-i-debug-what-influence-the-write-in-speed-from-logstash-input-file-to-es/67173/7 "2016-11-28T10:46:52Z")
This is the test.conf
```auto
input {
generator {
type => "generated"
}
}
filter {
if [type] == "generated" {
metrics {
meter => "events"
add_tag => "metric"
}
}
}
output {
if "metric" in [tags] {
stdout {
codec => line {
format => "rate: %{[events][rate_1m]}"
}
}
}
elasticsearch {
index => "ur_test_%{+YYYY.MM.dd}"
document_type => "kkk"
hosts => ["10.10.22.13:8200"]
workers => 16
}
}
```
If I test without output to es, the rate is about 20000/min
```auto
$ ./logstash -f conf/test.conf
Settings: Default pipeline workers: 16
Pipeline main started
rate: 0.0
rate: 18273.2
rate: 18615.553825440165
rate: 19008.827059558338
rate: 19918.175759533293
rate: 20440.175732344997
rate: 21039.14095380837
rate: 21438.843646149202
rate: 21849.22019386933
rate: 22468.25071231354
rate: 22695.56038678966
rate: 22992.998331666
```
If writing to es, the rate decreased deeply to 300/min
```auto
$ ./logstash -f conf/test.conf
Settings: Default pipeline workers: 16
Pipeline main started
rate: 399.4
rate: 383.45685627708707
rate: 364.7906766746546
rate: 351.6147416575027
rate: 331.4967377003855
```
And I've checked the network
```auto
$ ping 10.10.22.13
PING 10.10.22.13 (10.10.22.13) 56(84) bytes of data.
64 bytes from 10.10.22.13: icmp_seq=1 ttl=63 time=0.133 ms
64 bytes from 10.10.22.13: icmp_seq=2 ttl=63 time=0.104 ms
64 bytes from 10.10.22.13: icmp_seq=3 ttl=63 time=0.123 ms
64 bytes from 10.10.22.13: icmp_seq=4 ttl=63 time=0.107 ms
64 bytes from 10.10.22.13: icmp_seq=5 ttl=63 time=0.099 ms
64 bytes from 10.10.22.13: icmp_seq=6 ttl=63 time=0.232 ms
64 bytes from 10.10.22.13: icmp_seq=7 ttl=63 time=0.121 ms
64 bytes from 10.10.22.13: icmp_seq=8 ttl=63 time=0.201 ms
64 bytes from 10.10.22.13: icmp_seq=9 ttl=63 time=0.100 ms
64 bytes from 10.10.22.13: icmp_seq=10 ttl=63 time=0.105 ms
64 bytes from 10.10.22.13: icmp_seq=11 ttl=63 time=0.324 ms
```
And the cluster op engineer have told me the es cluster's average rate is 8000/sec
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [November 28, 2016, 10:56am UTC](https://discuss.elastic.co/t/how-can-i-debug-what-influence-the-write-in-speed-from-logstash-input-file-to-es/67173/8 "2016-11-28T10:56:37Z")
I believe the metrics shown are number of documents processed per second measured over a period of 1 minute, not documents per minute. Based on the information you have provided it would seem it is possible that Elasticsearch is the bottleneck. The fact that it is processing 8000 events/second does not necessarily mean it has any spare capacity to process additional events.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 26, 2016, 10:57am UTC](https://discuss.elastic.co/t/how-can-i-debug-what-influence-the-write-in-speed-from-logstash-input-file-to-es/67173/9 "2016-12-26T10:57:30Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.