# Logstash Peformance
**URL:** https://discuss.elastic.co/t/logstash-peformance/80966
**Category:** Logstash
**Created:** [April 3, 2017, 8:26am UTC](https://discuss.elastic.co/t/logstash-peformance/80966 "2017-04-03T08:26:52Z")
**Posts on this page:** 20
**Page:** 1
### Author:  [@Beuhlet\_Reseau](https://discuss.elastic.co/u/Beuhlet_Reseau)
#### Post date: [April 3, 2017, 8:26am UTC](https://discuss.elastic.co/t/logstash-peformance/80966/1 "2017-04-03T08:26:52Z")
Hello,
I would like know tips to increase performance of Logtash.
Currently i have :
OS RH 7.1 : 12 CPU & 24 Go RAM
Filebeat as shipper
Elasticsearch : Xms Xmx : 12g
Logstash (define in logstash.yml) :
Workers 12
Pipeline batch size : 2000
I haven't modify options like pipeline batch delay, pipeline output worker, Java heap..)
For Logstash, what do you precognized to better performance ?
PS : I have 1,5 millions of logs lines every five minutes.
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [April 3, 2017, 8:28am UTC](https://discuss.elastic.co/t/logstash-peformance/80966/2 "2017-04-03T08:28:58Z")
How have you gone about identifying that it is Logstash that is the bottleneck?
---
### Author:  [@Beuhlet\_Reseau](https://discuss.elastic.co/u/Beuhlet_Reseau)
#### Post date: [April 3, 2017, 8:30am UTC](https://discuss.elastic.co/t/logstash-peformance/80966/3 "2017-04-03T08:30:59Z")
Because Elasticsearch is a good DB.
Logstash is low (180k/minutes) to upload line in elasticsearch
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [April 3, 2017, 8:34am UTC](https://discuss.elastic.co/t/logstash-peformance/80966/4 "2017-04-03T08:34:09Z")
What is the specification of your Elasticsearch cluster? Have you performed any benchmark to see what the throughput limit of Elasticsearch is? Have you tried replacing the elasticsearch output in Logstash with a file output to see if this changes the Logstash throughput?
---
### Author:  [@Beuhlet\_Reseau](https://discuss.elastic.co/u/Beuhlet_Reseau)
#### Post date: [April 3, 2017, 8:36am UTC](https://discuss.elastic.co/t/logstash-peformance/80966/5 "2017-04-03T08:36:38Z")
> [@Christian\_Dahlqvist](#):
>
> What is the specification of your Elasticsearch cluster? Have you performed any benchmark to see what the throughput limit of Elasticsearch is? Have you tried replacing the elasticsearch output in Logstash with a file output to see if this changes the Logstash throughput?
I have only one node. Only one server to differents soft of elk.
I don't know how to get informations that you talk 😕
---
### Author:  [@Beuhlet\_Reseau](https://discuss.elastic.co/u/Beuhlet_Reseau)
#### Post date: [April 3, 2017, 8:39am UTC](https://discuss.elastic.co/t/logstash-peformance/80966/6 "2017-04-03T08:39:43Z")
The Java heap size for logstash have impact of performance ?
Currently i have :
-Xms256m
-Xmx1g
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [April 3, 2017, 8:46am UTC](https://discuss.elastic.co/t/logstash-peformance/80966/7 "2017-04-03T08:46:33Z")
What does CPU and disk I/O usage for Elasticsearch look like when you are indexing? Do you have monitoring installed?
---
### Author:  [@Beuhlet\_Reseau](https://discuss.elastic.co/u/Beuhlet_Reseau)
#### Post date: [April 3, 2017, 8:58am UTC](https://discuss.elastic.co/t/logstash-peformance/80966/8 "2017-04-03T08:58:07Z")
With iostat command line i view :
CPU :
```
avg-cpu: %user %nice %system %iowait %steal %idle
0.93 1.75 0.87 0.03 0.00 96.41
```
DISK :
```
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
dm 23.86 7.05 277.95 10208977 402426000
```
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [April 3, 2017, 9:05am UTC](https://discuss.elastic.co/t/logstash-peformance/80966/9 "2017-04-03T09:05:42Z")
Is this from the Elasticsearch node while you are indexing?
---
### Author:  [@Beuhlet\_Reseau](https://discuss.elastic.co/u/Beuhlet_Reseau)
#### Post date: [April 3, 2017, 9:19am UTC](https://discuss.elastic.co/t/logstash-peformance/80966/10 "2017-04-03T09:19:00Z")
Yes during indexing of documents
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [April 3, 2017, 9:33am UTC](https://discuss.elastic.co/t/logstash-peformance/80966/11 "2017-04-03T09:33:03Z")
What is the specification of your Elasticsearch host?
---
### Author:  [@Beuhlet\_Reseau](https://discuss.elastic.co/u/Beuhlet_Reseau)
#### Post date: [April 3, 2017, 9:37am UTC](https://discuss.elastic.co/t/logstash-peformance/80966/12 "2017-04-03T09:37:34Z")
I have use default installation.
I have just changed XMS and XMX java.
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [April 3, 2017, 9:38am UTC](https://discuss.elastic.co/t/logstash-peformance/80966/13 "2017-04-03T09:38:30Z")
How many CPU cores does your Elasticsearch host have? What type of storage?
---
### Author:  [@Beuhlet\_Reseau](https://discuss.elastic.co/u/Beuhlet_Reseau)
#### Post date: [April 3, 2017, 9:46am UTC](https://discuss.elastic.co/t/logstash-peformance/80966/14 "2017-04-03T09:46:24Z")
12 CPU, local storage (EXT4)
---
### Author:  [@Beuhlet\_Reseau](https://discuss.elastic.co/u/Beuhlet_Reseau)
#### Post date: [April 3, 2017, 12:04pm UTC](https://discuss.elastic.co/t/logstash-peformance/80966/15 "2017-04-03T12:04:32Z")
One things, when i parse and index data, my load average is low i find :
Load Average : 5,50 2,57 1,04
I don't know if it's important or no.
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [April 3, 2017, 12:10pm UTC](https://discuss.elastic.co/t/logstash-peformance/80966/16 "2017-04-03T12:10:37Z")
What does your Logstash configuration look like? Which version of Logstash are you using?
---
### Author:  [@Beuhlet\_Reseau](https://discuss.elastic.co/u/Beuhlet_Reseau)
#### Post date: [April 3, 2017, 12:38pm UTC](https://discuss.elastic.co/t/logstash-peformance/80966/17 "2017-04-03T12:38:10Z")
> [@Beuhlet\_Reseau](#):
>
> Logstash (define in logstash.yml) : Workers 12Pipeline batch size : 2000
I use 5.1 version.
Logstash conf parsing :
if [type] == "edr" {
if [message] =~ "\bCTEName\b" {
drop { }
} else {
csv {
columns =\> ["..."]
}
}
```
mutate {
convert => {
"edr_SrId" => "integer"
"edr_Uets" => "integer"
"edr_GtalOctets" => "integer"
"edr_Uimit" => "integer"
"edr_Jer" => "integer"
}
remove_field => ["message", "edr_D", "edr_IlltureFlag", "edr_UatureFlag", "edr_NP", "edr_Tpe", "edr_OSe", "edr_Oum", "edr_Oature", "edr_Monitor", "edr_Faig", "edr_Qbel", "edr_FlowStatus", "edr_Redabel", "edr_NoonID"]
}
```
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [April 3, 2017, 12:56pm UTC](https://discuss.elastic.co/t/logstash-peformance/80966/18 "2017-04-03T12:56:49Z")
How many fields do you have? What is the average size of your events?
---
### Author:  [@Beuhlet\_Reseau](https://discuss.elastic.co/u/Beuhlet_Reseau)
#### Post date: [April 3, 2017, 1:38pm UTC](https://discuss.elastic.co/t/logstash-peformance/80966/19 "2017-04-03T13:38:24Z")
I have 30 fields by line and about 1,2 GB per days of index.
Just if you tell me how change de memory buffer size. The documentation says that the value is static but i don't find where configure it.
By default it's 10% but i think i can grow at 20%
---
### Author:  [@AlexisAnzieu](https://discuss.elastic.co/u/AlexisAnzieu)
#### Post date: [April 5, 2017, 7:20am UTC](https://discuss.elastic.co/t/logstash-peformance/80966/20 "2017-04-05T07:20:25Z")
Hi,
Maybe this link could help you :
> **[Ben Passmore](https://www.passbe.com)**
>
> Incident response, coder and hacking all the things. Everything security.
[Next page](https://discuss.elastic.co/t/logstash-peformance/80966.md?page=2)