# Logstash 2.3.x throughput drops significantly compared to 2.2.0

**URL:** <https://discuss.elastic.co/t/logstash-2-3-x-throughput-drops-significantly-compared-to-2-2-0/55250>\
**Category:** Logstash\
**Created:** [July 12, 2016, 12:50am UTC](https://discuss.elastic.co/t/logstash-2-3-x-throughput-drops-significantly-compared-to-2-2-0/55250 "2016-07-12T00:50:03Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![anhlqn](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anhlqn/32/5454_2.png) [@anhlqn](https://discuss.elastic.co/u/anhlqn)\
**Post date:** [July 12, 2016, 12:50am UTC](https://discuss.elastic.co/t/logstash-2-3-x-throughput-drops-significantly-compared-to-2-2-0/55250/1 "2016-07-12T00:50:03Z")

</div>

Hi everyone,

Does anyone experience throughput drop in Logstash 2.3.x ( 2.3.0, 2.3.1, 2.3.3) compared to 2.2.0. I'm using Logstash to receive, process, and send IIS logs to Elasticsearch. On the same physical server with 1 Logstash instance with same Logstash config and same number of incoming events,

- Logstash 2.3.x sent 1500 EPS to ES
- Logstash 2.2.0 sent 6000 EPS to ES

I repeated the test multiple times and also got similar issue when indexing netflow. The Logstash config is very simple as below

```auto
input {	
	tcp {
		port => 5544
		codec => "json"
	}
}	

filter {

	#--------------------------- IISLogs Filters
	# Convert IIS log time stamp to local time and send to @timestamp field
	date {
		match => ["GMTTime", "yyyy-MM-dd HH:mm:ss"]
		timezone => "Etc/GMT"
	}
	
	# Parse user web browser
	useragent {
		source => "cs(User-Agent)"
		target => "user-agent"
	}
	
	# IIS Log client ip Geo info
	geoip {
		source => "c-ip"
		target => "client_geoip"
		fields => ["country_name", "real_region_name", "city_name", "location"]
	}
	
	geoip {
		source => "realip"
		target => "realip_geoip"
		fields => ["country_name", "real_region_name", "city_name", "location"]
	}
    
	#--------------------------- End IISLogs Filters
    
	#--------------------------- Perfmon Filters
	date {
		match => ["EventTime", "MM/dd/yyyy HH:mm:ss.SSS"]
		timezone => "America/Los_Angeles"
		locale => "en"
	}
    
    #--------------------------- End Perfmon Filters
	
	## Remove redundant fields
	mutate {	
		remove_field => ["@version", "host", "port", "EventTime", "cs(Cookie)", "sc-substatus", "sc-win32-status", "[user-agent][minor]", "[user-agent][os]", "[user-agent][patch]", "cs-version", "cs(User-Agent)", "cs-username"]
	}
	
	## Drop events if tags = "_jsonparsefailure"
	if "_jsonparsefailure" in [tags] {
		drop {}
	}
}

```

Is there a way to troubleshoot Logstash performance?

---

<div class="post-metadata">

**Author:** ![anhlqn](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anhlqn/32/5454_2.png) [@anhlqn](https://discuss.elastic.co/u/anhlqn)\
**Post date:** [July 12, 2016, 8:07pm UTC](https://discuss.elastic.co/t/logstash-2-3-x-throughput-drops-significantly-compared-to-2-2-0/55250/2 "2016-07-12T20:07:10Z")

</div>

Btw, is it safe to increase LS\_HEAP\_SIZE from the default 1G to 2 or 4G?

---

<div class="post-metadata">

**Author:** ![anhlqn](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anhlqn/32/5454_2.png) [@anhlqn](https://discuss.elastic.co/u/anhlqn)\
**Post date:** [July 15, 2016, 6:14am UTC](https://discuss.elastic.co/t/logstash-2-3-x-throughput-drops-significantly-compared-to-2-2-0/55250/3 "2016-07-15T06:14:34Z")

</div>

Just reply in case someone has similar issue as I do.

Same config and command line arguments worked fine for 2.2.0, but in LS 2.3.x I have to increase batch size in LS to 1000 or 1500 and increase the elasticsearch-output-plugins workers to the number of LS filter workers (both 24 in my case) and `flush_size` to 1500. Increasing elasticsearch output workers may not be necessary, but after that everything works fine.

> The default number of output workers in Logstash 2.2 is now equal to the number of pipeline workers (-w) unless overridden in the Logstash config file.  
> [https://www.elastic.co/guide/en/logstash/current/upgrading-logstash-2.2.html](https://www.elastic.co/guide/en/logstash/current/upgrading-logstash-2.2.html)

Do output workers also equal the number of TCP connections between LS and ES? If yes, then in LS 2.3.3 if I leave workers as default, there are 3 TCP connections between LS to each ES node even though the pipeline workers are 24.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:47am UTC](https://discuss.elastic.co/t/logstash-2-3-x-throughput-drops-significantly-compared-to-2-2-0/55250/4 "2017-07-06T04:47:57Z")

</div>