# How to improve performance of elasticsearch input in Logstash

**URL:** https://discuss.elastic.co/t/how-to-improve-performance-of-elasticsearch-input-in-logstash/58513
**Category:** Logstash
**Created:** [August 20, 2016, 7:13pm UTC](https://discuss.elastic.co/t/how-to-improve-performance-of-elasticsearch-input-in-logstash/58513 "2016-08-20T19:13:35Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![MikeM](https://avatars.discourse-cdn.com/v4/letter/m/41988e/32.png) [@MikeM](https://discuss.elastic.co/u/MikeM)
#### Post date: [August 20, 2016, 7:13pm UTC](https://discuss.elastic.co/t/how-to-improve-performance-of-elasticsearch-input-in-logstash/58513/1 "2016-08-20T19:13:35Z")

</div>

I want to use logstash to pull current elasticsearch documents, run them through filters, and then update them in the same cluster. I am trying to figure out how to get the best performance, and I'm first looking at the elasticsearch plugin. I know that pipeline workers only apply to the filter and output plugins.

I have 3 ES nodes all on the same subnet. I created a new Ubuntu 14.04 VM in the same subnet and installed LS 5.0-alpha5 from the debian package. The VM has 8GB of RAM and 4 cores.

Taking a tip from a [recent elastic blog post](https://www.elastic.co/blog/just_enough_redis_for_logstash), I decided to setup a logstash config file like the following:

```auto
input {
     elasticsearch {
         hosts => ["hostA","hostB","hostC"]
         index => "specific-index"
         docinfo => true
         query => '
         {
           "query": {
             "match": {
               "_type": {
                 "query": "MyType"
               }
             }
           }
          }
         '
     }
}

output {
      stdout { codec => dots }
}

```

Then I run the following command:

```auto
sudo ./logstash -f /etc/logstash/conf.d/logstash.conf --path.settings=/etc/logstash | pv -Wart > /dev/null

```

When I run this I get around 3 - 4 kB/s tops.

I'm wondering what I can do to improve the throughput of this logstash script / elasticsearch input plugin. Are there any settings that could improve this? Anything on the elasticsearch nodes that need to change?

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [August 22, 2016, 6:31am UTC](https://discuss.elastic.co/t/how-to-improve-performance-of-elasticsearch-input-in-logstash/58513/2 "2016-08-22T06:31:01Z")

</div>

Are you monitoring everything to see where it may be limited?

---

<div class="post-metadata">

### Author: ![MikeM](https://avatars.discourse-cdn.com/v4/letter/m/41988e/32.png) [@MikeM](https://discuss.elastic.co/u/MikeM)
#### Post date: [August 22, 2016, 1:08pm UTC](https://discuss.elastic.co/t/how-to-improve-performance-of-elasticsearch-input-in-logstash/58513/3 "2016-08-22T13:08:23Z")

</div>

@warkolm I'm not sure what you mean by monitoring. Are you referring to the Monitoring API's in LS 5.0? Or is there another way that should monitor this (on the elasticsearch side?)

I haven't used any of the monitoring API's in LS 5.0. Do you have any idea what API's would be the best to use to see what is happening on the input plugin?

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [August 22, 2016, 9:24pm UTC](https://discuss.elastic.co/t/how-to-improve-performance-of-elasticsearch-input-in-logstash/58513/4 "2016-08-22T21:24:35Z")

</div>

Monitoring anything really.  
How do you know it's LS or ES? Are you monitoring system resources?

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 4:42am UTC](https://discuss.elastic.co/t/how-to-improve-performance-of-elasticsearch-input-in-logstash/58513/5 "2017-07-06T04:42:09Z")

</div>


