# Elasticsearch Output - Request size exceeded error

**URL:** <https://discuss.elastic.co/t/elasticsearch-output-request-size-exceeded-error/56322>\
**Category:** Logstash\
**Created:** [July 25, 2016, 5:13pm UTC](https://discuss.elastic.co/t/elasticsearch-output-request-size-exceeded-error/56322 "2016-07-25T17:13:03Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![Stephen\_Purr](https://avatars.discourse-cdn.com/v4/letter/s/b19c9b/32.png) [@Stephen\_Purr](https://discuss.elastic.co/u/Stephen_Purr)\
**Post date:** [July 25, 2016, 5:13pm UTC](https://discuss.elastic.co/t/elasticsearch-output-request-size-exceeded-error/56322/1 "2016-07-25T17:13:03Z")

</div>

Hello everyone,

Occasionally we've been seeing our logstash server stop processing logs from our redis queue and start to show some errors in its logs about request size being too large. This causes our logstash server to stop working until a restart is done on the logstash\_server service. After that it will start to process logs again.

We're using the AWS Elasticsearch service as our ES backend and they have a limit of 10485760 bytes per request. I've tried lowering the `flush_size` option in our elasticsearch config to 50 to see if that would help but we still see this occurring periodically, about once a week right now. I'm going to try to lower it 20 but wanted to see if there's anywhere I can check to get more information on the batch of logs being sent so I can see where the offending log is coming from. Here's some more information on our logstash server and elasticsearch setup:

Logstash Version: `logstash 2.3.4`

Elasticsearch Output Plugin Config:

```auto
output {
  elasticsearch {
    'hosts' => ["logstash-elasticsearch.example.com:80"]
    'flush_size' => 50
  }
}

```

Logs of the Error:

```auto
{:timestamp=>"2016-07-25T16:50:02.153000+0000", :message=>"[413] {\"Message\":\"Request size exceeded 10485760 bytes\"}", :class=>"Elasticsearch::Transport::Transport::Errors::RequestEntityTooLarge", :backtrace=>["/opt/logstash/server/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.18/lib/elasticsearch/transport/transport/base.rb:201:in `__raise_transport_error'", "/opt/logstash/server/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.18/lib/elasticsearch/transport/transport/base.rb:312:in `perform_request'", "/opt/logstash/server/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.18/lib/elasticsearch/transport/transport/http/manticore.rb:67:in `perform_request'", "/opt/logstash/server/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.18/lib/elasticsearch/transport/client.rb:128:in `perform_request'", "/opt/logstash/server/vendor/bundle/jruby/1.9/gems/elasticsearch-api-1.0.18/lib/elasticsearch/api/actions/bulk.rb:90:in `bulk'", "/opt/logstash/server/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.7.1-java/lib/logstash/outputs/elasticsearch/http_client.rb:53:in `non_threadsafe_bulk'", "/opt/logstash/server/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.7.1-java/lib/logstash/outputs/elasticsearch/http_client.rb:38:in `bulk'", "org/jruby/ext/thread/Mutex.java:149:in `synchronize'", "/opt/logstash/server/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.7.1-java/lib/logstash/outputs/elasticsearch/http_client.rb:38:in `bulk'", "/opt/logstash/server/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.7.1-java/lib/logstash/outputs/elasticsearch/common.rb:172:in `safe_bulk'", "/opt/logstash/server/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.7.1-java/lib/logstash/outputs/elasticsearch/common.rb:101:in `submit'", "/opt/logstash/server/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.7.1-java/lib/logstash/outputs/elasticsearch/common.rb:86:in `retrying_submit'", "/opt/logstash/server/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.7.1-java/lib/logstash/outputs/elasticsearch/common.rb:29:in `multi_receive'", "org/jruby/RubyArray.java:1653:in `each_slice'", "/opt/logstash/server/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.7.1-java/lib/logstash/outputs/elasticsearch/common.rb:28:in `multi_receive'", "/opt/logstash/server/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/output_delegator.rb:130:in `worker_multi_receive'", "/opt/logstash/server/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/output_delegator.rb:129:in `worker_multi_receive'", "/opt/logstash/server/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/output_delegator.rb:114:in `multi_receive'", "/opt/logstash/server/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/pipeline.rb:301:in `output_batch'", "org/jruby/RubyHash.java:1342:in `each'", "/opt/logstash/server/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/pipeline.rb:301:in `output_batch'", "/opt/logstash/server/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/pipeline.rb:232:in `worker_loop'", "/opt/logstash/server/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/pipeline.rb:201:in `start_workers'"], :level=>:warn}
{:timestamp=>"2016-07-25T16:50:06.230000+0000", :message=>"Attempted to send a bulk request to Elasticsearch configured at '[\"http://logstash-elasticsearch.example.com\"]', but Elasticsearch appears to be unreachable or down!", :error_message=>"Broken pipe", :class=>"Manticore::SocketException", :level=>:error}

```

Thanks in advance,

Steve

---

<div class="post-metadata">

**Author:** ![niraj\_kumar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/niraj_kumar/32/4130_2.png) [@niraj\_kumar](https://discuss.elastic.co/u/niraj_kumar)\
**Post date:** [July 25, 2016, 7:06pm UTC](https://discuss.elastic.co/t/elasticsearch-output-request-size-exceeded-error/56322/2 "2016-07-25T19:06:04Z")

</div>

Does you elasticsearch run over port 80? Can you paste the entire config of logstash.

---

<div class="post-metadata">

**Author:** ![Stephen\_Purr](https://avatars.discourse-cdn.com/v4/letter/s/b19c9b/32.png) [@Stephen\_Purr](https://discuss.elastic.co/u/Stephen_Purr)\
**Post date:** [July 25, 2016, 7:28pm UTC](https://discuss.elastic.co/t/elasticsearch-output-request-size-exceeded-error/56322/3 "2016-07-25T19:28:19Z")

</div>

yes, we run our elasticsearch service over port 80. Here's the entire logstash config that we are using at the moment:

```auto
input {
  redis {
    'data_type' => "list"
    'host' => "logstash-redis.example.com"
    'key' => "logstash"
    'type' => "redis_input"
  }
}

input {
  file {
    type => "logstash-log"
    path => "/opt/logstash/server/log/logstash.log"
    start_position => "beginning"
  }
}

input {
  http {
    port => 19300
    user => user
    password => "XXXXXXXXXXXXX"
  }
}

filter {
  if [type] == "logstash-log" {
    ruby {
      code => "event['message'] = event['message'].gsub('=>', '=').gsub(':message', ':log_message')"
    }

    kv {
      add_field => {"app" => "logstash-server"}
      trimkey => "{:"
      trim => "}\""
      field_split => ", "
    }
  }
}

filter {
  if [type] == "rails" {
    ruby {
      'code' => "event['rails_params'] = event['params'].to_s if event.include?('params') && !event['params'].empty?"
      'remove_field' => ["params"]
    }
  }
}

output {
  elasticsearch {
    'hosts' => ["logstash-elasticsearch.example.com:80"]
    'flush_size' => 50
  }
}

```

---

<div class="post-metadata">

**Author:** ![niraj\_kumar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/niraj_kumar/32/4130_2.png) [@niraj\_kumar](https://discuss.elastic.co/u/niraj_kumar)\
**Post date:** [July 25, 2016, 9:00pm UTC](https://discuss.elastic.co/t/elasticsearch-output-request-size-exceeded-error/56322/4 "2016-07-25T21:00:18Z")

</div>

Can you add a protocol =\> http to it and give it a try.

---

<div class="post-metadata">

**Author:** ![Stephen\_Purr](https://avatars.discourse-cdn.com/v4/letter/s/b19c9b/32.png) [@Stephen\_Purr](https://discuss.elastic.co/u/Stephen_Purr)\
**Post date:** [July 26, 2016, 1:50pm UTC](https://discuss.elastic.co/t/elasticsearch-output-request-size-exceeded-error/56322/5 "2016-07-26T13:50:18Z")

</div>

It doesn't look like there is a `protocol` setting in the elasticsearch output plugin in logstash version 2.3.4. From their documentation:

```auto
This output only speaks the HTTP protocol. HTTP is the preferred protocol for interacting with Elasticsearch as of Logstash 2.0.

```

Is there something I'm missing there? Thanks for the response again.

---

<div class="post-metadata">

**Author:** ![niraj\_kumar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/niraj_kumar/32/4130_2.png) [@niraj\_kumar](https://discuss.elastic.co/u/niraj_kumar)\
**Post date:** [July 26, 2016, 7:06pm UTC](https://discuss.elastic.co/t/elasticsearch-output-request-size-exceeded-error/56322/6 "2016-07-26T19:06:09Z")

</div>

My bad. In the previous versions of LS there was a setting to manually specify the protocol. But seems like it is default now.

---

<div class="post-metadata">

**Author:** ![DiscussBuster](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/discussbuster/32/11042_2.png) [@DiscussBuster](https://discuss.elastic.co/u/DiscussBuster)\
**Post date:** [July 27, 2016, 2:54am UTC](https://discuss.elastic.co/t/elasticsearch-output-request-size-exceeded-error/56322/7 "2016-07-27T02:54:41Z")

</div>

Maybe you could do something in a ruby filter where you check the total size and add a metadata field if it is over a certain size, and conditionally write to a file output if that metadata field is set, else proceed with your normal output.

Just spitballin' here.

---

<div class="post-metadata">

**Author:** ![porcelani](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/porcelani/32/13531_2.png) [@porcelani](https://discuss.elastic.co/u/porcelani)\
**Post date:** [December 1, 2016, 12:39pm UTC](https://discuss.elastic.co/t/elasticsearch-output-request-size-exceeded-error/56322/8 "2016-12-01T12:39:13Z")

</div>

+1, same problem

---

<div class="post-metadata">

**Author:** ![mressler](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mressler/32/14313_2.png) [@mressler](https://discuss.elastic.co/u/mressler)\
**Post date:** [January 4, 2017, 7:19pm UTC](https://discuss.elastic.co/t/elasticsearch-output-request-size-exceeded-error/56322/9 "2017-01-04T19:19:56Z")

</div>

+1, same problem as well.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:29am UTC](https://discuss.elastic.co/t/elasticsearch-output-request-size-exceeded-error/56322/10 "2017-07-06T04:29:35Z")

</div>