# S3-plugin logstash "plugin had an unrecoverable error. Will restart this plugin."

**URL:** <https://discuss.elastic.co/t/s3-plugin-logstash-plugin-had-an-unrecoverable-error-will-restart-this-plugin/156526>\
**Category:** Logstash\
**Created:** [November 13, 2018, 6:14pm UTC](https://discuss.elastic.co/t/s3-plugin-logstash-plugin-had-an-unrecoverable-error-will-restart-this-plugin/156526 "2018-11-13T18:14:08Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![rpendela](https://avatars.discourse-cdn.com/v4/letter/r/3be4f8/32.png) [@rpendela](https://discuss.elastic.co/u/rpendela)\
**Post date:** [November 13, 2018, 6:14pm UTC](https://discuss.elastic.co/t/s3-plugin-logstash-plugin-had-an-unrecoverable-error-will-restart-this-plugin/156526/1 "2018-11-13T18:14:08Z")

</div>

I am using s3-input plugin to process the logs to logstash, I am receiving some logs of logfiles from s3 but not all of them. As far as I know, my configuration is correct because I am receiving some logs but something is blocking in plugin and getting message as "A plugin had an unrecoverable error. Will restart this plugin"

Here is the error log,

```
    logstash | Plugin: &lt;LogStash::Inputs::S3 bucket=&gt;"Bucket-name", prefix=&gt;"Bucket-name/", backup_to_bucket=&gt;"Bucket-name", backup_add_prefix=&gt;"Bucket-name-backup/", delete=&gt;true, additional_settings=&gt;{"force_path_style"=&gt;"true", "follow_redirects"=&gt;"false"}, id=&gt;"id-number", enable_metric=&gt;true, codec=&gt;&lt;LogStash::Codecs::Plain id=&gt;"plain_97f51ba7-9405-4698-9ebc-6d9517772d35", enable_metric=&gt;true, charset=&gt;"UTF-8"&gt;, region=&gt;"us-east-1", role_session_name=&gt;"logstash", interval=&gt;60, temporary_directory=&gt;"/tmp/logstash"&gt;
logstash | Error: 
logstash | Exception: Aws::S3::Errors::NotFound
logstash | Stack: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-core-2.11.67/lib/seahorse/client/plugins/raise_response_errors.rb:15:in `call'
logstash | /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-core-2.11.67/lib/aws-sdk-core/plugins/s3_sse_cpk.rb:19:in `call'
logstash | /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-core-2.11.67/lib/aws-sdk-core/plugins/s3_dualstack.rb:24:in `call'
logstash | /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-core-2.11.67/lib/aws-sdk-core/plugins/s3_accelerate.rb:34:in `call'
logstash | /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-core-2.11.67/lib/aws-sdk-core/plugins/jsonvalue_converter.rb:20:in `call'
logstash | /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-core-2.11.67/lib/aws-sdk-core/plugins/idempotency_token.rb:18:in `call'
logstash | /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-core-2.11.67/lib/aws-sdk-core/plugins/param_converter.rb:20:in `call'
logstash | /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-core-2.11.67/lib/seahorse/client/plugins/response_target.rb:21:in `call'
logstash | /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-core-2.11.67/lib/seahorse/client/request.rb:70:in `send_request'
logstash | /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-core-2.11.67/lib/seahorse/client/base.rb:207:in `block in define_operation_methods'
logstash | /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-resources-2.11.67/lib/aws-sdk-resources/request.rb:24:in `call'
logstash | /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-resources-2.11.67/lib/aws-sdk-resources/operations.rb:41:in `call'
logstash | /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-resources-2.11.67/lib/aws-sdk-resources/operations.rb:61:in `call'
logstash | /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-resources-2.11.67/lib/aws-sdk-resources/resource.rb:147:in `load'
logstash | /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-resources-2.11.67/lib/aws-sdk-resources/resource.rb:120:in `data'
logstash | /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-resources-2.11.67/lib/aws-sdk-resources/resource.rb:223:in `block in add_data_attribute'
logstash | /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-s3-3.3.5/lib/logstash/inputs/s3.rb:369:in `process_log'
logstash | /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-s3-3.3.5/lib/logstash/inputs/s3.rb:161:in `block in process_files'
logstash | org/jruby/RubyArray.java:1734:in `each'
logstash | /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-s3-3.3.5/lib/logstash/inputs/s3.rb:156:in `process_files'
logstash | /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-s3-3.3.5/lib/logstash/inputs/s3.rb:105:in `block in run'
logstash | /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/interval.rb:20:in `interval'
logstash | /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-s3-3.3.5/lib/logstash/inputs/s3.rb:104:in `run'
logstash | /usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:512:in `inputworker'
logstash | /usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:505:in `block in start_input'
logstash | [2018-11-13T16:29:42,881][WARN][logstash.inputs.s3] S3 input: Unable to download remote file {:remote_key=>"incapsula/641_827231.log", :message=>"The specified key does not exist."}
logstash | [2018-11-13T16:39:42,867][WARN][logstash.inputs.s3] S3 input: Unable to download remote file {:remote_key=>"incapsula/641_827259.log", :message=>"The specified key does not exist."}
logstash | [2018-11-13T16:43:44,305][ERROR][logstash.pipeline] A plugin had an unrecoverable error. Will restart this plugin.

```

My Logstash configuration here,

```
input {
  s3 {
    "bucket" => "bucket-name"
    "prefix" => "bucket-name/"
    "backup_to_bucket" => "bucket-name"
    "backup_add_prefix" => "bucket-name-backup/"
    "delete" => true
    "additional_settings" => {
      "force_path_style" => true
      "follow_redirects" => false
    }
  }
}

output {
       elasticsearch {
        hosts => ["https://host1:9200", "https://host2:9200", "https://host3:9200"]
        user => logstash_user
        password => password
        ssl => true
        cacert => '/etc/ca.crt'
        manage_template => false
        index => "incapsula-%{+YYYY.MM.dd}"
    }
    

}
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 11, 2018, 6:14pm UTC](https://discuss.elastic.co/t/s3-plugin-logstash-plugin-had-an-unrecoverable-error-will-restart-this-plugin/156526/2 "2018-12-11T18:14:16Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.