# Logstash S3 Input Plugin Error

**URL:** <https://discuss.elastic.co/t/logstash-s3-input-plugin-error/245104>\
**Category:** Logstash\
**Created:** [August 15, 2020, 6:28pm UTC](https://discuss.elastic.co/t/logstash-s3-input-plugin-error/245104 "2020-08-15T18:28:05Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![Kevin\_f](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kevin_f/32/72456_2.png) [@Kevin\_f](https://discuss.elastic.co/u/Kevin_f)\
**Post date:** [August 15, 2020, 6:28pm UTC](https://discuss.elastic.co/t/logstash-s3-input-plugin-error/245104/1 "2020-08-15T18:28:06Z")

</div>

Hi All,

I am using S3 plugin to ingest logs from Oracle Cloud Infrastructure bucket. They have advised that the S3 plugin is what can be used. So I have now configured the input config as:

```auto
input {
  s3 {
    bucket => "logs"
    endpoint => "https://<object_storage_namespace>.compat.objectstorage.<region>.oraclecloud.com"
    region => "uk-london-1"
    access_key_id => " *************************"
    secret_access_key => " ******************"
    proxy_uri => "http://x.x.x.x:3128"
    delete => false
    interval => 300 # seconds
    add_field => { "service" => "oci" }
    codec => "json"
  }
}

```

However in logstash logs I am getting the following error:  
A plugin had an unrecoverable error. Will restart this plugin.  
Pipeline\_id:main  
Plugin: \<LogStash::Inputs::S3 bucket=\>"logs", .............  
Error: certificate verify failed  
Exception: Seahorse::Client::NetworkingError  
Stack: uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:1002:in `connect' uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:924:in `do\_start'  
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:919:in `start' uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/delegate.rb:83:in `method\_missing'  
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.501/lib/seahorse/client/net\_http/connection\_pool.rb:285:in `start_session' /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.501/lib/seahorse/client/net_http/connection_pool.rb:92:in `session\_for'  
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.501/lib/seahorse/client/net\_http/handler.rb:119:in `session' /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.501/lib/seahorse/client/net_http/handler.rb:71:in `transmit'  
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.501/lib/seahorse/client/net\_http/handler.rb:45:in `call' /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.501/lib/seahorse/client/plugins/content_length.rb:12:in `call'

I can perform a successful curl on the endpoint https://\<object\_strorage\_namespace\>.compat.objectstorage..oraclecloud.com

Not sure what this error is about. Seems like some kind of authentication issue with the api endpoint? Any help would be much appreciated. Thank you.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 15, 2020, 8:25pm UTC](https://discuss.elastic.co/t/logstash-s3-input-plugin-error/245104/2 "2020-08-15T20:25:04Z")

</div>

> [@Kevin\_f](#):
>
> Error: certificate verify failed

I suggest that, _purely for debugging purposes_, that you use the additional\_settings option on the input to disable ssl\_verify\_peer. If that works then delete it and add one of

1. :ssl\_ca\_bundle
2. :ssl\_ca\_directory
3. :ssl\_ca\_store

to pass to the input the CA of the certificate the [oraclecloud.com](http://oraclecloud.com) endpoint is using.

---

<div class="post-metadata">

**Author:** ![Kevin\_f](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kevin_f/32/72456_2.png) [@Kevin\_f](https://discuss.elastic.co/u/Kevin_f)\
**Post date:** [August 15, 2020, 8:29pm UTC](https://discuss.elastic.co/t/logstash-s3-input-plugin-error/245104/3 "2020-08-15T20:29:15Z")

</div>

Hi Badger,

Thank you.

For the logstash s3 plugin input I do not see a parameter to disable ssl\_verify\_peer.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 15, 2020, 8:43pm UTC](https://discuss.elastic.co/t/logstash-s3-input-plugin-error/245104/4 "2020-08-15T20:43:32Z")

</div>

> [@Kevin\_f](#):
>
> For the logstash s3 plugin input I do not see a parameter to disable ssl\_verify\_peer.

Use the additonal\_settings option. This can be used to pass many options understood by the Seahorse client library. For example...

```
additional_options => { "ssl_verify_peer" => false }

```

or

```
additional_options => { "ssl_ca_directory" => "/path2/unbundledCAcerts/" }

```

---

<div class="post-metadata">

**Author:** ![Kevin\_f](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kevin_f/32/72456_2.png) [@Kevin\_f](https://discuss.elastic.co/u/Kevin_f)\
**Post date:** [August 15, 2020, 8:54pm UTC](https://discuss.elastic.co/t/logstash-s3-input-plugin-error/245104/5 "2020-08-15T20:54:30Z")

</div>

Hi Badger,

I updated the config as follows:

```auto
input {
  s3 {
    bucket => "logs"
    endpoint => "https://<object_storage_namespace>.compat.objectstorage.<region>.oraclecloud.com"
    region => "uk-london-1"
    access_key_id => " ******************"
    secret_access_key => " *************"
    proxy_uri => "http://x.x.x.x:3128"
    delete => false
    interval => 300 # seconds
    add_field => { "service" => "oci" }
    codec => "json"
    additional_options => { "ssl_verify_peer" => false }
  }
}

```

And when i do a config test using command:

```auto
/usr/share/logstash/bin/logstash --config.test_and_exit -f /etc/logstash/conf.d/input-oci-bucket.conf

```

I get the error:

[ERROR] 2020-08-15 20:50:04.975 [LogStash::Runner] s3 - Unknown setting 'additional\_options' for s3  
[FATAL] 2020-08-15 20:50:04.987 [LogStash::Runner] runner - The given configuration is invalid. Reason: Unable to configure plugins: (ConfigurationError) Something is wrong with your configuration.  
[ERROR] 2020-08-15 20:50:04.990 [LogStash::Runner] Logstash - java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 15, 2020, 9:21pm UTC](https://discuss.elastic.co/t/logstash-s3-input-plugin-error/245104/6 "2020-08-15T21:21:45Z")

</div>

Sorry, it is [additional\_settings](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-s3.html#plugins-inputs-s3-additional_settings), not additional\_options.

---

<div class="post-metadata">

**Author:** ![Kevin\_f](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kevin_f/32/72456_2.png) [@Kevin\_f](https://discuss.elastic.co/u/Kevin_f)\
**Post date:** [August 15, 2020, 9:26pm UTC](https://discuss.elastic.co/t/logstash-s3-input-plugin-error/245104/7 "2020-08-15T21:26:51Z")

</div>

Thank you Badger.

I made the change and still getting the same error as before unfortuantely. ☹

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 15, 2020, 11:08pm UTC](https://discuss.elastic.co/t/logstash-s3-input-plugin-error/245104/8 "2020-08-15T23:08:59Z")

</div>

Can you try

```
"additional_settings" => {
    "ssl_verify_peer" => false
    "http_wire_trace" => true
}

```

and see if the additional logging provides any clues?

---

<div class="post-metadata">

**Author:** ![Kevin\_f](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kevin_f/32/72456_2.png) [@Kevin\_f](https://discuss.elastic.co/u/Kevin_f)\
**Post date:** [August 16, 2020, 5:14am UTC](https://discuss.elastic.co/t/logstash-s3-input-plugin-error/245104/9 "2020-08-16T05:14:02Z")

</div>

Hi Badger,

Thanks. Unfortunately exact same error as before in the logs.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 16, 2020, 12:59pm UTC](https://discuss.elastic.co/t/logstash-s3-input-plugin-error/245104/10 "2020-08-16T12:59:43Z")

</div>

I expected the same error, but I also expected additional logging. Did you not see any additional logs?

---

<div class="post-metadata">

**Author:** ![Kevin\_f](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kevin_f/32/72456_2.png) [@Kevin\_f](https://discuss.elastic.co/u/Kevin_f)\
**Post date:** [August 16, 2020, 2:04pm UTC](https://discuss.elastic.co/t/logstash-s3-input-plugin-error/245104/11 "2020-08-16T14:04:17Z")

</div>

Hi Badger,

These are the logs.

[2020-08-16T06:55:32,440][ERROR][logstash.javapipeline][main][326a9d448dcf6ccc3a9b4aa774b705c4e99438831cad07ede980820129dfd759] A plugin had an unrecoverable error. Will restart this plugin.  
Pipeline\_id:main  
Plugin: \<LogStash::Inputs::S3 bucket=\>"logs", access\_key\_id=\>"\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*", add\_field=\>{"service"=\>"oci"}, codec=\>\<LogStash::Codecs::Plain id=\>"plain\_aff0212d-8e67-4f31-a0ef-cf7489efc511", enable\_metric=\>true, charset=\>"UTF-8"\>, endpoint=\>"https://\<object\_storage\_namespace\>.compat.objectstorage.uk-london-1.oraclecloud.com", additional\_settings=\>{"ssl\_verify\_peer"=\>"false", "http\_wire\_trace"=\>"true", "force\_path\_style"=\>"true"}, secret\_access\_key=\>, interval=\>300, id=\>"326a9d448dcf6ccc3a9b4aa774b705c4e99438831cad07ede980820129dfd759", region=\>"uk-london-1", delete=\>false, proxy\_uri=\>"[http://x.x.x.x:3128](http://x.x.x.x:3128)", enable\_metric=\>true, role\_session\_name=\>"logstash", watch\_for\_new\_files=\>true, temporary\_directory=\>"/tmp/logstash", include\_object\_properties=\>false, gzip\_pattern=\>".gz(ip)?$"\>  
Error: certificate verify failed  
Exception: Seahorse::Client::NetworkingError  
Stack: uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:1002:in `connect' uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:924:in `do\_start'  
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:919:in `start' uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/delegate.rb:83:in `method\_missing'  
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.501/lib/seahorse/client/net\_http/connection\_pool.rb:285:in `start_session' /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.501/lib/seahorse/client/net_http/connection_pool.rb:92:in `session\_for'  
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.501/lib/seahorse/client/net\_http/handler.rb:119:in `session' /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.501/lib/seahorse/client/net_http/handler.rb:71:in `transmit'  
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.501/lib/seahorse/client/net\_http/handler.rb:45:in `call' /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.501/lib/seahorse/client/plugins/content_length.rb:12:in `call'  
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.501/lib/aws-sdk-core/plugins/s3\_request\_signer.rb:88:in `call' /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.501/lib/aws-sdk-core/plugins/s3_request_signer.rb:23:in `call'

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 16, 2020, 2:45pm UTC](https://discuss.elastic.co/t/logstash-s3-input-plugin-error/245104/12 "2020-08-16T14:45:48Z")

</div>

What version of the s3 input are you running?

```auto
bin/logstash-plugin list --verbose logstash-input-s3

```

---

<div class="post-metadata">

**Author:** ![Kevin\_f](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kevin_f/32/72456_2.png) [@Kevin\_f](https://discuss.elastic.co/u/Kevin_f)\
**Post date:** [August 16, 2020, 3:06pm UTC](https://discuss.elastic.co/t/logstash-s3-input-plugin-error/245104/13 "2020-08-16T15:06:55Z")

</div>

> [@Badger](#):
>
> `n/logstash-plugin list --verbose logstash-input-s3`

logstash-input-s3 (3.5.0)

Which I believe is the latest.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 16, 2020, 3:18pm UTC](https://discuss.elastic.co/t/logstash-s3-input-plugin-error/245104/14 "2020-08-16T15:18:03Z")

</div>

Indeed, and it includes the fixes from 3.3.3. Before that the additional\_settings option was not effective. OK, I have no further suggestions on why those settings appear not to be applied.

---

<div class="post-metadata">

**Author:** ![Kevin\_f](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kevin_f/32/72456_2.png) [@Kevin\_f](https://discuss.elastic.co/u/Kevin_f)\
**Post date:** [August 16, 2020, 3:19pm UTC](https://discuss.elastic.co/t/logstash-s3-input-plugin-error/245104/15 "2020-08-16T15:19:56Z")

</div>

thank you badger.

Though I was wondering if http\_poller plugin would be able to work by calling the oracle cloud bucket api endpoint?

I tried the below, no errors in logstash logs and no result in Kibana ☹

```auto
input {
  http_poller {
    urls => {
      test1 => {
        method => get
        user => " *********"
        password => " *********"
        url => 'https://<bucket_namespace>.compat.objectstorage.uk-london-1.oraclecloud.com'
        #headers => {
        # Accept => "application/json"
        #}
      }
    }
    request_timeout => 60
    codec => "json"
    #schedule => { every => "20s"}
    schedule => { cron => "* * * * * UTC"}
    metadata_target => "http_poller_metadata"
    add_field => { "service" => "oci" }
    proxy => "http://x.x.x.x:3128"
  }
}

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 13, 2020, 3:20pm UTC](https://discuss.elastic.co/t/logstash-s3-input-plugin-error/245104/16 "2020-09-13T15:20:03Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.