# Error on S3 plugin accessing ELB logs - Logstash version 5

**URL:** https://discuss.elastic.co/t/error-on-s3-plugin-accessing-elb-logs-logstash-version-5/67663
**Category:** Logstash
**Created:** [November 30, 2016, 6:42pm UTC](https://discuss.elastic.co/t/error-on-s3-plugin-accessing-elb-logs-logstash-version-5/67663 "2016-11-30T18:42:52Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![Jimmy\_Etimos](https://avatars.discourse-cdn.com/v4/letter/j/d6d6ee/32.png) [@Jimmy\_Etimos](https://discuss.elastic.co/u/Jimmy_Etimos)
#### Post date: [November 30, 2016, 6:42pm UTC](https://discuss.elastic.co/t/error-on-s3-plugin-accessing-elb-logs-logstash-version-5/67663/1 "2016-11-30T18:42:52Z")

</div>

I am getting the following error while trying to read ELB logs from an S3 bucket.  
The instance running logstash is in same region a bucket and booted with a role that gives it full S3 access.

Error: can't convert nil into String  
[2016-11-30T18:35:47,828][ERROR][logstash.pipeline] A plugin had an unrecoverable error. Will restart this plugin

Here is my conf file:

input {  
s3 {  
type =\> "elb"  
bucket =\> "\*\*\*-test"  
region =\> "us-east-1"  
# use\_ssl =\> false  
}  
}

filter {  
if [type] == "elb" {  
grok {  
match =\> ["message", "%{TIMESTAMP\_ISO8601:timestamp} %{NOTSPACE:loadbalancer} %{IP:client\_ip}:%{NUMBER:client\_port:int} %{IP:backend\_ip}:%{NUMBER:backend\_port:int} %{NUMBER:request\_processing\_time:float} %{NUMBER:backend\_processing\_time:float} %{NUMBER:response\_processing\_time:float} %{NUMBER:elb\_status\_code:int} %{NUMBER:backend\_status\_code:int} %{NUMBER:received\_bytes:int} %{NUMBER:sent\_bytes:int} %{QS:request}"]  
}  
date {  
match =\> ["timestamp", "ISO8601"]  
}  
Add geolocalization attributes based on ip.  
geoip {  
source =\> "ip"  
}  
}  
}

output {  
elasticsearch {  
hosts =\> ["127.0.0.1:9200"]  
# sniffing =\> true  
manage\_template =\> false  
index =\> "elb-%{+YYYY.MM.dd}"  
# document\_type =\> "%{[@metadata][type]}"  
}

When I try to set the use\_ssl parameter, I also get an error:

[2016-11-30T18:14:44,310][ERROR][logstash.inputs.s3] Unknown setting 'use\_ssl' for s3  
... :reason=\>"Something is wrong with your configuration."}

---

<div class="post-metadata">

### Author: ![Evan\_Ravenelle](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/evan_ravenelle/32/45404_2.png) [@Evan\_Ravenelle](https://discuss.elastic.co/u/Evan_Ravenelle)
#### Post date: [November 30, 2016, 9:19pm UTC](https://discuss.elastic.co/t/error-on-s3-plugin-accessing-elb-logs-logstash-version-5/67663/2 "2016-11-30T21:19:55Z")

</div>

I was getting the same error until I added prefix =\> to the input:

```
input {
  s3 {
    access_key_id => [" ****"]
    secret_access_key => [" ****"]
    bucket => [" **** -access-logs"]
    type => ["elbaccesslog"]
    prefix => ["acct_number_here.*"]
  }

```

EDIT: It turns out that I read the s3 input plugin doc too quickly, as regexp is _not_ supported, so the above will not work. However, I would expect the below to work, and it does not.

```
input {
      s3 {
        access_key_id => [" ****"]
        secret_access_key => [" ****"]
        bucket => [" **** -access-logs"]
        type => ["elbaccesslog"]
        prefix => ["/path/to/individual/elb/log/2016/11/1/"]
      }

```

Both configs resolved the errors in the logstash log but I'm still not getting any logs read from my ELB logging bucket.

---

<div class="post-metadata">

### Author: ![Jimmy\_Etimos](https://avatars.discourse-cdn.com/v4/letter/j/d6d6ee/32.png) [@Jimmy\_Etimos](https://discuss.elastic.co/u/Jimmy_Etimos)
#### Post date: [November 30, 2016, 9:56pm UTC](https://discuss.elastic.co/t/error-on-s3-plugin-accessing-elb-logs-logstash-version-5/67663/3 "2016-11-30T21:56:26Z")

</div>

I have tried that before as well, can you verify your path is right?

If I add this:  
prefix =\> "AWSLogs/MYACCCOUNTNUMBER/elasticloadbalancing/us-east-1/"  
I get the same error

but if I make a "mistake":  
prefix =\> "AWSLogs/MYACCCOUNTNUMBER/elasticloadbalancing/us-west-1/"  
then the error disappears, but I am guessing its because it doesn't find anything to process

Any luck otherwise?

---

<div class="post-metadata">

### Author: ![Evan\_Ravenelle](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/evan_ravenelle/32/45404_2.png) [@Evan\_Ravenelle](https://discuss.elastic.co/u/Evan_Ravenelle)
#### Post date: [November 30, 2016, 9:58pm UTC](https://discuss.elastic.co/t/error-on-s3-plugin-accessing-elb-logs-logstash-version-5/67663/4 "2016-11-30T21:58:25Z")

</div>

No such luck. I just found the same thing you did -- when the path is accurate I get the `can't convert nil to String` message but when it doesn't match I get no error at all and, obviously, nothing in Elasticsearch.

---

<div class="post-metadata">

### Author: ![Jimmy\_Etimos](https://avatars.discourse-cdn.com/v4/letter/j/d6d6ee/32.png) [@Jimmy\_Etimos](https://discuss.elastic.co/u/Jimmy_Etimos)
#### Post date: [December 1, 2016, 4:50am UTC](https://discuss.elastic.co/t/error-on-s3-plugin-accessing-elb-logs-logstash-version-5/67663/5 "2016-12-01T04:50:56Z")

</div>

I think I got it. Adding a few more fields to the configuration seems to get it working.

```
s3 {
        type => "elb"
        bucket => " **** -test"
        region => "us-east-1"
        prefix => "AWSLogs/MYACCID/elasticloadbalancing/us-east-1/"
        interval => 20 # seconds
        sincedb_path => "/tmp/s3.sincedbappid"
        backup_add_prefix => "old/logstash-"
        backup_to_bucket => "jmfamily-test"
        delete => true
        codec => plain
}

```

I am pretty sure it is the **sincedb\_path** though. I turned on debugging on the logstash.yml settings file and it pointed to that function in s3 rb plugin.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 29, 2016, 4:51am UTC](https://discuss.elastic.co/t/error-on-s3-plugin-accessing-elb-logs-logstash-version-5/67663/6 "2016-12-29T04:51:11Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
