# Logstash s3 Plugin error

**URL:** <https://discuss.elastic.co/t/logstash-s3-plugin-error/60025>\
**Category:** Logstash\
**Created:** [September 8, 2016, 1:06am UTC](https://discuss.elastic.co/t/logstash-s3-plugin-error/60025 "2016-09-08T01:06:54Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Capt\_X](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/capt_x/32/11801_2.png) [@Capt\_X](https://discuss.elastic.co/u/Capt_X)\
**Post date:** [September 8, 2016, 1:06am UTC](https://discuss.elastic.co/t/logstash-s3-plugin-error/60025/1 "2016-09-08T01:06:54Z")

</div>

**Environment:**  
RHEL 6.8  
Logstash 2.4.0  
Kibana 4.5.4  
Amazon ElasticSearch 2.3

I've researched the error I am getting and while many have seen similar errors, I haven't found someone with the exact same error. I've tried numerous configs that worked for other people, but are not working for me. I was originally using Logstash 2.3.x, but upgraded to 2.4 hoping that might fix my problem, but I am still getting the same error.

I'm trying to ingest CloudTrail logs using the s3 plugin, cloudtrail codec and amazon\_es output plugin. I am getting the following error:

Settings: Default pipeline workers: 2  
Pipeline main started  
A plugin had an unrecoverable error. Will restart this plugin.  
Plugin: \<LogStash::Inputs::S3 bucket=\>"aws-logs", delete=\>false, interval=\>60, prefix=\>"AWSLogs/xxxxxxxxxxxx/CloudTrail-Digest/us-east-1/", type=\>"cloudtrail", codec=\>\<LogStash::Codecs::CloudTrail spool\_size=\>50\>, sincedb\_path=\>"/opt/logstash/sincedb", region=\>"us-east-1", use\_ssl=\>true, temporary\_directory=\>"/tmp/logstash"\>  
_Error: undefined method `each' for nil:NilClass {:level=\>:error}_

My config file is (I'm using an IAM role with full perms to s3, hence no creds in conf file):  
input{  
s3 {  
bucket =\> "aws-logs"  
region =\> "us-east-1"  
codec =\> "cloudtrail"  
type =\> "cloudtrail"  
prefix =\> "AWSLogs/xxxxxxxxxxxx/CloudTrail-Digest/us-east-1"  
}  
}  
filter {  
if [type] == "cloudtrail" {  
grok {  
match =\> ["@message", ""eventVersion":"%{GREEDYDATA:eventVersion}","errorCode":"%{GREEDYDATA:errorCode}","eventTime":"%{GREEDYDATA:eventTime}","requestParameters":"%{GREEDYDATA:requestParameters}","errorMessage":"%{GREEDYDATA:errorMessage}","responseElements":"%{GREEDYDATA:responseElements}","eventName":"%{GREEDYDATA:eventName}","userIdentity":"%{GREEDYDATA:userIdentity}","eventSource":"%{GREEDYDATA:eventSource}","userAgent":"%{GREEDYDATA:userAgent}","sourceIPAddress":"%{GREEDYDATA:sourceIPAddress}""]  
}  
geoip {  
source =\> "sourceIPAddress"  
target =\> "geoip"  
database =\> "/etc/logstash/GeoLiteCity.dat"  
add\_field =\> ["[geoip][coordinates]", "%{[geoip][longitude]}" ]  
add\_field =\> ["[geoip][coordinates]", "%{[geoip][latitude]}" ]  
}  
mutate {  
convert =\> ["[geoip][coordinates]", "float"]  
}  
}  
}  
output{  
amazon\_es {  
hosts =\> "[search-myesdomain.us-east-1.es.amazonaws.com](http://search-myesdomain.us-east-1.es.amazonaws.com)"  
region =\> "us-east-1"  
index =\> "cwl-%{+YYYY.MM.dd}"  
flush\_size =\> 250000  
port =\> 80  
protocol =\> "http"

I also have a Lambda function that pushes cloudtrail logs to Amazon ES and that works great, but when I turn off the lambda function and use Logstash it breaks. The purpose for using Logstash is so I can add geoip information to my logs. I don't know how to do that with Lambda.

I have gotten some logs in to ES with Logstash using this config, but it is usually a blip and then stops pushing data to ES.

Any help is much appreciated!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:39am UTC](https://discuss.elastic.co/t/logstash-s3-plugin-error/60025/2 "2017-07-06T04:39:24Z")

</div>