# Read aws cloud trail log from s3 bucket

**URL:** <https://discuss.elastic.co/t/read-aws-cloud-trail-log-from-s3-bucket/44820>\
**Category:** Logstash\
**Created:** [March 18, 2016, 9:28am UTC](https://discuss.elastic.co/t/read-aws-cloud-trail-log-from-s3-bucket/44820 "2016-03-18T09:28:46Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![mohankgm](https://avatars.discourse-cdn.com/v4/letter/m/f1d935/32.png) [@mohankgm](https://discuss.elastic.co/u/mohankgm)\
**Post date:** [March 18, 2016, 9:28am UTC](https://discuss.elastic.co/t/read-aws-cloud-trail-log-from-s3-bucket/44820/1 "2016-03-18T09:28:46Z")

</div>

Hi,

I'm reading the cloud trail log s from s3 bucket using the blow code. But only I getting log adding this  
" prefix =\> "AWSLogs/xxxxxxxxx/CloudTrail/us-west-2/2016/03/18""  
How get the log recursively from S3 subfolder.

input {  
s3 {  
access\_key\_id =\> ""  
bucket =\> "mohantestaws"  
secret\_access\_key =\> ""  
prefix =\> "AWSLogs/xxxxxxxxx/CloudTrail/us-west-2/2016/03/18"  
sincedb\_path =\> "/tmp/s3.sincedb"  
region =\> "us-west-2"  
codec =\> "json"  
add\_field =\> { source =\> gzfiles }  
}  
}  
filter {  
if [type] == "cloudtrail" {  
mutate {  
gsub =\> ["eventSource", ".amazonaws.com$", ""]  
add\_field =\> {  
"document\_id" =\> "%{eventID}"  
}  
}  
if ! [ingest\_time] {  
ruby {  
code =\> "event['ingest\_time'] = Time.now.utc.strftime '%FT%TZ'"  
}  
}  
ruby {  
code =\> "event.cancel if (Time.now.to\_f - event['@timestamp'].to\_f) \> (60 \* 60 \* 24 \* 1)"  
}  
ruby {  
code =\> "event['ingest\_delay\_hours'] = (Time.now.to\_f - event['@timestamp'].to\_f) / 3600"  
}

if [ingest\_delay\_hours] \> 24 {  
drop {}  
}

```
if [eventSource] == "elasticloadbalancing" and [eventName] == "describeInstanceHealth" and [userIdentity.userName] == "deploy-s3" {
  drop {}

```

}  
}  
}  
output {  
elasticsearch {  
hosts =\> localhost  
}  
stdout { codec =\> json }  
}

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [March 20, 2016, 12:29am UTC](https://discuss.elastic.co/t/read-aws-cloud-trail-log-from-s3-bucket/44820/2 "2016-03-20T00:29:01Z")

</div>

My understanding is that you cannot do recursive reads like this.  
The docs also don't mention this as being possible - [https://www.elastic.co/guide/en/logstash/current/plugins-inputs-s3.html](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-s3.html)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:06am UTC](https://discuss.elastic.co/t/read-aws-cloud-trail-log-from-s3-bucket/44820/3 "2017-07-06T05:06:16Z")

</div>