# Problem with S3 as Input

**URL:** <https://discuss.elastic.co/t/problem-with-s3-as-input/38280>\
**Category:** Logstash\
**Created:** [January 3, 2016, 7:57pm UTC](https://discuss.elastic.co/t/problem-with-s3-as-input/38280 "2016-01-03T19:57:27Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Nitz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nitz/32/14599_2.png) [@Nitz](https://discuss.elastic.co/u/Nitz)\
**Post date:** [January 3, 2016, 7:57pm UTC](https://discuss.elastic.co/t/problem-with-s3-as-input/38280/1 "2016-01-03T19:57:27Z")

</div>

Hi all,

I'm having the following configuration:

input {  
s3 {  
type =\> "cloudtrail"  
bucket =\> $BUCKET\_NAME  
prefix =\> $PREFIX  
access\_key\_id =\> $ACCESS\_KEY\_ID  
secret\_access\_key =\> $SECRET  
region =\> "eu-west-1"  
interval =\> 60  
codec =\> cloudtrail  
}  
}

filter {  
}

output {  
stdout { codec =\> rubydebug }  
}

(Of course instead of the variables with capital letters I have actual parameters there)

**_I don't get any output._**

I ran logstash like this:  
`bin/logstash -f logstash.conf -vvv`  
in order to see what happens.

I keep getting such messages:

S3 input: Found key {:key=\> SOME .JSON.GZ FILE FROM THE BUCKET, :level=\>:debug, :file=\>"logstash/inputs/s3.rb", :line=\>"111", :method=\>"list\_new\_files"}  
S3 input: Adding to objects[] {:key=\> SOME .JSON.GZ FILE FROM THE BUCKET, :level=\>:debug, :file=\>"logstash/inputs/s3.rb", :line=\>"116", :method=\>"list\_new\_files"}

the key (the json.gz file) keeps changing as it appears in the bucket. Meaning, it does scan the bucket but nothing is being outputted (as it should be in stdout).

I'm running Logstash 2.1.1.

Any ideas?  
I'll be glad for any help.

Thanks!

Nitz

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [January 3, 2016, 9:59pm UTC](https://discuss.elastic.co/t/problem-with-s3-as-input/38280/2 "2016-01-03T21:59:05Z")

</div>

At a guess it'd be probably this - [https://www.elastic.co/guide/en/logstash/current/plugins-inputs-s3.html#plugins-inputs-s3-sincedb\_path](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-s3.html#plugins-inputs-s3-sincedb_path)

---

<div class="post-metadata">

**Author:** ![Gregg\_Brzozowski](https://avatars.discourse-cdn.com/v4/letter/g/8edcca/32.png) [@Gregg\_Brzozowski](https://discuss.elastic.co/u/Gregg_Brzozowski)\
**Post date:** [February 7, 2017, 1:41pm UTC](https://discuss.elastic.co/t/problem-with-s3-as-input/38280/3 "2017-02-07T13:41:10Z")

</div>

Nitz, did you ever figure this out? I'm having the same problem.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:28am UTC](https://discuss.elastic.co/t/problem-with-s3-as-input/38280/4 "2017-07-06T04:28:49Z")

</div>