# Using S3 as input plugin

**URL:** <https://discuss.elastic.co/t/using-s3-as-input-plugin/28725>\
**Category:** Logstash\
**Created:** [September 5, 2015, 7:35pm UTC](https://discuss.elastic.co/t/using-s3-as-input-plugin/28725 "2015-09-05T19:35:38Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![niraj\_kumar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/niraj_kumar/32/4130_2.png) [@niraj\_kumar](https://discuss.elastic.co/u/niraj_kumar)\
**Post date:** [September 5, 2015, 7:35pm UTC](https://discuss.elastic.co/t/using-s3-as-input-plugin/28725/1 "2015-09-05T19:35:39Z")

</div>

Hi,

I am using s3 as the input plugin to download the logs in s3 bucket. The logs in my s3 bucket is organized in folders with a .tar.gz file in each of them. Can the S3 plugin help me achieve this. The .tar.gz file contains a JSON file.

I have used the following but somehow i don't seem to get anything in kibana.

input {  
s3 {  
bucket =\> "dev-metrics"  
access\_key\_id =\> "xxxxxxxxx"  
secret\_access\_key =\> "xxxxxxxxxxxxxxx"  
region =\> "us-east-1"  
sincedb\_path =\> "./last-s3-file"  
codec =\> "json"  
}  
}  
output {  
stdout {}  
elasticsearch\_http {  
host =\> "localhost"  
port =\> "9200"  
}  
}

Can anyone help me achieve this.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [September 5, 2015, 8:12pm UTC](https://discuss.elastic.co/t/using-s3-as-input-plugin/28725/2 "2015-09-05T20:12:14Z")

</div>

The S3 input supports gzipped plain files but not tarballs. You'll have to write a separate script that pulls the tarballs and unpacks them into a local directory that you can have Logstash monitor.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:29am UTC](https://discuss.elastic.co/t/using-s3-as-input-plugin/28725/3 "2017-07-06T05:29:57Z")

</div>