# S3 bucket data not processing correctly

**URL:** <https://discuss.elastic.co/t/s3-bucket-data-not-processing-correctly/103356>\
**Category:** Logstash\
**Created:** [October 10, 2017, 11:40am UTC](https://discuss.elastic.co/t/s3-bucket-data-not-processing-correctly/103356 "2017-10-10T11:40:31Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sunil\_M](https://avatars.discourse-cdn.com/v4/letter/s/9f8e36/32.png) [@Sunil\_M](https://discuss.elastic.co/u/Sunil_M)\
**Post date:** [October 10, 2017, 11:40am UTC](https://discuss.elastic.co/t/s3-bucket-data-not-processing-correctly/103356/1 "2017-10-10T11:40:31Z")

</div>

Hi All,

I have logstash code which reads the S3 bucket data.  
The data in the bucket was like  
{"Request":{"test":"BLABLA"},"timestamp":"1503404629272"}}  
{"Request":{"test":"BLABLA"},"timestamp":"1503404629272"}}

I use the code  
s3 {  
access\_key\_id =\> "KKKKK"  
bucket =\> "logs"  
region =\> "us-east-1"  
secret\_access\_key =\> "SESESE"  
prefix =\> "test\_"  
sincedb\_path =\> "/dev/null"  
codec =\> "json"  
delete =\> true  
}

I was able to get the data properly and was able to process correctly. Used to get required data as a single line.

But now the data in S3 bucket looks like the below. Its valid json. but seems like formatted.  
{  
"Request": {  
"test": "BLABLA",  
},  
"timestamp": "1506398327458"  
}

Due to this change in format the  
stdout { codec =\> rubydebug }  
gives me each line output for each line of data.  
How can i process the above data in logstash?

Help appreciated

Thanks

---

<div class="post-metadata">

**Author:** ![lueneburger](https://avatars.discourse-cdn.com/v4/letter/l/f475e1/32.png) [@lueneburger](https://discuss.elastic.co/u/lueneburger)\
**Post date:** [October 10, 2017, 3:03pm UTC](https://discuss.elastic.co/t/s3-bucket-data-not-processing-correctly/103356/2 "2017-10-10T15:03:00Z")

</div>

Hi @Sunil_M

try the filter plugin for json.

[Json Filter Plugin](https://www.elastic.co/guide/en/logstash/current/plugins-filters-json.html)

Cheers,  
Dirk

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 7, 2017, 3:03pm UTC](https://discuss.elastic.co/t/s3-bucket-data-not-processing-correctly/103356/3 "2017-11-07T15:03:06Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.