S3 bucket data not processing correctly

Hi All,

I have logstash code which reads the S3 bucket data.
The data in the bucket was like

I use the code
s3 {
access_key_id => "KKKKK"
bucket => "logs"
region => "us-east-1"
secret_access_key => "SESESE"
prefix => "test_"
sincedb_path => "/dev/null"
codec => "json"
delete => true

I was able to get the data properly and was able to process correctly. Used to get required data as a single line.

But now the data in S3 bucket looks like the below. Its valid json. but seems like formatted.
"Request": {
"test": "BLABLA",
"timestamp": "1506398327458"

Due to this change in format the
stdout { codec => rubydebug }
gives me each line output for each line of data.
How can i process the above data in logstash?

Help appreciated


Hi @Sunil_M

try the filter plugin for json.

Json Filter Plugin


This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.