S3 bucket data not processing correctly

Elastic Stack Logstash
1

Hi All,

I have logstash code which reads the S3 bucket data.
The data in the bucket was like
{"Request":{"test":"BLABLA"},"timestamp":"1503404629272"}}
{"Request":{"test":"BLABLA"},"timestamp":"1503404629272"}}

I use the code
s3 {
access_key_id => "KKKKK"
bucket => "logs"
region => "us-east-1"
secret_access_key => "SESESE"
prefix => "test_"
sincedb_path => "/dev/null"
codec => "json"
delete => true
}

I was able to get the data properly and was able to process correctly. Used to get required data as a single line.

But now the data in S3 bucket looks like the below. Its valid json. but seems like formatted.
{
"Request": {
"test": "BLABLA",
},
"timestamp": "1506398327458"
}

Due to this change in format the
stdout { codec => rubydebug }
gives me each line output for each line of data.
How can i process the above data in logstash?

Help appreciated

Thanks

2

Hi @Sunil_M

try the filter plugin for json.

Json Filter Plugin

Cheers,
Dirk

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.