Cloudtrail logs parsing with logstash

Elastic Stack Logstash
1

Hi All,

I am trying to parse Amazon Cloudtrail logs to Elasticsearch using logstash. I have my s3 plugin configured and output been set to Elasticsearch but i find out that there is no grok pattern defined as to pass the logs to Elasticsearch.

Has anyone been able to successfully ingest cloudtrail logs with logstash.

I had a look at the post Cloudtrail Codec but that surely didn't help. The installation is never successful.

I am currently using ES 5.2 with Logstash 5.2.

Help!!!

--
Niraj

2

I'm not sure if there is one. S3 buckets can be a lot of different things so it doesn't know how to do it natively.
If the message field holds just JSON data, then you can use the JSON filter to decode it onto separate fields. If it's text you may have to use GROK and do it yourself.

We are ingesting cloudtrail logs, but we are not getting directly via S3. Instead we are using a third party plugin to pull it from cloudwatch.