Logstash pipeline for aws cloudfront fixing timestamp issue

Elastic Stack Logstash
1

This took me ages to figure out so I thought it might be helpful for someone else.

I'm parsing AWS CloudFront standard logs in logstash (v8.x)

The included grok pattern worked fine for me apart from the timestamp, since that uses a tab \t character to separate date and time and I couldn't get the date filter to parse that correcly.

The working solution:

filter {
  grok {
    match => {
      "message" => "%{CLOUDFRONT_ACCESS_LOG}"
    }
  }
  mutate {
    gsub => [
      "timestamp", "\t", " "
    ]
  }

  date {
    locale => "en"
    timezone => "UTC"
    match => [ "timestamp", "yyyy-MM-dd HH:mm:ss" ]
    target => "@timestamp"
  }

  geoip {
    source => "[source][ip]"
    target => "[client]"
  }

}

I've also left in the geoip lookup.

If there is a better way of doing this let me know

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.