Can Logstash Base64 decode then parse json from a cloudfront log?

[chrisan]

Hello, we are using AWS's image resizer Serverless Image Handler | Implementations | AWS Solutions

To get an image, you need to base64 encode a json object such as

{"bucket":"my-s3-bucket","key":"path/to/your/image.jpg","edits":{"resize":{"width":200,"fit":"cover"}}}

which becomes eyJidWNrZXQiOiJteS1zMy1idWNrZXQiLCJrZXkiOiJwYXRoL3RvL3lvdXIvaW1hZ2UuanBnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjoyMDAsImZpdCI6ImNvdmVyIn19fQ==

So you would end up with a url of https://random-letters.cloudfront.net/eyJidWNrZXQiOiJteS1zMy1idWNrZXQiLCJrZXkiOiJwYXRoL3RvL3lvdXIvaW1hZ2UuanBnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjoyMDAsImZpdCI6ImNvdmVyIn19fQ== This url is what ends up in the cloudfront logs that are delivered to the s3 bucket.

What I'd like to do is have Logstash grab these cloudfront logs, decode url, and grab the "real" url path/to/your/image.jpg for Elasticsearch

[Badger]

You can use

    grok { match => { "url" => "/(?<[@metadata][uri]>[^/]*)$" } }
    ruby {
        code => '
            uri = event.get("[@metadata][uri]")
            if uri
                event.set("[@metadata][json]", Base64.decode64(uri))
            end
        '
    }
    json { source => "[@metadata][json]" target => "[@metadata][stuff]" }
    mutate { add_field => { "someField" => "%{[@metadata][stuff][key]}" } }

to process

       "url" => "https://random-letters.cloudfront.net/eyJidWNrZXQiOiJteS1zMy1idWNrZXQiLCJrZXkiOiJwYXRoL3RvL3lvdXIvaW1hZ2UuanBnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjoyMDAsImZpdCI6ImNvdmVyIn19fQ=="

and extract

 "someField" => "path/to/your/image.jpg",

[chrisan]

thank you, worked great!