With that config, you are parsing each line of JSON separately, which does not work. You need a multiline code on the input. Something like
input{
stdin{
codec => multiline {
pattern => "^}"
negate => true
what => next
max_lines => 20000
}
}
}
Then to parse it the following appears to do what you want
filter{
mutate { gsub => ["message", "\n", ""] }
json{ source => "message" }
split{ field => "Records" }
}
Lastly I would say that debugging this is much easier with a
output{
stdout{ codec=>"rubydebug" }
}
Personally I would add a filter to delete the message field provided the json parse works (i.e. no parse failure tag)
if "_jsonparsefailure" not in [tags] { mutate { remove_field => "message" } }