Dynamically create nested objects from a log record

i'm having trouble wrapping my head around how i would go about transforming some items in a log line into nested objects (or whether it's even worthwhile)

its a custom NGINX log that logs upstream connections and the time to response, and there are times when i end up with multiple upstream destinations and response times in one log line, like this:

2019-12-10T12:23:23-05:00 |, | [0.06, 0.07] | api.whatever.com

Where: and response time 0.06 are associated and and response time 0.07 are associated

so I was thinking an array of nested objects would be ideal here.

@timestamp: "2019-12-10T12:23:23-05:00", nginx.gateway.upstream: [ { ip:, response_time: 0.06 }, { ip:, response_time: 0.07 } ], host: "api.whatever.com"

when i parse the log, I can parse the IPs into a list variable: nginx.gateway.upstream.ip_list
and the response_times into a similar list: nginx.gateway.upstream.response_time_list

but i'm not sure how to go from those lists to nested objects. log entries may have only one upstream, or multiple.

i think its worth maintaining the associations so i can trace delays back to the appropriate upstream source.

i guess i should add that I'm using an ingest pipeline to do the parsing and transformations.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.