Getting lines of a log that surround an error message with kibana

[tnibbe]

Hello I am trying to use the ELK stack for error diagnosis with some testing output logs. I have a log structured as follows.

DateTime                  MessageType     MessageOrigin           Message
2021-10-28T00:11:16.385Z  INFO            fmwk.bll.base.task     Received signal KILL

The message type defines whether the message is an error, debug, info etc. What I am trying to do is get the 20 or so lines immediately preceding and following when an error message is found. I know I can't get the line number from the original log file, so I have just been using the log.offset which is fine since it doesn't have to be exactly 20 lines. I can filter by messageType and get the offsets of the error messages fairly easily but I don't know how to then get a the group of messages around those offsets.

So basically what I would like is to get the offset of an error message and then check for all messages in a +/- 4000ish range from it to get the context of the error message. Is there a way to chain queries together like that in KQL where the output of one query becomes the input of another?

[tsullivan (Tim Sullivan)]

@tnibbe

What I am trying to do is get the 20 or so lines immediately preceding and following when an error message is found

You can do this with "View Surrounding Fields" in Discover:

I'm not sure there is a way to do it in a query in Kibana. You would need the set of ranges that include error documents ahead of time. Maybe you could do this in a Python script where it re-indexes the documents adjacent to the errors.