we are using the classic ELK stack, mostly with Log4J messages. We have an average of about 200 log entries/second. We have been using syslog-ng with log rotation and still have that around. I appreciate the faceting, statistics and query capabilities of Kibana. However, there is one major use case for us for which I cannot find a good solution in Kibana, and that is:
- search for a message indicating or identifying a particular problem
- "look around", i.e. before and after, that message to see root causes and other things going on for that process
Now with a traditional log file + less this is very easy, using the search function in less I can get to the line I need and then I can just scroll up and down to see other messages at that time. If I searched for example for a session ID, this ID will be highlighted in less which makes it easy to see what happened in the session, but also any concurrent processes that might have interferred with anything.
With Kibana, I can find the message I look for very efficiently, but then "zooming out", i.e. checking what happened before and after that is very cumbersome, so I could adjust the time range to around that line and then remove the search query, but then how would I find and highlight the stuff that's interesting for me?
Is there any functionality that allows me to "scroll" through results while highlighting certain terms? Or can this "find message, see what was going on at that time" use case be done in Kibana?