How to search relationship between log lines

Elastic Stack Elasticsearch
1

Hello,
I’m using ElasticSearch to index log files coming from a system. The format of the log files is almost identical to syslog. It starts with a time stamp, then PID and TID of the logging thread, a log level and then the actual log content. Each of these fields is parsed and indexed.

Now users want to answer questions like this. “Consider three log lines L1,L2 and L3. Given a start time stamp and end time stamp, find all instances where L1 is followed by L2 within 10 seconds and then not followed by L3?”

I’m general users want to create arbitrary expressions of follows-by, AND, OR and NOT of log lines. Is such a query natively supported by ES or will I need to write some heavy-processing code that gets all such lines from ES and evaluates the expression?

2

Why it's hard and how to solve it: https://www.youtube.com/watch?v=yBf7oeJKH2Y

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.