Hi, i am new in elastic and i am working to send logs from filebeat to logstash, but i have a issue and i not found solution. the log have multiple related lines ID, information and status, i would like join them in single line to process more easy in elastic.
Please can you help me with my questions:
Is possible to do that i want?
what will be the procedure?
Are these lines 'intermixed' or always consecutive?
Any kind of start/stop pattern?
Hi, steffens, these lines are intermixed but always have the same pattern, for example
i could exclude the lines that not are relevant for me.
i would like have a single line with ID + status
Due to the lines being intermixed, but having an ID this is no plain multiline problem, but a correlation problem. You can try to solve this via aggregate filter in Logstash.
Hi, but how i can related the lines with the specific id, i have other line that say me if the record is inbound or outbound, a example is this
Assuming 3 events and lines being intermixed your log might look like this at worst:
The problem with this is: you can't even rely on order, as event_3 might surpass same lines from event_1 or event_2. In order to untangle these you need a correlation ID.
Your example suggest events are not intermixed, but written atomically. Like always:
If events are indeed written atomically, you can actually use multiline. Create a pattern looking for
^ID: (assuming ID is always the first entry) and use
match: after. See the table in the multiline docs.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.