Filebeat Multiline for logs starting in the same way

Elastic Stack Beats
1

Hello everybody,
I'm new to the community and I wanted to ask you a question. I have a log file that is sent to Logstash through Filebeat.
This log file is composed of lines like this:

2019.08.09 11:19:22 INFO [socket_c2s-thread-3]: com.plugin.apns.ApnsPlugin - Type :chat
2019.08.09 11:19:22 INFO [socket_c2s-thread-3]: com.plugin.apns.ApnsPlugin - from :8@srv..in/8
2019.08.09 11:19:22 INFO [socket_c2s-thread-3]: com.plugin.apns.ApnsPlugin - to :11@srv..in
2019.08.09 11:19:22 INFO [socket_c2s-thread-3]: com.plugin.apns.ApnsPlugin - MID :34B97E83785E
2019.08.09 11:19:22 INFO [socket_c2s-thread-3]: com.plugin.apns.ApnsPlugin - TID :CB0A4DA49BA1
2019.08.09 11:19:22 INFO [socket_c2s-thread-3]: com.plugin.apns.ApnsPlugin - body :hidden:q1bKTFGyUnJ3zigtVdJRKi5JLCktBgoUpSamAPkllQWpQF5ucTpUphYA
2019.08.09 11:19:22 ERROR [socket_c2s-thread-3]: android.util.Log - [HttpsUtils] conn.getResponseCode() = 200
2019.08.09 11:19:22 INFO [socket_c2s-thread-3]: com.plugin.apns.ApnsPlugin - RISPOSTA:{"ver":"1","err":0,"errd":"","ts":"1565342362","aw":"1"}
2019.08.09 11:19:22 INFO [socket_c2s-thread-3]: com.plugin.apns.ApnsPlugin - MESSAGE : hidden:q1bKTFGyUvLPLcvMVtJRKi5JLCktBgoUpS
2019.08.09 11:32:38 INFO [socket_c2s-thread-4]: com.plugin.apns.ApnsPlugin - Type :chat
2019.08.09 11:32:38 INFO [socket_c2s-thread-4]: com.plugin.apns.ApnsPlugin - from :8@srv.in/8
2019.08.09 11:32:38 INFO [socket_c2s-thread-4]: com.plugin.apns.ApnsPlugin - to :11@srv.in
2019.08.09 11:32:38 INFO [socket_c2s-thread-4]: com.plugin.apns.ApnsPlugin - MID :65AD0A4AC022
2019.08.09 11:32:38 INFO [socket_c2s-thread-4]: com.plugin.apns.ApnsPlugin - TID :4217AE6787FD
2019.08.09 11:32:38 INFO [socket_c2s-thread-4]: com.plugin.apns.ApnsPlugin - body :hidden:q1bKTFGyUvLPLcvMVtJRKi5JLCktBgoUpSamAPkllQWpQF5ucTpUphYA
2019.08.09 11:32:38 ERROR [socket_c2s-thread-4]: android.util.Log - [HttpsUtils] conn.getResponseCode() = 200
2019.08.09 11:32:38 INFO [socket_c2s-thread-4]: com.plugin.apns.ApnsPlugin - RISPOSTA:{"ver":"1","err":0,"errd":"","ts":"1565343158","aw":"1"}
2019.08.09 11:32:38 INFO [socket_c2s-thread-3]: com.plugin.apns.ApnsPlugin - MESSAGE : hidden:q1bKTFGyUvLPLcvMVtJRKi5JLCktBgoUpSamAPkllQWpQF5ucTpUphY

The log itself it's made up from 8 lines.
How can I manage this logs with Multiline method? I have some troubles because every line is beginning with a timestamp and I don't know how to merge them in groups of 8 lines.
Any ideas?

Thank you very much

2

Unfortunately, it's not possible to do this using Filebeat. I am not a Logstash export, but I think you can aggregate lines as you wish using Logstash: https://www.elastic.co/guide/en/logstash/current/plugins-filters-aggregate.html

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.