Respected Folks,
Good day 
May I please request your help w/ Filebeat conf.? I'm a newbie, so apologies in advance if I'm missing something.
My log file contains messages like,
<=- 20170414 12:58:39.768 PH2O incoming [135.46.166.47] << [135.46.166.47:7778] 3789 bytes
{"type":"eventEstablished","connection":"XXXXX","context":"ContextI",...}
-=>
<=- 20170414 07:04:49.889 PH2O incoming [135.46.166.59] << [135.46.166.59:7778] 2168 bytes
{"type":"eventReleased","connection":"XXXXX","context":"ContextI",...}
-=>
<=- 20170414 13:07:33.109 PH2O outgoing [135.46.166.47] >> [135.46.166.47:7778] 1886 bytes
{ "type": "requestMuteTransfer","connection":"XXXXX","context":"ContextI",...}
-=>
<=- 20170415 08:11:16.731 PH2O incoming [130.6.43.52] << [130.6.43.52:7776] 164 bytes
{ "context":"ContextI","sequence": 3,"type": "eventAuthenticated","content": { "loginid": "ABC123","authenticated": 1 } }
-=>
<=- 20170418 09:05:57.952 PH2O incoming [130.6.43.50] << [130.6.43.50:7778] 279 bytes
{"type":"eventError","connection":"XXXXX","context":"ContextI",...}
-=>
<=- 20170419 11:57:17.119 AH2O incoming [130.6.43.52] << [130.6.43.50:9977] 37 bytes
{"type":"pong","context":"reef2reef"}
-=>
What I'd like to retrieve through Filebeat?
- only those lines with "eventEstablished", "eventReleased", "requestMuteTransfer", "eventError" AND each line immediately above as that contains the date/time
Here's my Filebeat Conf.
document_type: coral_reef
tags: ["json"]
exclude_lines: ["AH2O incoming", "AH2O outgoing", "MH2O incoming", "PluginInitiated", "Administration", "ping", "pong", "eventInfo", "requestRegisterAddress", "hello", "requestAuthenticate", "requestAnswerCall", "RequestOpenStatistic", "eventRegistered", "pong", "ping", "reef2reef"]
include_lines: ["PH2O incoming", "PH2O outgoing"]
multiline:
pattern: ^{[[:space:]]|"type":[[:space:]]|"(eventEstablished|eventReleased|requestMuteTransfer|requestCompleteTransfer|eventAuthenticated|eventAgentLogin|eventAgentLogout|requestAgentLogout|eventHeld|eventRetrieved|goodbye|requestReleaseCall|requestAgentNotReady|eventAgentNotReady|eventError)"
negate: false
match: after
What's happening?
- all messages with the text in "exclude_lines" are not getting published - GOOD
- messages with text in multiline pattern like "eventReleased", "requestCompleteTransfer" are also not getting published - NOT GOOD
- messages that do not contain any of the text in multiline pattern are also getting published - NOT GOOD
Questions,
- What multiline pattern settings do I set to explicitly match messages with the keywords "eventEstablished ,eventReleased ,requestMuteTransfer etc.) and append the line above?
Thanks in advance for your time, patience and cooperation Wish you the very best.