Im trying to combine the following two log lines:
<133>Jul 27 14:36:19 mta.local amavis[6318]: (06318-14) Passed CLEAN {RelayedOutbound}, ORIGINATING LOCAL [1.1.1.1]:42718 <sender@xyz.ch> -> <marcneau@xyz.com>,<marco.sargious@xyz.com>,<marcorellisjohnson@xyz.com>,<marcosmarcomarc@xyz.com>,<marcumyoshiko@xyz.com>,<marcus_pinkney@xyz.com>,<marcuscollins2006@xyz.com>,<marcusjohnson3324@xyz.com>,<marcusl_ford@xyz.com>,<marcuslilskinny@xyz.com>,<marcusloveschauncy@xyz.com>,<marcusnr@xyz.com>,<marcusreaves83@xyz.com>,<marcussnoopyfrancis32@xyz.com>,<marcustrotter14@xyz.com>,<marcy_v85@xyz.com>,<mardelllacy@xyz.com>,<mardhand@xyz.com>,<mare53503@xyz.com>,<margarescott8903@xyz.com>,<margaret.benham@xyz.com>,<margaret_ugorcak@xyz.com>,<margaretcalh59@xyz.com>,<margaretcrittenden@xyz.com>,<margaretj_somber@xyz.com>,<margaretmfcc@xyz.com>,<margaretnyorkor@xyz.com>,<margaretp_smith@xyz.com>,<margaritagonzales17@xyz.com>,<margate4life@xyz.com>,<marge.snee@xyz.com>,<margiebear95@xyz.com>,<ma...
<133>Jul 27 14:36:19 mta.local amavis[6318]: (06318-14) ...rgieeubanks@xyz.com>,<margo.mcmaster@xyz.com>,<margurerite_pulliam@xyz.com>,<marhernandez1991@xyz.com>,<mari28roby@xyz.com>,<mari_mendez0322@xyz.com>,<maria.furtado@xyz.com>,<maria.krug@xyz.com>,<maria.rivera208@xyz.com>,<maria.seaberry@xyz.com>,<mariacrisanchez@xyz.com>,<mariadelaro@xyz.com>,<mariadlange@xyz.com>,<mariah.gayles@xyz.com>,<mariahhudleston@xyz.com>,<mariahulings@xyz.com>,<madelineandino@ymail.com>,<madelinep@ymail.com>, Queue-ID: 9F03619F4EE, Message-ID: <206732994.2427404@id>, mail_id: y11pcrA8D_Ey, Hits: -, size: 15716, queued_as: DA3261A0774, 877 ms
What is the correct way to achieve this?
multi line filter is not thread safe/deprecated, do I need to use the multi line codec?
I have a 3 node cluster with a central redis server as input, do I need to configure a specific multiline on the "global" input? (There are way more logs than just amavis....)
input {
redis {
host => "central-redis.local"
port => 6379
data_type => "list"
key => "logstash"
type => syslog
threads => 4
codec => multiline {
pattern => "%{AMAVISRESULT_MULTILINE2}"
negate => "false"
what => "previous"
patterns_dir => ["/etc/logstash/pattern"]
}
}
}
grok:
AMAVISRESULT_MULTILINE "%{AMAVIS_THREAD}%{SPACE}..."
AMAVISRESULT_MULTILINE2 (?m)(?:%{SYSLOGTIMESTAMP:syslog_timestamp}|%{DATA}%{DATESTAMP:syslog_timestamp}) %{SYSLOGHOST:syslog_hostname} %{SYSLOGPROG:syslog_program}(?:[%{POSINT:[syslog]pid}])?:(?:(%{SPACE}))%{AMAVISRESULT_MULTILINE}