Using aggregate to add modsecurity data to previous event

[admin_berlin (Tolga)]

Hi, I have a modsec logfile that looks like this:

--8afa774c-A--
[24/Feb/2023:04:34:53 +0100] Y-WoWiTsAtEsT123 12.139.152.111 14327 10.29.14.193 8080
--8afa774c-B--
POST /its/a/test/dude HTTP/1.1
Host: www.my-site.de
Content-Type: text/plain
Origin: https://www.my-site.com
Connection: keep-alive
Accept: */*
User-Agent: Mozilla/5.0 (iPhone; CPU iPhone OS 16_1_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.1 Mobile/15E148 Safari/604.1
Referer: https://www.my-site.com/
Content-Length: 129
Accept-Language: de-DE,de;q=0.9
X-Forwarded-For: 123.123.123.123
WL-Proxy-Client-IP: 123.123.123.123
Cookie: BIGipServerpool_pfe_https=!WoWiTsAtEsT123=; jsessionid=WoWiTsAtEsT123-WoWiTsAtEsT123-WoWiTsAtEsT123-WoWiTsAtEsT123!WoWiTsAtEsT123!8001!-1!-123123123!WoWiTsAtEsT123!8001!-1

--8afa774c-C--
data..

--8afa774c-F--
data..

--8afa774c-H--
data..

--8afa774c-Z--

We cannot use the JSON format for ModSecurity. Is there a way to combine this data with the respective header in logstash(--8afa774c-A--)?

We would like the following format:

--8afa774c-A-- [24/Feb/2023:04:34:53 +0100] Y-WoWiTsAtEsT123 12.139.152.111 14327 ...
--8afa774c-B-- POST /its/a/test/dude HTTP/1.1 Host: www.my-site.de Content-Type: text/plain ...

Thanks

[Badger]

You may be able to use a multiline codec. Something like

input { file { codec => { pattern => "^--" negate => true what => "previous" auto_flush_interval => 5 } ... } }

[admin_berlin (Tolga)]

My input typ is beats i think i cannot use multiline codec in there.

  beats{

    port => 5044
    codec => multiline {
      pattern => "^--"
      negate => true
      what => "previous"
      auto_flush_interval => 5 }
  }

Error:

 
[2023-02-28T19:55:17,130][ERROR][logstash.inputs.beats    ][main] Multiline codec with beats input is not supported. Please refer to the beats documentation for how to best manage multiline data. See https://www.elastic.co/guide/en/beats/filebeat/current/multiline-examples.html

[Badger]

Correct. In that case you would have to do the multiline combination in filebeat instead. You should ask for help in the filebeat forum.