Parsing log multiline

Elastic Stack Beats
1

Hi i need help with parsing my logs with logstash and grok.
I have a log like this:

141>1 2024-07-11T11:12:03.213953+02:00 TACP-VD-RDSH039.tac.priv takeinfo_reglog_CSV.ps1 14128 - - [2024-07-11 11:12:02.621][4465141][NOTICE]HKCU:\Volatile Environment\LOGONSERVER: \TACP-CS-DC002
[2024-07-11 11:12:02.621][4465141][NOTICE]HKCU:\Volatile Environment\USERDOMAIN: TAC
[2024-07-11 14:42:24.285][4459950][NOTICE]HKCU:\Volatile Environment\HOMEDRIVE: C:
[.. ecc there are several lines ..]

so it's combine with this part "141>1 2024-07-11T11:12:03.213953+02:00 TACP-VD-RDSH039.tac.priv takeinfo_reglog_CSV.ps1 14128 - -" + this lines "[2024-07-11 11:12:02.621][4465141][NOTICE]HKCU:\Volatile Environment\LOGONSERVER: \TACP-CS-DC002
[2024-07-11 11:12:02.621][4465141][NOTICE]HKCU:\Volatile Environment\USERDOMAIN: TAC
[2024-07-11 14:42:24.285][4459950][NOTICE]HKCU:\Volatile Environment\HOMEDRIVE: C:"

i tried follow this way:

input {
  udp {
    id => "tac-main-input"
    port => 5514
    type => "syslog"
  }
}
filter {
  if [type] == "syslog" {
        grok {
                match => {
                        "message" => "<%{POSINT:syslog_pri}>%{NUMBER:syslog_version} %{TIMESTAMP_ISO8601:syslog_timestamp} %{HOSTNAME:syslog_host} %{WORD:syslog_program}.%{WORD:extension} %{NUMBER:syslog_pid} (%{DATA:trattino_1})? (%{DATA:trattino_2})? %{GREEDYDATA:multi_msg}"
                }
        }
        mutate {
                split => { "multi_msg" => "\n" }
        }
        split {
                field => "multi_msg"
        }
        grok {
                match => { "multi_msg" => "\[%{TIMESTAMP_ISO8601:datetime}\]\[%{WORD:dem_id}\]\[%{WORD:dem_severity}\]%{DATA:dem_key}: %{GREEDYDATA:dem_value}" }
                break_on_match => false
        }
  }
}
output {
  elasticsearch {
    hosts => ["https://tacm-lg-esdn001.net:9200"]

  }
}

so I would like to process the first part up to 'dash_2' and would like the other part (which will contain the multiline logs) to be sent to multi_msg.
Then I would like to split multi_msg and process it.

With this pipeline configuration I get multi_msg correct, which contains all logs with multiple lines.
But dem_value (the last analysed field of multi_msg) contains the correct value + each other log line.
Es:
dem_value ->
\TACP-CS-DC002
[2024-07-11 14:42:24.285][4459950][NOTICE]HKCU:\Volatile Environment\USERDOMAIN: TAC
[2024-07-11 14:42:24.285][4459950][NOTICE]HKCU:\Volatile Environment\HOMEDRIVE: C

i would only dem_value -> \TACP-CS-DC002 (then other row log to parse)

(sorry for my bad english, i hope to expla

2

Hi,

Could you please share a simpler log sample with the expected result? I believe it will be easier to understand the problem. I got confused with the sample, not with the English :slight_smile: