Logstash cannot read new lines that are coming from .NET error exception msg

[theo003 (Theodoros)]

Hello,

Our system is throwing some error exceptions in the logs with the following format:

|17 01 2024 08:22:10,614| |ERROR| CreateSession API... File: "File_name" Line: 290System.InvalidOperationException: "Error_msg"
   at "Location" in "File":line 243
   at "Location" in "File":line 290

Logstash read only the first line of the log message. In the logstash.conf we tried to mutate the logmsg but still not working. Your assistance will be much appreciated.

filter {
  dissect {
    mapping => {
        "message" => "|%{time}| |%{level}| %{logmsg}"
      }
}
  kv {
    field_split => " "
}
  mutate {
    remove_field => ["event","input","ecs","version","name","@version","input","type","agent","offset","tags"]
    lowercase => [ "[host][name]" ]
    lowercase => [ "[level]" ]
    lowercase => [ "[Application_Name]" ]
    gsub => ["logmsg", "[\r\n]+", "line"]
    gsub => ["logmsg", "[\r]+", "line"]
    gsub => ["logmsg", "[\n]+", "line"]
}
}
  output {
      stdout {codec => rubydebug }
      elasticsearch {...}
}

[yago82]

@theo003

Hello,

Our system is throwing some error exceptions in the logs with the following format:

|17 01 2024 08:22:10,614| |ERROR| CreateSession API... File: "File_name" Line: 290System.InvalidOperationException: "Error_msg"
   at "Location" in "File":line 243
   at "Location" in "File":line 290

Logstash read only the first line of the log message. In the logstash.conf we tried to mutate the logmsg but still not working. Your assistance will be much appreciated.

filter {
  dissect {
    mapping => {
        "message" => "|%{time}| |%{level}| %{logmsg}"
      }
}
  kv {
    field_split => " "
}
  mutate {
    remove_field => ["event","input","ecs","version","name","@version","input","type","agent","offset","tags"]
    lowercase => [ "[host][name]" ]
    lowercase => [ "[level]" ]
    lowercase => [ "[Application_Name]" ]
    gsub => ["logmsg", "[\r\n]+", "line"]
    gsub => ["logmsg", "[\r]+", "line"]
    gsub => ["logmsg", "[\n]+", "line"]
}
}
  output {
      stdout {codec => rubydebug }
      elasticsearch {...}
}

Hi,

i think that the issue you're experiencing is likely due to multiline log messages.

Multiline codec plugin | Logstash Reference [8.12] | Elastic

Regards

[leandrojmp (Leandro Pereira)]

What is your input? You didn't share.

You have multiline logs, so you need to configure this in your input, if you are reading the files directly with Logstash, then you need to configure in the file input, if you are using beats to send the logs, then the multiline needs to be configured in beats, not logstash.

[theo003 (Theodoros)]

Hello,

Thank you for your help! We manage to sort this out with multiline in the filebeat .yml file configuration because we are using beats as input.

We used the following link for guide:

Manage multiline messages | Filebeat Reference [8.12] | Elastic

I will also share my multiline configuration for reference:
filebeat.yml:

filebeat.inputs:
- type: log
  paths:
    C:\...
  multiline.type: pattern
  multiline.pattern: '^\|'
  multiline.negate: true
  multiline.match: after

logstash.conf:

input {
  beats {
    port => 5050
  }
}