Cant parse xml file generated with log4net using logstash

Elastic Stack Logstash
1

Hello,
I have c# app that logs xml file
using log4net and log4net.Layout.XmlLayout
(see configuration below).

The generated log events contains
message ang name-value collection:

<log4net:**message**><Message from catch block<</log4net:message>
<log4net:properties>
<log4net:data name="RequestId" value="40000016-0007-ff00-b63f-84710c7967bb" />
<log4net:data name="**RequestPath**" value="/WeatherForecast" />" 
<log4net:data name="log4net:HostName" value="DESKTOP-XX" /> 
...
</log4net:properties>

My questions:
1.how can i get event where all the name-value keys are flat fields.
eg, message and RequestPath are both simple fields of the envent ?
2. do you recommand json ?

log4net configuration

<appender name="RollingLogFileAppender" type="log4net.Appender.RollingFileAppender">
  <file value=".\Log_AppName.log" />
  <appendToFile value="true" />
  <rollingStyle value="Size" />
  <maxSizeRoolBackups value="100" />
  <maximumFileSize value="10Mb" />
  <staticLogFileName value="true"/>
  <preserveLogFileNameExtension value="true"/>
  <layout type="log4net.Layout.XmlLayout" />
  <filter type="log4net.Filter.levelRangeFilter">
      <levelMin value="INFO" />
  </filter>
</appender>

Example of log entry

<log4netevent logger="WebApplication2.Controllers.WeatherForecastController" timestamp="2023-12-19T11:26:27.8096148+02:00" level="FATAL" thread="5" domain="WebApplication2" username="DESKTOP-LU6BPF1\Regina" xmlns:log4net="log4net"><log4net:message>Message from catch block</log4net:message><log4net:properties><log4net:data name="RequestId" value="40000016-0007-ff00-b63f-84710c7967bb" /><log4net:data name="RequestPath" value="/WeatherForecast" /><log4net:data name="SpanId" value="20d3c520cca51344" /><log4net:data name="TraceId" value="85283cc569e2b94283c6bc467d518f6d" /><log4net:data name="ActionName" value="WebApplication2.Controllers.WeatherForecastController.Get (WebApplication2)" /><log4net:data name="log4net:HostName" value="DESKTOP-LU6BPF1" /><log4net:data name="ParentId" value="0000000000000000" /><log4net:data name="ActionId" value="5c7d478e-ff22-4c2b-9f9a-01429a9e18a5" /></log4net:properties><log4net:exception>System.Exception: This is my Exception
   at WebApplication2.Controllers.WeatherForecastController.Get() in C:\R&amp;D\Solution1\WebApplication2\Controllers\WeatherForecastController.cs:line 37</log4net:exception></log4netevent>
2

To convert hashes with name/value entries you could use something like this.

If you are asking what log4net configuration would be easier to parse I cannot answer that.

3

Thank you so much for your replay.
It still does not work for me .
I only need the value of the HostName from the properties map.
any idea how to extract specifically that field ?
(assuming the properties map my contain any number of pairs, at any order).

<log4net:event>
<log4net:message>Application started</log4net:message><log4net:properties>
<log4net:data name="log4net:**HostName**" value="DESKTOP1" />
<log4net:data name="RequestId" value="84710c7967bb" />
</log4net:properties>
</log4net:event>
4

Hello,
I have been trying to use xpath (based onplease this link
However, logstash does not like it.
Any idea ?

xml


<log4netevent><log4netproperties><log4netdata name="log4netHostName" value="DESKTOP-LU6BPF1" /><log4netdata name="xxx" value="111" /></log4netproperties></log4netevent>
<log4netevent><log4netproperties><log4netdata name="log4netHostName" value="DESKTOP-LU6BPF1" /><log4netdata name="yyy" value="222" /></log4netproperties></log4netevent>

logstash.config

input
{
  	file
  	{
		path 		=> "c:/dockers/logstash/logstash-8.11.2/config/demo005e_input_file.txt"
    		sincedb_path 	=> "NUL"
    		start_position 	=> "beginning"
  	}
}
filter 
{
	xml 
	{
		remove_namespaces 	=> true
        	source 			=> "message"
		store_xml 		=> false		
		force_array 	=> "false"
		xpath 		=>
		{		"/log4netevent/log4netproperties/log4netdata[@name="log4netHostName"]/@value" => "LogHost"
		}
  	}
}
output
{
	#stdout
   	stdout { codec => rubydebug }
}

log error:
The given configuration is invalid. Reason: Expected one of [ \t\r\n], "#", "=>" at line 27, column 56 (byte 805) after filter

5

Replace that with

 '/log4netevent/log4netproperties/log4netdata[@name="log4netHostName"]/@value'
6

sweet ...
it works !
thank you, Dear Badger !

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.