Logstash xml input plugin - parsing log4net:event

Elastic Stack Logstash
1

Hello,
log4net generates xml file.
every event is stored in xml element called log4net:event.

The issue is that logstash cant parse the element with the ":" in it.
any idea ?

The xml


<log4net:event><log4netmessage>Message1</log4netmessage></log4net:event>
<log4net:event><log4netmessage>Message2</log4netmessage></log4net:event>

logstash config


input
{
	file
  	{
		path 		=> "c:/dockers/logstash/logstash-8.11.2/config/demo006b_input_file.txt"
    		sincedb_path 	=> "NUL"
    		start_position 	=> "beginning"
  	}
}
filter 
{
        xml 
	{
		remove_namespaces 	=> true
        	source 			=> "message"
		store_xml 		=> false		
		force_array 	=> "false"
		xpath 		=> [ "/log4netevent/log4netmessage/text()", "LogMessage" ]
  	}
}
output
{
	#stdout
   	stdout { codec => rubydebug }
}
2

If the XML uses a namespace then it needs to define it. So the XML should be

<log4net:event xmlns:log4net="http://www.example.com"><log4netmessage>Message1</log4netmessage></log4net:event>

and then you could parse it using

    xml {
        remove_namespaces   => true
        source              => "message"
        store_xml           => false
        force_array         => false
        xpath               => [ "/event/log4netmessage/text()", "LogMessage" ]
    }
3

I am sorry.
It is still not clear to me.

I am not working with schema.

do you suggest that if i will work with schema,
then logstash will know how to handle elements
that contains special characters in the element name ?
(eg: log4net:event)

4

Element names cannot contain colon unless it is used to separate the namespace from the element name. That may not have been true in the earliest versions of XML but I think you will find that modern XML processing software enforces it.

5

I used the remove_namespaces and it sees to work now .
see below.
Thank you for your kind help, Badger .

xml

<log4net:event><log4net:message>Message1</log4net:message></log4net:event>
<log4net:event><log4net:message>Message2</log4net:message></log4net:event>
<log4net:event><log4net:message>Message3</log4net:message></log4net:event>
<log4net:event><log4net:message>Message4</log4net:message></log4net:event>

configuration
Note: Xpath queries are using element names without namespace.


input
{
	#file input plugin. 
	#Note: pay attention to the slash direction in path.
  	file
  	{
		path 		=> "c:/dockers/logstash/logstash-8.11.2/config/demo006b_input_file.txt"
    		sincedb_path 	=> "NUL"
    		start_position 	=> "beginning"
	}
}
filter 
{
	xml 
	{
		source 			=> "message"	
		remove_namespaces 	=> true
		force_array 		=> "false"
		xpath 			=> 
		{
			 "/event/message/text()"	=> "LogMessage" 
		}
		store_xml 		=> false
  	}
}
output
{
	#stdout
   	stdout { codec => rubydebug }
}
6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.