Parsing logs with logstash

[Nejmeddine_Saidane]

I have this log

{"data" => "<Event xmlns='link'><System><Provider Name='Service Control Manager' Guid='{555908d1-a6d7-4695-8e1e-26931d2012f4}' EventSourceName='Service Control Manager'/><EventID Qualifiers='16384'>7036</EventID><Version>0</Version><Level>4</Level><Task>0</Task><Opcode>0</Opcode><Keywords>0x8080000000000000</Keywords><TimeCreated SystemTime='2024-02-06T16:01:08.593980800Z'/><EventRecordID>743861</EventRecordID><Correlation/><Execution ProcessID='644' ThreadID='9064'/><Channel>System</Channel><Computer>LPVSEIDW01.admsb3.log.intra.laposte.fr</Computer><Security/></System><EventData><Data Name='param1'>Service Configuration du réseau</Data><Data Name='param2'>en cours d’exécution</Data><Binary>4E0065007400530065007400750070005300760063002F0034000000</Binary></EventData><RenderingInfo Culture='fr-FR'><Message>Le service Service Configuration du réseau est entré dans l’état : en cours d’exécution.</Message><Level>Information</Level><Task></Task><Opcode></Opcode><Channel></Channel><Provider>Microsoft-Windows-Service Control Manager</Provider><Keywords><Keyword>Classique</Keyword></Keywords></RenderingInfo></Event>"}

I would delete this part of log

<Message>Le service Service Configuration du réseau est entré dans l’état : en cours d’exécution.</Message>

knowing that the value in the tag changes each time

[Badger]

Perhaps

mutate { gsub => [ "message", "<Message>[^<]*</Message>", "" ] }

[Nejmeddine_Saidane]

it does not work

[Badger]

OK, so show us what an event looks like when you use output { stdout { codec => rubydebug } } so that we do not have to guess your event structure.

[Rios]

It does work. I would say you have to put data instead of message in gsub.

input {
  generator {
       message => "<Event xmlns='link'><System><Provider Name='Service Control Manager' Guid='{555908d1-a6d7-4695-8e1e-26931d2012f4}' EventSourceName='Service Control Manager'/><EventID Qualifiers='16384'>7036</EventID><Version>0</Version><Level>4</Level><Task>0</Task><Opcode>0</Opcode><Keywords>0x8080000000000000</Keywords><TimeCreated SystemTime='2024-02-06T16:01:08.593980800Z'/><EventRecordID>743861</EventRecordID><Correlation/><Execution ProcessID='644' ThreadID='9064'/><Channel>System</Channel><Computer>LPVSEIDW01.admsb3.log.intra.laposte.fr</Computer><Security/></System><EventData><Data Name='param1'>Service Configuration du réseau</Data><Data Name='param2'>en cours d’exécution</Data><Binary>4E0065007400530065007400750070005300760063002F0034000000</Binary></EventData><RenderingInfo Culture='fr-FR'><Message>Le service Service Configuration du réseau est entré dans l’état : en cours d’exécution.</Message><Level>Information</Level><Task></Task><Opcode></Opcode><Channel></Channel><Provider>Microsoft-Windows-Service Control Manager</Provider><Keywords><Keyword>Classique</Keyword></Keywords></RenderingInfo></Event>"
	   count => 1
  }
}

filter {

  mutate { rename => { "message" => "data" } 
  gsub => [ "data", "<Message>[^<]*</Message>", "" ] }
}

output {
 stdout { codec => rubydebug{} }
}

Result:

<Event
	xmlns='link'>
	<System>
		<Provider Name='Service Control Manager' Guid='{555908d1-a6d7-4695-8e1e-26931d2012f4}' EventSourceName='Service Control Manager'/>
		<EventID Qualifiers='16384'>7036</EventID>
		<Version>0</Version>
		<Level>4</Level>
		<Task>0</Task>
		<Opcode>0</Opcode>
		<Keywords>0x8080000000000000</Keywords>
		<TimeCreated SystemTime='2024-02-06T16:01:08.593980800Z'/>
		<EventRecordID>743861</EventRecordID>
		<Correlation/>
		<Execution ProcessID='644' ThreadID='9064'/>
		<Channel>System</Channel>
		<Computer>LPVSEIDW01.admsb3.log.intra.laposte.fr</Computer>
		<Security/>
	</System>
	<EventData>
		<Data Name='param1'>Service Configuration du réseau</Data>
		<Data Name='param2'>en cours d?exécution</Data>
		<Binary>4E0065007400530065007400750070005300760063002F0034000000</Binary>
	</EventData>
	<RenderingInfo Culture='fr-FR'>
		<Level>Information</Level>
		<Task></Task>
		<Opcode></Opcode>
		<Channel></Channel>
		<Provider>Microsoft-Windows-Service Control Manager</Provider>
		<Keywords>
			<Keyword>Classique</Keyword>
		</Keywords>
	</RenderingInfo>
</Event>

[Nejmeddine_Saidane]

 gsub => ["data", "<Message>[^<]*<\/Message>", ""]

it work, thank you