Logstash replace string problem

(Arkadiusz Matalewski) #1


I'm new to the logstash and don't know if what I want to do is even possible. I need to replace some information in logs provided by an application. My example input is:
2017-09-18 10:45:25,404 ERROR some variable text some variable text
Task for me is to replace those IP's with other text. At first I tried to use below grok:

			match => {"log" => ["%{DATA}(?<ip>%{IPV4})%{GREEDYDATA}"]}
		if "_grokparsefailure" not in [tags]{
				gsub => ["log","%{ip}","SOMEIP"]

But this way I'm only replacing the first IP and the second one stays unchanged as grok takes first match and moves further, I tried to use %{IPV4} pattern inside mutate but it doesn't work. The main problem is that logs that I have to parse aren't standardized and I have to search for those IP inside a message. Also the occurrence of those IP inside a message varies.

(Dirk Lüneburger) #2

Hi @Arkadiusz,

first and important, play around with the grok debugger, its a powerfull tool for testing this kind of stuff :slight_smile:

Grok Debugger

then you should try something like this:

2017-09-18 10:45:25,404 ERROR some variable text some variable text


just as an example.

if [IPV4] {

if ![IPV4]
looks for "not exists"

if [IPV4]
looks for "exists"


(Arkadiusz Matalewski) #3

Hi @lueneburger
Thank you for fast response. I was using grok debugger but like I wrote the logs aren't standardized and this was only an example to show what I want to accomplish. And as the occurrence of the IP and the number of times it occurs varies so my Grok Pattern would have to look more like this:


Which I wanted to avoid, and what if there will be 5th IP etc. Best solution would be to use a loop but from what I found out there is no for statement for logstash.


The gsub function of the mutate filter might help, depending on what text you want to replace the IPs with.

(Arkadiusz Matalewski) #5

Yeah I have tried this but the problem is that IP's are different and mutate gsub replace one string with another, and when i tried to use pattern %{IPV4} with mutate it didn't worked.

(Magnus Bäck) #6

There's no need to use grok to locate the exact IP address first. Just use gsub:

mutate {
  gsub => ["message", "\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b", "FOO"]

(You could use a more exact regexp to match an IP address but this one's probably good enough.)

(Arkadiusz Matalewski) #7

Hi @magnusbaeck
Thank you this is exactly what I was looking for.

(system) #8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.