What filter should i use to remove the "\" from my logfile

Elastic Stack Logstash
1

this is an example of my log file..
{"LogMsg":"{"deviceId":"911490250554400","description":"P701","owner":"admin","passcodeEnabled":"false","batteryLevel":18.0,"internalAvailableMemory":2.78478848E8,"latitude":11.0131367,"longitude":76.9832415,"appName":"AnTuTu%20Benchmark","appMemoryUsage":34976,"appStatus":"true"}"}
{"LogMsg":"{"deviceId":"4604d6a6f15593eb","description":"iball8735m_9706","owner":"admin","passcodeEnabled":"false","batteryLevel":55.0,"internalAvailableMemory":3.57023744E8,"latitude":12.9719136,"longitude":80.2174182,"appName":"Asphalt%20Nitro","appMemoryUsage":0,"appStatus":"false"}"}
{"LogMsg":"{"deviceId":"911490250554400","description":"P701","owner":"admin","passcodeEnabled":"false","batteryLevel":18.0,"internalAvailableMemory":2.78478848E8,"latitude":11.0131367,"longitude":76.9832415,"appName":"AnTuTu%20Benchmark","appMemoryUsage":34976,"appStatus":"true"}"}
{"LogMsg":"{"deviceId":"4604d6a6f15593eb","description":"iball8735m_9706","owner":"admin","passcodeEnabled":"false","batteryLevel":55.0,"internalAvailableMemory":3.57023744E8,"latitude":12.9719136,"longitude":80.2174182,"appName":"Asphalt%20Nitro","appMemoryUsage":0,"appStatus":"false"}"}

2

Use the gsub setting in the mutate filter. I've used it the past. Here's the link and example from the logstash docs.

Good Luck!

https://www.elastic.co/guide/en/logstash/current/plugins-filters-mutate.html#plugins-filters-mutate-gsub

	filter {
  mutate {
    gsub => [
      # replace all forward slashes with underscore
      "fieldname", "/", "_",
      # replace backslashes, question marks, hashes, and minuses
      # with a dot "."
      "fieldname2", "[\\?#-]", "."
    ]
  }
}
1 Like
3

can you please tell what is error in my logstash filter ..
filter {
mutate {
gsub => [ "LogMsg", "", "." ]
}
}

4

i get this error

[2017-07-06T13:06:34,602][ERROR][logstash.agent ] Cannot create pipeline {:reason=>"Expected one of #, {, ,, ] at line 15, column 24 (byte 264) after filter {\n mutate {\n gsub => [\n # replace backslashes, question marks, hashes, and minuses\n # with a dot "."\n "message", "\", ""}
[2017-07-06T13:07:23,590][ERROR][logstash.agent ] Cannot create pipeline {:reason=>"Expected one of #, {, ,, ] at line 15, column 24 (byte 264) after filter {\n mutate {\n gsub => [\n # replace backslashes, question marks, hashes, and minuses\n # with a dot "."\n "message", "\", ""}

5

Be aware of escaping any backslash in the config file.

try this...

filter {
	mutate {
		gsub => [ "LogMsg", "\\", "." ]
	}
}
1 Like
6

i tried and i get this error

[2017-07-06T13:28:32,381][ERROR][logstash.agent ] Cannot create pipeline {:reason=>"Expected one of #, {, ,, ] at line 15, column 23 (byte 224) after filter {\n mutate {\n gsub => [\n # replace backslash\n # with a dot "."\n "LogMsg", "\", ""}

7

You don't have validate json because you're adding dots. Try this.

filter {
	mutate {
		gsub => [ "LogMsg", "\\", "" ]
	}
}

I suggest taking one of your example log lines and manually removing the \ then validate your json with https://jsonformatter.curiousconcept.com/

1 Like
8

i did.now it show this error

[2017-07-06T13:52:07,579][ERROR][logstash.agent ] Cannot create pipeline {:reason=>"Expected one of #, {, ,, ] at line 12, column 33 (byte 171) after filter {\n mutate {\n gsub => [ "LogMsg", '\\' , '"}

9

do we have to use Logmsg or only message in the gsub field

1 Like
10

The example log lines are from your log or logstash output? Can you share your logstash conf?

11

The contents of the LogMsg field is a JSON string so you should use a json filter to deserialize it. Do not use gsub.

2 Likes
12

Thank you so much it worked with JSON filter
filter {
json {
source => "message"
}
json {
source => "LogMsg"
}
}

Now its working correctly

13

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.