# What filter should i use to remove the "\\" from my logfile

**URL:** <https://discuss.elastic.co/t/what-filter-should-i-use-to-remove-the-from-my-logfile/92017>\
**Category:** Logstash\
**Created:** [July 6, 2017, 5:12am UTC](https://discuss.elastic.co/t/what-filter-should-i-use-to-remove-the-from-my-logfile/92017 "2017-07-06T05:12:15Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jonesthomas](https://avatars.discourse-cdn.com/v4/letter/j/f4b2a3/32.png) [@Jonesthomas](https://discuss.elastic.co/u/Jonesthomas)\
**Post date:** [July 6, 2017, 5:12am UTC](https://discuss.elastic.co/t/what-filter-should-i-use-to-remove-the-from-my-logfile/92017/1 "2017-07-06T05:12:15Z")

</div>

this is an example of my log file..  
{"LogMsg":"{"deviceId":"911490250554400","description":"P701","owner":"admin","passcodeEnabled":"false","batteryLevel":18.0,"internalAvailableMemory":2.78478848E8,"latitude":11.0131367,"longitude":76.9832415,"appName":"AnTuTu%20Benchmark","appMemoryUsage":34976,"appStatus":"true"}"}  
{"LogMsg":"{"deviceId":"4604d6a6f15593eb","description":"iball8735m\_9706","owner":"admin","passcodeEnabled":"false","batteryLevel":55.0,"internalAvailableMemory":3.57023744E8,"latitude":12.9719136,"longitude":80.2174182,"appName":"Asphalt%20Nitro","appMemoryUsage":0,"appStatus":"false"}"}  
{"LogMsg":"{"deviceId":"911490250554400","description":"P701","owner":"admin","passcodeEnabled":"false","batteryLevel":18.0,"internalAvailableMemory":2.78478848E8,"latitude":11.0131367,"longitude":76.9832415,"appName":"AnTuTu%20Benchmark","appMemoryUsage":34976,"appStatus":"true"}"}  
{"LogMsg":"{"deviceId":"4604d6a6f15593eb","description":"iball8735m\_9706","owner":"admin","passcodeEnabled":"false","batteryLevel":55.0,"internalAvailableMemory":3.57023744E8,"latitude":12.9719136,"longitude":80.2174182,"appName":"Asphalt%20Nitro","appMemoryUsage":0,"appStatus":"false"}"}

---

<div class="post-metadata">

**Author:** ![ritchierich](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ritchierich/32/4329_2.png) [@ritchierich](https://discuss.elastic.co/u/ritchierich)\
**Post date:** [July 6, 2017, 6:06am UTC](https://discuss.elastic.co/t/what-filter-should-i-use-to-remove-the-from-my-logfile/92017/2 "2017-07-06T06:06:37Z")

</div>

Use the gsub setting in the mutate filter. I've used it the past. Here's the link and example from the logstash docs.

Good Luck!

[https://www.elastic.co/guide/en/logstash/current/plugins-filters-mutate.html#plugins-filters-mutate-gsub](https://www.elastic.co/guide/en/logstash/current/plugins-filters-mutate.html#plugins-filters-mutate-gsub)

```
	filter {
  mutate {
    gsub => [
      # replace all forward slashes with underscore
      "fieldname", "/", "_",
      # replace backslashes, question marks, hashes, and minuses
      # with a dot "."
      "fieldname2", "[\\?#-]", "."
    ]
  }
}
```

---

<div class="post-metadata">

**Author:** ![Jonesthomas](https://avatars.discourse-cdn.com/v4/letter/j/f4b2a3/32.png) [@Jonesthomas](https://discuss.elastic.co/u/Jonesthomas)\
**Post date:** [July 6, 2017, 7:40am UTC](https://discuss.elastic.co/t/what-filter-should-i-use-to-remove-the-from-my-logfile/92017/3 "2017-07-06T07:40:45Z")

</div>

can you please tell what is error in my logstash filter ..  
filter {  
mutate {  
gsub =\> ["LogMsg", "", "."]  
}  
}

---

<div class="post-metadata">

**Author:** ![Jonesthomas](https://avatars.discourse-cdn.com/v4/letter/j/f4b2a3/32.png) [@Jonesthomas](https://discuss.elastic.co/u/Jonesthomas)\
**Post date:** [July 6, 2017, 7:41am UTC](https://discuss.elastic.co/t/what-filter-should-i-use-to-remove-the-from-my-logfile/92017/4 "2017-07-06T07:41:06Z")

</div>

i get this error

[2017-07-06T13:06:34,602][ERROR][logstash.agent] Cannot create pipeline {:reason=\>"Expected one of #, {, ,, ] at line 15, column 24 (byte 264) after filter {\n mutate {\n gsub =\> [\n # replace backslashes, question marks, hashes, and minuses\n # with a dot "."\n "message", "\", ""}  
[2017-07-06T13:07:23,590][ERROR][logstash.agent] Cannot create pipeline {:reason=\>"Expected one of #, {, ,, ] at line 15, column 24 (byte 264) after filter {\n mutate {\n gsub =\> [\n # replace backslashes, question marks, hashes, and minuses\n # with a dot "."\n "message", "\", ""}

---

<div class="post-metadata">

**Author:** ![ritchierich](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ritchierich/32/4329_2.png) [@ritchierich](https://discuss.elastic.co/u/ritchierich)\
**Post date:** [July 6, 2017, 7:59am UTC](https://discuss.elastic.co/t/what-filter-should-i-use-to-remove-the-from-my-logfile/92017/5 "2017-07-06T07:59:43Z")

</div>

Be aware of escaping any backslash in the config file.

try this...

```
filter {
	mutate {
		gsub => ["LogMsg", "\\", "."]
	}
}
```

---

<div class="post-metadata">

**Author:** ![Jonesthomas](https://avatars.discourse-cdn.com/v4/letter/j/f4b2a3/32.png) [@Jonesthomas](https://discuss.elastic.co/u/Jonesthomas)\
**Post date:** [July 6, 2017, 8:01am UTC](https://discuss.elastic.co/t/what-filter-should-i-use-to-remove-the-from-my-logfile/92017/6 "2017-07-06T08:01:13Z")

</div>

i tried and i get this error

[2017-07-06T13:28:32,381][ERROR][logstash.agent] Cannot create pipeline {:reason=\>"Expected one of #, {, ,, ] at line 15, column 23 (byte 224) after filter {\n mutate {\n gsub =\> [\n # replace backslash\n # with a dot "."\n "LogMsg", "\", ""}

---

<div class="post-metadata">

**Author:** ![ritchierich](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ritchierich/32/4329_2.png) [@ritchierich](https://discuss.elastic.co/u/ritchierich)\
**Post date:** [July 6, 2017, 8:22am UTC](https://discuss.elastic.co/t/what-filter-should-i-use-to-remove-the-from-my-logfile/92017/7 "2017-07-06T08:22:01Z")

</div>

You don't have validate json because you're adding dots. Try this.

```
filter {
	mutate {
		gsub => ["LogMsg", "\\", ""]
	}
}

```

I suggest taking one of your example log lines and manually removing the \ then validate your json with [https://jsonformatter.curiousconcept.com/](https://jsonformatter.curiousconcept.com/)

---

<div class="post-metadata">

**Author:** ![Jonesthomas](https://avatars.discourse-cdn.com/v4/letter/j/f4b2a3/32.png) [@Jonesthomas](https://discuss.elastic.co/u/Jonesthomas)\
**Post date:** [July 6, 2017, 8:23am UTC](https://discuss.elastic.co/t/what-filter-should-i-use-to-remove-the-from-my-logfile/92017/8 "2017-07-06T08:23:41Z")

</div>

i did.now it show this error

[2017-07-06T13:52:07,579][ERROR][logstash.agent] Cannot create pipeline {:reason=\>"Expected one of #, {, ,, ] at line 12, column 33 (byte 171) after filter {\n mutate {\n gsub =\> [ "LogMsg", '\\' , '"}

---

<div class="post-metadata">

**Author:** ![Jonesthomas](https://avatars.discourse-cdn.com/v4/letter/j/f4b2a3/32.png) [@Jonesthomas](https://discuss.elastic.co/u/Jonesthomas)\
**Post date:** [July 6, 2017, 9:18am UTC](https://discuss.elastic.co/t/what-filter-should-i-use-to-remove-the-from-my-logfile/92017/9 "2017-07-06T09:18:47Z")

</div>

do we have to use Logmsg or only message in the gsub field

---

<div class="post-metadata">

**Author:** ![ritchierich](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ritchierich/32/4329_2.png) [@ritchierich](https://discuss.elastic.co/u/ritchierich)\
**Post date:** [July 6, 2017, 2:39pm UTC](https://discuss.elastic.co/t/what-filter-should-i-use-to-remove-the-from-my-logfile/92017/10 "2017-07-06T14:39:02Z")

</div>

The example log lines are from your log or logstash output? Can you share your logstash conf?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [July 9, 2017, 3:48pm UTC](https://discuss.elastic.co/t/what-filter-should-i-use-to-remove-the-from-my-logfile/92017/11 "2017-07-09T15:48:42Z")

</div>

The contents of the `LogMsg` field is a JSON string so you should use a json filter to deserialize it. Do not use gsub.

---

<div class="post-metadata">

**Author:** ![Jonesthomas](https://avatars.discourse-cdn.com/v4/letter/j/f4b2a3/32.png) [@Jonesthomas](https://discuss.elastic.co/u/Jonesthomas)\
**Post date:** [July 12, 2017, 12:26pm UTC](https://discuss.elastic.co/t/what-filter-should-i-use-to-remove-the-from-my-logfile/92017/12 "2017-07-12T12:26:46Z")

</div>

Thank you so much it worked with JSON filter  
filter {  
json {  
source =\> "message"  
}  
json {  
source =\> "LogMsg"  
}  
}

Now its working correctly

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 9, 2017, 12:26pm UTC](https://discuss.elastic.co/t/what-filter-should-i-use-to-remove-the-from-my-logfile/92017/13 "2017-08-09T12:26:50Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
