# Removing special characters

**URL:** <https://discuss.elastic.co/t/removing-special-characters/309766>\
**Category:** Logstash\
**Tags:** docker\
**Created:** [July 15, 2022, 8:04pm UTC](https://discuss.elastic.co/t/removing-special-characters/309766 "2022-07-15T20:04:28Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![JeremyP](https://avatars.discourse-cdn.com/v4/letter/j/43a26b/32.png) [@JeremyP](https://discuss.elastic.co/u/JeremyP)\
**Post date:** [July 15, 2022, 8:04pm UTC](https://discuss.elastic.co/t/removing-special-characters/309766/1 "2022-07-15T20:04:28Z")

</div>

Hello,

I have some special characters in one of my fields which I'm having issues removing. They may be reserved characters so my logstash instance is failing. Here is the event in question...

```auto
"proof" => [
            [0] "\"Vulnerable software installed: Apache Log4j Core 2.11.1 (/var/lib/docker/overlay2/0edef2ae5c6fc053ca616bd84b6bfe81e39ebdc071ca44c9c29c378112693860/diff/usr/share/elasticsearch/lib/log4j-core-2.11.1.jar)No mitigation applied. JndiLookup class discovered.\"",
            [1] "\"Vulnerable software installed: Apache Log4j Core 2.14.0 (/var/lib/docker/overlay2/c2877f2cdbd6d80bcafb50907a452a74eefa6e95a34de050908632c0079c0b3a/diff/usr/share/logstash/logstash-core/lib/jars/log4j-core-2.14.0.jar)No mitigation applied. JndiLookup class discovered.\"",
            [2] "\"Vulnerable software installed: Apache Log4j Core 2.11.1 (/var/lib/docker/overlay2/9e668265b4cc434f8aa963a773a06c0662323ec76bdd91bf012b4db643298d3a/diff/usr/share/logstash/logstash-core/lib/jars/log4j-core-2.11.1.jar)No mitigation applied. JndiLookup class discovered.\"",
            [3] "\"Vulnerable software installed: Apache Log4j Core 2.11.1 (/var/lib/docker/overlay2/04b58f14d8ef5bab515849cb8ac65fd4843915ef905a3fadd97268a5dfd8b79a/diff/usr/share/logstash/logstash-core/lib/jars/log4j-core-2.11.1.jar)No mitigation applied. JndiLookup class discovered.\"",
            [4] "\"Vulnerable software installed: Apache Log4j Core 2.14.0 (/var/lib/docker/overlay2/b2efd746948b7132e32c25d9812d7ed46d27ca6ed4a98835386628d838c16e5b/diff/usr/share/logstash/logstash-core/lib/jars/log4j-core-2.14.0.jar)No mitigation applied. JndiLookup class discovered.\"",
            [5] "\"Vulnerable software installed: Apache Log4j Core 2.11.1 (/var/lib/docker/overlay2/3fca162b169ea32445c5a3e1f554891025dc1c24f9f9bf30ddc4af95a0670194/diff/usr/share/logstash/logstash-core/lib/jars/log4j-core-2.11.1.jar)No mitigation applied. JndiLookup class discovered.\""
        ],

```

My config (currently)

```auto
  mutate {
    gsub => [
      "proof", "[{}]", "",
    split => { "proof" => "," }
  }
  mutate { strip => ["proof"] }

```

In my output I have extra " and \ from the original data which was one message. I'm attempting to remove the special characters and performing a split on the event to create an array. So far the array works fine, I'm just left with additional characters so I need to clean up the data. Every time I attempt to add those characters to the gsub I get an error `exception=>#<RegexpError: premature end of char-class: /[\]/>`

Any assistance is appreciated!

Thank you.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [July 15, 2022, 9:26pm UTC](https://discuss.elastic.co/t/removing-special-characters/309766/2 "2022-07-15T21:26:43Z")

</div>

What characters do you want to remove?

---

<div class="post-metadata">

**Author:** ![JeremyP](https://avatars.discourse-cdn.com/v4/letter/j/43a26b/32.png) [@JeremyP](https://discuss.elastic.co/u/JeremyP)\
**Post date:** [July 16, 2022, 12:40am UTC](https://discuss.elastic.co/t/removing-special-characters/309766/3 "2022-07-16T00:40:13Z")

</div>

Hi Badger

The backslashes and extra double quotes.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [July 16, 2022, 12:45am UTC](https://discuss.elastic.co/t/removing-special-characters/309766/4 "2022-07-16T00:45:06Z")

</div>

There are no backslashes in your message. The rubydebug format inserts a backslash to escape the double quote.

```
split { field => "proof" }
mutate { gsub => ["proof", '"', ""] }

```

---

<div class="post-metadata">

**Author:** ![JeremyP](https://avatars.discourse-cdn.com/v4/letter/j/43a26b/32.png) [@JeremyP](https://discuss.elastic.co/u/JeremyP)\
**Post date:** [July 18, 2022, 2:18pm UTC](https://discuss.elastic.co/t/removing-special-characters/309766/5 "2022-07-18T14:18:35Z")

</div>

I think there is a backslash. I gave you the Ruby Debug output but here is the JSON data in the Elastic document when I do a \_doc query.

```auto
"proof" : [
        "\"Vulnerable software installed: Apache Log4j Core 2.11.1 (/var/lib/docker/overlay2/0edef2ae5c6fc053ca616bd84b6bfe81e39ebdc071ca44c9c29c378112693860/diff/usr/share/elasticsearch/lib/log4j-core-2.11.1.jar)No mitigation applied. JndiLookup class discovered.\"",
        "\"Vulnerable software installed: Apache Log4j Core 2.14.0 (/var/lib/docker/overlay2/c2877f2cdbd6d80bcafb50907a452a74eefa6e95a34de050908632c0079c0b3a/diff/usr/share/logstash/logstash-core/lib/jars/log4j-core-2.14.0.jar)No mitigation applied. JndiLookup class discovered.\"",
        "\"Vulnerable software installed: Apache Log4j Core 2.11.1 (/var/lib/docker/overlay2/9e668265b4cc434f8aa963a773a06c0662323ec76bdd91bf012b4db643298d3a/diff/usr/share/logstash/logstash-core/lib/jars/log4j-core-2.11.1.jar)No mitigation applied. JndiLookup class discovered.\"",
        "\"Vulnerable software installed: Apache Log4j Core 2.11.1 (/var/lib/docker/overlay2/04b58f14d8ef5bab515849cb8ac65fd4843915ef905a3fadd97268a5dfd8b79a/diff/usr/share/logstash/logstash-core/lib/jars/log4j-core-2.11.1.jar)No mitigation applied. JndiLookup class discovered.\"",
        "\"Vulnerable software installed: Apache Log4j Core 2.14.0 (/var/lib/docker/overlay2/b2efd746948b7132e32c25d9812d7ed46d27ca6ed4a98835386628d838c16e5b/diff/usr/share/logstash/logstash-core/lib/jars/log4j-core-2.14.0.jar)No mitigation applied. JndiLookup class discovered.\"",
        "\"Vulnerable software installed: Apache Log4j Core 2.11.1 (/var/lib/docker/overlay2/3fca162b169ea32445c5a3e1f554891025dc1c24f9f9bf30ddc4af95a0670194/diff/usr/share/logstash/logstash-core/lib/jars/log4j-core-2.11.1.jar)No mitigation applied. JndiLookup class discovered.\""
      ]

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [July 18, 2022, 3:17pm UTC](https://discuss.elastic.co/t/removing-special-characters/309766/6 "2022-07-18T15:17:38Z")

</div>

I still do not think there is a backslash in your field, but you can try

```
mutate { gsub => ["proof", '"', "", "proof", "[\\]", "" ] }

```

---

<div class="post-metadata">

**Author:** ![JeremyP](https://avatars.discourse-cdn.com/v4/letter/j/43a26b/32.png) [@JeremyP](https://discuss.elastic.co/u/JeremyP)\
**Post date:** [July 18, 2022, 4:40pm UTC](https://discuss.elastic.co/t/removing-special-characters/309766/7 "2022-07-18T16:40:07Z")

</div>

That did the trick. Thank you Badger.

```auto
"proof" : [
        "Vulnerable software installed: Apache Log4j Core 2.11.1 (/var/lib/docker/overlay2/0edef2ae5c6fc053ca616bd84b6bfe81e39ebdc071ca44c9c29c378112693860/diff/usr/share/elasticsearch/lib/log4j-core-2.11.1.jar)No mitigation applied. JndiLookup class discovered.",
        "Vulnerable software installed: Apache Log4j Core 2.14.0 (/var/lib/docker/overlay2/b2efd746948b7132e32c25d9812d7ed46d27ca6ed4a98835386628d838c16e5b/diff/usr/share/logstash/logstash-core/lib/jars/log4j-core-2.14.0.jar)No mitigation applied. JndiLookup class discovered.",
        "Vulnerable software installed: Apache Log4j Core 2.11.1 (/var/lib/docker/overlay2/04b58f14d8ef5bab515849cb8ac65fd4843915ef905a3fadd97268a5dfd8b79a/diff/usr/share/logstash/logstash-core/lib/jars/log4j-core-2.11.1.jar)No mitigation applied. JndiLookup class discovered.",
        "Vulnerable software installed: Apache Log4j Core 2.11.1 (/var/lib/docker/overlay2/9e668265b4cc434f8aa963a773a06c0662323ec76bdd91bf012b4db643298d3a/diff/usr/share/logstash/logstash-core/lib/jars/log4j-core-2.11.1.jar)No mitigation applied. JndiLookup class discovered.",
        "Vulnerable software installed: Apache Log4j Core 2.11.1 (/var/lib/docker/overlay2/3fca162b169ea32445c5a3e1f554891025dc1c24f9f9bf30ddc4af95a0670194/diff/usr/share/logstash/logstash-core/lib/jars/log4j-core-2.11.1.jar)No mitigation applied. JndiLookup class discovered.",
        "Vulnerable software installed: Apache Log4j Core 2.14.0 (/var/lib/docker/overlay2/c2877f2cdbd6d80bcafb50907a452a74eefa6e95a34de050908632c0079c0b3a/diff/usr/share/logstash/logstash-core/lib/jars/log4j-core-2.14.0.jar)No mitigation applied. JndiLookup class discovered."
      ],

```

```auto
  mutate {
    gsub => [
      "proof", '"', "", "proof", "[\\]", "", "proof", "[{}]", "" ]
    split => { "proof" => "," }
  }

```

```auto

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 15, 2022, 4:40pm UTC](https://discuss.elastic.co/t/removing-special-characters/309766/8 "2022-08-15T16:40:24Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.