# Issue with KV Filter when value ends with escape character

**URL:** https://discuss.elastic.co/t/issue-with-kv-filter-when-value-ends-with-escape-character/190618
**Category:** Logstash
**Created:** [July 15, 2019, 10:14pm UTC](https://discuss.elastic.co/t/issue-with-kv-filter-when-value-ends-with-escape-character/190618 "2019-07-15T22:14:50Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![MakoWish](https://avatars.discourse-cdn.com/v4/letter/m/e56c9b/32.png) [@MakoWish](https://discuss.elastic.co/u/MakoWish)
#### Post date: [July 15, 2019, 10:14pm UTC](https://discuss.elastic.co/t/issue-with-kv-filter-when-value-ends-with-escape-character/190618/1 "2019-07-15T22:14:50Z")

</div>

I am having an issue parsing some logs, and I would appreciate if someone could assist.

I have logs coming in that are primarily handled with a KV Filter. This works fine on about 99% of the messages, but one message type is giving me trouble. Here is a snippet from a sample log message giving me trouble:

```auto
File Name: foo.exe, Path: C:\, Drive Type: Internal Hard Drive

```

```auto
kv {
  transform_key => "lowercase"
  trim_key => " " # Trim any leading or trailing spaces
  value_split => ":"
  field_split => "\,"
}

# Substitutes spaces in the key names for underscores
ruby {
  code => "
    event.to_hash.each {
      |key, value|
      if key =~ / / then
        event.set(key.gsub(' ','_'), event.remove(key))
      end
    }
  "
}

```

Since the "Path" contains my split character ':' in the value, it is not parsing properly. How can I handle a value that contains the split character? Using the above example, the message would come through like this:

```auto
file_name: foo.exe
     path: D:\, Drive Type: Internal Hard Drive

```

When I want it to come through like this:

```auto
 file_name: foo.exe
      path: D:\
drive_type: Internal Hard Drive

```

I would appreciate any thoughts on this.

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [July 15, 2019, 10:53pm UTC](https://discuss.elastic.co/t/issue-with-kv-filter-when-value-ends-with-escape-character/190618/2 "2019-07-15T22:53:47Z")

</div>

The problem is not that the value contains the value\_split character, the problem is that the message contains the field\_split character escaped. It would help other folks if you adjusted the thread title.

The backslash in the message is escaping the comma so that it does not get matched by field\_split. You could try using a mutate+gsub to change it into one of

```
Path: C:\\,
Path: C:\ ,
```

---

<div class="post-metadata">

### Author: ![MakoWish](https://avatars.discourse-cdn.com/v4/letter/m/e56c9b/32.png) [@MakoWish](https://discuss.elastic.co/u/MakoWish)
#### Post date: [August 12, 2019, 8:51pm UTC](https://discuss.elastic.co/t/issue-with-kv-filter-when-value-ends-with-escape-character/190618/3 "2019-08-12T20:51:47Z")

</div>

@Badger

That was a poor example on my part. Let's say the path is 'C:\Program Files'. The backslash is not escaping any special characters in this instance. The issue is still the colon `:` Value Split character, not the comma.

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [August 12, 2019, 9:02pm UTC](https://discuss.elastic.co/t/issue-with-kv-filter-when-value-ends-with-escape-character/190618/4 "2019-08-12T21:02:49Z")

</div>

With this configuration

```
input { generator { count => 1 lines => ['File Name: foo.exe, Path: C:\Program Files, Drive Type: Internal Hard Drive'] } }
filter {
    kv {
        transform_key => "lowercase"
        trim_key => " " # Trim any leading or trailing spaces
        value_split => ":"
        field_split => "\,"
    }
}
output { stdout { codec => rubydebug { metadata => false } } }

```

I get

```
 "file name" => "foo.exe",
      "path" => "C:\\Program Files",
"drive type" => "Internal Hard Drive",

```

What issue do you have with that?

---

<div class="post-metadata">

### Author: ![MakoWish](https://avatars.discourse-cdn.com/v4/letter/m/e56c9b/32.png) [@MakoWish](https://discuss.elastic.co/u/MakoWish)
#### Post date: [August 12, 2019, 9:39pm UTC](https://discuss.elastic.co/t/issue-with-kv-filter-when-value-ends-with-escape-character/190618/5 "2019-08-12T21:39:18Z")

</div>

Since I could not get this figured out before, I have actually already moved away from the KV filter in favor of Grok patterns on this one. There are only five different patterns for this source, so it was quick and easy to do.

I just tested again, and you are correct. `D:\` and `D:\SomePath\` both cause the issue while `D:\SomePath` does not. The comma is being escaped as you mentioned.

I appreciate your responses on this one, and if I run into the issue again, I will use your gsub suggestion to ensure the field\_split character does not get escaped.

EDIT: Updated the post title.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [September 9, 2019, 9:47pm UTC](https://discuss.elastic.co/t/issue-with-kv-filter-when-value-ends-with-escape-character/190618/6 "2019-09-09T21:47:32Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
