# Space in key : remove\_char\_key is not working

**URL:** <https://discuss.elastic.co/t/space-in-key-remove-char-key-is-not-working/225873>\
**Category:** Logstash\
**Created:** [March 31, 2020, 1:46pm UTC](https://discuss.elastic.co/t/space-in-key-remove-char-key-is-not-working/225873 "2020-03-31T13:46:54Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![Travis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/travis/32/54079_2.png) [@Travis](https://discuss.elastic.co/u/Travis)\
**Post date:** [March 31, 2020, 1:46pm UTC](https://discuss.elastic.co/t/space-in-key-remove-char-key-is-not-working/225873/1 "2020-03-31T13:46:55Z")

</div>

Hello !

I have a nested part in a log I can't succeed to parse :

Here is my config :

```
filter {

#replace second "msg" field by "nested" field
  mutate {
    gsub => ["message", "Message=", " nested="]
  }

  kv {
    source => "message"
    field_split => " "
    value_split => "="
  }

  kv {
source => "nested"
remove_char_key => " "
value_split => ":"

  }
}

```

Result :

```auto
    {
        "<13>Mar 31 14:36:06 192.168.200.34 AgentDevice" => "WindowsLog",
                                              "Computer" => "WINCOLLECT_LOGSTASH",
                                          "RecordNumber" => "1467",
                                                 "Level" => "Log Always",
                                            "@timestamp" => 2020-03-31T12:36:14.998Z,
                                         "TimeGenerated" => "1585658165",
                                         "PluginVersion" => "7.2.9.96",
                                           "TimeWritten" => "1585658165",
                                                  "host" => "192.168.200.34",
                                              "Keywords" => "Audit Success",
                                               "message" => "<13>Mar 31 14:36:06 192.168.200.34 AgentDevice=WindowsLog\tAgentLogFile=Security\tPluginVersion=7.2.9.96\tSource=Microsoft-Windows-Security-Auditing\tComputer=WINCOLLECT_LOGSTASH\tOriginatingComputer=WINCOLLECT_LOGSTASH\tUser=\tDomain=\tEventID=4634\tEventIDCode=4634\tEventType=8\tEventCategory=12545\tRecordNumber=1467\tTimeGenerated=1585658165\tTimeWritten=1585658165\tLevel=Log Always\tKeywords=Audit Success\tTask=SE_ADT_LOGON_LOGOFF\tOpcode=Info\tnested=An account was logged off. Subject: Security ID: WINCOLLECT_LOGS\\Administrator Account Name: Administrator Account Domain: WINCOLLECT_LOGS Logon ID: 0xADD113 Logon Type: 7 This event is generated when a logon session is destroyed. It may be positively correlated with a logon event using the Logon ID value. Logon IDs are only unique between reboots on the same computer.",
                                          "AgentLogFile" => "Security",
                                               "EventID" => "4634",
                                                  "port" => 50120,
                                             "EventType" => "8",
                                               "Subject" => "Security",
                                                  "Type" => "7",
                                                "Domain" => "WINCOLLECT_LOGS",
                                   "OriginatingComputer" => "WINCOLLECT_LOGSTASH",
                                         "EventCategory" => "12545",
                                           "EventIDCode" => "4634",
                                              "@version" => "1",
                                                    "ID" => [
            [0] "WINCOLLECT_LOGS\\Administrator",
            [1] "0xADD113"
        ],
                                                  "Name" => "Administrator",
                                                "nested" => "An account was logged off. Subject: Security ID: WINCOLLECT_LOGS\\Administrator Account Name: Administrator Account Domain: WINCOLLECT_LOGS Logon ID: 0xADD113 Logon Type: 7 This event is generated when a logon session is destroyed. It may be positively correlated with a logon event using the Logon ID value. Logon IDs are only unique between reboots on the same computer.",
                                                  "Task" => "SE_ADT_LOGON_LOGOFF",
                                                  "type" => "syslog",
                                                "Opcode" => "Info",
                                                "Source" => "Microsoft-Windows-Security-Auditing"
    }

```

It shoud be "SecurityID" and not "ID"  
It should be "AccountDomain" and not "Domain"  
Ir should be "LogonType" and not "Type"

Thanks for your help !

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [March 31, 2020, 3:12pm UTC](https://discuss.elastic.co/t/space-in-key-remove-char-key-is-not-working/225873/2 "2020-03-31T15:12:30Z")

</div>

You have the field\_split character inside the keys. That will result in exactly what you see -- part of the key has no associated value and is therefore discarded.

---

<div class="post-metadata">

**Author:** ![Travis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/travis/32/54079_2.png) [@Travis](https://discuss.elastic.co/u/Travis)\
**Post date:** [March 31, 2020, 3:26pm UTC](https://discuss.elastic.co/t/space-in-key-remove-char-key-is-not-working/225873/3 "2020-03-31T15:26:14Z")

</div>

There is no field\_split setting for nested part (which the one I have issue with) :

```
  kv {
source => "nested"
remove_char_key => " "
value_split => ":"
  }

```

I taught that **remove\_char\_key** could remove the space _before_ the key (ID in this example) to transform

"Security ID: WINCOLLECT\_LOGS\Administrator" to "SecurityID: WINCOLLECT\_LOGS\Administrator"

But it doens't work. Or maybe I didn't understood the meaning of remove\_char\_key ?

Do you know a way to figure this out ?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [March 31, 2020, 3:42pm UTC](https://discuss.elastic.co/t/space-in-key-remove-char-key-is-not-working/225873/4 "2020-03-31T15:42:18Z")

</div>

> [@Travis](#):
>
> Do you know a way to figure this out ?

No, kv does not work the way you want it to work.

---

<div class="post-metadata">

**Author:** ![Travis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/travis/32/54079_2.png) [@Travis](https://discuss.elastic.co/u/Travis)\
**Post date:** [March 31, 2020, 3:51pm UTC](https://discuss.elastic.co/t/space-in-key-remove-char-key-is-not-working/225873/5 "2020-03-31T15:51:10Z")

</div>

Mmmk. Ok.

So I'm asking how users who have the same issue (space in the key) can deal with it ?

---

<div class="post-metadata">

**Author:** ![Travis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/travis/32/54079_2.png) [@Travis](https://discuss.elastic.co/u/Travis)\
**Post date:** [April 1, 2020, 5:47am UTC](https://discuss.elastic.co/t/space-in-key-remove-char-key-is-not-working/225873/6 "2020-04-01T05:47:12Z")

</div>

@Christian_Dahlqvist I've read some similar post from you but this does'nt apply to my needs. Can you please just confirm what I want to do is not possible using kv filter ?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [April 1, 2020, 7:53am UTC](https://discuss.elastic.co/t/space-in-key-remove-char-key-is-not-working/225873/7 "2020-04-01T07:53:32Z")

</div>

I agree with @Badger here.

---

<div class="post-metadata">

**Author:** ![Travis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/travis/32/54079_2.png) [@Travis](https://discuss.elastic.co/u/Travis)\
**Post date:** [April 1, 2020, 8:00am UTC](https://discuss.elastic.co/t/space-in-key-remove-char-key-is-not-working/225873/8 "2020-04-01T08:00:40Z")

</div>

Ok thanks. Is there exists any workaround ?

How could I parse this :

> "nested" =\> "An account was logged off. Subject: Security ID: WINCOLLECT\_LOGS\Administrator Account Name: Administrator Account Domain: WINCOLLECT\_LOGS Logon ID: 0xADD113 Logon Type: 7 This event is generated when a logon session is destroyed. It may be positively correlated with a logon event using the Logon ID value. Logon IDs are only unique between reboots on the same computer.",

In this :

Security ID: WINCOLLECT\_LOGS\Administrator  
Account Name: Administrator  
Account Domain: WINCOLLECT\_LOGS  
[...]

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [April 1, 2020, 8:03am UTC](https://discuss.elastic.co/t/space-in-key-remove-char-key-is-not-working/225873/9 "2020-04-01T08:03:43Z")

</div>

I would recommend either reformatting g tv bd message ahead of using the JVM filter or creating custom parsing through a ruby filter.

---

<div class="post-metadata">

**Author:** ![Travis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/travis/32/54079_2.png) [@Travis](https://discuss.elastic.co/u/Travis)\
**Post date:** [April 1, 2020, 8:08am UTC](https://discuss.elastic.co/t/space-in-key-remove-char-key-is-not-working/225873/10 "2020-04-01T08:08:27Z")

</div>

OK. I will give a try on ruby filter. Thank you @Christian_Dahlqvist and @Badger for your support 😉

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 29, 2020, 8:16am UTC](https://discuss.elastic.co/t/space-in-key-remove-char-key-is-not-working/225873/11 "2020-04-29T08:16:38Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
