# Logstash KV filter - Wrong parsing

**URL:** <https://discuss.elastic.co/t/logstash-kv-filter-wrong-parsing/282333>\
**Category:** Logstash\
**Created:** [August 24, 2021, 10:26am UTC](https://discuss.elastic.co/t/logstash-kv-filter-wrong-parsing/282333 "2021-08-24T10:26:52Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![vasek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vasek/32/136636_2.png) [@vasek](https://discuss.elastic.co/u/vasek)\
**Post date:** [August 24, 2021, 10:26am UTC](https://discuss.elastic.co/t/logstash-kv-filter-wrong-parsing/282333/1 "2021-08-24T10:26:52Z")

</div>

Hello,  
I'm using KV filter in Logstash. Logstash configuration:

```auto
filter {

  #PARSING LOG
  #------------------------------------------

  grok {
      patterns_dir => ["/etc/logstash/grok-patterns"]
      break_on_match => true
      match => { "message" => ["%{CYBERARC_LOG}"] }
      overwrite => ["message"]
      tag_on_failure => ["not_parsed", "not_parsed_cyberarc"]
  }

  #MAPPING DATE
  #------------------------------------------

  if [timestamp] {

    #https://discuss.elastic.co/t/logstash-date-parse-failure-for-jdbc-input/92713/2
    #https://github.com/logstash-plugins/logstash-filter-date/issues/95 

    mutate {
      convert => { "timestamp" => "string" }
    }

    date {
      locale => "en"
      match => ["timestamp", "ISO8601", "yyyy-MM-dd'T'HH:mm:ss"]
      target => "@timestamp"
      remove_field => ["timestamp"]
    }
  }

  # KV FILTER
  #------------------------------------------

  if [cef_message] {
    kv {
      source => "cef_message"
      transform_key => "lowercase"
      trim_key => "no"
      trim_value => "no"
      field_split => " "
      whitespace => "strict"
      prefix => "kv_"
      #remove_field => ["cef_message"]
    }
  }
}

```

There is a log:

```auto
<5>1 2021-08-24T09:13:26Z CDIS000PHANT642 CEF:0|Cyber-Ark|Vault|12.2.0000|99|Open File|5|act=\"Open File\" fname=\"Root\\Policies\\Policy-CyberArkPTA.ini\" cs1Label=\"Affected User Name\" cs1=\"test\" cs2Label=\"Safe Name\" cs2=\"PasswordManagerShared\" cs3Label=\"Device Type\" cs3=\"test\" cs4Label=\"Other information message\" cs4=\"information\" cs5Label=\"Other information\" cs5=\"test\" cn1Label=\"Request Id\" cn1=\"test\" cn2Label=\"Ticket Id\" cn2=\"test\" 

```

From message was parsed a field with name **cef\_message**.

```auto
"cef_message": "act=\"Open File\" fname=\"Root\\Policies\\Policy-CyberArkPTA.ini\" cs1Label=\"Affected User Name\" cs1=\"test\" cs2Label=\"Safe Name\" cs2=\"PasswordManagerShared\" cs3Label=\"Device Type\" cs3=\"test\" cs4Label=\"Other information message\" cs4=\"information\" cs5Label=\"Other information\" cs5=\"test\" cn1Label=\"Request Id\" cn1=\"test\" cn2Label=\"Ticket Id\" cn2=\"test\" "

```

KV filter was applied on **cef\_field** field.

Very strange behaving is that information is trimmed to informati.

```auto
    "kv_act": "Open File"
    "kv_cn1": "test",
    "kv_cn1label": "Request Id",
    "kv_cn2": "test",
    "kv_cn2label": "Ticket Id",
    "kv_cs1": "test",
    "kv_cs1label": "Affected User Name",
    "kv_cs2": "PasswordManagerShared",
    "kv_cs2label": "Safe Name",
    "kv_cs3": "test",
    "kv_cs3label": "Device Type",
    "kv_cs4": "informati",
    "kv_cs4label": "Other information message",
    "kv_cs5": "test",
    "kv_cs5label": "Other informati",
    "kv_fname": "Root\\Policies\\Policy-CyberArkPTA.ini",

```

Look at fields:

```auto
  "kv_cs5label": "Other informati"
  "kv_cs4label": "Other information message"
  "kv_cs4": "informati"

```

If the text ends with "on", it removes the text "on".

I'm using Logstash 7.8.1. Can anyone help me to resolve this problem?

---

<div class="post-metadata">

**Author:** ![Cad](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cad/32/86661_2.png) [@Cad](https://discuss.elastic.co/u/Cad)\
**Post date:** [August 24, 2021, 12:14pm UTC](https://discuss.elastic.co/t/logstash-kv-filter-wrong-parsing/282333/2 "2021-08-24T12:14:32Z")

</div>

Hi,

I think the error is because of the `trim_value => "no"`. Indeed, the [code](https://github.com/logstash-plugins/logstash-filter-kv/blob/371d9e1e2e731ba8657abe5afee56d5a99a3cece/lib/logstash/filters/kv.rb#L374) for the trim with your values give a regex like this `^[no]+|[no]+$` and this regex on the word `information` match the two last values.  
So the trim\_value is a string who contains character you want to remove, it don't only remove the string itself.

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [August 24, 2021, 12:23pm UTC](https://discuss.elastic.co/t/logstash-kv-filter-wrong-parsing/282333/3 "2021-08-24T12:23:22Z")

</div>

Also, your input seems to be a syslog message with a CEF part, you can try the `cef` codec directly in your input, this way you won't need neither `grok` nor `kv`, the `cef` codec would parse your message.

---

<div class="post-metadata">

**Author:** ![vasek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vasek/32/136636_2.png) [@vasek](https://discuss.elastic.co/u/vasek)\
**Post date:** [August 24, 2021, 12:36pm UTC](https://discuss.elastic.co/t/logstash-kv-filter-wrong-parsing/282333/4 "2021-08-24T12:36:31Z")

</div>

Hi @Cad,  
**thank you** for your solution! I thought that `trim_value` was supposed to take the value of a boolean - aaa, such as mistake 🙉.

---

<div class="post-metadata">

**Author:** ![vasek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vasek/32/136636_2.png) [@vasek](https://discuss.elastic.co/u/vasek)\
**Post date:** [August 24, 2021, 12:37pm UTC](https://discuss.elastic.co/t/logstash-kv-filter-wrong-parsing/282333/5 "2021-08-24T12:37:26Z")

</div>

Hi @leandrojmp . Thank you for the tip 😉 I'll try it.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 21, 2021, 12:38pm UTC](https://discuss.elastic.co/t/logstash-kv-filter-wrong-parsing/282333/6 "2021-09-21T12:38:19Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.