# Logstash kv field split problem

**URL:** https://discuss.elastic.co/t/logstash-kv-field-split-problem/259552
**Category:** Logstash
**Created:** [December 24, 2020, 6:38am UTC](https://discuss.elastic.co/t/logstash-kv-field-split-problem/259552 "2020-12-24T06:38:19Z")
**Posts on this page:** 9
**Page:** 1

<div class="post-metadata">

### Author: ![abu.sayeed](https://avatars.discourse-cdn.com/v4/letter/a/dec6dc/32.png) [@abu.sayeed](https://discuss.elastic.co/u/abu.sayeed)
#### Post date: [December 24, 2020, 6:38am UTC](https://discuss.elastic.co/t/logstash-kv-field-split-problem/259552/1 "2020-12-24T06:38:19Z")

</div>

Logs like this:

**topic = abc, partition = 2, offset = 4386, serialized key size = -1, serialized value size = 1139, key = null, value = {"Name":"pc-1","test":false,"message":"elk test - Your pc-1 better. Thank you.","Count":1,"valid":true,"includeResend":false,}**

My logstash config file looks like below:

**kv {  
value\_split =\> ","  
value\_split =\> "="  
allow\_duplicate\_values =\> false  
}**

Fields look like below:  
topic = abc  
partition = 2  
offset = 4386  
serialized key size = -1  
serialized value size = 1139  
key = null  
value = {"Name":"pc-1"

But I need fields below:

topic = abc  
partition = 2  
offset = 4386  
serialized key size = -1  
serialized value size = 1139  
key = null  
value = {"Name":"pc-1","test":false,"message":"elk test - Your pc-1 better. Thank you.","Count":1,"valid":true,"includeResend":false,}

So I wish help to solve this. Thanks.

---

<div class="post-metadata">

### Author: ![fadjar340](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fadjar340/32/43610_2.png) [@fadjar340](https://discuss.elastic.co/u/fadjar340)
#### Post date: [December 24, 2020, 8:44am UTC](https://discuss.elastic.co/t/logstash-kv-field-split-problem/259552/2 "2020-12-24T08:44:10Z")

</div>

Better you use `Dissect` to archive what you want.  
Below thew snippet:

```auto
"dissect": {
        "mapping" => {
        "message" => "%{*topic} = %{&topic}, %{*partition} = %{&partition}, %{*offset} = %{&offset}, %{*serialized_key->} = %{&serialized_key}, %{*serialized_value->} = %{&serialized_value}, %{*key} = %{&key}, %{*value} = %{&value->}"
         }
  }

```

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [December 24, 2020, 4:34pm UTC](https://discuss.elastic.co/t/logstash-kv-field-split-problem/259552/3 "2020-12-24T16:34:40Z")

</div>

You could also use grok to pull out [value], then mutate+gsub to remove it before running the message through the kv filter.

```
grok { match => { "message" => "value = %{DATA:value}$" } }
mutate { gsub => ["message", "value = .*$", ""] }
kv ...
```

---

<div class="post-metadata">

### Author: ![fadjar340](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fadjar340/32/43610_2.png) [@fadjar340](https://discuss.elastic.co/u/fadjar340)
#### Post date: [December 25, 2020, 8:25am UTC](https://discuss.elastic.co/t/logstash-kv-field-split-problem/259552/4 "2020-12-25T08:25:23Z")

</div>

@Badger  
Which one is fastest? Grok or dissect?  
Need your comment to make my parsers work more efficient... 🙂

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [December 25, 2020, 4:03pm UTC](https://discuss.elastic.co/t/logstash-kv-field-split-problem/259552/5 "2020-12-25T16:03:28Z")

</div>

> [@fadjar340](#):
>
> Which one is fastest? Grok or dissect?

No way to know for sure without benchmarking it. In general dissect has much less functionality so it tends to be cheaper. If grok has to do a lot of back-tracking it can be really expensive (hence the default 30 second time limit to match a pattern), but I would expect a pattern that is anchored at the beginning (by "value =") and then captures the rest of the string to be cheaper than most groks.

---

<div class="post-metadata">

### Author: ![ylasri](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ylasri/32/86120_2.png) [@ylasri](https://discuss.elastic.co/u/ylasri)
#### Post date: [December 25, 2020, 4:10pm UTC](https://discuss.elastic.co/t/logstash-kv-field-split-problem/259552/6 "2020-12-25T16:10:45Z")

</div>

Here is an [article](https://web.leandrojmp.com/posts/en/2020/08/logstash-grok-vs-dissect) that give an idea

---

<div class="post-metadata">

### Author: ![fadjar340](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fadjar340/32/43610_2.png) [@fadjar340](https://discuss.elastic.co/u/fadjar340)
#### Post date: [December 25, 2020, 4:24pm UTC](https://discuss.elastic.co/t/logstash-kv-field-split-problem/259552/7 "2020-12-25T16:24:20Z")

</div>

Thanks @ylasri.

Regards,  
Fadjar

---

<div class="post-metadata">

### Author: ![abu.sayeed](https://avatars.discourse-cdn.com/v4/letter/a/dec6dc/32.png) [@abu.sayeed](https://discuss.elastic.co/u/abu.sayeed)
#### Post date: [December 27, 2020, 10:00am UTC](https://discuss.elastic.co/t/logstash-kv-field-split-problem/259552/8 "2020-12-27T10:00:10Z")

</div>

Wow its working better. Thanks all for helping me.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 24, 2021, 10:00am UTC](https://discuss.elastic.co/t/logstash-kv-field-split-problem/259552/9 "2021-01-24T10:00:25Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
