# Need help to exlcude messages which are not having kv pair

**URL:** https://discuss.elastic.co/t/need-help-to-exlcude-messages-which-are-not-having-kv-pair/231894
**Category:** Logstash
**Created:** [May 10, 2020, 5:20pm UTC](https://discuss.elastic.co/t/need-help-to-exlcude-messages-which-are-not-having-kv-pair/231894 "2020-05-10T17:20:41Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![Nithani25](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nithani25/32/33457_2.png) [@Nithani25](https://discuss.elastic.co/u/Nithani25)
#### Post date: [May 10, 2020, 5:20pm UTC](https://discuss.elastic.co/t/need-help-to-exlcude-messages-which-are-not-having-kv-pair/231894/1 "2020-05-10T17:20:41Z")

</div>

HI Team,  
I have split last field of my message has greedydata

Example Message:

%{ts:timestampe} %{loglevel} %{DATA} %{GREEDYDATA:Logmessage}

My "Logmessage" here at times contains some kv pair in them and i am able to extract them via kv pair extraction as below.

kv { source =\> "Logmessage" field\_split =\> "," value\_split =\> "=" }

But this kv is just applicable only for the messages which contains the correct kv. there are some other messages which doesn't meet the criteria and i want to eliminate them and want them to be treated as Logmessage only.

Problem: these unwanted messages are creating multiple fields in elasticsearch.

---

<div class="post-metadata">

### Author: ![Luca\_Belluccini](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/luca_belluccini/32/33239_2.png) [@Luca\_Belluccini](https://discuss.elastic.co/u/Luca_Belluccini)
#### Post date: [May 10, 2020, 11:42pm UTC](https://discuss.elastic.co/t/need-help-to-exlcude-messages-which-are-not-having-kv-pair/231894/2 "2020-05-10T23:42:28Z")

</div>

Hello @Nithani25

Do the events which are not `kv` fail the `kv` filter or they go through it?

When the `kv` fails, the event is tagged (see [tag\_on\_failure](https://www.elastic.co/guide/en/logstash/current/plugins-filters-kv.html#plugins-filters-kv-tag_on_failure)).

If they're tagged, you can identify such events.

If you're able to distinguish between "correct" KV and not-KV `Logmessages` using a regular expression, I think the best would be to check it with the regular expression before applying the filter.

---

<div class="post-metadata">

### Author: ![Nithani25](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nithani25/32/33457_2.png) [@Nithani25](https://discuss.elastic.co/u/Nithani25)
#### Post date: [May 11, 2020, 6:15am UTC](https://discuss.elastic.co/t/need-help-to-exlcude-messages-which-are-not-having-kv-pair/231894/3 "2020-05-11T06:15:10Z")

</div>

Sample Log messages:

```
    CurrentURL = [/google/abc] , UserID = nishanth

    EVENTS[1ms]("RenderingManager.getRequestContext")
    {
      [getRequestContext: Creating RequestContext]=0
      [getRequestContext: Setting up RequestContext]=0
      [getRequestContext: Setting up OpenSessionsInViewHelper]=1
    }

{
  [cleanup: doCleanupSessionsInView]=0
}

```

while apply below kv filter

kv { source =\> "Logmessage" field\_split =\> "," value\_split =\> "=" }

I am seeing multiple fields being created, i want only the CurrentURL and UserId to captured and corresponding fields created for them.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [June 8, 2020, 6:15am UTC](https://discuss.elastic.co/t/need-help-to-exlcude-messages-which-are-not-having-kv-pair/231894/4 "2020-06-08T06:15:13Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
