# Using kv and Grok Pattern in Ingest Pipelines

**URL:** https://discuss.elastic.co/t/using-kv-and-grok-pattern-in-ingest-pipelines/263997
**Category:** Elasticsearch
**Tags:** ingest-pipeline
**Created:** [February 11, 2021, 10:50am UTC](https://discuss.elastic.co/t/using-kv-and-grok-pattern-in-ingest-pipelines/263997 "2021-02-11T10:50:22Z")
**Posts on this page:** 2
**Page:** 1

<div class="post-metadata">

### Author: ![geek876](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/geek876/32/9329_2.png) [@geek876](https://discuss.elastic.co/u/geek876)
#### Post date: [February 11, 2021, 10:50am UTC](https://discuss.elastic.co/t/using-kv-and-grok-pattern-in-ingest-pipelines/263997/1 "2021-02-11T10:50:22Z")

</div>

I am trying to extract a log-pattern

`[LEVEL] (COMPONENT) MESSAGE [msgid: MESSAGE ID]`

example message:

```auto
[INFO] (core.workflow) Start scan, workflow_id='lms::workflow::WorkflowExecutor(0x259c99f94b0, name = "WF-127.0.0.1:60708")', dataId='4d66c5069365476a8f36431e6369ab0b', fileName='winlogbeat.yml' [msgid: 3491]

```

In the pipeline I have two filters:

(1) Grok: That extracts level, component, kvmessage and mid - This works.  
(2) kv: I want to now split values like `workflow_id`, `name` etc as key value pairs. However, this is not working. Would appreciate some assistance.

```auto
    [
      {
        "grok": {
          "field": "message",
          "patterns": [
            "\\[%{WORD:level}\\s*\\]: \\(%{COMPONENT:component}\\) %{MSG:kvmessage}\\[%{GREEDYDATA:mid}\\]"
          ],
          "pattern_definitions": {
            "COMPONENT": "\\w*.\\w*",
            "STATUS": "\\w*\\s\\w*",
            "MSG": "[^\\[]*"
          },
          "ignore_missing": true
        }
      },
      {
        "kv": {
          "field": "kvmessage",
          "field_split": ",",
          "value_split": "="
        }
      }
    ]

```

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 11, 2021, 10:51am UTC](https://discuss.elastic.co/t/using-kv-and-grok-pattern-in-ingest-pipelines/263997/2 "2021-03-11T10:51:00Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
