# Ingest pipeline grok processor on\_failure

**URL:** <https://discuss.elastic.co/t/ingest-pipeline-grok-processor-on-failure/257755>\
**Category:** Elasticsearch\
**Tags:** ingest-pipeline\
**Created:** [December 5, 2020, 10:30pm UTC](https://discuss.elastic.co/t/ingest-pipeline-grok-processor-on-failure/257755 "2020-12-05T22:30:46Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![vsv0001](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vsv0001/32/124730_2.png) [@vsv0001](https://discuss.elastic.co/u/vsv0001)\
**Post date:** [December 5, 2020, 10:30pm UTC](https://discuss.elastic.co/t/ingest-pipeline-grok-processor-on-failure/257755/1 "2020-12-05T22:30:46Z")

</div>

Hi,  
I have an elastic ingest pipeline with grok processor defined along with error handling

```auto
{
  "my_ingest" : {
    "description" : "parse multiple patterns",
    "processors" : [
      {
        "grok" : {
          "field" : "message",
          "patterns" : [
            """^\[end] %{DATA:method} \'%{GREEDYDATA:url}' %{DATA:status} :: Duration: %{DATA:duration} ms""",
            """^\[start] %{DATA:method} \'%{GREEDYDATA:url}' :: Start Time:%{GREEDYDATA:starttime}""",
            "%{GREEDYDATA:message}"
          ]
        }
      }
    ],
    "on_failure" : [
      {
        "set" : {
          "field" : "_index",
          "value" : "failed-{{ _index }}"
        }
      }
    ]
  }
}

```

i am referring to this pipeline in my filebeat.yml  
the grok filters work when i do a simulate. in dev tools. But when i run the actual logging i do not see the log statements. it looks like they are failing to get parsed and not visible in kibana. i also don't see a new index created where i am hoping to see the errors logged as defined on on\_failure.  
can some one please suggest or give pointers for debugging the issue.

thanks

---

<div class="post-metadata">

**Author:** ![vsv0001](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vsv0001/32/124730_2.png) [@vsv0001](https://discuss.elastic.co/u/vsv0001)\
**Post date:** [December 7, 2020, 8:36pm UTC](https://discuss.elastic.co/t/ingest-pipeline-grok-processor-on-failure/257755/2 "2020-12-07T20:36:12Z")

</div>

Hi,  
how do i access this on\_failure\_processor\_type and on\_failure\_message from the metadata ?

```auto
"on_failure" : [
          {
            "set" : {
              "field" : "error",
              "value" : "{{_ingest.on_failure_processor_type }}-{{ _ingest.on_failure_message }}"
            }
          }
        ]

```

Thanks

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 4, 2021, 8:36pm UTC](https://discuss.elastic.co/t/ingest-pipeline-grok-processor-on-failure/257755/3 "2021-01-04T20:36:37Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.