# Ingest pipeline not working correctly?

**URL:** <https://discuss.elastic.co/t/ingest-pipeline-not-working-correctly/253291>\
**Category:** Elasticsearch\
**Created:** [October 26, 2020, 10:45am UTC](https://discuss.elastic.co/t/ingest-pipeline-not-working-correctly/253291 "2020-10-26T10:45:51Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![iamtheliquor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/iamtheliquor/32/60472_2.png) [@iamtheliquor](https://discuss.elastic.co/u/iamtheliquor)\
**Post date:** [October 26, 2020, 10:45am UTC](https://discuss.elastic.co/t/ingest-pipeline-not-working-correctly/253291/1 "2020-10-26T10:45:52Z")

</div>

Hi everybody,

We have two pipelines, that are identical, one is for staging and one is for production logs;

```auto
    {
      "staging_pipeline" : {
        "description" : "Staging Pipeline",
        "processors" : [
          {
            "grok" : {
              "field" : "message",
              "patterns" : [
                "%{GREEDYDATA:log}"
              ],
              "on_failure" : [
                {
                  "set" : {
                    "field" : "error",
                    "value" : "{{ _ingest.on_failure_message }}"
                  }
                }
              ]
            }
          },
          {
            "json" : {
              "field" : "log",
              "on_failure" : [
                {
                  "set" : {
                    "field" : "error",
                    "value" : "{{ _ingest.on_failure_message }}"
                  }
                }
              ]
            }
          }
        ]
      }
    }

```

The difference is literally the word staging is production.

In the staging environment this does what we expect, and creates fields called log.whatever.is.in.the.json. However, in production, we get the following error;

> Provided Grok expressions do not match field value:

Then the contents of the message, which is all in standard json format, and we can verify that in the actual message field.

I don't know why it's complaining about the grok expression in one but not the other, as the pipelines are identical, as are the logs coming in. We have tried completely deleting the production pipeline and recreating it, but the same thing happens.

Any thoughts would be much appreciated.

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [October 26, 2020, 12:25pm UTC](https://discuss.elastic.co/t/ingest-pipeline-not-working-correctly/253291/2 "2020-10-26T12:25:06Z")

</div>

can you provide a **fully reprodicible example** with both pipelines and a [simulate pipeline](https://www.elastic.co/guide/en/elasticsearch/reference/7.9/simulate-pipeline-api.html) call for each of them showing the problem? That would help a lot!

Thanks!

---

<div class="post-metadata">

**Author:** ![iamtheliquor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/iamtheliquor/32/60472_2.png) [@iamtheliquor](https://discuss.elastic.co/u/iamtheliquor)\
**Post date:** [October 27, 2020, 10:36am UTC](https://discuss.elastic.co/t/ingest-pipeline-not-working-correctly/253291/3 "2020-10-27T10:36:55Z")

</div>

Okay, so I think I am doing this correctly, here is the example log coming from the application;

```auto
{"message":"Notified event \"kernel.request\" to listener \"Symfony\\Component\\HttpKernel\\EventListener\\ValidateRequestListener::onKernelRequest\".","context":{"event":"kernel.request","listener":"Symfony\\Component\\HttpKernel\\EventListener\\ValidateRequestListener::onKernelRequest"},"level":100,"level_name":"DEBUG","channel":"event","datetime":{"date":"2020-10-27 10:19:40.978821","timezone_type":3,"timezone":"UTC"},"extra":[]}`

```

Here is me simulating it through the pipelines;

Staging;

```auto
POST /_ingest/pipeline/staging_pl/_simulate
{
  "docs": [
    {
        "_index": "index",
        "_id": "id",
        "_source": {"message":"Notified event \"kernel.request\" to listener \"Symfony\\Component\\HttpKernel\\EventListener\\ValidateRequestListener::onKernelRequest\".","context":{"event":"kernel.request","listener":"Symfony\\Component\\HttpKernel\\EventListener\\ValidateRequestListener::onKernelRequest"},"level":100,"level_name":"DEBUG","channel":"event","extra":[]}
    }
  ]
}

```

Response;

```auto
{
  "docs" : [
    {
      "doc" : {
        "_index" : "index",
        "_type" : "_doc",
        "_id" : "id",
        "_source" : {
          "level_name" : "DEBUG",
          "level" : 100,
          "log" : """Notified event "kernel.request" to listener "Symfony\Component\HttpKernel\EventListener\ValidateRequestListener::onKernelRequest".""",
          "extra" : [],
          "context" : {
            "listener" : """Symfony\Component\HttpKernel\EventListener\ValidateRequestListener::onKernelRequest""",
            "event" : "kernel.request"
          },
          "channel" : "event",
          "message" : """Notified event "kernel.request" to listener "Symfony\Component\HttpKernel\EventListener\ValidateRequestListener::onKernelRequest".""",
          "error" : """Unrecognized token 'Notified': was expecting 'null', 'true', 'false' or NaN\n at [Source: org.elasticsearch.common.bytes.BytesReference$MarkSupportingStreamInputWrapper@322f9cd2; line: 1, column: 10]"""
        },
        "_ingest" : {
          "timestamp" : "2020-10-27T10:22:01.769671Z"
        }
      }
    }
  ]
}

```

Production;

```auto
POST /_ingest/pipeline/production_pl/_simulate
{
  "docs": [
    {
      "_index": "index",
      "_id": "id",
      "_source": {"message":"Notified event \"kernel.request\" to listener \"Symfony\\Component\\HttpKernel\\EventListener\\ValidateRequestListener::onKernelRequest\".","context":{"event":"kernel.request","listener":"Symfony\\Component\\HttpKernel\\EventListener\\ValidateRequestListener::onKernelRequest"},"level":100,"level_name":"DEBUG","channel":"event","extra":[]}
    }
  ]
}

```

Response;

```auto
{
  "docs" : [
    {
      "doc" : {
        "_index" : "index",
        "_type" : "_doc",
        "_id" : "id",
        "_source" : {
          "level_name" : "DEBUG",
          "level" : 100,
          "log" : """Notified event "kernel.request" to listener "Symfony\Component\HttpKernel\EventListener\ValidateRequestListener::onKernelRequest".""",
          "extra" : [],
          "context" : {
            "listener" : """Symfony\Component\HttpKernel\EventListener\ValidateRequestListener::onKernelRequest""",
            "event" : "kernel.request"
          },
          "channel" : "event",
          "message" : """Notified event "kernel.request" to listener "Symfony\Component\HttpKernel\EventListener\ValidateRequestListener::onKernelRequest".""",
          "error" : """Unrecognized token 'Notified': was expecting 'null', 'true', 'false' or NaN\n at [Source: org.elasticsearch.common.bytes.BytesReference$MarkSupportingStreamInputWrapper@463937d9; line: 1, column: 10]"""
        },
        "_ingest" : {
          "timestamp" : "2020-10-27T10:31:58.385232Z"
        }
      }
    }
  ]
}

```

Both response look exactly the same to me. They are complaining about something;

```auto
"error" : """Unrecognized token 'Notified': was expecting 'null', 'true', 'false' or NaN\n at [Source: org.elasticsearch.common.bytes.BytesReference$MarkSupportingStreamInputWrapper@463937d9; line: 1, column: 10]""" },

```

But still, staging populates the fields correctly, but production does not.

Perhaps I am not doing something right here?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 24, 2020, 10:37am UTC](https://discuss.elastic.co/t/ingest-pipeline-not-working-correctly/253291/4 "2020-11-24T10:37:06Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.