# Multiple Grok Processors in pipeline: "\[processors\] required property is missing"

**URL:** <https://discuss.elastic.co/t/multiple-grok-processors-in-pipeline-processors-required-property-is-missing/236741>\
**Category:** Elasticsearch\
**Created:** [June 11, 2020, 4:02pm UTC](https://discuss.elastic.co/t/multiple-grok-processors-in-pipeline-processors-required-property-is-missing/236741 "2020-06-11T16:02:54Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![KookieMeister](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kookiemeister/32/70182_2.png) [@KookieMeister](https://discuss.elastic.co/u/KookieMeister)\
**Post date:** [June 11, 2020, 4:02pm UTC](https://discuss.elastic.co/t/multiple-grok-processors-in-pipeline-processors-required-property-is-missing/236741/1 "2020-06-11T16:02:54Z")

</div>

Hi there,

I'm currently using the following pipeline with success:

```auto

{
    "ingestpipeline" : {
      "description" : "...",
      "processors" : [
        {
          "grok" : {
            "field" : "message",
            "patterns" : [
              "(?<loglevel>([Tt]race|TRACE|[Dd]ebug|DEBUG|[Nn]otice|NOTICE|[Ii]nformation|INFO|[Ww]arn?(?:ing)?|WARN?(?:ING)?|Error|ERROR|[Cc]rit?(?:ical)?|CRIT?(?:ICAL)?|[Ff]atal|FATAL|[Ss]evere|SEVERE)).*?(?<useremail>[a-zA-Z0-9_.+=:-]+@[0-9A-Za-z][0-9A-Za-z-]{0,62}(?:\\.(?:[0-9A-Za-z][0-9A-Za-z-]{0,62}))*)"
            ],
            "on_failure" : [
              {
                "grok" : {
                  "field" : "message",
                  "patterns" : [
                    "(?<useremail>[a-zA-Z0-9_.+=:-]+@[0-9A-Za-z][0-9A-Za-z-]{0,62}(?:\\.(?:[0-9A-Za-z][0-9A-Za-z-]{0,62}))*)"
                  ],
                  "on_failure" : [
                    {
                      "grok" : {
                        "field" : "message",
                        "patterns" : [
                          "(?<loglevel>([Tt]race|TRACE|[Dd]ebug|DEBUG|[Nn]otice|NOTICE|[Ii]nformation|INFO|[Ww]arn?(?:ing)?|WARN?(?:ING)?|Error|ERROR|[Cc]rit?(?:ical)?|CRIT?(?:ICAL)?|[Ff]atal|FATAL|[Ss]evere|SEVERE))"
                        ],
                        "ignore_failure" : true
                      }
                    }
                  ]
                }
              }
            ]
          }
        }
      ]
    }
  }

```

After using that, I decided to add a second grok processor in the pipeline for extraction from a different source field: log. But whenever I try to use that new pipeline config, I'm given the error message:

```auto
{
  "error": {
    "root_cause": [
      {
        "type": "parse_exception",
        "reason": "[processors] required property is missing",
        "property_name": "processors"
      }
    ],
    "type": "parse_exception",
    "reason": "[processors] required property is missing",
    "property_name": "processors"
  },
  "status": 400
}

```

Here is the pipeline configuration I'm trying to use:

```auto
{
    "ingestpipeline" : {
      "description" : "...",
      "processors" : [
        {
          "grok" : {
            "field" : "message",
            "patterns" : [
              "(?<loglevel>([Tt]race|TRACE|[Dd]ebug|DEBUG|[Nn]otice|NOTICE|[Ii]nformation|INFO|[Ww]arn?(?:ing)?|WARN?(?:ING)?|Error|ERROR|[Cc]rit?(?:ical)?|CRIT?(?:ICAL)?|[Ff]atal|FATAL|[Ss]evere|SEVERE)).*?(?<useremail>[a-zA-Z0-9_.+=:-]+@[0-9A-Za-z][0-9A-Za-z-]{0,62}(?:\\.(?:[0-9A-Za-z][0-9A-Za-z-]{0,62}))*)"
            ],
            "on_failure" : [
              {
                "grok" : {
                  "field" : "message",
                  "patterns" : [
                    "(?<useremail>[a-zA-Z0-9_.+=:-]+@[0-9A-Za-z][0-9A-Za-z-]{0,62}(?:\\.(?:[0-9A-Za-z][0-9A-Za-z-]{0,62}))*)"
                  ],
                  "on_failure" : [
                    {
                      "grok" : {
                        "field" : "message",
                        "patterns" : [
                          "(?<loglevel>([Tt]race|TRACE|[Dd]ebug|DEBUG|[Nn]otice|NOTICE|[Ii]nformation|INFO|[Ww]arn?(?:ing)?|WARN?(?:ING)?|Error|ERROR|[Cc]rit?(?:ical)?|CRIT?(?:ICAL)?|[Ff]atal|FATAL|[Ss]evere|SEVERE))"
                        ],
                        "ignore_failure" : true
                      }
                    }
                  ]
                }
              }
            ]
          }
        },
        {
            "grok" : {
                "field" : "log",
                "patterns" : [
                  "(?<loglevel>([Tt]race|TRACE|[Dd]ebug|DEBUG|[Nn]otice|NOTICE|[Ii]nformation|INFO|[Ww]arn?(?:ing)?|WARN?(?:ING)?|[Ee]rror|ERROR|[Cc]rit?(?:ical)?|CRIT?(?:ICAL)?|[Ff]atal|FATAL|[Ss]evere|SEVERE)).*?(?<useremail>[a-zA-Z0-9_.+=:-]+@[0-9A-Za-z][0-9A-Za-z-]{0,62}(?:\\.(?:[0-9A-Za-z][0-9A-Za-z-]{0,62}))*)"
                ],
                "on_failure" : [
                  {
                    "grok" : {
                      "field" : "log",
                      "patterns" : [
                        "(?<useremail>[a-zA-Z0-9_.+=:-]+@[0-9A-Za-z][0-9A-Za-z-]{0,62}(?:\\.(?:[0-9A-Za-z][0-9A-Za-z-]{0,62}))*)"
                      ],
                      "on_failure" : [
                        {
                          "grok" : {
                            "field" : "log",
                            "patterns" : [
                              "(?<loglevel>([Tt]race|TRACE|[Dd]ebug|DEBUG|[Nn]otice|NOTICE|[Ii]nformation|INFO|[Ww]arn?(?:ing)?|WARN?(?:ING)?|[Ee]rror|ERROR|[Cc]rit?(?:ical)?|CRIT?(?:ICAL)?|[Ff]atal|FATAL|[Ss]evere|SEVERE))"
                            ],
                            "ignore_failure" : true
                          }
                        }
                      ]
                    }
                  }
                ]
            }
        }
      ]
    }
  }

```

I'm using AWS Elasticsearch Service OpenDistro version 7.1

---

<div class="post-metadata">

**Author:** ![KookieMeister](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kookiemeister/32/70182_2.png) [@KookieMeister](https://discuss.elastic.co/u/KookieMeister)\
**Post date:** [June 12, 2020, 1:28pm UTC](https://discuss.elastic.co/t/multiple-grok-processors-in-pipeline-processors-required-property-is-missing/236741/2 "2020-06-12T13:28:47Z")

</div>

I'm going to close this issue. It turned out that I was adding in an additional field in the file that I didn't need, the ingestpipeline name with definition. I was using GET to get the current pipeline but was not removing the name definition before PUTing the new modified version. It should have looked like the below:

```auto
{
      "description" : "Ingest pipeline for all incoming elasticsearch data",
      "processors" : [
        {
          "grok" : {
            "field" : "message",
            "patterns" : [
              "(?<loglevel>([Tt]race|TRACE|[Dd]ebug|DEBUG|[Nn]otice|NOTICE|[Ii]nformation|INFO|[Ww]arn?(?:ing)?|WARN?(?:ING)?|Error|ERROR|[Cc]rit?(?:ical)?|CRIT?(?:ICAL)?|[Ff]atal|FATAL|[Ss]evere|SEVERE)).*?(?<useremail>[a-zA-Z0-9_.+=:-]+@[0-9A-Za-z][0-9A-Za-z-]{0,62}(?:\\.(?:[0-9A-Za-z][0-9A-Za-z-]{0,62}))*)"
            ],
            "on_failure" : [
              {
                "grok" : {
                  "field" : "message",
                  "patterns" : [
                    "(?<useremail>[a-zA-Z0-9_.+=:-]+@[0-9A-Za-z][0-9A-Za-z-]{0,62}(?:\\.(?:[0-9A-Za-z][0-9A-Za-z-]{0,62}))*)"
                  ],
                  "on_failure" : [
                    {
                      "grok" : {
                        "field" : "message",
                        "patterns" : [
                          "(?<loglevel>([Tt]race|TRACE|[Dd]ebug|DEBUG|[Nn]otice|NOTICE|[Ii]nformation|INFO|[Ww]arn?(?:ing)?|WARN?(?:ING)?|Error|ERROR|[Cc]rit?(?:ical)?|CRIT?(?:ICAL)?|[Ff]atal|FATAL|[Ss]evere|SEVERE))"
                        ],
                        "ignore_failure" : true
                      }
                    }
                  ]
                }
              }
            ]
          }
        },
        {
          "grok" : {
            "field" : "log",
            "patterns" : [
              "(?<loglevel>([Tt]race|TRACE|[Dd]ebug|DEBUG|[Nn]otice|NOTICE|[Ii]nformation|INFO|[Ii]nfo|[Ww]arn?(?:ing)?|WARN?(?:ING)?|[Ee]rror|ERROR|[Cc]rit?(?:ical)?|CRIT?(?:ICAL)?|[Ff]atal|FATAL|[Ss]evere|SEVERE)).*?(?<useremail>[a-zA-Z0-9_.+=:-]+@[0-9A-Za-z][0-9A-Za-z-]{0,62}(?:\\.(?:[0-9A-Za-z][0-9A-Za-z-]{0,62}))*)"
            ],
            "on_failure" : [
              {
                "grok" : {
                  "field" : "log",
                  "patterns" : [
                    "(?<useremail>[a-zA-Z0-9_.+=:-]+@[0-9A-Za-z][0-9A-Za-z-]{0,62}(?:\\.(?:[0-9A-Za-z][0-9A-Za-z-]{0,62}))*)"
                  ],
                  "on_failure" : [
                    {
                      "grok" : {
                        "field" : "log",
                        "patterns" : [
                          "(?<loglevel>([Tt]race|TRACE|[Dd]ebug|DEBUG|[Nn]otice|NOTICE|[Ii]nformation|INFO|[Ii]nfo|[Ww]arn?(?:ing)?|WARN?(?:ING)?|[Ee]rror|ERROR|[Cc]rit?(?:ical)?|CRIT?(?:ICAL)?|[Ff]atal|FATAL|[Ss]evere|SEVERE))"
                        ],
                        "ignore_failure" : true
                      }
                    }
                  ]
                }
              }
            ]
          }
        }
      ]
    }
  

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 10, 2020, 1:33pm UTC](https://discuss.elastic.co/t/multiple-grok-processors-in-pipeline-processors-required-property-is-missing/236741/3 "2020-07-10T13:33:23Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.