# \[processors\] required property is missing

**URL:** <https://discuss.elastic.co/t/processors-required-property-is-missing/271554>\
**Category:** Elasticsearch\
**Created:** [April 28, 2021, 8:33pm UTC](https://discuss.elastic.co/t/processors-required-property-is-missing/271554 "2021-04-28T20:33:00Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![ankitdevnalkar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ankitdevnalkar/32/46158_2.png) [@ankitdevnalkar](https://discuss.elastic.co/u/ankitdevnalkar)\
**Post date:** [April 28, 2021, 8:33pm UTC](https://discuss.elastic.co/t/processors-required-property-is-missing/271554/1 "2021-04-28T20:33:00Z")

</div>

I am creating a ingest pipeline for custom index for Azure activity log with same configurations it has for Filebeat index. I copied pipeline content from `_ingest/pipeline`. Due to character limit, I am unable to paste the entire file here.

The error I am facing is :

```
{
  "error" : {
    "root_cause" : [
      {
        "type" : "parse_exception",
        "reason" : "[processors] required property is missing",
        "property_name" : "processors"
      }
    ],
    "type" : "parse_exception",
    "reason" : "[processors] required property is missing",
    "property_name" : "processors"
  },
  "status" : 400
}
```

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [April 28, 2021, 8:47pm UTC](https://discuss.elastic.co/t/processors-required-property-is-missing/271554/2 "2021-04-28T20:47:30Z")

</div>

You can share it on [gist.GitHub.com](http://gist.GitHub.com) and link here.

---

<div class="post-metadata">

**Author:** ![ankitdevnalkar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ankitdevnalkar/32/46158_2.png) [@ankitdevnalkar](https://discuss.elastic.co/u/ankitdevnalkar)\
**Post date:** [April 28, 2021, 8:55pm UTC](https://discuss.elastic.co/t/processors-required-property-is-missing/271554/3 "2021-04-28T20:55:53Z")

</div>

@dadoonet thanks ! here is the link

> <https://gist.github.com/ankitdevnalkar/6fbc7fcfe5c60a4d45f0be118f6ae04a>

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [April 28, 2021, 11:22pm UTC](https://discuss.elastic.co/t/processors-required-property-is-missing/271554/4 "2021-04-28T23:22:29Z")

</div>

You can't define 2 pipelines with one call.  
You need 2 calls, one for each pipeline.

See [Create or update pipeline API | Elasticsearch Guide [7.12] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-pipeline-api.html)

---

<div class="post-metadata">

**Author:** ![ankitdevnalkar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ankitdevnalkar/32/46158_2.png) [@ankitdevnalkar](https://discuss.elastic.co/u/ankitdevnalkar)\
**Post date:** [April 29, 2021, 4:02am UTC](https://discuss.elastic.co/t/processors-required-property-is-missing/271554/5 "2021-04-29T04:02:14Z")

</div>

I tried creating first one with the following and still getting same error.

```
PUT _ingest/pipeline/signals-azure-activitylogs-azure-shared-pipeline
{
  "signals-azure-activitylogs-azure-shared-pipeline" : {
    "description" : "Pipeline for parsing azure activity logs.",
    "processors" : [
      {
        "set" : {
          "field" : "cloud.provider",
          "value" : "azure"
        }
      },
      {
        "grok" : {
          "ignore_failure" : true,
          "field" : "azure.resource_id",
          "patterns" : [
            "/SUBSCRIPTIONS/%{SUBID:azure.subscription_id}/RESOURCEGROUPS/%{GROUPID:azure.resource.group}/PROVIDERS/%{PROVIDERNAME:azure.resource.provider}/NAMESPACES/%{NAMESPACE:azure.resource.namespace}/AUTHORIZATIONRULES/%{RULE:azure.resource.authorization_rule}"
          ],
          "pattern_definitions" : {
            "GROUPID" : ".+",
            "PROVIDERNAME" : ".+",
            "NAMESPACE" : ".+",
            "RULE" : ".+",
            "SUBID" : """(\{){0,1}[0-9a-fA-F]{8}\-[0-9a-fA-F]{4}\-[0-9a-fA-F]{4}\-[0-9a-fA-F]{4}\-[0-9a-fA-F]{12}(\}){0,1}"""
          }
        }
      },
      {
        "grok" : {
          "field" : "azure.resource_id",
          "patterns" : [
            "/SUBSCRIPTIONS/%{SUBID:azure.subscription_id}/RESOURCEGROUPS/%{GROUPID:azure.resource.group}/PROVIDERS/%{PROVIDERNAME:azure.resource.provider}/%{NAME:azure.resource.name}"
          ],
          "pattern_definitions" : {
            "SUBID" : """(\{){0,1}[0-9a-fA-F]{8}\-[0-9a-fA-F]{4}\-[0-9a-fA-F]{4}\-[0-9a-fA-F]{4}\-[0-9a-fA-F]{12}(\}){0,1}""",
            "GROUPID" : ".+",
            "PROVIDERNAME" : """([A-Z])\w+.([A-Z])\w+/([A-Z])\w+.""",
            "NAME" : "((?!AUTHORIZATIONRULES).)*$"
          },
          "ignore_failure" : true
        }
      },
      {
        "grok" : {
          "field" : "azure.resource_id",
          "patterns" : [
            "/providers/%{PROVIDER:azure.resource.provider}"
          ],
          "pattern_definitions" : {
            "PROVIDER" : ".+"
          },
          "ignore_failure" : true
        }
      },
      {
        "rename" : {
          "field" : "azure.resource_id",
          "target_field" : "azure.resource.id",
          "ignore_missing" : true
        }
      },
      {
        "lowercase" : {
          "ignore_missing" : true,
          "field" : "event.outcome"
        }
      }
    ],
    "on_failure" : [
      {
        "set" : {
          "field" : "error.message",
          "value" : "{{ _ingest.on_failure_message }}"
        }
      }
    ]
  }
}
```

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [April 29, 2021, 5:30am UTC](https://discuss.elastic.co/t/processors-required-property-is-missing/271554/6 "2021-04-29T05:30:07Z")

</div>

Read carefully the documentation I linked to.

You will see that you must remove:

```
signals-azure-activitylogs-azure-shared-pipeline" : {
```

---

<div class="post-metadata">

**Author:** ![ankitdevnalkar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ankitdevnalkar/32/46158_2.png) [@ankitdevnalkar](https://discuss.elastic.co/u/ankitdevnalkar)\
**Post date:** [April 29, 2021, 7:17am UTC](https://discuss.elastic.co/t/processors-required-property-is-missing/271554/7 "2021-04-29T07:17:26Z")

</div>

Thanks @dadoonet

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 27, 2021, 7:18am UTC](https://discuss.elastic.co/t/processors-required-property-is-missing/271554/8 "2021-05-27T07:18:21Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.