"processor [remove] doesn't support one or more provided configuration parameters [ignore_missing]

Elastic Stack Elasticsearch
1

I'm studying the ingest node and just write a simple pipeline,

PUT _ingest/pipeline/my-lab1
{
    "description": "to parse the filebeat message",
    "version": 101, 
    "processors": [
      {
        "kv": {
            "field": "message",
            "field_split": ",",
            "value_split": "="
        }
      },
      {
      "rename": {
        "field": "foo",
        "target_field": "foobar",
        "ignore_missing": false
        }
      },
      {
      "remove": {
        "field": "foo",
        "ignore_missing": false
        }
      },
      {
        "date_index_name": {
          "field" : "lot_start_time",
          "index_name_prefix": "tester-data-",
          "date_rounding" : "M"
        }

      }
    ]
}

But this return the error:

{
  "error": {
    "root_cause": [
      {
        "type": "parse_exception",
        "reason": "processor [remove] doesn't support one or more provided configuration parameters [ignore_missing]",
        "header": {
          "processor_type": "remove"
        }
      }
    ],
    "type": "parse_exception",
    "reason": "processor [remove] doesn't support one or more provided configuration parameters [ignore_missing]",
    "header": {
      "processor_type": "remove"
    }
  },
  "status": 400
}

But the doc says, it does support this option, who can help to explain why the error?

Remove Processoredit

Removes existing fields. If one field doesn’t exist, an exception will be thrown.

Table 45. Remove Options

Name Required Default Description
field yes - Fields to be removed
ignore_missing no false If true and field does not exist or is null , the processor quietly exits without modifying the document
2

Which version of Elasticsearch are you using? If I look at the documentation for various versions it looks like this parameter was introduced in Elasticsearch 6.4.

3

ok, my es version is 6.3 and I just noticed it does not support this option! thanks.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.