# Using the Remove processor for ingest node **URL:** \ **Category:** Elasticsearch\ **Created:** [July 27, 2016, 10:05am UTC](https://discuss.elastic.co/t/using-the-remove-processor-for-ingest-node/56500 "2016-07-27T10:05:24Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![ori.rubinfeld](https://avatars.discourse-cdn.com/v4/letter/o/7c8e57/32.png) [@ori.rubinfeld](https://discuss.elastic.co/u/ori.rubinfeld)\ **Post date:** [July 27, 2016, 10:05am UTC](https://discuss.elastic.co/t/using-the-remove-processor-for-ingest-node/56500/1 "2016-07-27T10:05:24Z")
Hi, I am trying to remove some fields from the document using the REMOVE processor in the pipeline. I tried: remove { "field"=\>"a", "field"=\>"b", "field"=\>"c", "field"=\>"d", "field"=\>"e", "ignore\_failure" : true } And: remove { "field"=\>"a", "ignore\_failure" : true }, remove { "field"=\>"b", "ignore\_failure" : true }, remove { "field"=\>"c", "ignore\_failure" : true }, remove { "field"=\>"d", "ignore\_failure" : true }, remove { "field"=\>"e", "ignore\_failure" : true } But every time it just remove the last one in the processors chain. Is there another syntax ? Or am I using it wrong ? Thanks, Ori ---
**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\ **Post date:** [July 27, 2016, 11:30am UTC](https://discuss.elastic.co/t/using-the-remove-processor-for-ingest-node/56500/2 "2016-07-27T11:30:45Z")
Hi @ori.rubinfeld, here is a complete example that worked for me: ```auto DELETE /_ingest/pipeline/remove-fields PUT /_ingest/pipeline/remove-fields { "description": "remove a couple of fields", "processors": [ { "remove": { "field": "foo" } }, { "remove": { "field": "bar" } } ] } DELETE /sample-index PUT /sample-index POST /sample-index/type?pipeline=remove-fields { "foo": 1, "bar": 2, "baz": 3 } POST /sample-index/type?pipeline=remove-fields { "foo": 4, "bar": 5, "baz": 6 } GET /sample-index/type/_search { "query": { "match_all": {} } } ``` The query returns: ```auto { "took": 37, "timed_out": false, "_shards": { "total": 5, "successful": 5, "failed": 0 }, "hits": { "total": 2, "max_score": 1, "hits": [ { "_index": "sample-index", "_type": "type", "_id": "AVYsHLDoFKBGyZwFuEKG", "_score": 1, "_source": { "baz": 3 } }, { "_index": "sample-index", "_type": "type", "_id": "AVYsHM4xFKBGyZwFuEKH", "_score": 1, "_source": { "baz": 6 } } ] } } ``` As you can see, both returned documents only contain the `baz` field. Daniel ---
**Author:** ![mvg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mvg/32/98890_2.png) [@mvg](https://discuss.elastic.co/u/mvg)\ **Post date:** [July 27, 2016, 12:25pm UTC](https://discuss.elastic.co/t/using-the-remove-processor-for-ingest-node/56500/3 "2016-07-27T12:25:02Z")
Daniel's example is how a pipeline should be specified. (an array element per processor) Unfortunately specifying a pipeline like Ori doesn't fail... so I think that should be fixed. This isn't ingest specific, but the json parser just allows this. ---
**Author:** ![ori.rubinfeld](https://avatars.discourse-cdn.com/v4/letter/o/7c8e57/32.png) [@ori.rubinfeld](https://discuss.elastic.co/u/ori.rubinfeld)\ **Post date:** [July 28, 2016, 8:54am UTC](https://discuss.elastic.co/t/using-the-remove-processor-for-ingest-node/56500/4 "2016-07-28T08:54:48Z")
Can you please write an example ? "an array element per processor" - Does it means that it should not be as follows: "processoers": [ { { "grok":..... }, { "remove":...... }, { "remove":..... } } ] How should it be written then ? Ori ---
**Author:** ![mvg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mvg/32/98890_2.png) [@mvg](https://discuss.elastic.co/u/mvg)\ **Post date:** [July 28, 2016, 9:15am UTC](https://discuss.elastic.co/t/using-the-remove-processor-for-ingest-node/56500/5 "2016-07-28T09:15:51Z")
It should just be like this: ```auto "processors": [ { "grok": { // processor options } }, { "remove": { // processor options } }, { "remove": { // processor options } } ] ``` ---
**Author:** ![ori.rubinfeld](https://avatars.discourse-cdn.com/v4/letter/o/7c8e57/32.png) [@ori.rubinfeld](https://discuss.elastic.co/u/ori.rubinfeld)\ **Post date:** [July 28, 2016, 9:19am UTC](https://discuss.elastic.co/t/using-the-remove-processor-for-ingest-node/56500/6 "2016-07-28T09:19:35Z")
This is the exact way how I wrote the processors, and yet, it only removes the last field mentioned in the pipeline. If I will add another remove, it then will remove only the new one. I am sending data from filebeat, can it be related to bulks ? ---
**Author:** ![mvg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mvg/32/98890_2.png) [@mvg](https://discuss.elastic.co/u/mvg)\ **Post date:** [July 28, 2016, 9:51am UTC](https://discuss.elastic.co/t/using-the-remove-processor-for-ingest-node/56500/7 "2016-07-28T09:51:00Z")
> [@ori.rubinfeld](#): > > This is the exact way how I wrote the processors, and yet, it only removes the last field mentioned in the pipeline. This has nothing to de with bulk. At least what you shared is different: ```auto remove { "field"=>"a", "ignore_failure" : true }, remove { "field"=>"b", "ignore_failure" : true }, remove { "field"=>"c", "ignore_failure" : true }, remove { "field"=>"d", "ignore_failure" : true }, remove { "field"=>"e", "ignore_failure" : true } ``` Each processor needs to be wrapped in a extra json object in the `processors` array. So this: ```auto { "remove" { "field"=>"e", "ignore_failure" : true } } ``` instead of this: ```auto "remove": { "field"=>"e", "ignore_failure" : true } ``` This behavior you're seeing with this format is a bug. ES should fail when processors are accepted in this way; instead of accepting it and only using the last processor. ---
**Author:** ![ori.rubinfeld](https://avatars.discourse-cdn.com/v4/letter/o/7c8e57/32.png) [@ori.rubinfeld](https://discuss.elastic.co/u/ori.rubinfeld)\ **Post date:** [July 28, 2016, 10:33am UTC](https://discuss.elastic.co/t/using-the-remove-processor-for-ingest-node/56500/8 "2016-07-28T10:33:09Z")
This is correct!!!! Thanks!!! works great now. Had a similar problem with gsub, will check it too. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 10:32pm UTC](https://discuss.elastic.co/t/using-the-remove-processor-for-ingest-node/56500/9 "2017-07-05T22:32:10Z")