# Regarding Ingest Node Set Processor

**URL:** <https://discuss.elastic.co/t/regarding-ingest-node-set-processor/187606>\
**Category:** Elasticsearch\
**Created:** [June 26, 2019, 3:05pm UTC](https://discuss.elastic.co/t/regarding-ingest-node-set-processor/187606 "2019-06-26T15:05:18Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![dawiro](https://avatars.discourse-cdn.com/v4/letter/d/71e660/32.png) [@dawiro](https://discuss.elastic.co/u/dawiro)\
**Post date:** [June 26, 2019, 3:05pm UTC](https://discuss.elastic.co/t/regarding-ingest-node-set-processor/187606/1 "2019-06-26T15:05:18Z")

</div>

Hi,  
I'm looking at upgrading from 5.6.3 to 6.8.x and have come across a behaviour in an ingest node pipeline which I need to address before upgrading...

I have the following processor defined in one of our pipelines:

```auto
{
        "set" : {
          "field" : "pipeline",
          "value" : "metadata"
        }
}

```

In our 5.6.3 cluster this ends up being indexed as:

```auto
        "pipeline" : {
            "properties" : {
              "name" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
          }
        }

```

While in another 6.8.0 cluster:

```auto
          "pipeline" : {
            "type" : "keyword",
            "ignore_above" : 256
          }

```

This will lead to problems on upgrade. I have seen that logstash 6.8.x will fail to write to this pipeline if the index contains the 5.6.3 mapping shown above. Our current logstashes are at 5.6.9.

Why is this happening and how do you recommend I handle it? Remove that processor until the upgrade is complete?

Regards,  
D

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [June 27, 2019, 7:32am UTC](https://discuss.elastic.co/t/regarding-ingest-node-set-processor/187606/2 "2019-06-27T07:32:16Z")

</div>

Hey,

this sounds less like an ingest node processor issue (as the ingest node only changes the JSON document), but more like a mapping issue. Are you missing or using a different index template on the different elasticsearch versions?

--Alex

---

<div class="post-metadata">

**Author:** ![dawiro](https://avatars.discourse-cdn.com/v4/letter/d/71e660/32.png) [@dawiro](https://discuss.elastic.co/u/dawiro)\
**Post date:** [June 27, 2019, 9:09am UTC](https://discuss.elastic.co/t/regarding-ingest-node-set-processor/187606/3 "2019-06-27T09:09:13Z")

</div>

Figured it out this morning. Someone is shipping rogue fields to that cluster outside of the pipeline..

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 25, 2019, 9:09am UTC](https://discuss.elastic.co/t/regarding-ingest-node-set-processor/187606/4 "2019-07-25T09:09:25Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.