# Remove field create by processor split target\_field

**URL:** <https://discuss.elastic.co/t/remove-field-create-by-processor-split-target-field/154595>\
**Category:** Elasticsearch\
**Created:** [October 30, 2018, 9:26am UTC](https://discuss.elastic.co/t/remove-field-create-by-processor-split-target-field/154595 "2018-10-30T09:26:15Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Izek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/izek/32/37113_2.png) [@Izek](https://discuss.elastic.co/u/Izek)\
**Post date:** [October 30, 2018, 9:26am UTC](https://discuss.elastic.co/t/remove-field-create-by-processor-split-target-field/154595/1 "2018-10-30T09:26:15Z")

</div>

Here is my pipeline setting

```json
{
    "processors": [
      {
        "split": {
          "field": "service",
          "separator": "/",
          "target_field": "test"
        }
      }
      {
        "script": {
          "source": """
           if (ctx.service.contains("serverid")) { /// for serverid
              ctx.metric_type = ctx.test[1]; 
             ctx.serverid = ctx.instancename;
            }
            else {
             ctx.metric_type = ctx.service;
            }
"""
        }
      },
      {
        "remove": {
          "field": [
            "test"
          ],
          "ignore_failure": true
        }
      }
    ]
}

```

I try to remove test field after whole process finished.  
but it didn't.  
Is there any way to do it?

---

<div class="post-metadata">

**Author:** ![jakelandis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jakelandis/32/36163_2.png) [@jakelandis](https://discuss.elastic.co/u/jakelandis)\
**Post date:** [October 30, 2018, 2:13pm UTC](https://discuss.elastic.co/t/remove-field-create-by-processor-split-target-field/154595/2 "2018-10-30T14:13:53Z")

</div>

hmm.. I tried your example and other then a missing `,` after split and before script, it seems to work.

What version of Elasticsearch ? Can you provide an example input ?

(copied your example, added the missing `,` and called the pipeline `test`)

```auto
POST _ingest/pipeline/test/_simulate
  {
  "docs": [
    {
      "_source": {
        "service" : "foo/bar/serverid/1/foo/baz",
        "instancename": "myinstance"
      }
    }
  ]
}

```

Results in :

```auto
{
  "docs" : [
    {
      "doc" : {
        "_index" : "_index",
        "_type" : "_type",
        "_id" : "_id",
        "_source" : {
          "instancename" : "myinstance",
          "service" : "foo/bar/serverid/1/foo/baz",
          "metric_type" : "bar",
          "serverid" : "myinstance"
        },
        "_ingest" : {
          "timestamp" : "2018-10-30T14:12:03.38385Z"
        }
      }
    }
  ]
}

```

---

<div class="post-metadata">

**Author:** ![Izek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/izek/32/37113_2.png) [@Izek](https://discuss.elastic.co/u/Izek)\
**Post date:** [October 31, 2018, 1:29am UTC](https://discuss.elastic.co/t/remove-field-create-by-processor-split-target-field/154595/3 "2018-10-31T01:29:12Z")

</div>

My ES version is 6.4.2  
Here is my simulator

```json
POST _ingest/pipeline/_simulate
{
  "pipeline": {
    "description": "test",
    "processors": [
      {
        "split": {
          "field": "service",
          "separator": "/",
          "target_field": "test"
        }
      },
      {
        "gsub": {
          "field": "service",
          "pattern": "_ld",
          "replacement": ""
        }
      },
      {
        "script": {
          "source": """
            if (ctx.service.contains("routerid")) { /// for routerid
             ctx.metric_type = ctx.test[1];
             ctx.routerid = ctx.instancename;
            } else {
             ctx.metric_type = ctx.service;
            }
"""
        }
      },
      {
        "remove": {
          "field": [
            "instancename",
            "test"
          ],
          "ignore_failure": true
        }
      }
    ]
  },
  "docs": [
    {
      "_index": "my-index",
      "_type": "my-type",
      "_id": "my-id",
      "_source": {
        "service": "routerid/connected_players",
        "instancename": 2141007
      }
    }
  ]
}

```

And result

```json
{
  "docs": [
    {
      "doc": {
        "_index": "my-index",
        "_type": "my-type",
        "_id": "my-id",
        "_source": {
          "test": [
            "routerid",
            "connected_players"
          ],
          "service": "routerid/connected_players",
          "routerid": 2141007,
          "metric_type": "connected_players"
        },
        "_ingest": {
          "timestamp": "2018-10-31T01:24:44.560Z"
        }
      }
    }
  ]
}

```

The test field still exist.

---

<div class="post-metadata">

**Author:** ![jakelandis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jakelandis/32/36163_2.png) [@jakelandis](https://discuss.elastic.co/u/jakelandis)\
**Post date:** [October 31, 2018, 2:46pm UTC](https://discuss.elastic.co/t/remove-field-create-by-processor-split-target-field/154595/4 "2018-10-31T14:46:19Z")

</div>

This is really odd.. I ran the exact simulate in and got the following:

```auto
{
  "docs": [
    {
      "doc": {
        "_index": "my-index",
        "_type": "my-type",
        "_id": "my-id",
        "_source": {
          "metric_type": "connected_players",
          "service": "routerid/connected_players",
          "routerid": 2141007
        },
        "_ingest": {
          "timestamp": "2018-10-31T14:42:17.690913Z"
        }
      }
    }
  ]
}

```

To ensure I didn't have anything odd locally, i ran this from the following docker compose

```auto
version: '3'
services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:6.4.2
    container_name: elasticsearch
    environment:
      - cluster.name=docker-cluster
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
      - node.ingest=false
      - node.name=es1
    ulimits:
      memlock:
        soft: -1
        hard: -1
    ports:
      - 9200:9200
    networks:
      - esnet
  kibana:
    image: docker.elastic.co/kibana/kibana:6.4.2
    ports:
      - "5601:5601"
    networks:
      - esnet
    depends_on:
      - elasticsearch

networks:
  esnet:

```

Can you post the result of:

```auto
GET /?filter_path=version

```

and the result of ?verbose simulation

```auto
POST _ingest/pipeline/_simulate?verbose
... your example above ...

```

---

<div class="post-metadata">

**Author:** ![Izek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/izek/32/37113_2.png) [@Izek](https://discuss.elastic.co/u/Izek)\
**Post date:** [November 1, 2018, 2:05am UTC](https://discuss.elastic.co/t/remove-field-create-by-processor-split-target-field/154595/5 "2018-11-01T02:05:38Z")

</div>

```json
GET /?filter_path=version

{
  "version": {
    "number": "6.4.2",
    "build_flavor": "default",
    "build_type": "rpm",
    "build_hash": "04711c2",
    "build_date": "2018-09-26T13:34:09.098244Z",
    "build_snapshot": false,
    "lucene_version": "7.4.0",
    "minimum_wire_compatibility_version": "5.6.0",
    "minimum_index_compatibility_version": "5.0.0"
  }
}

```

It is very odd. Because it works now....  
And I already done the reindex, the test field still exist.  
Anyway seems fine now. I will try that again

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 29, 2018, 2:05am UTC](https://discuss.elastic.co/t/remove-field-create-by-processor-split-target-field/154595/6 "2018-11-29T02:05:42Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.