# Elasticsearch ingest processor to test for field type

**URL:** https://discuss.elastic.co/t/elasticsearch-ingest-processor-to-test-for-field-type/97325
**Category:** Elasticsearch
**Created:** [August 16, 2017, 10:52pm UTC](https://discuss.elastic.co/t/elasticsearch-ingest-processor-to-test-for-field-type/97325 "2017-08-16T22:52:36Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![htandra](https://avatars.discourse-cdn.com/v4/letter/h/f17d59/32.png) [@htandra](https://discuss.elastic.co/u/htandra)
#### Post date: [August 16, 2017, 10:52pm UTC](https://discuss.elastic.co/t/elasticsearch-ingest-processor-to-test-for-field-type/97325/1 "2017-08-16T22:52:36Z")

</div>

Hello all,

I am looking for a way to check if a field is "array" or "object" datatype before a document is ingested. For example,

PUT my\_id2/\_create {  
"locations" : {  
"first" : { .... }  
}  
}

PUT my\_id1/\_create {  
"locations" : ["abc","xyz"]  
}

I want to use this to remove in-consistencies in the input data. Rename "locations" field if it is an array so there is no mapping error when the documents are inserted.

Can I do this using "scripting" or "Grok" processor ?  
Any help or suggestions will be very helpful.

Thanks,  
H.

---

<div class="post-metadata">

### Author: ![abdon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abdon/32/9195_2.png) [@abdon](https://discuss.elastic.co/u/abdon)
#### Post date: [August 17, 2017, 9:55am UTC](https://discuss.elastic.co/t/elasticsearch-ingest-processor-to-test-for-field-type/97325/2 "2017-08-17T09:55:53Z")

</div>

You could use a script processor for that, and make use of the fact that an object is a HashMap and an array is a LinkedList. For example, the pipeline in the `_simulate` request below will first rename the `location` field to `location.original` and next create a new field `location.array` if the field is an array (`instance of List`), or a new field `location.object` if the field is an object (`instanceof Map`).

```
POST _ingest/pipeline/_simulate
{
  "pipeline": {
    "description": "Test",
    "processors": [
      {
        "rename": {
          "field": "location",
          "target_field": "location.original",
          "ignore_missing": true
        }
      },
      {
        "script": {
          "lang": "painless",
          "inline": "if (ctx.location.original instanceof List) { ctx.location.array = ctx.location.original } else if (ctx.location.original instanceof Map) { ctx.location.object = ctx.location.original } "
        }
      }
    ]
  },
  "docs": [
    {
      "_index": "my_index",
      "_type": "doc",
      "_id": "1",
      "_source": {
        "location": [
          "abc",
          "xyz"
        ]
      }
    },
    {
      "_index": "my_index",
      "_type": "doc",
      "_id": "2",
      "_source": {
        "location": {
          "first": {
            "foo": "bar"
          }
        }
      }
    },
    {
      "_index": "my_index",
      "_type": "doc",
      "_id": "3",
      "_source": {
        "location": "test"
      }
    }
    ,
    {
      "_index": "my_index",
      "_type": "doc",
      "_id": "4",
      "_source": {
        "location": []
      }
    }
  ]
}
```

---

<div class="post-metadata">

### Author: ![htandra](https://avatars.discourse-cdn.com/v4/letter/h/f17d59/32.png) [@htandra](https://discuss.elastic.co/u/htandra)
#### Post date: [August 17, 2017, 11:12pm UTC](https://discuss.elastic.co/t/elasticsearch-ingest-processor-to-test-for-field-type/97325/3 "2017-08-17T23:12:06Z")

</div>

Thank you so much for your detailed replied. It is most helpful !

Thanks a bunch again.  
-H

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [September 14, 2017, 11:12pm UTC](https://discuss.elastic.co/t/elasticsearch-ingest-processor-to-test-for-field-type/97325/4 "2017-09-14T23:12:12Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.