# When creating an index in Elasticsearch is there a way to map \_id field to incoming data field

**URL:** <https://discuss.elastic.co/t/when-creating-an-index-in-elasticsearch-is-there-a-way-to-map-id-field-to-incoming-data-field/204256>\
**Category:** Elasticsearch\
**Created:** [October 18, 2019, 4:54pm UTC](https://discuss.elastic.co/t/when-creating-an-index-in-elasticsearch-is-there-a-way-to-map-id-field-to-incoming-data-field/204256 "2019-10-18T16:54:46Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![bkleynbok](https://avatars.discourse-cdn.com/v4/letter/b/d2c977/32.png) [@bkleynbok](https://discuss.elastic.co/u/bkleynbok)\
**Post date:** [October 18, 2019, 4:54pm UTC](https://discuss.elastic.co/t/when-creating-an-index-in-elasticsearch-is-there-a-way-to-map-id-field-to-incoming-data-field/204256/1 "2019-10-18T16:54:46Z")

</div>

My team is using ELK Stack 7.3.2

I have tried many ways to show what we need in Kibana.

Lets say I have some JSON coming in from Kafka topic.  
ex. {"processName":"SM1","processType":"serviceManager","status":"UP","update":"2h14m","hostname":"devvm20"}

Then I see a message that says:  
{"processName":"SM1","processType":"serviceManager","status":"DOWN","update":"2h14m","hostname":"devvm20"}

Here is what I have tried to create an index in Elasticseach.  
curl -X PUT "nyuatalcasvm01.sscnydirect.local:9200/boris-index?pretty" -H 'Content-Type: application/json' -d'  
{  
"mappings": {  
"properties": {  
"processName": { "type": "keyword", "index": false, "fielddata": true },  
"hostname":{"type"},  
"processType": { "type": "text", "fielddata": true },  
"status":{"type":"text"},  
"update":{"type":"text"},  
}  
}  
}  
'  
Is there a way for me to set \_id from field processName.

All I want to is to get the latest record from the message that will be sent via Kafka and not all of the records that come in.

Basically the same thing as lets say as  
SELECT \* FROM my\_table GROUP BY (processName)

After that I would like to see if the process is UP or DOWN.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [October 18, 2019, 5:25pm UTC](https://discuss.elastic.co/t/when-creating-an-index-in-elasticsearch-is-there-a-way-to-map-id-field-to-incoming-data-field/204256/2 "2019-10-18T17:25:08Z")

</div>

You should be able to do that using an ingest node pipeline.

---

<div class="post-metadata">

**Author:** ![ylasri](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ylasri/32/86120_2.png) [@ylasri](https://discuss.elastic.co/u/ylasri)\
**Post date:** [October 18, 2019, 5:40pm UTC](https://discuss.elastic.co/t/when-creating-an-index-in-elasticsearch-is-there-a-way-to-map-id-field-to-incoming-data-field/204256/3 "2019-10-18T17:40:29Z")

</div>

Yes this can be done with an ingest pipleline.

```
PUT _ingest/pipeline/my_pipeline
{
    "processors": [
      {
        "set" : {
          "field" : "_id",
          "value" : "{{processName}}"
        }
      }
    ]
}

POST foo/_doc?pipeline=my_pipeline&refresh=true
{
  "processName": "SM1",
  "processType": "serviceManager",
  "status": "DOWN",
  "update": "2h14m",
  "hostname": "devvm20"
}
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 15, 2019, 5:40pm UTC](https://discuss.elastic.co/t/when-creating-an-index-in-elasticsearch-is-there-a-way-to-map-id-field-to-incoming-data-field/204256/4 "2019-11-15T17:40:38Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.