# How to get json subfield from input and use it as document\_id elastic?

**URL:** <https://discuss.elastic.co/t/how-to-get-json-subfield-from-input-and-use-it-as-document-id-elastic/52334>\
**Category:** Logstash\
**Created:** [June 9, 2016, 1:28pm UTC](https://discuss.elastic.co/t/how-to-get-json-subfield-from-input-and-use-it-as-document-id-elastic/52334 "2016-06-09T13:28:50Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![breezzz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/breezzz/32/18245_2.png) [@breezzz](https://discuss.elastic.co/u/breezzz)\
**Post date:** [June 9, 2016, 1:28pm UTC](https://discuss.elastic.co/t/how-to-get-json-subfield-from-input-and-use-it-as-document-id-elastic/52334/1 "2016-06-09T13:28:50Z")

</div>

Good day!  
I have this work config for logstash:

> input {  
> kafka {  
> zk\_connect =\> "kafka:2181"  
> topic\_id =\> "requests"  
> group\_id =\> "kafka\_requests"  
> consumer\_threads =\> 1  
> }  
> }  
> output {  
> elasticsearch {  
> codec =\> "json"  
> index =\> "kafka\_requests-%{+YYYY.MM.dd}"  
> }  
> }

But elastic not contain offset kafka message (number of message).  
I add

> decorate\_events =\> true

to input kafka section and have now

> "kafka":{"msg\_size":150,"topic":"requests","consumer\_group":"kafka\_requests",  
> "partition":2,"offset":1411769133,"key":null}}

Question:  
How i may use subfield "kafka": "offset":1411769133 from input as document\_id ( \_id ) in output elasticsearch.

This need for messages non duplicate, for test all messages put to elastic, and so.

Thank you!

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 10, 2016, 5:51am UTC](https://discuss.elastic.co/t/how-to-get-json-subfield-from-input-and-use-it-as-document-id-elastic/52334/2 "2016-06-10T05:51:45Z")

</div>

```nohighlight
elasticsearch {
  index => "kafka_requests-%{+YYYY.MM.dd}"
  document_id => "%{[kafka][offset]}"
}

```

---

<div class="post-metadata">

**Author:** ![breezzz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/breezzz/32/18245_2.png) [@breezzz](https://discuss.elastic.co/u/breezzz)\
**Post date:** [June 10, 2016, 11:33am UTC](https://discuss.elastic.co/t/how-to-get-json-subfield-from-input-and-use-it-as-document-id-elastic/52334/3 "2016-06-10T11:33:21Z")

</div>

> [@magnusbaeck](#):
>
> document\_id =\> "%{[kafka][offset]}"

Thank you! It's work!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:53am UTC](https://discuss.elastic.co/t/how-to-get-json-subfield-from-input-and-use-it-as-document-id-elastic/52334/4 "2017-07-06T04:53:40Z")

</div>

Closed on July 6, 2017, 4:53am UTC