How to create custom document _id in logstash?


(Roshni Kasliwal) #1

This is sample logstash output filter.

output
{
   elasticsearch
   {
        hosts => ["localhost"]
        sniffing => true
        manage_template => false
        index => "mqtt-index-%{+YYYY.MM.dd}"
        document_id => "%{parsedMessage.device_id}"
        document_type => "iot_data"
   }
}

This code changes existing _id as %{parsedMessage.device_id} as it is. How to change _id by another field?


(Teddy) #2

you can replace "parsedMessage.device_id" in document_id => "%{parsedMessage.device_id}"
by any other field you like.


(Roshni Kasliwal) #3

thank you devil_srj7......i got it, but the point is that it takes this string as it is as _id like,

_id = %{parsedMessage.device_id}


(Magnus Bäck) #4

See https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html#logstash-config-field-references for the syntax of nested fields.


(Roshni Kasliwal) #5

Thank you magnusbaeck. But, this is not nested field. The field name itself is parsedMessage.device_id


(Magnus Bäck) #6

Replace the elasticsearch output with a stdout { codec => rubydebug } output and show an example event produced by Logstash.


(Roshni Kasliwal) #7

issue solved


(system) #8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.