Creating the document_id in logstash by combining the two columns coming from the input jdbc filter

Elastic Stack Logstash
1

Right now this is how it looks,

elasticsearch {
user => "admin"
password => "elastic"
index => "customs"
manage_template => false
template_name => "template1"
document_type => "test"
document_id => "%{item_id}"
hosts => "https://localhost:9200/"
flush_size => 1000
}

There is one more field with class_Id, how can combine item_id and class_id to form one single document_id in logstash

Thank You

2

I suppose that depends on what you mean by "combine." Elasticsearch only cares that document_id is unique within an index and type (though it should be unique across the index regardless of type).

In that sense, you can concatenate your item_id and class_id fields as follows:

document_id => "%{class_id}%{item_id}"

If you wanted to do some kind of fingerprint of the two combined, look into the fingerprint plugin. You could use the resulting field as the id.

2 Likes
3

Thanks theuntergeek :smiley: