How to create my own document_id in logstash?

Elastic Stack Logstash
3

Thanks very much for this. Just what I wanted. Just to help anyone finding this question, here is the code I actually used:

ruby {
  code => "require 'digest/md5';
  event['computed_id'] = Digest::MD5.hexdigest(event['ip'] + event['sha1_fingerprint'])"
}

Then

document_id => "%{computed_id}"

Can I ask two more related questions:

  1. How can I remove the field after I have set the document_id? I don't want it in my stored data. eg; remove event['computed_id']

  2. Its seems that my index is BIGGER doing it this way? Any ideas? I thought that deduplication would save space? It can't be because of the extra 'computed_id' field can it?

Thanks

2 Likes