How to remove duplicate logs in logstash?

Elastic Stack Logstash
1

How to remove duplicate logs in logstash ?

input {
  beats {
    port => 5044
    tags => "%{[fields][log_type]}"
  }
}

}
filter {
  fingerprint {
    source => "message"
    target => "[@metadata][fingerprint]"
    method => "SHA256"
  }
}

output {
  elasticsearch {
    hosts => ["192.168.0.90:9200"]
    manage_template => false
    index => "index-%{+YYYY.MM.dd}"
    document_id => "%{[@metadata][fingerprint]}"
    user => User
    password => Pass
	}
}
2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.