I'd like elastic search to dynamically create different document_types based on input json variable property in the config. For quite some time I've been struggling to get logstash replace the elastic search document_type field with the JSON property. As this is a common feature in logstash appreciate your help with identifying what I'm doing wrong with my configuration. I can see each of the input json show up in elastic search index however the document_type shows up as %{testid} instead of "abc123"
My conf files are: (logstash filebeat config)
input {
beats {
port => 5044
type => "json"
ssl => false
}
}
filter {
json {
source => "message"
}
}
output {
elasticsearch {
hosts => ["192.168.99.103:9200"]
sniffing => true
manage_template => false
index => "logs"
document_type => "%{testid}"
}
}
filebeat.yml (snippet)
document_type: json
json.message_key: log
The input jso
{ "URI": "http://test.com/test", "file": "testgo:1475", "level": "warn", "method": "GET", "msg": "just a test", "package": "test", "payload": "", "time": "2017-04-05T01:10:50.175580776Z", "testid": "abc123", "someid": "1234", "creationdt": "2017-04-06" }