Failure to replace elastic search document_type using logstash config

Elastic Stack Logstash
1

I'd like elastic search to dynamically create different document_types based on input json variable property in the config. For quite some time I've been struggling to get logstash replace the elastic search document_type field with the JSON property. As this is a common feature in logstash appreciate your help with identifying what I'm doing wrong with my configuration. I can see each of the input json show up in elastic search index however the document_type shows up as %{testid} instead of "abc123"

My conf files are: (logstash filebeat config)

input {
beats {
port => 5044
type => "json"
ssl => false

}
}

filter {
json {
source => "message"
}
}

output {
elasticsearch {
hosts => ["192.168.99.103:9200"]
sniffing => true
manage_template => false
index => "logs"
document_type => "%{testid}"
}
}

filebeat.yml (snippet)
document_type: json
json.message_key: log

The input jso
{ "URI": "http://test.com/test", "file": "testgo:1475", "level": "warn", "method": "GET", "msg": "just a test", "package": "test", "payload": "", "time": "2017-04-05T01:10:50.175580776Z", "testid": "abc123", "someid": "1234", "creationdt": "2017-04-06" }

2

Please show an example of what an event stored in Elasticsearch looks like. Copy/paste from Kibana's JSON tab (in the Discover panel) or us a stdout { codec => rubydebug } output in Logstash.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.