Hi, I manually put an index template in Elasticsearch with doc_type doc. But when Logstash tries to index an event using this template. it shows failed to index due to multiple doc types. I think the template I defined is the same as the doc type in the event, which is doc. Can anyone explain why it failed to index?
Another question:
If I update the index pattern, will Logstash use the new index pattern or still cache the old index pattern?
[2019-02-18T22:37:05,637][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"log-2019", :_type=>"doc", :_routing=>nil}, #<LogStash::Event:0xdc6d769>], :response=>{"index"=>{"_index"=>"log-2019", "_type"=>"doc", "_id"=>"4yZ4BGkBCjBT-EkD5FW0", "status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"Rejecting mapping update to [log-2019] as the final mapping would have more than 1 type: [_doc, doc]"}}}}
Template defined use Template API:
PUT _template/log-success
{
"index_patterns": ["log-*"],
"settings": {
"number_of_shards": 1,
"number_of_replicas" : 0,
"index.refresh_interval": "5s"
},
"mappings": {
"doc": {
"dynamic": "strict",
"properties": {
"@timestamp" : {"type" : "date"},
"@version" : {"type" : "keyword"},
"source" : {"type" : "keyword"}
}
}
}
}
Logstash pipeline.conf:
input {
beats {
port => "5044"
}
}
filter {
ruby {
path => "./parse.rb"
}
date {
match => ["time", "EEE MMM dd HH:mm:ss z yyyy"]
remove_field => ["time"]
}
}
output {
stdout { codec => rubydebug }
if [parsingError] {
elasticsearch {
manage_template => false
template_name => "log-failure"
hosts => [ "localhost:9200" ]
index => "failure-log-%{+yyyy}"
}
}
else {
elasticsearch {
manage_template => false
template_name => "log-success"
hosts => [ "localhost:9200" ]
index => "log-%{+yyyy}"
}
}
}