@magnusbaeck I have a question, but if you have time I wanted to ask your opinion before testing.
Is there a difference between these 2?
Would this possibly stop my duplication issue?
Original:
output { if [type] == "cloudtrail" { elasticsearch { hosts => "MYESCLUSTER" <======== REMOVE HERE
index => "logstash-cloudtrail" } } else { elasticsearch { hosts => "MYESCLUSTER" <======== REMOVE HERE
index => "wtf-are-these-logs" } } stdout { codec => "rubydebug" } }
Revised:
output { if [type] == "cloudtrail" { elasticsearch { index => "logstash-cloudtrail" } } else { elasticsearch { index => "wtf-are-these-logs" } } elasticsearch { <=============== ADD HERE
hosts => "MYESCLUSTER" <======== ADD HERE
} <============================ ADD HERE
stdout { codec => "rubydebug" } }