I'm also trying to get filebeat to talk to AWS Elasticsearch and running into the same errors. I'm running filebeat 5.0.0 on Ubuntu 14.04 and my config is basically the same as OP's. With additional debugging enabled, I see entries like this:
2016/10/27 13:13:11.847333 output.go:109: DBG output worker: publish 50 events
2016/10/27 13:13:11.847391 client.go:615: DBG ES Ping(url=http://search-domain.us-east-1.es.amazonaws.com:80, timeout=1m30s)
2016/10/27 13:13:11.848870 spooler.go:118: DBG Flushing spooler because spooler full. Events flushed: 2048
2016/10/27 13:13:11.856899 client.go:639: DBG Ping status code: 200
2016/10/27 13:13:11.856929 client.go:640: INFO Connected to Elasticsearch version 2.3.2
2016/10/27 13:13:11.856950 output.go:214: INFO Trying to load template for client: http://search-domain.us-east-1.es.amazonaws.com:80
2016/10/27 13:13:11.856971 client.go:655: DBG HEAD http://search-domain.us-east-1.es.amazonaws.com:80/_template/filebeat <nil>
2016/10/27 13:13:11.873497 output.go:235: INFO Template already exists and will not be overwritten.
2016/10/27 13:13:11.904051 client.go:232: ERR Failed to perform any bulk index operations: 400 Bad Request
2016/10/27 13:13:11.904085 single.go:91: INFO Error publishing events (retrying): 400 Bad Request
2016/10/27 13:13:11.904109 single.go:156: DBG send fail
I've tried about every permutation of host configuration I can think of: "http://search-domain...es.amazonaws.com:80", "search-domain...es.amazonaws.com:80", "https://search-domain...es.amazonaws.com:443", etc.
From the log entries, I can see that it's connecting, getting a successful ping, detecting the version of Elasticsearch, but then something is going wrong and it's generating bad requests.
If I curl http://search-domain...es.amazonaws.com/_template/filebeatI can see that there is a filebeat template there that looks correct.
Has anyone succeeded in getting filebeat to talk to AWS Elasticsearch?
Is there anything else I can try?