Hi Axrayn
thank you for your suggestion. I had already tried. Indeed I think I have tried almost anything. Based on your suggestion I removed the https:// prefix and I removed the HTTP_POST env variable (I am using docker containers) and this in turn would remove it from the generic configuration. Also XPACK_MONITORING_ENABLED=false
In the conf.d configuration I also change the output sections to
output {
elasticsearch {
hosts => ["myawses.us-east-1.es.amazonaws.com:443"]
And now i get
logstash_1 | 2018/10/08 01:49:53 Setting 'http.host' from environment.
logstash_1 | 2018/10/08 01:49:53 Setting 'xpack.monitoring.enabled' from environment.
logstash_1 | Sending Logstash's logs to /usr/share/logstash/logs which is now configured via log4j2.properties
logstash_1 | [2018-10-08T01:50:59,238][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.queue", :path=>"/usr/share/logstash/data/queue"}
logstash_1 | [2018-10-08T01:50:59,266][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.dead_letter_queue", :path=>"/usr/share/logstash/data/dead_letter_queue"}
logstash_1 | [2018-10-08T01:51:00,353][INFO ][logstash.agent ] No persistent UUID file found. Generating new UUID {:uuid=>"69eba438-f345-4bb8-8a84-afcfbe4c96ef", :path=>"/usr/share/logstash/data/uuid"}
logstash_1 | [2018-10-08T01:51:01,215][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"6.3.2"}
logstash_1 | [2018-10-08T01:51:09,442][INFO ][logstash.pipeline ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
logstash_1 | [2018-10-08T01:51:11,683][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://myawses.us-east-1.es.amazonaws.com:443/]}}
logstash_1 | [2018-10-08T01:51:11,695][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://myawses.us-east-1.es.amazonaws.com:443/, :path=>"/"}
logstash_1 | [2018-10-08T01:51:12,232][WARN ][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"http://myawses.us-east-1.es.amazonaws.com:443/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError, :error=>"Got response code '400' contacting Elasticsearch at URL 'http://myawses.us-east-1.es.amazonaws.com:443/'"}
logstash_1 | [2018-10-08T01:51:12,258][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//myawses.us-east-1.es.amazonaws.com:443"]}
logstash_1 | [2018-10-08T01:51:12,283][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://myawses.us-east-1.es.amazonaws.com:443/]}}
logstash_1 | [2018-10-08T01:51:12,284][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://myawses.us-east-1.es.amazonaws.com:443/, :path=>"/"}
logstash_1 | [2018-10-08T01:51:12,512][WARN ][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"http://myawses.us-east-1.es.amazonaws.com:443/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError, :error=>"Got response code '400' contacting Elasticsearch at URL 'http://myawses.us-east-1.es.amazonaws.com:443/'"}
logstash_1 | [2018-10-08T01:51:12,514][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//myawses.us-east-1.es.amazonaws.com:443"]}
logstash_1 | [2018-10-08T01:51:13,852][INFO ][logstash.inputs.beats ] Beats inputs: Starting input listener {:address=>"0.0.0.0:5044"}
logstash_1 | [2018-10-08T01:51:13,913][INFO ][logstash.pipeline ] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0x40e1709b run>"}
logstash_1 | [2018-10-08T01:51:14,232][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
logstash_1 | [2018-10-08T01:51:14,425][INFO ][org.logstash.beats.Server] Starting server on port: 5044
logstash_1 | [2018-10-08T01:51:14,849][FATAL][logstash.runner ] An unexpected error occurred! {:error=>#<SocketError: initialize: name or service not known>, :backtrace=>["org/jruby/ext/socket/RubyTCPServer.java:124:in `initialize'", "org/jruby/RubyIO.java:875:in `new'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/puma-2.16.0-java/lib/puma/binder.rb:234:in `add_tcp_listener'", "(eval):2:in `add_tcp_listener'", "/usr/share/logstash/logstash-core/lib/logstash/webserver.rb:88:in `start_webserver'", "/usr/share/logstash/logstash-core/lib/logstash/webserver.rb:44:in `block in run'", "org/jruby/RubyRange.java:485:in `each'", "org/jruby/RubyEnumerable.java:1067:in `each_with_index'", "/usr/share/logstash/logstash-core/lib/logstash/webserver.rb:39:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:353:in `block in start_webserver'"]}
logstash_1 | [2018-10-08T01:51:14,908][ERROR][org.logstash.Logstash ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit
As you can see logstash doesnt use https unless explicitly set. Is the above change in configuration what your suggestion was about?