# Logstash can't push data to AWS ES **URL:** \ **Category:** Logstash\ **Created:** [April 12, 2018, 6:23am UTC](https://discuss.elastic.co/t/logstash-cant-push-data-to-aws-es/127739 "2018-04-12T06:23:45Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![samwzm](https://avatars.discourse-cdn.com/v4/letter/s/ac91a4/32.png) [@samwzm](https://discuss.elastic.co/u/samwzm)\ **Post date:** [April 12, 2018, 6:23am UTC](https://discuss.elastic.co/t/logstash-cant-push-data-to-aws-es/127739/1 "2018-04-12T06:23:45Z")
Hi, I config AWS ES for the elasticsearch service. I can use curl or python to load single doc to it without any problem. However, when I tried to upload data through logstash (I use docker image from another EC2), I always got mainly two kinds of errors: first, the healthy check error, because of can't connect to port 9200, but I would assume this error won't affect function second, An unexpected error occurred! {:error=\>#\,...... (you can get the detailed information from below log). I use latest ES and logstash. Below is my logstash conf: input { stdin { } } filter { grok { match =\> { "message" =\> "%{COMBINEDAPACHELOG}" } } date { match =\> ["timestamp" , "dd/MMM/yyyy:HH:mm:ss Z"] } } output { elasticsearch { hosts =\> ["[search-es-5-5-wxzz2y5456vntppkwwkntfe7b4.us-east-1.es.amazonaws.com](http://search-es-5-5-wxzz2y5456vntppkwwkntfe7b4.us-east-1.es.amazonaws.com)"] manage\_template =\> false index =\> "logstash-%{+YYYY.MM.dd}" ssl =\> true } stdout { codec =\> rubydebug } } Even I installed the plugin "logstash-output-amazon\_es", I got the same error. I would greatly appreciate your help! Thanks, Sam bash-4.2$ logstash -f ./conf/command.conf Sending Logstash's logs to /usr/share/logstash/logs which is now configured via log4j2.properties [2018-04-12T06:12:44,394][INFO][logstash.modules.scaffold] Initializing module {:module\_name=\>"fb\_apache", :directory=\>"/usr/share/logstash/modules/fb\_apache/configuration"} [2018-04-12T06:12:44,415][INFO][logstash.modules.scaffold] Initializing module {:module\_name=\>"netflow", :directory=\>"/usr/share/logstash/modules/netflow/configuration"} [2018-04-12T06:12:45,040][WARN][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified [2018-04-12T06:12:45,671][INFO][logstash.runner] Starting Logstash {"logstash.version"=\>"6.2.3"} [2018-04-12T06:12:46,029][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=\>9600} [2018-04-12T06:12:49,982][INFO][logstash.pipeline] Starting pipeline {:pipeline\_id=\>"main", "pipeline.workers"=\>4, "pipeline.batch.size"=\>125, "pipeline.batch.delay"=\>50} [2018-04-12T06:12:50,554][INFO][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=\>{:removed=\>[], :added=\>[[https://search-es-5-5-wxzz2y5456vntppkwwkntfe7b4.us-east-1.es.amazonaws.com:9200/](https://search-es-5-5-wxzz2y5456vntppkwwkntfe7b4.us-east-1.es.amazonaws.com:9200/)]}} [2018-04-12T06:12:50,565][INFO][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck\_url=\>[https://search-es-5-5-wxzz2y5456vntppkwwkntfe7b4.us-east-1.es.amazonaws.com:9200/](https://search-es-5-5-wxzz2y5456vntppkwwkntfe7b4.us-east-1.es.amazonaws.com:9200/), :path=\>"/"} [2018-04-12T06:13:10,791][WARN][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=\>"[https://search-es-5-5-wxzz2y5456vntppkwwkntfe7b4.us-east-1.es.amazonaws.com:9200/](https://search-es-5-5-wxzz2y5456vntppkwwkntfe7b4.us-east-1.es.amazonaws.com:9200/)", :error\_type=\>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=\>"Elasticsearch Unreachable: [[https://search-es-5-5-wxzz2y5456vntppkwwkntfe7b4.us-east-1.es.amazonaws.com:9200/](https://search-es-5-5-wxzz2y5456vntppkwwkntfe7b4.us-east-1.es.amazonaws.com:9200/)][Manticore::ConnectTimeout] connect timed out"} [2018-04-12T06:13:10,811][INFO][logstash.outputs.elasticsearch] New Elasticsearch output {:class=\>"LogStash::Outputs::ElasticSearch", :hosts=\>["[//search-es-5-5-wxzz2y5456vntppkwwkntfe7b4.us-east-1.es.amazonaws.com](https://search-es-5-5-wxzz2y5456vntppkwwkntfe7b4.us-east-1.es.amazonaws.com)"]} [2018-04-12T06:13:11,082][INFO][logstash.pipeline] Pipeline started succesfully {:pipeline\_id=\>"main", :thread=\>"#\"} The stdin plugin is now waiting for input: [2018-04-12T06:13:11,190][INFO][logstash.agent] Pipelines running {:count=\>1, :pipelines=\>["main"]} [2018-04-12T06:13:15,805][INFO][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck\_url=\>[https://search-es-5-5-wxzz2y5456vntppkwwkntfe7b4.us-east-1.es.amazonaws.com:9200/](https://search-es-5-5-wxzz2y5456vntppkwwkntfe7b4.us-east-1.es.amazonaws.com:9200/), :path=\>"/"} [2018-04-12T06:13:35,834][WARN][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=\>"[https://search-es-5-5-wxzz2y5456vntppkwwkntfe7b4.us-east-1.es.amazonaws.com:9200/](https://search-es-5-5-wxzz2y5456vntppkwwkntfe7b4.us-east-1.es.amazonaws.com:9200/)", :error\_type=\>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=\>"Elasticsearch Unreachable: [[https://search-es-5-5-wxzz2y5456vntppkwwkntfe7b4.us-east-1.es.amazonaws.com:9200/](https://search-es-5-5-wxzz2y5456vntppkwwkntfe7b4.us-east-1.es.amazonaws.com:9200/)][Manticore::ConnectTimeout] connect timed out"} [2018-04-12T06:13:36,835][INFO][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck\_url=\>[https://search-es-5-5-wxzz2y5456vntppkwwkntfe7b4.us-east-1.es.amazonaws.com:9200/](https://search-es-5-5-wxzz2y5456vntppkwwkntfe7b4.us-east-1.es.amazonaws.com:9200/), :path=\>"/"} 127.0.0.1 - - [11/Dec/2013:00:01:45 -0800] "GET /xampp/status.php HTTP/1.1" 200 3891 "[http://cadenza/xampp/navi.php](http://cadenza/xampp/navi.php)" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:25.0) Gecko/20100101 Firefox/25.0" [2018-04-12T06:13:49,973][FATAL][logstash.runner] An unexpected error occurred! {:error=\>#\, :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.0.3-java/lib/logstash/outputs/elasticsearch/common.rb:213:in`get\_event\_type'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.0.3-java/lib/logstash/outputs/elasticsearch/common.rb:165:in `event_action_params'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.0.3-java/lib/logstash/outputs/elasticsearch/common.rb:39:in`event\_action\_tuple'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.0.3-java/lib/logstash/outputs/elasticsearch/common.rb:34:in `block in multi_receive'", "org/jruby/RubyArray.java:2486:in`map'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.0.3-java/lib/logstash/outputs/elasticsearch/common.rb:34:in `multi_receive'", "/usr/share/logstash/logstash-core/lib/logstash/output_delegator_strategies/shared.rb:13:in`multi\_receive'", "/usr/share/logstash/logstash-core/lib/logstash/output\_delegator.rb:49:in `multi_receive'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:479:in`block in output\_batch'", "org/jruby/RubyHash.java:1343:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:478:in`output\_batch'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:430:in `worker_loop'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:385:in`block in start\_workers'"]} [2018-04-12T06:13:50,131][ERROR][org.logstash.Logstash] java.lang.IllegalStateException: org.jruby.exceptions.RaiseException: (SystemExit) exit bash-4.2$ ---
**Author:** ![samwzm](https://avatars.discourse-cdn.com/v4/letter/s/ac91a4/32.png) [@samwzm](https://discuss.elastic.co/u/samwzm)\ **Post date:** [April 12, 2018, 2:34pm UTC](https://discuss.elastic.co/t/logstash-cant-push-data-to-aws-es/127739/2 "2018-04-12T14:34:58Z")
Anyone can help? Thanks! ---
**Author:** ![JKhondhu](https://avatars.discourse-cdn.com/v4/letter/j/ed655f/32.png) [@JKhondhu](https://discuss.elastic.co/u/JKhondhu)\ **Post date:** [April 19, 2018, 8:07pm UTC](https://discuss.elastic.co/t/logstash-cant-push-data-to-aws-es/127739/3 "2018-04-19T20:07:49Z")
> [@samwzm](#): > > Elasticsearch Unreachable Your logstash output is unable to reach your ES instance.... Is your cluster setup with aws\_access\_key\_id, aws\_secret\_access\_key? ref: [GitHub - awslabs/logstash-output-amazon\_es: Logstash output plugin to sign and export logstash events to Amazon Elasticsearch Service](https://github.com/awslabs/logstash-output-amazon_es#configuration-for-amazon-elasticsearch-output-plugin) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 17, 2018, 8:08pm UTC](https://discuss.elastic.co/t/logstash-cant-push-data-to-aws-es/127739/4 "2018-05-17T20:08:14Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.