LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError, :message=>"Got response code '401'

[Diego_T (Diego)]

when I run logstash on my local windows pointing to my remote elastic account I get the following error

Using LS_JAVA_HOME defined java: C:\Program Files\Java\jdk-17.0.2
WARNING: Using LS_JAVA_HOME while Logstash distribution comes with a bundled JDK.
Sending Logstash logs to C:/logstash-8.1.0/logs which is now configured via log4j2.properties
[2022-03-15T16:36:17,659][INFO ][logstash.runner          ] Log4j configuration path used is: C:\logstash-8.1.0\config\log4j2.properties
[2022-03-15T16:36:17,665][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"8.1.0", "jruby.version"=>"jruby 9.2.20.1 (2.5.8) 2021-11-30 2a2962fbd1 Java HotSpot(TM) 64-Bit Server VM 17.0.2+8-LTS-86 on 17.0.2+8-LTS-86 +indy +jit [mswin32-x86_64]"}
[2022-03-15T16:36:17,666][INFO ][logstash.runner          ] JVM bootstrap flags: [-Xms1g, -Xmx1g, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djruby.compile.invokedynamic=true, -Djruby.jit.threshold=0, -Djruby.regexp.interruptible=true, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true, --add-opens=java.base/java.security=ALL-UNNAMED, --add-opens=java.base/java.io=ALL-UNNAMED, --add-opens=java.base/java.nio.channels=ALL-UNNAMED, --add-opens=java.base/sun.nio.ch=ALL-UNNAMED, --add-opens=java.management/sun.management=ALL-UNNAMED]
[2022-03-15T16:36:17,721][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2022-03-15T16:36:19,149][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false}
[2022-03-15T16:36:19,631][INFO ][org.reflections.Reflections] Reflections took 46 ms to scan 1 urls, producing 120 keys and 417 values
[2022-03-15T16:36:20,230][INFO ][logstash.javapipeline    ] Pipeline `main` is configured with `pipeline.ecs_compatibility: v8` setting. All plugins in this pipeline will default to `ecs_compatibility => v8` unless explicitly configured otherwise.
[2022-03-15T16:36:20,255][INFO ][logstash.outputs.elasticsearch][main] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["https://interconomy.es.us-central1.gcp.cloud.es.io:9243"]}
[2022-03-15T16:36:20,439][INFO ][logstash.outputs.elasticsearch][main] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[https://elastic:xxxxxx@interconomy.es.us-central1.gcp.cloud.es.io:9243/]}}
[2022-03-15T16:36:21,533][WARN ][logstash.outputs.elasticsearch][main] Attempted to resurrect connection to dead ES instance, but got an error {:url=>"https://elastic:xxxxxx@interconomy.es.us-central1.gcp.cloud.es.io:9243/", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError, :message=>"Got response code '401' contacting Elasticsearch at URL 'https://interconomy.es.us-central1.gcp.cloud.es.io:9243/'"}
[2022-03-15T16:36:21,554][INFO ][logstash.outputs.elasticsearch][main] Config is not compliant with data streams. `data_stream => auto` resolved to `false`
[2022-03-15T16:36:21,555][WARN ][logstash.outputs.elasticsearch][main] Elasticsearch Output configured with `ecs_compatibility => v8`, which resolved to an UNRELEASED preview of version 8.0.0 of the Elastic Common Schema. Once ECS v8 and an updated release of this plugin are publicly available, you will need to update this plugin to resolve this warning.
[2022-03-15T16:36:21,562][INFO ][logstash.filters.json    ][main] ECS compatibility is enabled but `target` option was not specified. This may cause fields to be set at the top-level of the event where they are likely to clash with the Elastic Common Schema. It is recommended to set the `target` option to avoid potential schema conflicts (if your data is ECS compliant or non-conflicting, feel free to ignore this message)
[2022-03-15T16:36:21,567][WARN ][logstash.filters.grok    ][main] ECS v8 support is a preview of the unreleased ECS v8, and uses the v1 patterns. When Version 8 of the Elastic Common Schema becomes available, this plugin will need to be updated
[2022-03-15T16:36:21,676][INFO ][logstash.javapipeline    ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>6, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>750, "pipeline.sources"=>["C:/logstash-8.1.0/config/heroku-elastic.conf"], :thread=>"#<Thread:0x9b4c94f run>"}
[2022-03-15T16:36:22,183][INFO ][logstash.javapipeline    ][main] Pipeline Java execution initialization time {"seconds"=>0.51}
[2022-03-15T16:36:22,278][INFO ][logstash.javapipeline    ][main] Pipeline started {"pipeline.id"=>"main"}
[2022-03-15T16:36:22,283][INFO ][logstash.inputs.tcp      ][main][f776fb44fb757dfb7aeb84ecda40ccb5b5dd2f486e8f7093cd4ebfab7f867c7d] Starting tcp input listener {:address=>"0.0.0.0:1514", :ssl_enable=>false}
[2022-03-15T16:36:22,312][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2022-03-15T16:36:26,816][WARN ][logstash.outputs.elasticsearch][main] Attempted to resurrect connection to dead ES instance, but got an error {:url=>"https://elastic:xxxxxx@interconomy.es.us-central1.gcp.cloud.es.io:9243/", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError, :message=>"Got response code '401' contacting Elasticsearch at URL 'https://interconomy.es.us-central1.gcp.cloud.es.io:9243/'"}

[jayparmar]

I am facing a similar issue for connecting Logstash and Elastisearch. It is mainly because "It is not authenticating incoming user". Here is a snippet logstash.conf file of my by which I resolved this error.

input {  
   beats {
      port => 5044
  }
}

output {
  elasticsearch {
     hosts=> ["http://localhost:9200/"]
     index=> "your-index-name"
     user=> "elasticsearch-username"
     password => "elasticsearch-user-password"
  }
}

Hope you find it useful

[Diego_T (Diego)]

thank you!