Transport Error Unauthorized

I upgraded our cluster to 7.14, and after the update for Logstash I received "Pipeline error {:pipeline_id=>"main", :exception=>#<Elasticsearch::Transport::Transport::Errors::Unauthorized: [401]". I was able to isolate it to pipeline configs using the logstash-filter-elasticsearch plugin. The Elasticsearch output plugin works fine, as I see data being indexed. The cluster is using basic auth and TLS. Steps I took to try to fix the issue:

  • Verified credentials with the _authenticate API.
  • Verified the role in Kibana had index: read and cluster: manage set
  • Tried with the superuser account to rule out missing permissions
  • Updated the logstash-filter-elasticsearch plugin

I do have a second Logstash server running 7.13.4 with the same config, and it is not exhibiting issue. This seems to be an issue with the latest version the this specific plugin. Any help is appreciated.

1 Like

+1

I just upgraded to Logstash 7.14.0 and am also seeing this. I have one Logstash server that performs lookups via the Logstash Elasticsearch Filter Plugin, and this plugin is now causing Logstash to crash since the upgrade from 7.13.3 to 7.14.0.

Aug 05 11:16:17 LS1 ServerName[10839]: [2021-08-05T11:16:17,173][ERROR][logstash.javapipeline    ][main] Pipeline error {:pipeline_id=>"main", :exception=>#<Elasticsearch::Transport::Transport::Errors::Unauthorized: [401] >, :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/elasticsearch-transport-7.5.0/lib/elasticsearch/transport/transport/base.rb:205:in `__raise_transport_error'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/elasticsearch-transport-7.5.0/lib/elasticsearch/transport/transport/base.rb:333:in `perform_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/elasticsearch-transport-7.5.0/lib/elasticsearch/transport/transport/http/manticore.rb:71:in `perform_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/elasticsearch-transport-7.5.0/lib/elasticsearch/transport/client.rb:152:in `perform_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/elasticsearch-api-7.5.0/lib/elasticsearch/api/actions/ping.rb:23:in `ping'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-elasticsearch-3.9.3/lib/logstash/filters/elasticsearch.rb:310:in `test_connection!'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-elasticsearch-3.9.3/lib/logstash/filters/elasticsearch.rb:117:in `register'", "org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java:75:in `register'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:228:in `block in register_plugins'", "org/jruby/RubyArray.java:1820:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:227:in `register_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:586:in `maybe_setup_out_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:240:in `start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:185:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:137:in `block in start'"], "pipeline.sources"=>["/etc/logstash/conf.d/redacted1.conf", "/etc/logstash/conf.d/redacted2.conf", "/etc/logstash/conf.d/redacted3.conf", "/etc/logstash/conf.d/redacted4.conf"], :thread=>"#<Thread:0x198c08c8 run>"}

Yep, same here.

Going to do a rollback, raise a ticket..

@Kaj_Noppen,

Did a rollback work for you? I rolled Logstash back to the previous version we were running (7.13.3), but the issue did not go away.

EDIT:

Just completely purged Logstash from the system, installed 7.13.3, and still having the issue. :frowning:

Yep, Rollback fixed this instantly. Support is looking for the issue.

Just an FYI, Elastic created some Github issues, so it's a confirmed bug and should be fixed sometime :smiley:

1 Like

Thanks for the update!

same issue. just upgraded to 7.14.0 and
all job failed which has filter { eleasticsearch and Variable for username/password

The git page says it is fixed. How do I fix this?

The git page says it is fixed in 7.14.1, which has not released yet.

what should I do mean time? that ingestion is not super crtical. hence I was thinking to do some manual work this then if I can't fix it.

should I move to manual fix?

@elasticforme
Running /usr/share/logstash/bin/logstash-plugin update logstash-filter-elasticsearch fixed it for my v7.14.0 host.

2 Likes

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.