Logstash - Elasticsearch filter plugin with basic authentication

Elastic Stack Logstash
1

Hi all,

I have a problem with the Logstash elasticsearch filter plugin when using basic authentication. I have created a specific
user called "logstash_internal_reader" which has the built-in logstash_admin role and a custom role called logstash_reader :

{
  "logstash_reader" : {
    "cluster" : [ ],
    "indices" : [
      {
        "names" : [
          "*"
        ],
        "privileges" : [
          "read",
          "view_index_metadata"
        ],
        "allow_restricted_indices" : false
      }
    ],
    "applications" : [ ],
    "run_as" : [ ],
    "metadata" : { },
    "transient_metadata" : {
      "enabled" : true
    }
  }
}

I'm testing the elasticsearch filter plugin with a very simple pipeline:

input { stdin { } }

filter {
        elasticsearch {
                hosts => ["<my-host>"]
                index => "test"
                query => "message:Hello"
                fields => { "message" => "doc_message" }
                user => "logstash_internal_reader"
                password => "<my-password>"
                enable_sort => false
        }
}

output {
  stdout { codec => rubydebug }
}

The pipeline is not working. Logstash gives me the following error message:

Pipeline aborted due to error {:pipeline_id=>"MY-PIPELINE", :exception=>#<Elasticsearch::Transport::Transport::Errors::Forbidden: [403] >, :backtrace=>["/app/elk/LOGSTASH/vendor/bundle/jruby/2.5.0/gems/elasticsearch-transport-5.0.5/lib/elasticsearch/transport/transport/base.rb:202:in __raise_transport_error'", "/app/elk/LOGSTASH/vendor/bundle/jruby/2.5.0/gems/elasticsearch-transport-5.0.5/lib/elasticsearch/transport/transport/base.rb:319:inperform_request'", "/app/elk/LOGSTASH/vendor/bundle/jruby/2.5.0/gems/elasticsearch-transport-5.0.5/lib/elasticsearch/transport/transport/http/manticore.rb:67:in perform_request'", "/app/elk/LOGSTASH/vendor/bundle/jruby/2.5.0/gems/elasticsearch-transport-5.0.5/lib/elasticsearch/transport/client.rb:131:inperform_request'", "/app/elk/LOGSTASH/vendor/bundle/jruby/2.5.0/gems/elasticsearch-api-5.0.5/lib/elasticsearch/api/actions/ping.rb:20:in ping'", "/app/elk/LOGSTASH/vendor/bundle/jruby/2.5.0/gems/logstash-filter-elasticsearch-3.6.0/lib/logstash/filters/elasticsearch.rb:192:intest_connection!'", "/app/elk/LOGSTASH/vendor/bundle/jruby/2.5.0/gems/logstash-filter-elasticsearch-3.6.0/lib/logstash/filters/elasticsearch.rb:74:in register'", "org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java:56:inregister'", "/app/elk/LOGSTASH/logstash-core/lib/logstash/java_pipeline.rb:195:in block in register_plugins'", "org/jruby/RubyArray.java:1800:ineach'", "/app/elk/LOGSTASH/logstash-core/lib/logstash/java_pipeline.rb:194:in register_plugins'", "/app/elk/LOGSTASH/logstash-core/lib/logstash/java_pipeline.rb:468:inmaybe_setup_out_plugins'", "/app/elk/LOGSTASH/logstash-core/lib/logstash/java_pipeline.rb:207:in start_workers'", "/app/elk/LOGSTASH/logstash-core/lib/logstash/java_pipeline.rb:149:inrun'", "/app/elk/LOGSTASH/logstash-core/lib/logstash/java_pipeline.rb:108:in `block in start'"], :thread=>"#<Thread:0x2dfae9b0 run>"}

NB: Using the same logstash_internal_reader user I can:
1. Successfully query documents via CURL
2. Successfully query documents via Elasticsearch input plugin

2

Hi guys,
any news about this issue? Or is there something wrong in our implementation? I would like to be sure, before submitting a github issue.
Thanks

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

4

Hello @ea1987
Late answer, but starting from 3.5.0 (ES Filter plugin), we introduced health checks which require the rights to monitor the cluster (it will perform a HEAD /).
Please add the monitor cluster permission.

2 Likes