Logstash new install: response code '403' contacting Elasticsearch

Posting this for the record, should anyone else run into the same problem...

LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError: Got response code '403' contacting Elasticsearch at URL 'https://es01:9200/logstash'
                    perform_request at /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.8.1-java/lib/logstash/outputs/elasticsearch/http_client/manticore_adapter.rb:80

I've run through a couple of new installations of Elastic Stack over the past two months, with a particular issue of not being able to get authentication for publishing to Elasticsearch from Logstash to work. Works fine with a "superuser" userid but not with any other credential that doesn't have this level of privileges.

As I said, a new install; I hadn't done any work with this stuff in a few years but I had some background; followed the guides and documentation online to get a three-cluster (container, in my case) ES cluster running, with Kibana and a Logstash container.

The reason for the background is that I specifically backed off from where I started, and was using the documentation for Configuring Security in Logstash to the point of pasting in the examples for defining a role, creating a user with that role, and creating an API key to use.

The issue seems to be this: while all of the configuration for Logstash uses the string logstash-* as an index template - as in all of the Kibana references, the abovementioned examples, and other questions I saw posted while researching this, in my environment it causes this 403 issue, and the fix is to express the pattern as logstash* (i.e., no hyphen). This was implied in at least one other post, but not confirmed, so I wish to confirm it.

It's also the case that although I didn't quite get it on first (and fourth) reading, creating an API key doesn't have any relationship to a user - meaning that for setting access for an API key, the same logstash* string needs to be used to specify the index pattern for access, or else a 403 results when trying to use that API key for access.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.