Logstash connection to Elastic search


I installed logstash, elastic search and kibana from debs on my system.
Version 6.2.4

I then installed the x-pack plugin on all components.
I used pretty much the defaults (exception being labels of cluster and such)
Everything works, data flows.

However, log stash will only connect to elasticsearch if I provide it the username and password of the "elastic" superuser. In which case it all works.

If I try using the internal logstash_system user. Making a new user with the role of logstash_admin.
It always gives this error when logstash starts:

Failed to install template. {:message=>"Got response code '403' contacting Elasticsearch at URL 'http://:9200/_template/logstash'"

This is my output config:

        elasticsearch {
            hosts => [ "<IP>:9200" ]
            index => "logstash-%{+YYYY.MM.dd}"
            user => "logstash_system"
            password => "password"

I've reset the password for the logstash account and this curl gives the following output:
(works against public IP as well)

curl 'http://localhost:9200/?pretty' -u logstash_system

Enter host password for user 'logstash_system':
"name" : "name",
"cluster_name" : "cluster_name",
"cluster_uuid" : "cluster_uuid",
"version" : {
"number" : "6.2.4",
"build_hash" : "ccec39f",
"build_date" : "2018-04-12T20:37:28.497551Z",
"build_snapshot" : false,
"lucene_version" : "7.2.1",
"minimum_wire_compatibility_version" : "5.6.0",
"minimum_index_compatibility_version" : "5.0.0"
"tagline" : "You Know, for Search"

Trying to access the templates:

curl -u logstash_system http://localhost:9200/_template/logstash
Enter host password for user 'logstash_system':
{"error":{"root_cause":[{"type":"security_exception","reason":"action [indices:admin/template/get] is unauthorized for user [logstash_system]"}],"type":"security_exception","reason":"action [indices:admin/template/get] is unauthorized for user [logstash_system]"},"status":403}

I made the logstash_admin user and gave it the internal logstash_admin role which through the kibana interface says it has

for the indices on the system, but trying to do the curl above also fails with that user.

curl -u logstash_admin http://localhost:9200/_template/logstash
Enter host password for user 'logstash_admin':
{"error":{"root_cause":[{"type":"security_exception","reason":"action [indices:admin/template/get] is unauthorized for user [logstash_admin]"}],"type":"security_exception","reason":"action [indices:admin/template/get] is unauthorized for user [logstash_admin]"},"status":403

Gives me an error. I haven't change any real settings.

If I use the "elastic" superuser in the logstash config everything works fine, but it seems to me that this shouldn't be required. Not to mention I need to manage the roles more distinctly in physical space as well.

Can someone point in the right direction to get this sorted out?


(CJ Cenizal) #2

Hi there Chris,

Someone else recently had a similar problem and posted their solution: Unable to modify roles or see logstash indices

Can you take a look at https://www.elastic.co/guide/en/logstash/6.x/ls-security.html and try following the steps for creating a logstash_writer role, logstash_internal user, and configuring Logstash to authenticate as that user? Please let me know if this helps.



Thank you very much. This was exactly what I needed.

I was missing some cluster permissions it seems.

And the linked document is immensely helpful in configuring other aspects.

(CJ Cenizal) #4

I'm glad I could help!


(system) #5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.