Getting data into Elasticsearch

[pace]

Hi all,

So i'm finding out that the documentation for the elasticstak is really not user friendly at all.

I have managed to get to grips with beats and logstash, but when it comes to getting my data into elasticsearch i'm hitting a brick wall.

From the default installation of elasticsearch, I understand that in order for me to get this data into elasticsearch, I first need to setup a user with the correct privileges.

I have read over this document more times that I would have liked, to no avail.

Basic authentication

Does anyone have an easy to read process for simply getting my data into elasticsearch.

Kind regards,

[Christian_Dahlqvist]

Have you got X-Pack installed? How have you configured it? What errors are you seeing?

[pace]

x-pack is installed for elasticsearch and kibana. Both elasticsearch and kibana are running.

bin/elasticsearch
bin/kibana

I login to kibana at http://localhost:5601/

From a terminal I use the following commands for testing:

curl -XPUT 'localhost:9200/idx'

{"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication token for REST request [/idx]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}}],"type":"security_exception","reason":"missing authentication token for REST request [/idx]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}},"status":401}

From here I created a logstash writer role

POST _xpack/security/role/logstash_writer
{
"cluster": ["manage_index_templates", "monitor"],
"indices": [
{
"names": [ "logstash-*" ],
"privileges": ["write","delete","create_index"]
}
]
}

Then a logstash internal user:

POST _xpack/security/user/logstash_internal
{
"password" : "changeme",
"roles" : [ "logstash_writer"],
"full_name" : "Internal Logstash User"
}

I then ran the following command from the terminal to test the setup:

curl --user logstash_internal:changeme -XPUT 'localhost:9200/idx'

{"error":{"root_cause":[{"type":"security_exception","reason":"action [indices:admin/create] is unauthorized for user [logstash_internal]"}],"type":"security_exception","reason":"action [indices:admin/create] is unauthorized for user [logstash_internal]"},"status":403}

If I add the superuser role to logstash_internal and run:

curl --user logstash_internal:changeme -XPUT 'localhost:9200/idx'

{"acknowledged":true,"shards_acknowledged":true}

How can I get this to work without the superuser role?

[Christian_Dahlqvist]

I moved this to the X-Pack category.

[TimV (Tim Vernum)]

"names": [ "logstash-*" ],

Your logstash_writer role only grants access to indices that match the logstash-* pattern.

[quote=", post:3, topic:88010"]
-XPUT 'localhost:9200/idx'
[/quote]

But here you are trying to create an index named idx.

You either need to change your role to allow access to other index names, or PUT something like logstash-test

[pace]

Tim, thank you so much, i'm such an idiot !

You saved me from pulling out anymore of my hair