Error adding the marvel template after Shield installed

Elastic Stack Elasticsearch
1

I have a one node Elasticsearch (1.7.1) cluster setting up with Marvel and Kibana and everything working OK. Then I followed the Shield (1.3.2) online doc to install the product, created an admin user, marvel_agent and marvel_user. But I got the following error when I restart ES:

error adding the marvel template to [x.x.x.x:8200] response code [401 Unauthorized]. content: [{"error":"AuthenticationException[missing authentication token for REST request [/_template/marvel]]","status":401}]

could not connect to any configured elasticsearch instances: [x.x.x.x:8200]

After I login to Marvel UI as admin, I am not able to see any index in the Overview page. What's wrong?

2

I think you may need to provide credentials for the Marvel exporter and the user querying the marvel indices as described in using marvel with shield.

3

Thanks for the reply, Jay. If you are referring the following credentials, I already confirmed that these are the default setting in the roles.yml right after I installed the Shield. All I did was to create the user marvel_agent and marvel_user after the install and restart ES according to the install doc. But it didn't work. If you are referring other credentials than this. please be more specific, may be I am missing something. Appreciate your help - Ming

on the marvel indices

marvel_user:
cluster: cluster:monitor/nodes/info, cluster:admin/plugin/license/get
indices:
'.marvel-*': all

Marvel Agent users

marvel_agent:
cluster: indices:admin/template/get, indices:admin/template/put
indices:
'.marvel-*': indices:data/write/bulk, create_index

4

Hi Ming,

Sorry I wasn't more specific; in your elasticsearch.yml file did you specify the marvel.agent.exporter.es.hosts setting with a user that has the marvel_agent role? If not, you will need to do that so that the exporter can index into the cluster.

Jay

5

Hi Jay,

I didn't do that change. When I read that section, somehow I got caught in the langue that it is only for configuring HTTPS. Once I make the change I am able to see all the indices. But one more question, the password of that user is in plain text in the elasticsearch.yml. which is a security concern. How to encrypt the password in the yml file, or I didn't do it correctly?

Thanks.
Ming

6

Hi Ming,

You have done it correctly. The password is in plaintext and that is a limitation that currently exists. We're looking at ways to provide better security for this, but there are some issues. If you have an encrypted value, then a key is needed to decrypt the value, and if the key is right next to the encrypted data then there really is not much added security.

One option, could be to use the elasticsearch prompt support, and start elasticsearch in the foreground. This will require you to enter the value for the marvel.agent.exporter.es.hosts when you start elasticsearch. This way your passwords are not stored in the elasticsearch.yml file. More information can be found at https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-configuration.html#styles (look for ${prompt.text} and ${prompt.secret}).

Jay

7

Hi Jay,

Thanks for the reply. While the suggestion is a work around to the plain text, we then can't practically start ES automatically, e.g. after a server reboot. So I will just wait for the enhancement on this issue.

Thanks.
Ming