Problem with monitoring x-pack

Elastic Stack Elasticsearch
1

hello!
i actually have a problem with monitoring, i can't access it, i have the following message :

image.png1222×424 14.6 KB

here is what the logs says, i don't understand at all:

Bad Request :: {"path":"_xpack/security/user/_has_privileges","body":"{"index":[{"names":[".monitoring--2-"],"privileges":["read"]}]}","statusCode":400,"response":"No handler found for uri [_xpack/security/user/_has_privileges] and method [POST]"}
at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:295:15)
at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:254:7)
at HttpConnector. (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:159:7)
at IncomingMessage.bound (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/dist/lodash.js:729:21)
at emitNone (events.js:91:20)
at IncomingMessage.emit (events.js:185:7)
at endReadableNT (_stream_readable.js:974:12)
at _combinedTickCallback (internal/process/next_tick.js:80:11)
at process._tickDomainCallback (internal/process/next_tick.js:128:9)

if you have an idea it would be really helpfull!!!!
thank you!

2

Hi again, I believe you asked a similar question Elasticsearch doesn't work after installing xpack

By default when you install x-pack, it expects you already set up your security "layers" between the components. If you haven't set up the security between the components then they won't be able to communicate with each other because they will simply reject each others' request communicate.

So in this case, try disabling the security in your cluster.

3

Did you install X-Pack on both Elasticsearch and Kibana? From this error it looks like X-Pack was installed for Kibana but not for Elasticsearch

4

yes x-pack is install on elasticsearch and i have already set up the security to false :

mayber i forgot Something, but i don't know what

5

Did you set security to false in both Elasticsearch and also in Kibana? If you disable security in elasticsearch.yml you need to also disable it in Kibana's kibana.yml file

6

yes i also set security to false in kibana.yml

7

Hi @Benjamin_Peere,

Can you share your settings from your kibana.yml (feel free to obfuscate any secret values)? Can you do the same with your elasticsearcy.yml?

Thanks

8

hi, no problem,
here is the kibana.yml

server.port: 5602
server.host: "my-host"

xpack.security.enabled: false
xpack.monitoring.enabled: true
9

and here is my elasticsearch.yml :

cluster.name: elastic
node.name: elasticsearch
network.host: 0.0.0.0
http.port: 9200
xpack.security.enabled: false
xpack.monitoring.enabled: true

xpack.notification.email.account:
    gmail_account:
        profile: gmail
        smtp:
            auth: true
            starttls.enable: true
            host: smtp.gmail.com
            port: 587
            user: ********@gmail.com
            password: *********

thanks for your reply!

10

Hi @Benjamin_Peere,

Nothing seems very weird in either configuration. As a side note, Kibana will dynamically determine if it needs to use security by talking to Elasticsearch., but it shouldn't matte if you are explicit.

Which version of Kibana / Elasticsearch are you running? Also, which license level are you using?

Thanks,
Chris

11

i'm actually using kibana and elasticsearch version 6.0.0, and i'm using a free license,
thanks

12

i'm actually using kibana and elasticsearch version 6.0.0, and i'm using a free license,

Ah! I missed it in the screenshot. Sorry.

Ugh, so I have bad but also good news. You caught a bug in alpha2! The Monitoring UI is not checking that API only when security is in used. This is already fixed, but it obviously doesn't save you here.

Fortunately there is a workaround: you can leave Security enabled in Elasticsearch, but turn on anonymous access via a superuser role.

xpack.security.authc.anonymous.roles: superuser

Naturally this would not be a solution for a production setup, but you should never be running an alpha build in a production scenario.

Hope that helps,
Chris

13

thank you i will try this!
i didn't paid attention that i was in a beta version, do you know how i can do to downgrade my version without loosing my data?

14

You can not downgrade existing indices, so you will need to reindex the data.

15

ok, it's not a problem, is there a way to downgrade easily or do i have absolutely uninstal kibana/elasticsearch and reinstal it?

16

You will need to reinstall completely as Elasticsearch 6.0 uses a newer version of Lucene for storage that Elasticsearch 5.x can not read. If you want to reindex your data from Elasticsearch 6.0, you should be able to install a separate instance of Elasticsearch side by side and use the reindex from remote API.

17

Thank you for all your answer, i have reinstall elasticsearch and kibana and reindex everything.

and i have another problem, always with monitoring,
i have the following message :

image.png1142×204 12 KB

the thing is that i don't have text fields in my index :

image.png346×770 19.9 KB

if you have an idea of what is the problem that would be awesome!
thanks!

18

@Benjamin_Peere

It's complaining about a field named cluster_uuid. This is most likely coming from X-Pack monitoring indices and it most commonly happens because the templates were deleted / overwritten.

It's hard to say what version you are seeing this issue under now, but the easiest thing to do is to open up Dev Tools within Kibana and then Console. In there, run this command:

DELETE /.monitoring-*

Alternatively you can do this from cURL (or your favorite command line tool):

curl -XDELETE localhost:9200/.monitoring-*

This will delete any X-Pack monitoring-generated indices. They will be recreated, but there should be no funny business with their templates being deleted in the middle. If you reindexed your data from the 6.0 cluster and simply brought over indices from that cluster, then it's critical to always pre-create mappings before creating indices. If you ended up copying over .monitoring-* indices, it's safe to just delete them and to allow it to start fresh.

As a completely unrelated suggestion, you should look into time-based indices for your own data so that they are easier to curate.

We have a few places that discuss it, including the new "rollover" feature:

19

hi!
thanks for your answer,
when i moved from 6.0 to 5.4.3 i had delete all my indices and recreated them.
i tried : DELETE /.monitoring-*
when i do this, it work for maybe 30 seconds and it does the samething after, i also tried to delete my index and recreate it, and it doesn't work.

20

@pickypg -- Does there happen to be a release date for the next XPack build or can I get a copy of the build [6.0.0-alpha2] with the bug fix? I am running [xpack.security.enabled: false] because I don't want to go through the headache of configuring all of the TLS and User permissions stuff on my test cluster every time I delete/start over with stress testing.

If there's a way to override the boostrap security checks for a test cluster (not single node) I'm all ears, and then I can use the workaround you mentioned.

Thanks for your help!

~Jer