(solved) Marvel cpu always 0

Elastic Stack Kibana
1

ignore below as the system back to normal with no reason.

Hello,

The marvel data looks abnormal
1.cpu of all nodes are always 0%, however the min and max cup is showing, cluster status still shows yellow even it is already green

2.The data interval looks too large although I'm using default config. Index rate is abnormal as the system is indexing all the time and the index size is increasing.

Some background not sure whether it is related:

  • The cluster is just deployed in Azure using the template provided by Elastic
  • It is a 3 master nodes + 10 data nodes + 1 kibana cluster running ubuntu
  • I disabled shield from all elasticsearch nodes by config and commented out the shield user name and password in Kibana config
  • Indexing and searching is good

The other question I want to ask is how to shutdown Kibana in ubuntu, I use "service kibana stop", but the returned message shows kibana is running, and the thread id id different. Does that mean kibana automatically start after shut down?

2

Hi Tony,

This certainly looks weird. Specifically, it looks like Marvel is missing data based the document count graph. All of those flat lines imply that data is not being received. There's a few possibilities here:

  1. Your cluster is not able to send out Marvel documents fast enough because it's starving.
  2. The monitoring cluster, if you had an external one, is not able to keep up with the documents that it's receiving.

The data to the right of the CPU percentage implies that data is there given that it's able to find non-zero values for min and max (and the same goes for memory and load average, etc.).

As a best practice, it's best to offload Marvel from your production cluster in order to isolate performance issues. As I noted above, it's possible that the production side is impacting Marvel and vice versa.