Kibana Infrastructure shows no data

[Shivarhce (Shiva)]

Hi Team,

Kibana Infrastructure shows no data, but I can see data sending by metricbeat its working fine.
what could be the problem to show data in Infrastructure UI.

Thank you & regards
Shiva S

[Shivarhce (Shiva)]

Hi Team,

I have pasted logs for the metricbeat. I pasted recent logs from my host.

2019-08-29T09:52:43.009+0800 INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":524210,"time":{"ms":263}},"total":{"ticks":1158440,"time":{"ms":567},"value":1158440},"user":{"ticks":634230,"time":{"ms":304}}},"handles":{"limit":{"hard":4096,"soft":1024},"open":7},"info":{"ephemeral_id":"c85dd131-e173-4404-a106-fd2e68028b6b","uptime":{"ms":59760074}},"memstats":{"gc_next":13876544,"memory_alloc":10056928,"memory_total":123502583760},"runtime":{"goroutines":43}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"acked":44,"batches":6,"total":44},"read":{"bytes":10110},"write":{"bytes":58565}},"pipeline":{"clients":3,"events":{"active":0,"published":44,"total":44},"queue":{"acked":44}}},"metricbeat":{"system":{"cpu":{"events":3,"success":3},"load":{"events":3,"success":3},"memory":{"events":3,"success":3},"network":{"events":6,"success":6},"process":{"events":23,"success":23},"process_summary":{"events":3,"success":3},"socket_summary":{"events":3,"success":3}}},"system":{"load":{"1":0.26,"15":0.36,"5":0.4,"norm":{"1":0.0325,"15":0.045,"5":0.05}}}}}}
2019-08-29T09:53:13.009+0800 INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":524470,"time":{"ms":267}},"total":{"ticks":1159020,"time":{"ms":583},"value":1159020},"user":{"ticks":634550,"time":{"ms":316}}},"handles":{"limit":{"hard":4096,"soft":1024},"open":7},"info":{"ephemeral_id":"c85dd131-e173-4404-a106-fd2e68028b6b","uptime":{"ms":59790072}},"memstats":{"gc_next":13249632,"memory_alloc":10541664,"memory_total":123564541496},"runtime":{"goroutines":43}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"acked":53,"batches":6,"total":53},"read":{"bytes":12027},"write":{"bytes":66863}},"pipeline":{"clients":3,"events":{"active":0,"published":53,"total":53},"queue":{"acked":53}}},"metricbeat":{"system":{"cpu":{"events":3,"success":3},"filesystem":{"events":8,"success":8},"fsstat":{"events":1,"success":1},"load":{"events":3,"success":3},"memory":{"events":3,"success":3},"network":{"events":6,"success":6},"process":{"events":23,"success":23},"process_summary":{"events":3,"success":3},"socket_summary":{"events":3,"success":3}}},"system":{"load":{"1":0.33,"15":0.36,"5":0.41,"norm":{"1":0.0413,"15":0.045,"5":0.0513}}}}}}

[jbudz (Jon Budzenski)]

Can you verify a few things

• you can see metricbeat-* data in kibana (e.g. by adding the index pattern in discover)
• the metricbeat version matches kibana (not 100% necessary but if they're majorly different that may be a reason)
• the infra ui is configured to look for the metricbeat-* or similar index pattern, you can get there by checking the infra ui settings

Also can you share your Kibana version?

[Shivarhce (Shiva)]

Hi Jbudz,

Thank you for your response

It got it fixed by rebuild my lab setup once again from the beginning. I don't see any issue now with Infra UI.

My lab setup:
OS Platform: Redhat Linux 6.8
Java: JDK12
ELK Stack Version: 7.3

Hostname Applications
elasticsearch-server1 - elasticsearch
elasticsearch-server2 - elasticsearch
elasticsearch-server3 - elasticsearch
kibana-server - kibana
beats-server - filebeat, metricbeat and auditbeat.

Cluster Health:

I have another issue now with timestamp and logs are missing while I see log in kibana. I have attached screen shot below for better understand. for example I marked both log lines of /var/log/messages. Last log line timestamp shows in kibana and server log as same timestamp. but above of last line in kibana log screen shot shows different timestamp its not match with server log.
And In between that two log lines are missing in kibana log. Please compare it both screen shot and let me know how to correct/fix it, for other server I dont see any issue, I am getting this timestamp issue for only this two servers (elasticsearch-server1 and elasticsearch-server2).

Server Log:

Kibana Infra UI Log:

Infra UI Screen shot:

[Shivarhce (Shiva)]

okay I found it.

It cause of filebeat system modules and elasticsearch modules