Investigating why all logs aren't visible in Kibana

Hi all,

I've put "Kibana" as the category for the post, but realistically it could be any part of the Elastic stack that I've misconfigured.

Elastic stack structure
Azure-hosted VMs running the following; part of the same IP range as the DCOS platform:

  • Elastic-master
  • Elastic-data-1
  • Elastic-data-2
  • Logstash
  • Kibana

Application infrastructure

  • Master Nodes (Azure-hosted Linux VMs) running DCOS container orchestration platform
  • Agent Nodes (Azure-hosted Linux VM Scale Set) running containers, orchestrated by DCOS
  • Application comprised of NodeJS microservices each running as a container
  • Microservices log to container's STDOUT
  • Containers' STDOUT entries are stored in log files on whichever Agent Node they're running on


  • Filebeats container running on each Agent Node
  • Scrapes logs from Agent Node
  • Passes to Logstash
  • Logstash pipeline filters/categorises log entries and passes to Elasticsearch

This setup works as intended in our Development environment. In Kibana, I can see all of our containers' logs/log entries and they're parsed correctly.

I've just recreated the ELK cluster in our Pre-Production environment and encountered the following problems in Kibana:

  • Not all log entries appear to be coming through; I can only see 5 of our ~20 microservices
  • If I click on "Logs", I see the error "Looks like you don't have any logging indices." yet if I go to "Management" then "Elasticsearch Index Management", I can see my indices with green health status

In an attempt to diagnose these problems, I would like to know:

  • How can I be certain that all of the Elastic nodes are working together and processing data?
  • In my Logstash pipeline, should I be specifying the IP addresses of all of my Elastic nodes (Master and 2x Data) or just the Master? Currently I'm just specifying hosts: [] which is the IP of the Master
  • Is there a way that I can see the log entries that Logstash is sending to Elasticsearch, and a way that I can see the output that Kibana is accessing from Elasticsearch?

Thanks in advance

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.