Kibana 6.5.1 intermittently stops displaying data

[byoungman (Bill Youngman)]

We are having an issue where Kibana will just stop displaying data this can go on for a couple of hours or it can last 2 - 3 days there seems to be no pattern to it that I can ascertain.

We have alerts set up so that if documents stop being indexed into Elasticsearch we'll get alerted and we are not being alerted at all. The data just stops being displayed and the index sizes are all about the same size as data that is being displayed.

What could cause this behavior - it makes analysis and trouble shooting difficult as we have to go to the raw logs in order to do this.

TIA,
Bill Youngman

[jbudz (Jon Budzenski)]

Strange. Any queries? Which page do you see data not being displayed? In some cases discover can freeze if there's too much text to process.

[byoungman (Bill Youngman)]

I've attached some screen grabs of what we're experiencing - 2 for Kibana showing this gaps in the data display and 2 for Elasticsearch showing the indexes for those days. You'll notice that the document count for the May time period is similar for the June indexes so it doesn't appear that it's missing any documents.

Also I seem to have lost the functionality to display the Kibana query that was run for the current time range and can't find where to get it back.

Thanks
-b

[byoungman (Bill Youngman)]

Updated our stack to have Filebeat do all log handling and since that change we have not seen any more problems with Kibana not showing data.

Marking this as closed / resolved.

-Bill