Hi Elastic Sorcerers,
I’ve recently set up a network monitoring system where data about network activity is archived in an Elasticsearch v 7.4.0 Docker container. On the same physical host (Ubuntu 16.04), I also have a Kibana v 7.4.0 Docker container, which manages and displays the ES data. As I’m new to the Elastic suite of products, everything is pretty much running with minimal (and default) configurations.
The Kibana application works great… except I’ve noticed there is about a three minute lag between when data arrives in Elasticsearch and when that same data appears in my Kibana dashboard.
In other words: Suppose I run a test where Host A sends some traffic to Host B across my network. When I run this test and then do an SQL-like query against Elasticsearch, I can see data about the traffic appear almost immediately. But I must wait three minutes before I see that same data appear in my Kibana Visualizations.
I’ve run a lot of tests, and there is no discrepancy between the ES-reported data and the Kibana-reported data. The only issue is that Kibana seems to be three minutes behind Elasticsearch.
I hope to use my system to monitor network activity in real time, so this three minute lag is a serious problem. But I have no idea where to start troubleshooting.
As I mentioned, I set up my ES / Kibana containers using a tutorial, doing a minimal configuration. I’m sure I’m using a lot of default settings. Is this a Kibana configuration issue, and if so, where would I look?
Or could this be a system problem? When I do a simple “uptime” on the command line of both containers, it seems that they have the same system time.
Any advice or help is appreciated! Thanks….