Not seeing data in Kibana (suddenly)

[kprojects]

This will be a bit long, but hopefully have enough information to get sorted out..

Yesterday, I set up Elasticsearch, Logstash and Kibana on a clean CentOS 7 VM (hostname kp3).

I have two other servers (Both CentOS 6.. hostnames kp1, kp2) in which I set up with filebeat to send /var/log/secure and /var/log/messages to this server.

I followed this 'howto':

How To Install Elasticsearch, Logstash, and Kibana (ELK Stack) on CentOS 7 |...

Versions:
elasticsearch-2.4.6-1
logstash-2.2.4-1
kibana-4.4.2-1

Surprisingly, it worked right off the bat, then, it didn't.

I can see data being added in /var/lib/elasticsearch/elasticsearch/nodes/0/indices/

[root@kp3 ~]# du -sh /var/lib/elasticsearch/elasticsearch/nodes/0/indices/*
452K	/var/lib/elasticsearch/elasticsearch/nodes/0/indices/filebeat-2018.05.19
7.5M	/var/lib/elasticsearch/elasticsearch/nodes/0/indices/filebeat-2018.10.14
5.5M	/var/lib/elasticsearch/elasticsearch/nodes/0/indices/filebeat-2018.10.15
322M	/var/lib/elasticsearch/elasticsearch/nodes/0/indices/filebeat-2018.10.16
21M	/var/lib/elasticsearch/elasticsearch/nodes/0/indices/filebeat-2018.10.17
[root@kp3 ~]# ls -alrt /var/lib/elasticsearch/elasticsearch/nodes/0/indices/
total 0
drwxr-xr-x. 4 elasticsearch elasticsearch  29 Oct 16 18:56 .kibana
drwxr-xr-x. 8 elasticsearch elasticsearch  65 Oct 16 19:18 filebeat-2018.10.16
drwxr-xr-x. 8 elasticsearch elasticsearch  65 Oct 16 19:24 filebeat-2018.10.14
drwxr-xr-x. 8 elasticsearch elasticsearch  65 Oct 16 19:52 filebeat-2018.05.19
drwxr-xr-x. 8 elasticsearch elasticsearch  65 Oct 16 19:52 filebeat-2018.10.15
drwxr-xr-x. 8 elasticsearch elasticsearch 156 Oct 17 04:00 .
drwxr-xr-x  8 elasticsearch elasticsearch  65 Oct 17 04:00 filebeat-2018.10.17
drwxr-xr-x. 4 elasticsearch elasticsearch  52 Oct 17 19:09 ..

Going further into today's filebeat directory, I can see files like _2in.cfs, etc... that have data in them and were just written to this last minute..

In Kibana, using the sample dashboards, it says to use 'filebeat-*' index pattern. I see some data from yesterday, but only for about a half hour or so and nothing aftrewards.

I even looked through my history on the server to see what I must have changed/edited in that time that it stopped working and can't figure it out.

Any idea?

THanks!

[kprojects]

Fixed this - for the time being at least.. started nptd on each of the 3 servers, changed them all to the same timezone.

[Court (Court Ewing)]

Thanks for posting your resolution.

For what it's worth, I strongly encourage you to use a more recent version of the stack if you're just getting started. The versions from that tutorial have all long passed their EOL date and no longer get any updates, including security fixes. The latest version of the stack is 6.4.2.

If it helps, there is detailed installation documentation, including guides for rpm install for each product in the stack here: https://www.elastic.co/guide/en/elastic-stack/current/installing-elastic-stack.html