This will be a bit long, but hopefully have enough information to get sorted out..
Yesterday, I set up Elasticsearch, Logstash and Kibana on a clean CentOS 7 VM (hostname kp3).
I have two other servers (Both CentOS 6.. hostnames kp1, kp2) in which I set up with filebeat to send /var/log/secure and /var/log/messages to this server.
I followed this 'howto':
Versions:
elasticsearch-2.4.6-1
logstash-2.2.4-1
kibana-4.4.2-1
Surprisingly, it worked right off the bat, then, it didn't.
I can see data being added in /var/lib/elasticsearch/elasticsearch/nodes/0/indices/
[root@kp3 ~]# du -sh /var/lib/elasticsearch/elasticsearch/nodes/0/indices/*
452K /var/lib/elasticsearch/elasticsearch/nodes/0/indices/filebeat-2018.05.19
7.5M /var/lib/elasticsearch/elasticsearch/nodes/0/indices/filebeat-2018.10.14
5.5M /var/lib/elasticsearch/elasticsearch/nodes/0/indices/filebeat-2018.10.15
322M /var/lib/elasticsearch/elasticsearch/nodes/0/indices/filebeat-2018.10.16
21M /var/lib/elasticsearch/elasticsearch/nodes/0/indices/filebeat-2018.10.17
[root@kp3 ~]# ls -alrt /var/lib/elasticsearch/elasticsearch/nodes/0/indices/
total 0
drwxr-xr-x. 4 elasticsearch elasticsearch 29 Oct 16 18:56 .kibana
drwxr-xr-x. 8 elasticsearch elasticsearch 65 Oct 16 19:18 filebeat-2018.10.16
drwxr-xr-x. 8 elasticsearch elasticsearch 65 Oct 16 19:24 filebeat-2018.10.14
drwxr-xr-x. 8 elasticsearch elasticsearch 65 Oct 16 19:52 filebeat-2018.05.19
drwxr-xr-x. 8 elasticsearch elasticsearch 65 Oct 16 19:52 filebeat-2018.10.15
drwxr-xr-x. 8 elasticsearch elasticsearch 156 Oct 17 04:00 .
drwxr-xr-x 8 elasticsearch elasticsearch 65 Oct 17 04:00 filebeat-2018.10.17
drwxr-xr-x. 4 elasticsearch elasticsearch 52 Oct 17 19:09 ..
Going further into today's filebeat directory, I can see files like _2in.cfs, etc... that have data in them and were just written to this last minute..
In Kibana, using the sample dashboards, it says to use 'filebeat-*' index pattern. I see some data from yesterday, but only for about a half hour or so and nothing aftrewards.
I even looked through my history on the server to see what I must have changed/edited in that time that it stopped working and can't figure it out.
Any idea?
THanks!