Filebeat is not sending a continuous stream to Logstash

Elastic Stack Beats
1

I'm running a development set of servers (not production). I have an ELK server, a web server, and a reverse proxy server. Data from the web server and the reverse proxy have successfully been sent into Logstash on the ELK server. Kibana's filebeat discovery does show the hostname of all three servers. So I know the configuration is correct.
Running the following command: sudo filebeat -e -c /etc/filebeat/filebeat.yml test output on both remote servers (the web server and the reverse proxy server) has the following response:

logstash: 192.168.1.6:5044... 
  connection... 
    parse host... OK 
   dns lookup... OK 
  addresses: 192.168.1.6 
  dial up... OK 
TLS... WARN secure connection disabled 
talk to server... OK

Running the command to see the status of the filebeat service (on both remote servers) reflects the service is active and does have recent timestamps of log data being collected. I did this a few times over a span of 15 minutes, and I can continuously see new time stamps of log collection by file beat on the remote servers.

But for some reason, the only time filebeat sends data to Logstash on the ELK server is upon rebooting the remote servers. And only just the once. Filebeat data is not a continuous stream of data into Logstash.

So my question is: How is it possible that testing filebeat output shows a successful connection, that filebeat data is successfully seen in elasticsearch, and that Kibana discovery has valid data for my servers? But, filebeat is not sending continuously into logstash.

Oh, one more thing. The ELK server does, in fact, have a continuous stream of filebeat data from itself (the ELK server's own filebeat is continuously streaming data to logstash). It's the other servers that are not feeding continuously.

I'm running Elasticstack version 7.17.13 on Ubuntu servers. ELK server is Ubuntu server 20.04 (minimal), the web server is Ubuntu server 20.04 (minimal), and the reverse proxy server is Ubuntu server 22.04 (minimal). All are on the same subnet. Running as virtual servers in an ESXi VMware platform.

I've tried purging filebeat from the web server and the reverse proxy server and ensuring the filebeat configurations are deleted.

Then, reinstalling filebeat and putting in a fresh configuration to send all data to logstash on the ELK server and test the filebeat outputs.

Everything looks OK, but I'm seeing only the one feed of data into logstash from remote servers, and that is when rebooting the remote servers! Ughhhhh!

2

I have determined why the Filebeat service is not streaming to ELK logstash. It is as simple as the logs do not exist to be sent. Apparently, a Ubuntu server (minimized) doesn’t have “rsyslog” installed, and therefore "auth.log, kern.log, and syslog don’t exist. Therefore, there are no logs that would change over time in the path /var/log/, except upon reboot. This makes sense, I suppose, because a minimized system is not expecting users to log into the system.

However, I must ensure that no rogue users are getting into the system, and therefore, I must see user logins, user sudo commands, etc., especially on the reverse proxy.

So I ran sudo apt-get install rsyslog on the web server and the reverse proxy server. Now, I see a regular stream of data into logstash from those servers because now the data actually exists.

Still don’t see sudo commands yet. Hopefully, I can figure that one out. At least I have streaming data from remote servers into Logstash now.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.