Can't get logging to happen

Elastic Stack Beats
1

Hello,

I'm trying to verify that both topbeat and packetbeat can connect to the logstash host over port 5000.

I've enabled logging in the yaml files for both and for some reason the logging directories remain empty after a service restart.

Using topbeat for an example, but the result is the same for both:

First I stop topbeat:

[root@ops1:/etc/topbeat] #systemctl stop topbeat
[root@ops1:/etc/topbeat] #systemctl status topbeat
topbeat.service - topbeat
   Loaded: loaded (/usr/lib/systemd/system/topbeat.service; enabled)
   Active: failed (Result: start-limit) since Tue 2016-02-16 00:03:19 EST; 50s ago
     Docs: https://www.elastic.co/guide/en/beats/topbeat/current/index.html
  Process: 29854 ExecStart=/usr/bin/topbeat -c /etc/topbeat/topbeat.yml (code=exited, status=1/FAILURE)
 Main PID: 29854 (code=exited, status=1/FAILURE)

Feb 16 00:03:19 ops1 systemd[1]: topbeat.service: main process exited, code=exited, stat...LURE
Feb 16 00:03:19 ops1 systemd[1]: Unit topbeat.service entered failed state.
Feb 16 00:03:19 ops1 systemd[1]: topbeat.service holdoff time over, scheduling restart.
Feb 16 00:03:19 ops1 systemd[1]: Stopping topbeat...
Feb 16 00:03:19 ops1 systemd[1]: Starting topbeat...
Feb 16 00:03:19 ops1 systemd[1]: topbeat.service start request repeated too quickly, ref...art.
Feb 16 00:03:19 ops1 systemd[1]: Failed to start topbeat.
Feb 16 00:03:19 ops1 systemd[1]: Unit topbeat.service entered failed state.
Feb 16 00:04:02 ops1 systemd[1]: Stopped topbeat.
Hint: Some lines were ellipsized, use -l to show in full.

Then I enable logging in the config:

[root@ops1:/etc/topbeat] #egrep -v "^$|^(.*)#|^#" topbeat.yml
input:
  period: 10
  procs: [".*"]
  stats:
    system: true
    proc: true
    filesystem: true
    cpu_per_core: false
output:
  elasticsearch:
  logstash:
    hosts: ["logs.example.com:5000"]
    tls:
      certificate_authorities: ["/etc/pki/CA/certs/ca.crt"]
shipper:
  tags: ["jokefire-prod", "web-tier"]
logging:
  to_syslog: false
  to_files: true
  files:
    path: /var/log/topbeat
    name: topbeat.log
    keepfiles: 7

And when I start it back up again:

[root@ops1:~] #systemctl start topbeat
[root@ops1:~] #systemctl status topbeat
topbeat.service - topbeat
   Loaded: loaded (/usr/lib/systemd/system/topbeat.service; enabled)
   Active: active (running) since Tue 2016-02-16 00:08:34 EST; 6s ago
     Docs: https://www.elastic.co/guide/en/beats/topbeat/current/index.html
 Main PID: 30592 (topbeat)
   CGroup: /system.slice/topbeat.service
           └─30592 /usr/bin/topbeat -c /etc/topbeat/topbeat.yml

Feb 16 00:08:34 ops1 systemd[1]: Starting topbeat...
Feb 16 00:08:34 ops1 systemd[1]: Started topbeat.

The log directory for topbeat remians empty!!

[root@ops1:~] #ls -lh /var/log/topbeat/
total 0

As I've said I've seen the exact same behavior for packebeat. And this is an experiment I tried with both services on 3 different hosts. I can't seem to get logging to happen on any of the hosts for either packetbeat or topbeat.

Just wondering what the problem could be and how to correct that!

Also , I have my beats input on my logstash host listening on port 5000. How can all 3 services: filebeat, topbeat, and packetbeat all send messags to the logstash host on the same port?

2

For the logging: The default level is "warning", so it could be the log file is not created because there was no warning yet. Can you change the log level to info an try again? https://www.elastic.co/guide/en/beats/topbeat/current/configuration-logging.html#_level

About the LS connection: Is it working that all 3 send their data to LS or is it a question if it will work?

3

Hi Thanks,

Yeah, I changed my logging level to 'debug' and after I restarted the agent, the log file appeared and started filling with data. So that was the problem, I had forgotten to set the log level.

As to my question regarding sending data to LS, I want to know if all 3 beats (filebeat, topbeat and packetbeat) can all share the same port. Which is 5000 in my case. I never knew a case before where 3 different daemons could share the same port. So that's why I'm asking.

Thanks

4

As 5000 is the port they are sending information to, this should not be an issue. Let us know if there are any issues.