Not finding my log file

Elastic Stack Beats
1

There is some newbie thing I have missed here. I have configured filebeat 1.0.1 on the latest ubuntu 14.04, installed via the elastic.co repository. Stripped of comments and blank lines, my filebeat.yml file looks like this:

filebeat:
  prospectors:
    -
      paths:
        - /var/log/appname/file.log
      input_type: log
  registry_file: /var/lib/filebeat/registry
output:
  logstash:
    hosts: ["logstash.example.com:4018"]
shipper:
logging:
  files:

/var/log/appname/file.log is not empty, but nothing turns up on my logstash system, to which I have added this input: (It works with other inputs.)

beats {
    port => 4018
    codec => json_lines {}
}

/var/lib/filebeat/registry contains just

{}

which makes me think that it has not found /var/log/appname/file.log, though I can see access it using tail, etc.

filebeat is running as root and using the right .yml file:

root      5994     1  0 02:09 pts/2    00:00:00 /usr/bin/filebeat-god -r / -n -p /var/run/filebeat.pid -- /usr/bin/filebeat -c /etc/filebeat/filebeat.yml
root      5995  5994  0 02:09 pts/2    00:00:00 /usr/bin/filebeat -c /etc/filebeat/filebeat.yml

Any ideas?

2

can you try to delete the registry file and run filebeat with debug output:

filebeat -e -v -c /etc/filebeat.yml -d '*'

The -d option enabled all debug selectors. If output is too noise you can try to run with '-v -e' only.

3

Hi Steffen,

Thanks for that. With the help of the debugging output I realised that I had failed to open a hole in a firewall that filebeat needed in order to get to logstash. I focused on the registry file because I saw no errors for filebeat in syslog, and because I didn't expect it to be empty.

Regards,
Oliver