Apache module and var.paths not working

Elastic Stack Beats
1

All,

I'm facing an issue with the apache module paths variable.
When enabling the module without any customization, filebeat starts as expected sending the default access-log and error.log files to a logstash/eleastic server.
As soon as I'm adding var.paths in the configuration to also process my other apache logs, filebeat is not able to start.
The error message is
"Fileset apache/var is configured but doesn't exist"

The config is:

  • module: apache
    access:
    enabled: true
    var.paths: ["/var/log/apache2/.access"]
    error:
    enabled: true
    var.paths: ["/var/log/apache2/    .err"]

Am I missing something?

Thanks
René

2

Hey @Hausmeister, welcome to discuss :slight_smile:

Indentation is relevant in Beats configuration, in the case of filebeat modules, vars have to be defined under the fileset, like this:

- module: apache
  access:
    enabled: true
    var.paths: ['/var/log/apache2/*.access']
  error:
    enabled: true
    var.paths: ['/var/log/apache2/*.err']
3

Hi @jsoriano,

thanks for your reply.
I had the asterisks in my config. Due to format limits they were not shown in my post.
Tested with single quotes as you posted but still not working.
Receiving the same error message. As from the debug log:
'May 11 07:41:26 webserver systemd[1]: Started Filebeat sends log files to Logstash or directly to Elasticsearch..
May 11 07:41:26 webserver filebeat[98144]: 2020-05-11T07:41:26.216+0200#011INFO#011instance/beat.go:622#011Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]
May 11 07:41:26 webserver filebeat[98144]: 2020-05-11T07:41:26.218+0200#011DEBUG#011[beat]#011instance/beat.go:674#011Beat metadata path: /var/lib/filebeat/meta.json
May 11 07:41:26 webserver filebeat[98144]: 2020-05-11T07:41:26.218+0200#011INFO#011instance/beat.go:630#011Beat ID: 99743ca0-0c4f-477f-8d0d-2d7f6d71c6e1
May 11 07:41:26 webserver filebeat[98144]: 2020-05-11T07:41:26.219+0200#011DEBUG#011[processors]#011processors/processor.go:101#011Generated new processors: add_host_metadata=[netinfo.enabled=[false], cache.ttl=[5m0s]]
May 11 07:41:26 webserver filebeat[98144]: 2020-05-11T07:41:26.220+0200#011DEBUG#011[seccomp]#011seccomp/seccomp.go:117#011Loading syscall filter#011{"seccomp_filter": {"no_new_privs":true,"flag":"tsync","policy":{"default_action":"errno","syscalls":[{"names":["accept","accept4","access","arch_prctl","bind","brk","chmod","clock_gettime","clone","close","connect","dup","dup2","epoll_create","epoll_create1","epoll_ctl","epoll_pwait","epoll_wait","exit","exit_group","fchdir","fchmod","fchown","fcntl","fdatasync","flock","fstat","fstatfs","fsync","ftruncate","futex","getcwd","getdents","getdents64","geteuid","getgid","getpeername","getpid","getppid","getrandom","getrlimit","getrusage","getsockname","getsockopt","gettid","gettimeofday","getuid","inotify_add_watch","inotify_init1","inotify_rm_watch","ioctl","kill","listen","lseek","lstat","madvise","mincore","mkdirat","mmap","mprotect","munmap","nanosleep","newfstatat","open","openat","pipe","pipe2","poll","ppoll","pread64","pselect6","pwrite64","read","readlink","readlinkat","recvfrom","recvmmsg","recvmsg","rename","renameat","rt_sigaction","rt_sigprocmask","rt_sigreturn","sched_getaffinity","sched_yield","sendfile","sendmmsg","sendmsg","sendto","set_robust_list","setitimer","setsockopt","shutdown","sigaltstack","socket","splice","stat","statfs","sysinfo","tgkill","time","tkill","uname","unlink","unlinkat","wait4","waitid","write","writev"],"action":"allow"}]}}}
May 11 07:41:26 webserver filebeat[98144]: 2020-05-11T07:41:26.221+0200#011INFO#011[seccomp]#011seccomp/seccomp.go:124#011Syscall filter successfully installed
May 11 07:41:26 webserver filebeat[98144]: 2020-05-11T07:41:26.221+0200#011INFO#011[beat]#011instance/beat.go:958#011Beat info#011{"system_info": {"beat": {"path": {"config": "/etc/filebeat", "data": "/var/lib/filebeat", "home": "/usr/share/filebeat", "logs": "/var/log/filebeat"}, "type": "filebeat", "uuid": "99743ca0-0c4f-477f-8d0d-2d7f6d71c6e1"}}}
May 11 07:41:26 webserver filebeat[98144]: 2020-05-11T07:41:26.222+0200#011INFO#011[beat]#011instance/beat.go:967#011Build info#011{"system_info": {"build": {"commit": "d57bcf8684602e15000d65b75afcd110e2b12b59", "libbeat": "7.6.2", "time": "2020-03-26T05:23:38.000Z", "version": "7.6.2"}}}
May 11 07:41:26 webserver filebeat[98144]: 2020-05-11T07:41:26.222+0200#011INFO#011[beat]#011instance/beat.go:970#011Go runtime info#011{"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":1,"version":"go1.13.8"}}}
May 11 07:41:26 webserver filebeat[98144]: 2020-05-11T07:41:26.223+0200#011INFO#011[beat]#011instance/beat.go:974#011Host info#011{"system_info": {"host": {"architecture":"x86_64","boot_time":"2020-03-31T15:07:46+02:00","containerized":false,"name":"webserver","ip":["127.0.0.1/8","::1/128","192.168.1.25/24","2003:c6:2f10:b900:215:5dff:fe01:501/64","fe80::215:5dff:fe01:501/64"],"kernel_version":"5.3.0-45-generic","mac":["00:15:5d:01:05:01"],"os":{"family":"debian","platform":"ubuntu","name":"Ubuntu","version":"18.04.4 LTS (Bionic Beaver)","major":18,"minor":4,"patch":4,"codename":"bionic"},"timezone":"CEST","timezone_offset_sec":7200,"id":"1655dd9026684064ac0fcd75d4ee6bb5"}}}
May 11 07:41:26 webserver filebeat[98144]: 2020-05-11T07:41:26.223+0200#011INFO#011[beat]#011instance/beat.go:1003#011Process info#011{"system_info": {"process": {"capabilities": {"inheritable":null,"permitted":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"effective":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"bounding":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"ambient":null}, "cwd": "/", "exe": "/usr/share/filebeat/bin/filebeat", "name": "filebeat", "pid": 98144, "ppid": 1, "seccomp": {"mode":"filter","no_new_privs":true}, "start_time": "2020-05-11T07:41:25.630+0200"}}}
May 11 07:41:26 webserver filebeat[98144]: 2020-05-11T07:41:26.223+0200#011INFO#011instance/beat.go:298#011Setup Beat: filebeat; Version: 7.6.2
May 11 07:41:26 webserver filebeat[98144]: 2020-05-11T07:41:26.224+0200#011DEBUG#011[beat]#011instance/beat.go:324#011Initializing output plugins
May 11 07:41:26 webserver filebeat[98144]: 2020-05-11T07:41:26.226+0200#011DEBUG#011[publisher]#011pipeline/consumer.go:137#011start pipeline event consumer
May 11 07:41:26 webserver filebeat[98144]: 2020-05-11T07:41:26.226+0200#011INFO#011[publisher]#011pipeline/module.go:110#011Beat name: webserver
May 11 07:41:26 webserver filebeat[98144]: 2020-05-11T07:41:26.226+0200#011INFO#011instance/beat.go:412#011filebeat stopped.
May 11 07:41:26 webserver filebeat[98144]: 2020-05-11T07:41:26.226+0200#011ERROR#011instance/beat.go:933#011Exiting: Fileset apache/var is configured but doesn't exist
May 11 07:41:26 webserver filebeat[98144]: Exiting: Fileset apache/var is configured but doesn't exist
May 11 07:41:26 webserver systemd[1]: filebeat.service: Main process exited, code=exited, status=1/FAILURE
May 11 07:41:26 webserver systemd[1]: filebeat.service: Failed with result 'exit-code'.
May 11 07:41:26 webserver systemd[1]: filebeat.service: Service hold-off time over, scheduling restart.
May 11 07:41:26 webserver systemd[1]: filebeat.service: Scheduled restart job, restart counter is at 5.
May 11 07:41:26 webserver systemd[1]: Stopped Filebeat sends log files to Logstash or directly to Elasticsearch..
May 11 07:41:26 webserver systemd[1]: filebeat.service: Start request repeated too quickly.
May 11 07:41:26 webserver systemd[1]: filebeat.service: Failed with result 'exit-code'.
May 11 07:41:26 webserver systemd[1]: Failed to start Filebeat sends log files to Logstash or directly to Elasticsearch..'

Thanks
René

4

All,

found the issue.
There were some tabs missing before "enabled" and "var.paths".
Now it's working.
Thanks @jsoriano

Cheers
René

1 Like
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.