Filebeat as a non-root user

Trying to get filebeat to run as a non-root user on a rpm distro.

So far this is what I have:

#created a user
useradd filebeat -u 5044 -c "Filebeat Service Account" -d /dev/null -s /sbin/nologin

#edited the filebeat.yml file by adding the runoptions parameter

https://www.elastic.co/guide/en/beats/filebeat/current/configuration-run-options.html

#edited the /lib/systemd/system/filebeat.target file

added the User=filebeat under the [Service] stanza

#Added some facl's for access for the filebeat user:
###read access for the log files specified in the prospector
setfacl -m u:filebeat:r /var/log/mylogfile
###read access for the filebeat.yml
setfacl -m u:filebeat:r /etc/filebeat/filebeat.yml
###execute access for filebeat executable
setfacl -m u:filebeat:x /usr/bin/filebeat
###write access for the filebeat log file
setfacl -m u:filebeat:w /var/log/filebeat/filebeat.log
###read/write access for the registry file
setfacl -m u:filebeat:rw /var/lib/filebeat/registry

However when I do a systemctl daemon-reload and restart the filebeat service it is stuck and not actively prospecting the log file. Is there a permission that I missed that is needed for proper functioning of filebeat?

To reach the right people, edit your post and change the category from Elasticsearch to Filebeat.

Thanks, didn't realize!

Start Filebeat with -d '*' to get full debug logs. That should give clues about what's going on.

I already have filebeat set to log in debug mode in my filebeat.yml file. This made no difference as the output is the same:

[root@myvm ~]# service filebeat start -d '*'
Starting filebeat (via systemctl):                         [  OK  ]

The last log entry I have is:

2016-08-25T10:15:44-04:00 INFO Cleaning up filebeat before shutting down.

Nothing further, no acknowledgement of the process starting even though it has:

[root@myvm ~]# systemctl status filebeat.service
● filebeat.service - filebeat
Loaded: loaded (/usr/lib/systemd/system/filebeat.service; disabled; vendor preset: disabled)
Active: active (running) since Thu 2016-08-25 10:24:17 EDT; 2h 27min ago
Docs: Filebeat Reference [8.11] | Elastic
Main PID: 5627 (filebeat)
CGroup: /system.slice/filebeat.service
└─5627 /usr/bin/filebeat -c /etc/filebeat/filebeat.yml

Aug 25 12:51:35 myvm /usr/bin/filebeat[5627]: prospector.go:185: Start next scan
Aug 25 12:51:35 myvm /usr/bin/filebeat[5627]: prospector.go:261: scan path /var/log/mylogfile
Aug 25 12:51:40 myvm /usr/bin/filebeat[5627]: spooler.go:97: Flushing spooler because of tim...: 0
Aug 25 12:51:45 myvm /usr/bin/filebeat[5627]: prospector.go:185: Start next scan
Aug 25 12:51:45 myvm /usr/bin/filebeat[5627]: prospector.go:261: scan path /var/log/mylogfile
Aug 25 12:51:47 myvm /usr/bin/filebeat[5627]: spooler.go:97: Flushing spooler because of tim...: 0
Aug 25 12:51:55 myvm /usr/bin/filebeat[5627]: spooler.go:97: Flushing spooler because of tim...: 0
Aug 25 12:51:55 myvm /usr/bin/filebeat[5627]: prospector.go:185: Start next scan
Aug 25 12:51:55 myvm /usr/bin/filebeat[5627]: prospector.go:261: scan path /var/log/mylogfile
Aug 25 12:52:02 myvm /usr/bin/filebeat[5627]: spooler.go:97: Flushing spooler because of tim...: 0

Is it working and just not writing output to the log file? I would suspect it still isn't working because prompting changes to the file does not prompt changes in my output plugin.

No, don't give -d '*' to the service command. It's Filebeat that should get the option. I can't give specific advice in the systemd case.

Did you mean the ExecStart in the /lib/systemd/system/filebeat.service perhaps? I changed this:

ExecStart=/usr/bin/filebeat -c /etc/filebeat/filebeat.yml

to this:

ExecStart=/usr/bin/filebeat -c /etc/filebeat/filebeat.yml -d '*'

Same result

If you mean in the filebeat configuration, I already have that set by having the logging level set to debug (excuse the indenting; it isn't copying over well but I assure you it is correct in my configuration):

logging:
to_files: true
files:
path: /var/log/filebeat
name: filebeat.log
rotateeverybytes: 10485760 # = 10MB
keepfiles: 3
level: debug

Getting slightly closer to an answer here. Instead of facl's I changed ownership of the /var/lib/filebeat and /etc/filebeat and /var/log/filebeat/ directories to my filebeat user. Oh and I took out the runoptions, it got a little confusing when it kept causing some sort of log file rotation. Now I am getting output but nothing is presumably being scanned? Just a continuous loop of the following three lines, even when I change the contents of the log file:

2016-08-25T15:18:19-04:00 DBG  Start next scan
2016-08-25T15:18:19-04:00 DBG  scan path /var/log/mylogfile
2016-08-25T15:18:24-04:00 DBG  Flushing spooler because of timeout. Events flushed: 0

When I should actually be seeing something like this:

> 2016-08-25T15:51:56-04:00 DBG  Start next scan
> 2016-08-25T15:51:56-04:00 DBG  scan path /var/log/mylogfile
> 2016-08-25T15:51:56-04:00 DBG  Check file for harvesting: /var/log/mylogfile
> 2016-08-25T15:51:56-04:00 DBG  Same file as before found. Fetch the state.
> 2016-08-25T15:51:56-04:00 DBG  Update existing file for harvesting: /var/log/mylogfile
> 2016-08-25T15:51:56-04:00 DBG  Not harvesting, file didn't change: /var/log/mylogfile
> 2016-08-25T15:51:59-04:00 DBG  Flushing spooler because of timeout. Events flushed: 0
> 2016-08-25T15:52:01-04:00 DBG  End of file reached: /var/log/mylogfile; Backoff now.
> 2016-08-25T15:52:06-04:00 DBG  Flushing spooler because of timeout. Events flushed: 0

And I am getting some filebeat messages in the messages log file:

Aug 25 15:25:04 myvm /usr/bin/filebeat[7685]: prospector.go:133: Set backoff duration to 1s
Aug 25 15:25:04 myvm /usr/bin/filebeat[7685]: prospector.go:133: Set max_backoff duration to 10s
Aug 25 15:25:04 myvm /usr/bin/filebeat[7685]: prospector.go:113: force_close_file is disabled
Aug 25 15:25:04 myvm /usr/bin/filebeat[7685]: prospector.go:143: Starting prospector of type: log
Aug 25 15:25:04 myvm /usr/bin/filebeat[7685]: crawler.go:78: All prospectors initialised with 3 states to persist
Aug 25 15:25:04 myvm /usr/bin/filebeat[7685]: registrar.go:87: Starting Registrar
Aug 25 15:25:04 myvm /usr/bin/filebeat[7685]: publish.go:88: Start sending events to output
Aug 25 15:25:04 myvm /usr/bin/filebeat[7685]: spooler.go:77: Starting spooler: spool_size: 2048; idle_timeout: 5s

Any further suggestions?

The log messages you posted above are only from the startup of filebeat. There should be mor log messages following, especially if you enabled debug level. Can you post these?

ruflin,

Thank you for your response and yes there are more messages but didn't feel they were relevant to the question I am asking. Ultimately, I think I still have a permissions issue with filebeat being run as a non-root user; I simply took a repeating loop from the log and only included one iteration of the loop. When trying to get my non-root user to function, the filebeat user fails to check the file for harvesting among other things. This is curious because I set a facl for read access to the log file. Does it need more than read access? Do you have guidance on what additional files/directories a non-root user would need to function correctly?

What happens if you become the filebeat user that you added (sudo su filebeat)? Can you read the log files? Can you read the directories containing the log files?

Filebeat needs to be able to stat the log file. According to the docs for stat, execute (search) permissions are required on any directories that lead to the file.

If you are familiar with strace, you could use it to look for permission errors in Filebeat.

Not sure if you have altered your logs here for security but it seems it is checking for /var/log/mylogfile.

Typically /var/log is owned by root and a non-root user won't be able to create files under /var/log. In my experience filebeat will rotate the logs and create a new filebeat log file with every restart. So you need to make sure that the filebeat user has write permissions to the log directory. Creating a directory /var/log/filebeat and having that directory be owned by filebeat should do the trick if this is the issue.

I run my filebeat with non-root user under RHEL with no issues. I am not using RPM deployment though. The way I accomplish it is by making sure the service start script runs as the non-root user so the start command

ExecStart=/usr/bin/filebeat -c /etc/filebeat/filebeat.yml -d '*'

will get triggered by the non-root user and it all works fine.

Here is the checkuser function of my service start shell script:

RUN_AS="filebeat"
# if this script is not yet running as the right user use su to run it again as the correct user
# This will prevent output files from being created with wrong user ownership
checkuser() {
  # Check to see if we are running as the right user. Else switch, and re-exec
  if test `/usr/bin/id -un` = $RUN_AS; then
    #echo $0 --- Already running as correct user $RUN_AS
    :
  elif test `/usr/bin/id -un` = "root"; then
    echo "$0 --- Running as `/usr/bin/id -un`. Re-exec to run as $RUN_AS"
    exec /bin/su - $RUN_AS -c "$0 $ARGS"
    exit 0
  else
    echo "Only root or $RUN_AS can start $EXECUTABLE"
    exit 1
  fi

}

...

ARGS=$*;
case $1 in 
'start')
    checkuser
    for CONFIG in "${config_list[@]}"; do
        startproc
    done
    ;;

...

@andrewkroh; yes I do have read permissions on the file and I did test that :slight_smile: I was not aware of execute permissions being needed for collecting log files but adding that to the facl seemed to fix this for me. The only problem I seem to have now is filebeat is not giving verbose logging (see above) when set to debug. Any idea on what the root cause of this might be?

@anefassa Thank you for your response, I took a slightly different approach but I think I have it mostly figured out :wink: