Filebeat setup command can not find environment variables

Elastic Stack Beats
1

Hi there,

I've read up a lot of the various posts on this topic but cannot seem to find something specific to my particular issue.

I've got filebeat running on centos as a service with systemd as per the docs. I'm using a dropin file to add some environment variables in filebeat's config (filebeat.yml). Filebeat starts no problem. However, when I try to run the filebeat setup -e command I get the dreaded

Exiting: error unpacking config data: Can not find FOO environment variable accessing 'fields.env' (source:'/etc/filebeat/filebeat.yml')

I've found that I can run setup with an environment variable if I set it by adding it to somewhere like /etc/profile.d/var-script.sh. Then restart filebeat (sudo systemctl restart filebeat) and then run the setup with sudo -E filebeat setup -e but this feels odd to have to add it in the system dropin file for running filebeat and a second location for the setup command.

To recap:

  • I can add systemd environment variables and filebeat will start properly
  • The setup command does not seem to be able to find these variables

Any ideas what i'm doing wrong?

2

Hello @user8753123 , the reason is more around how environment variables work, and in which scope they are available.

When running commands as a user, or sudo, you do not get access to environment variables meant for the service (when it starts from systemd).

When using drop-in files or defining environment variables directly in the systemd service files, the environment variables are only available to that application for security purposes.

When you run the setup command as a user, or through sudo, it will only look for environment variables that is accessible by your user, as you are not using systemd when running setup. Which is why it only works if you add your environment variables either to /etc/profile.d or your own bash_profile file for your user.

The filebeat setup command is something you only need to run once, and should be run before the service starts up for the first time, after that you only need to run setup again on a single filebeat instance if you upgrade to a newer version and want to apply some newer changes.

3

Hi @Marius_Iversen , thanks for your response, I really appreciate it. What you're saying makes sense and to be honest, it's also really nice to validate what I was finding with someone and stop me from proceeding down that rabbit hole.

My current implementation based on this is to add the necessary environment variables in 2 spots.

  1. in a systemd override.conf for the service to use when it starts up and is running
  2. in a spot a user can find them when running the initial (or other) filebeat commands

As you clarified, the duplication makes a bit more sense when thinking about who is running a given command (or the context in which they are being run).

For others possibly reading this, there's other ways to set variables when running filebeat commands as a user (e.g. you can pass them in with -E as in --E "SETTING_NAME=VALUE", you can set them elsewhere as long as they are accessible for the user running the command). If you're then running filebeat as a service, the service also needs access to these variables.

1 Like
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.