Environment variable error

[YvorL]

Hi!

There isn't too many documents or threads on this issue and I reckon it isn't used too much.

I've several external configuration files to make updating more dynamic. I'd like to add environment variables to these which are being reloaded every X minutes.
Unfortunately, when I tried to add a custom variable, I get this in the log:

ERROR [reload] cfgfile/list.go:104 Error creating runner from config: missing field accessing ...

External configuration file:

- type: log
  paths:
    - /var/log/*.log
  scan_frequency: 1m
  close_renamed: true
  fields_under_root: true
  fields:
    custom_var: "${FOO}"

_.profile

...
export FOO="BAR"

Filebeat version: 6.4.3
OS: Ubuntu 18.04
Filebeat is running as a service by root.
The variable is defined in root's '.profile', bash is returning the value.
The variable is intentionally available for only root.

Thank you

[steffens (Steffen Siering)]

The '.profile' file is read by bash only. I think with ubuntu 18.04 the systemd service file is used to start filebeat. In order to add environment variables to a process managed by systemd use the Environment=FOO=BAR directive or use the EnvironmentFile directive, so you can manage environment variables in a different file.

[YvorL]

I'm sorry, but could you please be a bit more specific? I'm not well versed in this topic and not sure what you are referring to by those two directives. For Ubuntu I found an "/etc/environment" file, but that's system wide which isn't implementable for us since non-root users mustn't get those information.
As I wrote earlier, want to spare the time to overwrite changes in all existing files.

Thank you!

[YvorL]

If anyone wonders how to achieve what steffens wrote, here's one way. Please note, that

• You need root privilege
• In case, you don't know what implications does this flow have, please consult with a system administrator (as I have)

1. Create a file where you'll store your environment variables (e.g., env_vars.conf or environmentVars).
2. Populate the file with variables (as many as you need):

MyVar1="This is one"
myvar2="polar bear v1.2"

_Note_:
Environment variables are case sensitive!
There are predefined variables which you shouldn't overwrite (consult with a sysadmin/read manual).
It's a good practice to have all variables encapsulated by quotation mark (**"**).

3. Execute systemctl edit filebeat.service in the terminal and add these lines:

[Service]
EnvironmentFile=/etc/somefolder/env_vars.conf

4. Add the environment variables to your Filebeat configuration (using ${MyVar1} format).
5. Restart Filebeat service: systemctl restart filebeat.service

And you're good to go. I'm not sure that this is the best way to achieve dynamic variables, but it works for my case
Important note: Every time you change these variables (update the file), you need to restart the Filebeat service since those are read once at startup.

[steffens (Steffen Siering)]

Sorry, I totally missed you asking how to do.

This is a very good solutions for users running systemd. Thank you for sharing.

Also check the ownership and access rights to the file with environment variables. Filebeat normally checks it is the only owner of it's config file (belongs to root and no other can write to it). As the file with environment variables is read by systemd, we can't do these checks.