Logstash cannot read /var/log/maillog

[robs]

Hi all,

I can't find this problem or a related answer in recent postings (after 2014) so I drop this here.
I'm running this:

• CentOS 6.8 (final)
• Logstash 6.2.3
• Elasticsearch 6.2.3
  To allow logstash to read log files I added the user logstash to all groups of (service-)users that create log files I want to monitor and added, where needed, group read rights to log files.
  This worked until I tried to monitor /var/log/maillog.
  User logstash is a member of the root group and /var/log/maillog (and the intermediate directories) are accessible to logstash for reading.
  Proof: When I give user logstash a login shell, after logging in 'cat /var/log/maillog' works perfectly.
  But: After starting the logstash service I get (in logstash-plain.log):

[WARN ][logstash.inputs.file ] failed to open /var/log/maillog: Permission denied - /var/log/maillog

This has me baffled. Tips anyone? Your help will be appreciated.

[magnusbaeck (Magnus Bäck)]

Perhaps only the user's primary group is kept? I know that was an old bug when using Logstash with an Upstart-based init.

[robs]

Hi Magnus,

Thanks for the suggestion. Wouldn't have thought of it. The system is indeed using Upstart and I'll look into this, although I'm not sure at this moment how to correct this if indeed it is the cause. But It would be nice to at least understand what goes wrong.

[robs]

Hi Magnus,

You put me on the right track. There was something fishy with group-IDs.
First, the installation left me with logstash group-ID in /etc/password different from logstash group-ID in /etc/group. Fixed that (took the correct group-ID from the installed files)
Second, your suggestion, not a bug but apparently a feature: in /etc/init/logstash.conf I found:

exec chroot --userspec logstash:logstash / {rest of the startup line}

I changed this to:

exec chroot --userspec=logstash:logstash --groups=all,the,additional,group,ids / {rest of the startup line}

That did the trick.
So thank you for saving me lots of time & frustrations.

Regards, Rob