Logstash Troubleshooting: ERROR - (EACCES) Permission denied to output-file-path

Elastic Stack Logstash
1

Hello

I've been using Logstash for integration with Wazuh.
Usually I hadn't have any issues by this since the pipeline it's pretty straight forward.
My usuall config is the next one:

input {
   syslog {
      port => 514
   }
}

output {
   file {
      path => "/home/user/logs-wazuh.json"
      codec => "line"
   }
}

But, when I run logstash with the same config on a new updated VM Ubuntu 24.04, I get the same 4 next error messages:

[ERROR][logstash.javapipeline    ][main] Pipeline worker error, the pipeline will be stopped {:pipeline_id=>"main", :error=>"(EACCES) Permission denied - /home/user/logs-wazuh.json" <more stuff...> sleep/run

There's a lot more data of the error that I coined as <more sutff...> in order to simplify my post. I can resume that info as the next key values:

  • Exception Type: Java::OrgJrubyExceptions::SystemCallError
  • Output Plugin (from backtrace): logstash-output-file (specifically version 4.3.0)
  • Relevant Ruby File (from backtrace): /usr/share/logstash/vendor/bundle/jruby/3.1.0/gems/logstash-output-file-4.3.0/lib/logstash/outputs/file.rb
  • Logstash Core File (from backtrace): /usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb
  • Thread: #<Thread:0x32746c51 /usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:138
  • Final param: sleep or run >

Since the error is cause permission denial, I explored the next configs in order to give it permission:
Note: systemd's logstash.service file has user:logstash and group:logstash. Keep that in mind.

  • Change output file permissions to 666 with user:user = Same errors
  • Change output file permissions to 666 with user:logstash = Same errors
  • Change output file permissions to 666 with logstash:logstash = Same errors
  • Change output file permissions to 666 with root:root = Same errors
  • Change logstash.service file's user/group values to user:user = Same errors
  • Change logstash.service file's user/group values to root:root = Can't run as root

I really don't get why I get this error since every time I had implemented this short pipeline It works.

Is there some similar documentation someone can help me retrive in order to troubleshoot this error?
Or is there a correct way to analyze the debug plain log in order to learn what's happening?

Thanks in advance,
Diego

2

Hi @MonkeyDono Welcome to the community.

The permission denied probably has to do with not being able to access write to the parent directories. Not specific file ...

Just because the file has the correct permissions, it does not mean the user can access it if the parent directories above it are not permitted.

1 Like
3

This is not exactly a Logstash error, it is a Linux permission issue, directories under /home have 750 (rwxr-x---) permissions, changing the permissions of the file has no impact because the logstash user has no permissions on the parent directory.

So you would need to change the user home folder to 770 and change the group ownership to the logstash group, I would not recommend that, it would be better to change the path to a directory under /opt or /var/opt.

Another issue that you may have is that your input would bind on port 514, which is a privileged port and the logstash user would not be able to bind to it per default.

You would need to add this capability to the java process running logstash using setcap, I'm not exactly sure of what is the command, but this is pretty easy to find.

I would use another port, like 1514 and leave 514 free to be used for rsyslog if required.

Also, you mentioned that you changed the logstash service user and group to root:root , this may have messed up with the permissions and give you errors when trying to run it again as a service with the logstash user, you may have to fix the permissions of the files inside /var/lib/logstash and /var/log/logstash.

1 Like
4

Hello @leandrojmp , absolutely!

You were rigth, thanks for the tip.
I was focusing only on the output file and logstash config that I didn't consider the parent directories.

I changed the path of my logs file to /opt, the permissions of the /opt directory to 777 and gave the new file path to my pipeline to the next one /opt/logs-wazuh.json.

Pasted image 20250423104731
Pasted image 20250423104731837×441 35 KB

With that, no more errors were triggered!

Fortunately, the changes I made on the service config file didn't break the logstash process. I checked the /var/lib&log/logstash and there weren't any error besides the next WARNING:
[2025-04-23T10:48:50,415][WARN ][logstash.filters.grok ][main] ECS v8 support is a preview of the unreleased ECS v8, and uses the v1 patterns.

I just stay with 1 question, that is I've never need to config the parent's directory or output file permissions on the same deployment. Why does this time I needed to do it?
I mean, of course is cause' the permission config. But since I deployed the same OS, with the same config and resources on the same paths... I just don't get what was the keystone.
I want to know in order to enhance my linux sysadmin skills, so I'd appreciate any feedback you can give me.

The problem was solved, thanks for the help. :grin:

5

Absolutely, that was the problem.

I took also the considerations of @leandrojmp and was able to solve the problem I had.

Thanks for the comment @stephenb ! :smiley:

1 Like
6

You need it everytime, a user can only access a file inside a directory if it also has access to the directory.

If you didn't change any permissions before is because the user/directory/file you used already had the necessary permissions in some way.

Do you have any evidence on other deployments where you didn't had to change permissions, to check what are the permissions?