Error=>logstash Pipeline worker error :"(EACCES) Permission denied

[vaibhav.ubale]

Hi Team/Everyone,

I am facing a pipeline error and my pipeline is terminating randomly with error=>"(EACCES) Permission denied - /var/myrepo/devops/mytask_watcher.csv"

My logstash output conf is as below

output {

csv {
      fields => [ "message"]
      path => "/var/repository/devops/mytask_watcher.csv"
   }
}

and Operating System:
NAME="CentOS Linux"
VERSION="7 (Core)"

Folder Permission
10:51:45 Mon Feb 06 ->: $ ls -ltrh /var/myrepo/ |grep devops
drwxrwxr-x 2 logstash logstash 6 Feb 6 10:50 devops

Logstash Version : logstash 7.16.2

I have added logstash to sudoers group as well but still unable to resolve this error.

Detailed error is as follows:

[2023-02-02T12:55:02,625][ERROR][logstash.javapipeline    ][watcher-mytask] Pipeline worker error, the pipeline will be stopped {:pipeline_id=>"watcher-mytask", :error=>"(EACCES) Permission denied - /var/myrepo/devops/mytask_watcher.csv", :exception=>Java::OrgJrubyExceptions::SystemCallError, :backtrace=>["org.jruby.RubyIO.sysopen(org/jruby/RubyIO.java:1237)", "org.jruby.RubyFile.initialize(org/jruby/RubyFile.java:365)", "org.jruby.RubyIO.new(org/jruby/RubyIO.java:876)", "usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_output_minus_file_minus_4_dot_3_dot_0.lib.logstash.outputs.file.open(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-file-4.3.0/lib/logstash/outputs/file.rb:276)", "usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_output_minus_csv_minus_3_dot_0_dot_8.lib.logstash.outputs.csv.multi_receive_encoded(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-csv-3.0.8/lib/logstash/outputs/csv.rb:46)", "org.jruby.RubyHash.each(org/jruby/RubyHash.java:1415)", "usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_output_minus_csv_minus_3_dot_0_dot_8.lib.logstash.outputs.csv.multi_receive_encoded(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-csv-3.0.8/lib/logstash/outputs/csv.rb:45)", "org.jruby.ext.thread.Mutex.synchronize(org/jruby/ext/thread/Mutex.java:164)", "usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_output_minus_csv_minus_3_dot_0_dot_8.lib.logstash.outputs.csv.multi_receive_encoded(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-csv-3.0.8/lib/logstash/outputs/csv.rb:44)", "usr.share.logstash.logstash_minus_core.lib.logstash.outputs.base.multi_receive(/usr/share/logstash/logstash-core/lib/logstash/outputs/base.rb:103)", "org.logstash.config.ir.compiler.OutputStrategyExt$AbstractOutputStrategyExt.multi_receive(org/logstash/config/ir/compiler/OutputStrategyExt.java:143)", "org.logstash.config.ir.compiler.AbstractOutputDelegatorExt.multi_receive(org/logstash/config/ir/compiler/AbstractOutputDelegatorExt.java:121)", "usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.start_workers(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:299)"], :thread=>"#<Thread:0x7b3fd5df@/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:54 sleep>"}
[2023-02-02T12:55:03,535][INFO ][logstash.javapipeline    ][watcher-mytask] Pipeline terminated {"pipeline.id"=>"watcher-mytask"}

Can some one help me over here or atleast point in right direction.
Thanks

PS: I have other two pipelines with same output path for csv in same folder which are not having any such problem.

[Rios]

Have you checked permissions for /var/myrepo/devops/mytask_watcher.csv?
Should be set for logstash user, maybe you should change ownership for mytask_watcher.csv.

[vaibhav.ubale]

Hi Rios,

Thanks for your reply

The file is having the permission as below when ever it is created by logstash.

 06:17:15 Tue Feb 07 ->: $ ls -ltr /var/myrepo/devops/
total 4
-rw-r--r-- 1 logstash logstash 273 Feb  7 06:18 mytask_watcher.csv

but I can see the following logs in logstash-plain.log

[2023-02-07T06:15:01,090][INFO ][logstash.outputs.csv     ][watcher-mytask][516cd5c0ac218c746620a3a46c4b8c92b6fd85856b5fa2445be8b5a461441419] Opening file {:path=>"/var/myrepo/devops/mytask_watcher.csv"}
[2023-02-07T06:15:28,563][INFO ][logstash.outputs.csv     ][watcher-mytask][516cd5c0ac218c746620a3a46c4b8c92b6fd85856b5fa2445be8b5a461441419] Closing file /var/myrepo/devops/mytask_watcher.csv