Logstash failed to load files with permission denied

Elastic Stack Logstash
1

Logstash failed to load files with permission denied.
In my setup Logstash running as service with command of "service logstash start"
folder also having full permissions
-rwxrwxrwx..
Please suggest what can be the cause.

Thanks in advance.

2

Please provide the full error, the version you are running and your config.

3

Running logstash as a service with these system properties:
-b3500 -w 20
Logstash version:2.3.2
ERROR:
{:timestamp=>"2016-05-30T05:37:50.006000+0100", :message=>"failed to open /log/__302042___15482766983234091.log_10.done.completed: Permission denied - /log/__302042___15482766983234091.log_10.done.completed", :level=>:warn}
{:timestamp=>"2016-05-30T05:37:50.007000+0100", :message=>"failed to open /log/__302042___15850459744045356.log_1.done.completed: Permission denied - /log/__302042___15850459744045356.log_1.done.completed", :level=>:warn}
{:timestamp=>"2016-05-30T05:37:50.008000+0100", :message=>"failed to open /log/__302042___14980760853726598.log_5.done.completed: Permission denied - /log/__302042___14980760853726598.log_5.done.completed", :level=>:warn}leted", :level=>:warn}

CONFIG FILE:

input {
file {
path => ["/data/302001_"]
start_position => "beginning"
type=>"API"
delimiter=>"^"
close_older=>"600"
max_open_files=>"25000"
discover_interval=>5
ignore_older=>2592000
sincedb_path=>"/apps/sincedb/API/api.log"
}
}
filter{
if [message] =~ /^\s$/ {
drop { }
}
if[type]=="API"{
csv {
quote_char => "`"
separator=>""
autogenerate_column_names=>"false"
columns=>["id","cor_id","key","name","ip","data","state","datetime"]
}
if[datetime] and [datetime]!='null' and [datetime]!='NULL'{
#ruby {
# code => "event['indexdate'] = event['datetime'][0..6]"
#}
date {
match => [ "datetime", "YYYY-MM-dd HH:mm:ss.SSS","YYYY-MM-dd HH:mm:ss" ]
target =>"datetime"
timezone=>"Europe/London"
}
grok {
match => { "datetime" => "(?[0-9]{4}-[0-9]{2})"}
}
}
if![datetime] or [datetime]=='null' or [datetime]=='NULL'{
grok {
match => { "@timestamp" => "(?[0-9]{4}-[0-9]{2})"}
}
ruby {
code => "event['datetime'] = nil"
}
}
mutate {
convert => { "state" => "integer" }
}
}
}
output {
if[type]=="API"{
elasticsearch {
workers =>15
hosts => ["localhost:9200","localhost:9201"]
manage_template => true
template_name => "template"
template => "/apps/template.json"
template_overwrite=>true
index=>"api-%{indexdate}"
#index=>"api-%{+YYYY.MM}"
}
}
}

4

What are the permissions on the files you are trying to process?

5

these are my file permissions

sh-4.1$ ls -ltr /data/__302001___15763039509869681.log_1.done.completed
-rwxrwxrwx. 1 root root 256 May 26 16:18 /log/__302001___15763039509869681.log_1.done.completed

Manually i given full permissions with below command chmod -R 777 folder name
Then also same error came

6

Can you please suggest why logstash behaving like this

7

Logstash seems to complain about files in the /log directory rather than the /data directory that you have configured. Do you have any other configuration files being picked up?