Logstash cannot upload files, "failed to open Permission Denied"

I am facing an issue with uploading XML-files to an index in the cloud using Logstash. I have a directory with around 10.000 files and configured a Logstash in which all the contents of that directory are indicated. When executing the .conf - file in the Terminal, it uploads some documents (around 200) successfully to the index. However, after that I get a lot of permission denied messages and the process stops.

This is the .conf file:

input {
file {
path => "/Users/arietwigt/Documents/Archief_test/*"
start_position => "beginning"
sincedb_path => "/Users/arietwigt/Documents/Archief_test/sincedb"
codec => multiline {
pattern => ""
negate => true
what => "previous"
max_lines => 15000
max_open_files => 2500000

filter {
filter stuff

output {
elasticsearch {
hosts => "https://xxx.eu-west-1.aws.found.io:9243"
user => "elastic"
password => "xxxxx"
index => "my_index"
document_type => "my_type"
stdout {
codec => rubydebug


And this is the message I get in the Terminal:

[2017-02-27T08:20:22,235][WARN ][logstash.inputs.file ] failed to open /Users/arietwigt/Documents/Archief_test/ECLI_NL_RVS_2016_60.xml: Permission denied - /Users/arietwigt/Documents/Archief_test/ECLI_NL_RVS_2016_60.xml
[2017-02-27T08:20:22,238][WARN ][logstash.inputs.file ] failed to open /Users/arietwigt/Documents/Archief_test/ECLI_NL_RVS_2016_61.xml: Permission denied - /Users/arietwigt/Documents/Archief_test/ECLI_NL_RVS_2016_61.xml
[2017-02-27T08:20:22,239][WARN ][logstash.inputs.file ] failed to open /Users/arietwigt/Documents/Archief_test/ECLI_NL_RVS_2016_62.xml: Permission denied - /Users/arietwigt/Documents/Archief_test/ECLI_NL_RVS_2016_62.xml

... and so fort

Has anyone faced this problem before or can somebody notice if I am missing something here?

What does ls -l /Users/arietwigt/Documents/Archief_test/ECLI_NL_RVS_2016_62.xml return?

Hey Magnus, thanks for your quick response :slight_smile:

ls -l /Users/arietwigt/Documents/Archief_test/ECLI_NL_RVS_2016_62.xml


-rwxrwxrwx@ 1 logstash staff 15155 5 jan 14:42 /Users/arietwigt/Documents/Archief_test/ECLI_NL_RVS_2016_62.xml

That file has extended attributes (as evident by the @ sign). Perhaps something to look into? Otherwise I don't know what's up.

Okay, I will look into the attributes of the files. Thanks for your help, I will let you know.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.