I am facing an issue with uploading XML-files to an index in the cloud using Logstash. I have a directory with around 10.000 files and configured a Logstash in which all the contents of that directory are indicated. When executing the .conf - file in the Terminal, it uploads some documents (around 200) successfully to the index. However, after that I get a lot of permission denied messages and the process stops.
This is the .conf file:
input {
file {
path => "/Users/arietwigt/Documents/Archief_test/*"
start_position => "beginning"
sincedb_path => "/Users/arietwigt/Documents/Archief_test/sincedb"
codec => multiline {
pattern => ""
negate => true
what => "previous"
max_lines => 15000
}
max_open_files => 2500000
}
}
filter {
filter stuff
}
output {
elasticsearch {
hosts => "https://xxx.eu-west-1.aws.found.io:9243"
user => "elastic"
password => "xxxxx"
index => "my_index"
document_type => "my_type"
}
stdout {
codec => rubydebug
}
}
And this is the message I get in the Terminal:
[2017-02-27T08:20:22,235][WARN ][logstash.inputs.file ] failed to open /Users/arietwigt/Documents/Archief_test/ECLI_NL_RVS_2016_60.xml: Permission denied - /Users/arietwigt/Documents/Archief_test/ECLI_NL_RVS_2016_60.xml
[2017-02-27T08:20:22,238][WARN ][logstash.inputs.file ] failed to open /Users/arietwigt/Documents/Archief_test/ECLI_NL_RVS_2016_61.xml: Permission denied - /Users/arietwigt/Documents/Archief_test/ECLI_NL_RVS_2016_61.xml
[2017-02-27T08:20:22,239][WARN ][logstash.inputs.file ] failed to open /Users/arietwigt/Documents/Archief_test/ECLI_NL_RVS_2016_62.xml: Permission denied - /Users/arietwigt/Documents/Archief_test/ECLI_NL_RVS_2016_62.xml
... and so fort
Has anyone faced this problem before or can somebody notice if I am missing something here?
