Not able to read large csv file in logstash

Elastic Stack Logstash
1

I am using below config to read large file of 1gb size having around 5m records, it is not giving any output or errors

input {

file {
path => "/filepath/filename.txt"
}
}
filter {
csv {
separator => "|"
}
}
output {
stdout {
}
}

I tried keeping only 10 records in file, it is working fine
I tried with --debug option, found below log with 10 records file
[2017-08-05T11:41:15,011][DEBUG][logstash.inputs.file ] _globbed_files:
[2017-08-05T11:41:15,018][DEBUG][logstash.inputs.file ] _discover_file:
[2017-08-05T11:41:15,023][DEBUG][logstash.inputs.file ] _open_file:

with 1gb file only below log was coming (discover_file and open_file is not coming up)
[2017-08-05T11:41:15,011][DEBUG][logstash.inputs.file ] _globbed_files:

I tried even deleting all sincedb files

2

Given it's 1GB it's probably reading the file, how long did you wait?

3

1 hour

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.