Logstash Input file is not reading file

Elastic Stack Logstash
1

Hello,
I want first to apologize for my english level. I'm using Logstash 5.5.0 and I'm facing an issue with my configuration. It works just fine with the stdin plugin but not with the file pugin.
I have the following configuration for the file plugin:
input {
file {
path => "/home/salam/ELK/test/load/igbayilola.csv"
start_position => "beginning"
sincedb_path => "/dev/null"
ignore_older => 0
}
}

It is supposed to create an index in elasticsearch at the output.

Thanks for your help,
Igbayilola.

2

Hello, if it could hepl, this is an extract of the debug info

12:36:03.864 [Api Webserver] DEBUG logstash.agent - Starting puma
12:36:03.864 [Api Webserver] DEBUG logstash.agent - Trying to start WebServer {:port=>9600}
12:36:03.867 [Api Webserver] DEBUG logstash.api.service - [api-service] start
12:36:03.867 [[main]<file] DEBUG logstash.inputs.file - _globbed_files: /etc/logstash/igbayilola.csv: glob is: ["/etc/logstash/igbayilola.csv"]
12:36:03.867 [[main]<file] DEBUG logstash.inputs.file - _discover_file: /etc/logstash/igbayilola.csv: new: /etc/logstash/igbayilola.csv (exclude is [])
12:36:03.868 [[main]<file] DEBUG logstash.inputs.file - _open_file: /etc/logstash/igbayilola.csv: opening
12:36:03.868 [[main]<file] DEBUG logstash.inputs.file - /etc/logstash/igbayilola.csv: initial create, no sincedb, seeking to beginning of file
12:36:03.869 [[main]<file] DEBUG logstash.inputs.file - writing sincedb (delta since last write = 1499686563)
12:36:03.871 [[main]<file] DEBUG logstash.inputs.file - each: file grew: /etc/logstash/igbayilola.csv: old size 0, new size 7294
12:36:03.894 [Api Webserver] DEBUG logstash.agent - Trying to start WebServer {:port=>9601}
12:36:03.894 [Api Webserver] DEBUG logstash.api.service - [api-service] start
12:36:03.898 [Api Webserver] INFO logstash.agent - Successfully started Logstash API endpoint {:port=>9601}
12:36:08.856 [Ruby-0-Thread-14: /usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:532] DEBUG logstash.pipeline - Pushing flush onto pipeline
12:36:13.856 [Ruby-0-Thread-14: /usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:532] DEBUG logstash.pipeline - Pushing flush onto pipeline
12:36:13.872 [[main]<file] DEBUG logstash.inputs.file - each: file grew: /etc/logstash/igbayilola.csv: old size 0, new size 7294
12:36:18.855 [Ruby-0-Thread-14: /usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:532] DEBUG logstash.pipeline - Pushing flush onto pipeline
12:36:23.855 [Ruby-0-Thread-14: /usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:532] DEBUG logstash.pipeline - Pushing flush onto pipeline
12:36:23.873 [[main]<file] DEBUG logstash.inputs.file - each: file grew: /etc/logstash/igbayilola.csv: old size 0, new size 7294
12:36:28.854 [Ruby-0-Thread-14: /usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:532] DEBUG logstash.pipeline - Pushing flush onto pipeline
12:36:33.854 [Ruby-0-Thread-14: /usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:532] DEBUG logstash.pipeline - Pushing flush onto pipeline
12:36:33.875 [[main]<file] DEBUG logstash.inputs.file - each: file grew: /etc/logstash/igbayilola.csv: old size 0, new size 7294
12:36:38.854 [Ruby-0-Thread-14: /usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:532] DEBUG logstash.pipeline - Pushing flush onto pipeline
12:36:43.853 [Ruby-0-Thread-14: /usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:532] DEBUG logstash.pipeline - Pushing flush onto pipeline
12:36:43.876 [[main]<file] DEBUG logstash.inputs.file - each: file grew: /etc/logstash/igbayilola.csv: old size 0, new size 7294
12:36:48.852 [Ruby-0-Thread-14: /usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:532] DEBUG logstash.pipeline - Pushing flush onto pipeline
12:36:53.851 [Ruby-0-Thread-14: /usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:532] DEBUG logstash.pipeline - Pushing flush onto pipeline
12:36:53.877 [[main]<file] DEBUG logstash.inputs.file - each: file grew: /etc/logstash/igbayilola.csv: old size 0, new size 7294
12:36:58.850 [Ruby-0-Thread-14: /usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:532] DEBUG logstash.pipeline - Pushing flush onto pipeline
12:37:03.849 [Ruby-0-Thread-14: /usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:532] DEBUG logstash.pipeline - Pushing flush onto pipeline
12:37:03.878 [[main]<file] DEBUG logstash.inputs.file - each: file grew: /etc/logstash/igbayilola.csv: old size 0, new size 7294
12:37:08.849 [Ruby-0-Thread-14: /usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:532] DEBUG logstash.pipeline - Pushing flush onto pipeline
12:37:13.849 [Ruby-0-Thread-14: /usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:532] DEBUG logstash.pipeline - Pushing flush onto pipeline
12:37:13.881 [[main]<file] DEBUG logstash.inputs.file - each: file grew: /etc/logstash/igbayilola.csv: old size 0, new size 7294

3

Please post your complete conf file. Also use the </> to format text so it'll be easier to read.

4

Thanks you for your answer. This is the complete conf file:

input {
    file {
        path => "/etc/logstash/igbayilola.csv"
        start_position => "beginning"
        sincedb_path => "/dev/null"
        ignore_older => 0
    }
}

filter {
    csv {
        columns => ["ID", "SWITCH", "TYPE", "NAME"]
        separator => ";"
        remove_field => ["message"]
    }
}

output {
    elasticsearch {
       hosts => ["localhost:9200"]
        index => "ressources"
    }
}
5

Hi,

I've the same output error:

[Ruby-0-Thread-27: /usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:532] DEBUG logstash.pipeline - Pushing flush onto pipeline

But I don't use ruby code in my configuration, like it seem the reason.

Please, could you help us?

Thanks

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.