Logstash Parsing File (Without Filebeat)

Elastic Stack Logstash
1

Trying to simply have logstash parse a file. I've checked that the file input path is correct, and when I tail the logtash log file, there is no update/activity. Is there anything else that I can check to ensure that it's parsing as it should be?

2

Could you write your input configuration?
It will make troubleshooting easier.

3 
file {
        path => ["<pathToFile>/*.log"]
        type => "example_type"
        start_position => beginning
        close_older => 180
        codec => multiline {
            auto_flush_interval => 1
            patterns_dir => [ "/etc/logstash/conf.d/patterns" ]
            negate => "true"
            pattern => "^ERROR:\s+"
            what => "previous"
            max_lines => 2
        }
    }

You mean this input configuration? It should be similar to the other configurations, which are working correctly. This one on the other hand is the only one that doesn't register for some reason. I've triple checked the path to ensure that it's the correct directory, too

4

Set the log.level to TRACE and see what filewatch has to say.

5

How do I access filewatch? Is that a separate tool within Elasticsearch?

I did get it to TRACE, now I see this line:

[2020-01-23T19:21:15,496][DEBUG][logstash.inputs.file] _globbed_files: <pathToFile>/*.log: glob is: ["<pathToFile>/log_name.log"]

So at least the glob is not empty, but it still does not parse it

6

You should be seeing lines like these. filewatch is the library that the file input uses to read files.

7

@Badger what should I do if I do not see any line with filewatch? Is it something that I need to set up then, or should it already be included?

8

If you have log.level set to TRACE (not DEBUG) then if the path option glob matches any files I cannot conceive of filewatch not logging anything.

9

I do have it on trace, though (not debug)

log.level: trace
path.logs: /var/log/logstash

Maybe I'm tailing the wrong file? I'm tailing /var/log/logstash/logstash-plain.log

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.