Logstash not reading remote log files

Elastic Stack Logstash
1

Hi,

I have some log files in a remote system, I need to send those to elasticsearch. But logstash is not reading the remote log files. I have specified the following configuration for logstash.

input {
file{
path => ["\\web1\C$\msu\logs\prod-hickory\prod-admin-debug.log","\\web1\C$\msu\logs\prod-hickory\prod-admin-error.log"]
start_position => "beginning"
sincedb_path => "demo3_since.db"
codec => multiline {
pattern => "^%{TIMESTAMP_ISO8601} "
negate => true
what => previous
}
}
}
output{
elasticsearch{
hosts => ["xxx.xx.xx.xx:9200"]
}
stdout{
codec => rubydebug
}
}

Can you please suggest if anything is wrong in this config file?

2

You might want to try forward slashes instead of backslashes in the path. If that doesn't help, try increasing the logging verbosity by starting Logstash with --verbose or even --debug and look for clues.

3

Those look like Windows remote share paths, so \ is probably valid.

4

Windows is fine with backslashes (and slashes), but I'm not sure Logstash is. I wouldn't trust its configuration lexer to not mangle them.

5

Are you sure there should be a \ between C and $? C$ seems to be more likely as it is the hidden share for the C-drive.

6

Nothing is working, tried everything, do we have any other possible solution?

7

Are the input files older than 24 hours? Then you need to adjust the file input's ignore_older option.

Also, did you try the suggestion I gave about starting Logstash with --verbose or --debug to get additional clues?

8

Does it have anything to do with the authentication of the remote system?

9

Possibly. If you start Logstash with --verbose it'll tell you if it doesn't have permissions to open a file. It'll also tell you if it's not able to find any files (which could also be caused by a lack of permissions) but that may require debug-level logging (i.e. starting with --debug), I don't remember OTOH.

10

No such message was seen in the debug mode or verbose mode.

11

So it does discover the files? There should be a log entry containing "discover" and the paths to the files you won't to monitor.

Also, what's in your sincedb file? If Logstash already think its current position is at the end of the file it obviously won't do anything.

13

I would like to know, how can I set the same path for linux system.

14

@netaibiswas, please start a new thread for your unrelated question.