Logstash input-> file-> path…wildcard * not working

Elastic Stack Logstash
1

Working on a Windows 10 system and for the life of me I cannot get this system to recognize a wildcard path statement. ELK stack is 6.4.0

I have had this work in the past on different versions and/or systems and I cannot seem to determine what is going wrong.

To start, logstash will process a single file, but will not recognize a wildcard at the end of the file name. I have tried both forward slash / backward slash, different file names (p8_server*, p8_server_error.*, etc.) and it will only pick up the first file.

I have also tried the 'debug' and/or 'verbose' options and it doesn't show me (or I can't see it from the messaging) why it might be picking up only the first file.

Can someone help me better troubleshoot this and/or determine why wildcards might not be working properly?

Thanks much.

2

You're going to need to include errors and config in this post.

3

The input string looks as follows:

input {
file {
path => "C:/Temp/p8_server_error.log*"

AGain, there are no errors as it does successfully process the 'p8_server_error.log' file. But there is a log.1, log.2, etc that are not processed yet resident in that filesystem.

What way can I determine what input-->file-->path is interpreting and/or why it wouldn't be recognizing everything in that path?

Thanks.

4

I have also confirmed that this does work properly on Linux and other Windows system but not the one I'm currently working on.

What (environmentally) could be hanging Logstash up from recognizing asterisks?

5

I assume these files are no longer written to, so have you specified start_position as beginning?

6

I have, thanks for the reply.

We found out an interesting discovery today as we were troubleshooting.

We had modified our file path to SystemOut*.*log and it only captured a single file out of the 20. As a quick test I took a file that had underscores and periods in it and was going to copy it and remove those characters from the file name. As the copy completed logstash actually picked up the file with all of the characters in it and processed the file correctly ?!

So as another test I took all of the hundreds of files and then copied them all to another (new) folder and logstash processed them all. We did a file comparison and the security looks identical, etc.

Thus, my initial 'move' of the files into a new folder seemed to cause or perpetuate the issue. As soon as I performed a 'copy' to another folder all of the files now process successfully.

It has been an interesting experiment and I appreciate those who commented.

1 Like
7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.