Inputting XML from File (Windows)

[Tim_Mobley]

I am attempting to use the File input plugin to ingest XML into Logstash running on a Windows host. At this point, I just want to verify that the input section of the pipeline is working. Here is the input section of my pipeline config:

input {
  file {
    path => [ "C:\temp\SCAP\*.xml" ]
  }
}

And here is the what I see in the logstash-plain.log:

[2021-10-24T11:23:49,953][INFO ][logstash.inputs.file     ][scap-results] No sincedb_path set, generating one based on the "path" setting {:sincedb_path=>"P:/Elastic/Logstash/logstash-7.14.1/data/plugins/inputs/file/.sincedb_24f630408b5fe29cf9ccb9e55db97036", :path=>["C:\\temp\\SCAP\\*.xml"]}

Other than this, I don't see any ERROR or WARN logs. The part I'm wanting to confirm is that it is reading the path correctly. Examples of the path array are for Linux, so I am not sure if I should make the slashes '/' or '' in Windows or escape them. The same documentation shows '*' for globbing, so I assume this would work for Windows as well, but if I could get a second opinion, that'd be great.

[leandrojmp (Leandro Pereira)]

You are missing the start_position => "beginning" in your file input, without it logstash will start reading your file from the end, which means only when new events are written to that file.

Not sure if you need to change the slashes to forward slashes as I do not use windows, but I know that filebeat has some issues if you are not using forward slashes.

Try this:

input {
  file {
    path => [ "C:\temp\SCAP\*.xml" ]
    start_position => "beginning"
  }
}

Also, you will need to delete this file to make logstash read your source file again:

P:/Elastic/Logstash/logstash-7.14.1/data/plugins/inputs/file/.sincedb_24f630408b5fe29cf9ccb9e55db97036

If you want, you can use sincedb_path => "NUL" in the file input to force logstash to always reread files when started.

[Badger]

Do not use backslash in the path option of a file input, they are treated as escapes. Use / or \\.

[Tim_Mobley]

Thank you both for your replies. @leandrojmp your suggestion of adding the start_position setting did the trick. I also changed the slash direction as @Badger suggested.