I am attempting to use the File input plugin to ingest XML into Logstash running on a Windows host. At this point, I just want to verify that the input section of the pipeline is working. Here is the input section of my pipeline config:
And here is the what I see in the logstash-plain.log:
[2021-10-24T11:23:49,953][INFO ][logstash.inputs.file ][scap-results] No sincedb_path set, generating one based on the "path" setting {:sincedb_path=>"P:/Elastic/Logstash/logstash-7.14.1/data/plugins/inputs/file/.sincedb_24f630408b5fe29cf9ccb9e55db97036", :path=>["C:\\temp\\SCAP\\*.xml"]}
Other than this, I don't see any ERROR or WARN logs. The part I'm wanting to confirm is that it is reading the path correctly. Examples of the path array are for Linux, so I am not sure if I should make the slashes '/' or '' in Windows or escape them. The same documentation shows '*' for globbing, so I assume this would work for Windows as well, but if I could get a second opinion, that'd be great.
You are missing the start_position => "beginning" in your file input, without it logstash will start reading your file from the end, which means only when new events are written to that file.
Not sure if you need to change the slashes to forward slashes as I do not use windows, but I know that filebeat has some issues if you are not using forward slashes.
Thank you both for your replies. @leandrojmp your suggestion of adding the start_position setting did the trick. I also changed the slash direction as @Badger suggested.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.