fileBeat isn't harvesting the logs from the last path's

Elastic Stack Beats
8

Update,
I managed to solve the problem by opennnig to more services of filebeat:
with the following configurations:

paths:
    - D:\logs\*\pid_*_rr_*
  input_type: log
  document_type: A 
  multiline.pattern: '<\?xml version="1\.0" encoding="UTF-8"\?>'
  multiline.negate: true
  multiline.match: after
  close_eof: true

And because of your warning i checked the difference in the harvesting amount, and it actually doubled
and I even took a day to check if now we are harvesting all the files -and it seems like yes (not 100% sure but i checked on all the last files on each path).

In conclusion-open an extra service to special cases like this - might be the solution (unless you change something on filebeat 6.0.0 - that we didn't upgrade to it yet)

P.S
sorry on the delay - it took me time to verify my solution.
Also I posted this questions band answer in stack overflow for the common good https://stackoverflow.com/q/47182771/8868108

1 Like