When FileBeat collects a certain file, only a portion is collected, and the rest is not collected further. This problem only exists on a certain machine. Here is my FileBeat.yml, which writes a large amount of data at a certain point in time. Taking the following data as an example, only the last data of 9.40 will be collected, and the rest will not be collected. I checked log.json and found that the last offset also stays here, At the end of the file, I also checked if there is a problem with Windows. This platform is unable to upload the file, otherwise I can provide the original file
aaa1 9.40
aaa2 9.40
xxxx .......
bbb1 11.40
bbb2 11.40
xxxx .....
filebeat.inputs:
- type: filestream
enabled: true
paths:
- C:\Users\wh\Desktop\position-test\*.txt
fields:
code: haiheng # 新增默认参数
output.kafka:
hosts: ["172.19.0.139:9092"]
topic: "receive_item_position"
namespace: 'kafka_output'