I'm trying to get a near-realtime logstash output from filebeat on windows. The logs are updated frequently, but with periods of no updates for 1-120 seconds at a time. The logging application never produces more than 20 lines/sec.
With my current setup, I am seeing delays of up to 5 seconds between the time the log line is written to when it is shipped to logstash. My project requires this delay to be under 1 second, is this possible?
Here is my config.
filebeat.prospectors: - input_type: log scan_frequency: 1s close_inactive: 10m ignore_older: 30m tail_files: true backoff: 0.5s max_backoff: 1s document_type: hearthstone paths: - C:/Program Files (x86)/Hearthstone/Logs/Power.log - C:/Program Files (x86)/Hearthstone/Logs/Zone.log output.logstash: hosts: ["localhost:5044"] workers: 4