Filebeat is not auto refreshing the log files

Elastic Stack Beats
1

My file beat is running on my app server where my application code is generating , the following is the YML file for beat.

---
filebeat.inputs:
  - type: log
    id: node-api
    enabled: true
    paths:
      - C:\\node-api-14-10-2024-logs.log
    multiline.pattern: ^[0-9]{2}-[0-9]{2}-[0-9]{4}
    multiline.negate: true
    multiline.match: after
    scan_frequency: 5s
filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false
setup.template.settings:
  index.number_of_shards: 1
setup.kibana:
  host: <Kibana_Server>:5601
output.elasticsearch:
  hosts:
    - <Elastic_server>:9200
  username: <User_Name>
  password: <Password>
  preset: balanced
processors:
  - add_host_metadata:
      when.not.contains.tags: forwarded
  - add_cloud_metadata: null
  - add_docker_metadata: null
  - add_kubernetes_metadata: null

And this is working fine when there is log generated and I am restarting the filebeat server, then I see the logs in the Kiabana dashboard. Though I have configured the scan_frequency: 5s it is not working only after restarting the logs are populating.

How to solve it ?

2

Hi @JyotiPrakash, Default scan_frequency is 10s, So there could be some delay but log visible as soon as they pushed to Elasticsearch.

Could you check the document count while ingesting the logs?