# Filebeat Config file

**URL:** <https://discuss.elastic.co/t/filebeat-config-file/330724>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [April 25, 2023, 10:05am UTC](https://discuss.elastic.co/t/filebeat-config-file/330724 "2023-04-25T10:05:57Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Dasher](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dasher/32/137841_2.png) [@Dasher](https://discuss.elastic.co/u/Dasher)\
**Post date:** [April 25, 2023, 10:05am UTC](https://discuss.elastic.co/t/filebeat-config-file/330724/1 "2023-04-25T10:05:57Z")

</div>

I Need a help with the file beat config:  
I'm trying to read the data from a log file whose size remains the same and its modification time is changed every 30 mins.During these 30 mins interval new logs are added in place of the old logs.  
Can someone suggest sometime,I tried multiple things but its not reading the newly added logs in the file.  
Filebeat - 7.16.2  
Running on Windows Server 2019

Filebeat Config:

- type: filestream  
enabled: true  
paths : 'path of the file'  
tags: ['console\_data']

Filebeat Logs:

After the initial read it keeps giving me the same response

2023-04-24T06:19:33.495Z INFO [file\_watcher] filestream/fswatch.go:137 Start next scan  
2023-04-24T06:19:33.496Z DEBUG [file\_watcher] filestream/fswatch.go:204 Found 1 paths  
2023-04-24T06:19:33.496Z DEBUG [input.filestream] filestream/prospector.go:164 File C:\Lotus\Domino\Data\IBM\_TECHNICAL\_SUPPORT\console.log has been updated {"id": "41C27034C04F35E3", "prospector": "file\_prospector", "operation": "write", "source\_name": "native::153485312-87546-3234102977", "os\_id": "153485312-87546-3234102977", "new\_path": "C:\Lotus\Domino\Data\IBM\_TECHNICAL\_SUPPORT\console.log", "old\_path": "C:\Lotus\Domino\Data\IBM\_TECHNICAL\_SUPPORT\console.log"}  
2023-04-24T06:19:33.496Z DEBUG [input.filestream] input-logfile/harvester.go:145 Starting harvester for file {"id": "41C27034C04F35E3", "source": "filestream::.global::native::153485312-87546-3234102977"}  
2023-04-24T06:19:33.497Z DEBUG [input.filestream] input-logfile/harvester.go:181 Stopped harvester for file {"id": "41C27034C04F35E3", "source": "filestream::.global::native::153485312-87546-3234102977"}  
2023-04-24T06:19:43.322Z DEBUG [input.filestream] filestream/filestream.go:131 End of file reached: C:\Lotus\Domino\Data\IBM\_TECHNICAL\_SUPPORT\console.log; Backoff now. {"id": "41C27034C04F35E3", "source": "filestream::.global::native::153485312-87546-3234102977", "path": "C:\Lotus\Domino\Data\IBM\_TECHNICAL\_SUPPORT\console.log", "state-id": "native::153485312-87546-3234102977"}

---

<div class="post-metadata">

**Author:** ![PRASHANT\_MEHTA](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/prashant_mehta/32/101764_2.png) [@PRASHANT\_MEHTA](https://discuss.elastic.co/u/PRASHANT_MEHTA)\
**Post date:** [April 25, 2023, 12:56pm UTC](https://discuss.elastic.co/t/filebeat-config-file/330724/2 "2023-04-25T12:56:57Z")

</div>

Hi @Dasher ,  
Below I've shared filebeat example yml.  
Read documentation and try with **ignore\_older**.Any file modification done within 1 hour will always be read and after 1 hour logs altered wont be read.

```auto
filebeat.inputs:
- type: log
  enabled: true
  paths:
   - /p/logs/mis/**/*.log
  ignore_older: 1h
  include_lines: 
  - ^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}.[0-9]{3}Z ?(.*)
  multiline.type: pattern
  multiline.pattern: ^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}.[0-9]{3}Z
  multiline.negate: true
  multiline.match: after
  scan_frequency: 30s
  harvester_limit: 100
  close_inactive: 30m
  close_removed: true
  clean_removed: true
filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false
setup.ilm.enabled: true
setup.ilm.check_exists: true
setup.ilm.rollover_alias: mis-log
setup.ilm.pattern: '{now/d}-000001'
setup.ilm.overwrite: false
setup.kibana:
  host: http://abc:5601
output.elasticsearch:
  hosts:
  - http://abc:9200
processors:
- add_host_metadata: null
- drop_fields:
    when:
      equals:
        agent.type: filebeat
    fields:
    - agent.hostname
    - agent.id
    - agent.type
    - agent.ephemeral_id
    - agent.version
    - log.offset
    - log.flags
    - input.type
    - ecs.version
    - host.os
    - host.id
    - host.mac
    - host.architecture
monitoring.enabled: true
monitoring.elasticsearch: null

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 23, 2023, 2:57pm UTC](https://discuss.elastic.co/t/filebeat-config-file/330724/3 "2023-05-23T14:57:29Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.