Filebeat 6.6.0 doesn't harvest old logfile (even at first time run)


#1

Hello. I use filebeat->logstash->elasticsearch <- kibana . I am facing an issue with filebeat, it doesn't harvest logfile, even at first run. I tried to stop filebeat, remove registry file and start filebeat again without success. I am getting this message in filebeat log with debug level:

2019-02-07T15:57:32.947+0200    DEBUG   [input] log/input.go:404        Check file for harvesting: /pathtolog/log.txt
2019-02-07T15:57:32.947+0200    DEBUG   [input] log/input.go:555        Ignore file because ignore_older reached: /pathtolog/log.txt
2019-02-07T15:57:32.947+0200    DEBUG   [input] file/states.go:68       New state added for /pathtolog/log.txt

So filebeat thinks that ignore_older setting is set, but it is not. I have tried to set it to 0 and 2160h. It didn't help. Also I have tried to use close_eof: true setting, no succes. Every time I removed registry file. Log file has been modified 2019-01-23 last time. Os version: Red Hat Enterprise Linux Server release 7.6 (Maipo). What else can I try?
my filebeat.yml:

output.logstash:
  enabled: true
  hosts: ["hostname:5044"]
  loadbalance: false
  worker: 10
  compression_level: 9
  escape_html: true
  ssl.certificate_authorities: ["/etc/ca.crt"]
  ssl.certificate: "/etc/beat.crt"
  ssl.key: "/etc/beat.key"
  bulk_max_size: 2048      
output.elasticsearch:
  enabled: false
  hosts: ["hostname:9200"]
filebeat.config.modules:
  enabled: true
  reload.enabled: true
  reload.period: 30s
  path: ${path.config}/modules.d/*.yml     
queue.mem:
  events: 4096
  flush.min_events: 512
  flush.timeout: 5s
  logging.metrics.enabled: false
logging.json: false
#logging.level: info
logging.level: debug
logging.to_files: true
logging.files:
  path: /var/log/filebeat
  name: filebeat
  keepfiles: 7
  permissions: 0644
  rotateeverybytes: 10485760 # = 10MB
filebeat.inputs:
- type: log
  enabled: true
#  close_eof: true
#  ignore_older: 0
  ignore_older: 2160h
  tail_files: true
  paths:
    - /pathtolog/log.txt
  exclude_files: ['\.gz$','\.zip$','\.tmp$','\.hprof$']
  symlinks: true
  multiline.pattern: '(^\w{3}[[:space:]][0-9]{1,},[[:space:]][0-9]{4}[[:space:]][0-9]{1,}:[0-9]{1,}:[0-9]{1,})|(^\[\w{3}[[:space:]]\w{3}[[:space:]][0-9]{1,}[[:space:]][0-9]{1,}:[0-9]{1,}:[0-9]{1,})|(^\[[0-9]{1,}\/\w{3}\/[0-9]{1,}:[0-9]{1,}:[0-9]{1,})|(^[0-9]{4}-[0-9]{1,2
}-[0-9]{1,2}[[:space:]][0-9]{1,2}:[0-9]{1,2}:[0-9]{1,2})|(^[0-9]{4}-[0-9]{1,2}-[0-9]{1,2}[[:space:]][0-9]{1,2}:[0-9]{1,2}:[0-9]{1,2}.[0-9]{1,3})|([0-9]{1,2}:[0-9]{1,2}:[0-9]{1,2},[0-9]{3})|(^[0-9]{1,2}.[0-9]{1,2}.[0-9]{4}[[:space:]][0-9]{1,2}:[0-9]{1,2}:[0-9]{1,2})'
  multiline.negate: false
  multiline.match: before