Unable to send newly updated events form filebeat to logstash

Elastic Stack Beats
1 
 ------------------------ filebeat config -----------------------------------

filebeat.inputs:
- type: log

  # Change to true to enable this input configuration.
  enabled: true

  # Paths that should be crawled and fetched. Glob based paths.
  paths:
    - C:\Office\Work\processed_fileChaser_2020-05-22_12_56.log
    
  close_eof: true
  close_inactive: 10s
  scan_frequency: 15s
  tail_files: true
  
output.logstash:
  # The Logstash hosts
  hosts: ["XX.XXX.XXX.XX:5044"]
  
  
 ------------------------ logsatsh config -----------------------------------
 
input {
  beats {
    port => 5044
  }
}

filter {
  json {
    source => "message" 
  }
  mutate {
    remove_field => ["tags", "ecs", "log", "agent", "message", "input"]
  }
}
output {
  file {
    path => "C:/Softwares/logstash-7.5.2/output/dm_filechaser.json"
  }
}

Sample input:

{"fileName": "DM.YYYYMMDD.TXT", "country": "TW", "type": "TW ", "app_code": "DM", "upstream": "XXXX", "insertionTime": "2020-05-20T13:01:04+08:00"}
{"fileName": "TRANSACTION.YYYYMMDD.TXT", "country": "TW", "type": "TW", "app_code": "DM", "upstream": "XXXX", "insertionTime": "2020-05-22T13:01:04+08:00"}

Versions:
Logstash: 7.5.2
Filebeat: 7.5.2

filebeat unable to send new data and always sends same data to logstash from file beat.
Can you please help how to send newly updated data from log to logsatsh.

Filebeat logs:
2020-05-22T15:39:30.630+0530	ERROR	logstash/async.go:256	Failed to publish events caused by: read tcp 10.10.242.48:53400->10.10.242.48:5044: i/o timeout
2020-05-22T15:39:30.632+0530	ERROR	logstash/async.go:256	Failed to publish events caused by: client is not connected
2020-05-22T15:39:32.605+0530	ERROR	pipeline/output.go:121	Failed to publish events: client is not connected
2020-05-22T15:39:32.605+0530	INFO	pipeline/output.go:95	Connecting to backoff(async(tcp://10.10.242.48:5044))
2020-05-22T15:39:32.606+0530	INFO	pipeline/output.go:105	Connection to backoff(async(tcp://10.10.242.48:5044)) established
2020-05-22T15:39:44.588+0530	INFO	[monitoring]	log/log.go:145	Non-zero metrics in the last 30s	{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":1203,"time":{"ms":32}},"total":{"ticks":1890,"time":{"ms":48},"value":1890},"user":{"ticks":687,"time":{"ms":16}}},"handles":{"open":280},"info":{"ephemeral_id":"970ad9cc-16a8-4bf1-85ed-4dd774f0df42","uptime":{"ms":61869}},"memstats":{"gc_next":11787664,"memory_alloc":8709864,"memory_total":16065440,"rss":2510848},"runtime":{"goroutines":26}},"filebeat":{"events":{"active":2,"added":2},"harvester":{"closed":1,"open_files":0,"running":0,"started":1}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"batches":2,"failed":4,"total":4},"read":{"errors":1},"write":{"bytes":730}},"pipeline":{"clients":1,"events":{"active":2,"filtered":2,"retry":6,"total":2}}},"registrar":{"states":{"current":1}}}}}
2020-05-22T15:39:44.657+0530	INFO	log/harvester.go:251	Harvester started for file: C:\Office\Work\processed_fileChaser_2020-05-22_12_56.log
2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.