Error creating input: Can only start an input when all related states are finished

Elastic Stack Beats
1

I run filebeat with system module enabled and filebeat log like this..

2018-11-19T07:18:31.662+0900 INFO log/input.go:138 Configured paths: [/var/log/secure*]
2018-11-19T07:18:31.666+0900 ERROR fileset/factory.go:105 Error creating input: Can only start an input when all related states are finished: {Id:17600537-64768 Finished:false Fileinfo:0xc4203f0b60 Source:/var/log/messages Offset:1950248 Timestamp:2018-11-19 07:18:26.962331439 +0900 KST m=+428.768260414 TTL:-1ns Type:log Meta:map FileStateOS:17600537-64768}
2018-11-19T07:18:31.666+0900 ERROR [reload] cfgfile/list.go:96 Error creating runner from config: Can only start an input when all related states are finished: {Id:17600537-64768 Finished:false Fileinfo:0xc4203f0b60 Source:/var/log/messages Offset:1950248 Timestamp:2018-11-19 07:18:26.962331439 +0900 KST m=+428.768260414 TTL:-1ns Type:log Meta:map FileStateOS:17600537-64768}

elasticsearch cannot parsing system log exactly.
[Filebeat System] Syslog dashboard cannot show system log and auth log.
I have many logs between 04:43 to 07:10 but dashboard cannot recognize that logs.

kibana_dashboard
kibana_dashboard1680×939 162 KB

kibana_dashboard_02
kibana_dashboard_021680×938 374 KB

here is my config

/etc/filebeat/filebeat.yml

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/log/secure*
    - /var/log/messages*
    - /var/log/yum.log*
    - /var/log/maillog*
    - /var/log/cron*
filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: true
  reload.period: 10s
setup.template.settings:
  index.number_of_shards: 1
setup.kibana:
  host: "snip"
cloud.id: starwars:snip
output.elasticsearch:
  hosts: ["snip"]
  protocol: "https"
  username: "elastic"
  password: "snip"
processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~

/etc/filebeat/modules.d/system.yml

- module: system
  syslog:
    enabled: true
    var.paths: ["/var/log/messages*"]
  auth:
    enabled: true
    var.paths: ["/var/log/secure*"]
    var.convert_timezone: true

anyone know why this situations?

2

@Hoon_Cho

Looking at your configuration and your error message, I think the following is happening,
your manual configuration and system module are watching the same files.

if you remove the following lines in your defined inputs it should solve the problem.

 - /var/log/secure*
- /var/log/messages*
3

thanks for reply

you mean that remove paths from filebeat.yml? not in system.yml

like this?

/etc/filebeat/filebeat.yml

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/log/yum.log*
    - /var/log/maillog*
    - /var/log/cron*
4

@Hoon_Cho Yes exactly

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.