Using filebeat modules with custom fields

[Burga]

Hi guys , I'm wondering , can I enable module and use fields in filebeat.yml file?

I just configured filebeat input

filebeat.inputs:

- type: log

  paths:

    - /var/log/messages

  document_type: syslog
  fields:
        log_type: "syslog"
  enable: true
  fields_under_root: true
  multiline.pattern: '^[0-9][0-9]:[0-9][0-9]:'
  multiline.negate: true
  multiline.match: after

` `` 

but I can't start filebeat with module enabled .

Best Regards.

[shaunak (Shaunak Kashyap)]

@Burga

but I can't start filebeat with module enabled .

Could you post the errors you are getting when you try to start Filebeat?

[Burga]

Apr 20 21:12:08 ild-tlv-mongodb02.checkpoint.com filebeat[45466]: Exiting: Can only start an input when all related states are finished: {Id:34379986-64768 Finished:false Fileinfo:0xc00057c270 Sourc...79986-64768}
Apr 20 21:12:08 ild-tlv-mongodb02.checkpoint.com systemd[1]: filebeat.service: main process exited, code=exited, status=1/FAILURE
Apr 20 21:12:08 ild-tlv-mongodb02.checkpoint.com systemd[1]: Unit filebeat.service entered failed state.
Apr 20 21:12:08 ild-tlv-mongodb02.checkpoint.com systemd[1]: filebeat.service failed.
Apr 20 21:12:08 ild-tlv-mongodb02.checkpoint.com systemd[1]: filebeat.service holdoff time over, scheduling restart.
Apr 20 21:12:08 ild-tlv-mongodb02.checkpoint.com systemd[1]: Stopped Filebeat sends log files to Logstash or directly to Elasticsearch..
Apr 20 21:12:08 ild-tlv-mongodb02.checkpoint.com systemd[1]: start request repeated too quickly for filebeat.service
Apr 20 21:12:08 ild-tlv-mongodb02.checkpoint.com systemd[1]: Failed to start Filebeat sends log files to Logstash or directly to Elasticsearch..
Apr 20 21:12:08 ild-tlv-mongodb02.checkpoint.com systemd[1]: Unit filebeat.service entered failed state.
Apr 20 21:12:08 ild-tlv-mongodb02.checkpoint.com systemd[1]: filebeat.service failed.

[Burga]

I managed to "solve " the issue , just changed syslog.yml by adding "- /var/log/*.log "

type: log
paths:
**#{{ range $i, $path := .paths }}**
**# - {{$path}}**
**#{{ end }}**
  - /var/log/*.log
exclude_files: [".gz$"]
multiline:
  pattern: "^\\s"
  match: after
processors:
- add_locale: ~

but somehow filebeat although the system module is enabled it running threw the filebeat.yml input and takes fields values

- type: log

  paths:

    - /var/log/messages

  document_type: syslog
  fields:
        log_type: "syslog"

why is that ? if system module is enabled shouldn't it use only modules input parameters ? how can I check if it uses system module ?

Best Regards.