Connect filebeat to elasticsearch

Hello,
i've a filebeat-6.4.1-1 and try connect to ELK (elasticsearch-5.6 and kibana-5.6.12) but not connect and not index create in to Kibana. My filebeat.yml is:

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/log/auth.log
filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false
setup.template.settings:
  index.number_of_shards: 3
setup.kibana:
  host: 192.168.x.x:5601
output.elasticsearch:
  hosts: ["http://192.168.x.x:9200"]
  template.name: filebeat
  template.path: filebeat.template.json

And the filebeat message:

2018-09-27T14:28:44.900-0400    WARN    elasticsearch/client.go:520     Cannot index event publisher.Event{Content:beat.Event{Timestamp:time.Time{wall:0xbee36636f39b2454, ext:280057474517, loc:(*time.Location)(0x1f61700)}, Meta:common.MapStr(nil), Fields:common.MapStr{"beat":common.MapStr{"name":"monitor.example.lab", "hostname":"monitor.example.lab", "version":"6.4.1"}, "host":common.MapStr{"name":"monitor.example.lab"}, "source":"/var/log/secure", "offset":30490, "message":"Sep 27 14:28:42 monitor sshd[15486]: pam_unix(sshd:session): session opened for user zimbra4 by (uid=0)", "prospector":common.MapStr{"type":"log"}, "input":common.MapStr{"type":"log"}}, Private:file.State{Id:"", Finished:false, Fileinfo:(*os.fileStat)(0xc4204ff380), Source:"/var/log/secure", Offset:30594, Timestamp:time.Time{wall:0xbee365f0f2e05811, ext:45232592, loc:(*time.Location)(0x1f61700)}, TTL:-1, Type:"log", Meta:map[string]string(nil), FileStateOS:file.StateOS{Inode:0x89646, Device:0xfd00}}}, Flags:0x1} (status=400): {"type":"mapper_parsing_exception","reason":"Failed to parse mapping [_default_]: Mapping definition for [error] has unsupported parameters:  [properties : {code={type=long}, type={ignore_above=1024, type=keyword}, message={norms=false, type=text}}]","caused_by":{"type":"mapper_parsing_exception","reason":"Mapping definition for [error] has unsupported parameters:  [properties : {code={type=long}, type={ignore_above=1024, type=keyword}, message={norms=false, type=text}}]"}}
[root@elk ~]# curl localhost:9200/_cat/indices?
yellow open .kibana                    ej-lfzbnRw2mt2a-WoFn-w 5 1   34 15 189.2kb 189.2kb
yellow open auditbeat-6.4.0-2018.09.27 4v8j_1WkS8WAhedEm2nQ4g 3 1 6625  0   5.9mb   5.9mb
yellow open logstash-2018.09.27        lV7Wxbe-TkWsdSA9ox0UWQ 5 1  316  0 590.6kb 590.6kb

Thanks

Hello @mcoa

Looking at the error in the log this look like a mapping error.

Mapping definition for [error] has unsupported parameters:

Looking at your configuration, you are providing a custom template with these options:

 template.name: filebeat
 template.path: filebeat.template.json

Unless you really need it I would remove the two options above and use the default templates that filebeat will manage for you.

Also, I see that you are parsing auth log did you take a look at the system module which will take care of parsing the auth log and creating dashboard for it?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.