Connect filebeat to elasticsearch

Elastic Stack Beats
1

Hello,
i've a filebeat-6.4.1-1 and try connect to ELK (elasticsearch-5.6 and kibana-5.6.12) but not connect and not index create in to Kibana. My filebeat.yml is:

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/log/auth.log
filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false
setup.template.settings:
  index.number_of_shards: 3
setup.kibana:
  host: 192.168.x.x:5601
output.elasticsearch:
  hosts: ["http://192.168.x.x:9200"]
  template.name: filebeat
  template.path: filebeat.template.json

And the filebeat message:

2018-09-27T14:28:44.900-0400    WARN    elasticsearch/client.go:520     Cannot index event publisher.Event{Content:beat.Event{Timestamp:time.Time{wall:0xbee36636f39b2454, ext:280057474517, loc:(*time.Location)(0x1f61700)}, Meta:common.MapStr(nil), Fields:common.MapStr{"beat":common.MapStr{"name":"monitor.example.lab", "hostname":"monitor.example.lab", "version":"6.4.1"}, "host":common.MapStr{"name":"monitor.example.lab"}, "source":"/var/log/secure", "offset":30490, "message":"Sep 27 14:28:42 monitor sshd[15486]: pam_unix(sshd:session): session opened for user zimbra4 by (uid=0)", "prospector":common.MapStr{"type":"log"}, "input":common.MapStr{"type":"log"}}, Private:file.State{Id:"", Finished:false, Fileinfo:(*os.fileStat)(0xc4204ff380), Source:"/var/log/secure", Offset:30594, Timestamp:time.Time{wall:0xbee365f0f2e05811, ext:45232592, loc:(*time.Location)(0x1f61700)}, TTL:-1, Type:"log", Meta:map[string]string(nil), FileStateOS:file.StateOS{Inode:0x89646, Device:0xfd00}}}, Flags:0x1} (status=400): {"type":"mapper_parsing_exception","reason":"Failed to parse mapping [_default_]: Mapping definition for [error] has unsupported parameters:  [properties : {code={type=long}, type={ignore_above=1024, type=keyword}, message={norms=false, type=text}}]","caused_by":{"type":"mapper_parsing_exception","reason":"Mapping definition for [error] has unsupported parameters:  [properties : {code={type=long}, type={ignore_above=1024, type=keyword}, message={norms=false, type=text}}]"}}

[root@elk ~]# curl localhost:9200/_cat/indices?
yellow open .kibana                    ej-lfzbnRw2mt2a-WoFn-w 5 1   34 15 189.2kb 189.2kb
yellow open auditbeat-6.4.0-2018.09.27 4v8j_1WkS8WAhedEm2nQ4g 3 1 6625  0   5.9mb   5.9mb
yellow open logstash-2018.09.27        lV7Wxbe-TkWsdSA9ox0UWQ 5 1  316  0 590.6kb 590.6kb

Thanks

2

Hello @mcoa

Looking at the error in the log this look like a mapping error.

Mapping definition for [error] has unsupported parameters:

Looking at your configuration, you are providing a custom template with these options:

 template.name: filebeat
 template.path: filebeat.template.json

Unless you really need it I would remove the two options above and use the default templates that filebeat will manage for you.

Also, I see that you are parsing auth log did you take a look at the system module which will take care of parsing the auth log and creating dashboard for it?

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.