Issue with filebeat 6.1.2 and elasticsearch

Hello,

I updated elasticsearch to version 6.1.2 I also have two version of filebeat in my infrastructure 5.6.2 and 6.1.2 I see this message in the filebeat log and the log never reach elasticsearc

2018-04-06T12:54:26.688Z        WARN    elasticsearch/client.go:502     Cannot index event publisher.Event{Content:beat.Event{Timestamp:time.Time{wall:0xbea9f960688b4b59, ext:55801757983034, loc:(*time.Location)(0x200c1a0)}, Meta:common.MapStr(nil), Fields:common.MapStr{"message":"Apr  6 12:54:23 mslog puppet-agent[20048]: Applied catalog in 3.66 seconds", "prospector":common.MapStr{"type":"log"}, "beat":common.MapStr{"hostname":"mslog.oscaddie.net", "version":"6.2.3", "name":"mslog.oscaddie.net"}, "source":"/var/log/messages", "offset":1555985}, Private:file.State{Id:"", Finished:false, Fileinfo:(*os.fileStat)(0xc4205d9ee0), Source:"/var/log/messages", Offset:1555985, Timestamp:time.Time{wall:0xbea9c2e2426b28c7, ext:1118342988, loc:(*time.Location)(0x200c1a0)}, TTL:-1, Type:"log", FileStateOS:file.StateOS{Inode:0x100f8a9, Device:0xca01}}}, Flags:0x1} (status=400): {"type":"mapper_parsing_exception","reason":"Failed to parse mapping [doc]: Mapping definition for [error] has unsupported parameters:  [properties : {code={type=long}, type={ignore_above=1024, type=keyword}, message={norms=false, type=text}}]","caused_by":{"type":"mapper_parsing_exception","reason":"Mapping definition for [error] has unsupported parameters:  [properties : {code={type=long}, type={ignore_above=1024, type=keyword}, message={norms=false, type=text}}]"}}

For filebeat deployment, I use a puppet module: GitHub - pcfens/puppet-filebeat

Filebeat configuration managed by Puppet


shutdown_timeout: 0
name: mslog.oscaddie.net
tags:
fields: {}
fields_under_root: false
filebeat:
registry_file: "/var/lib/filebeat/registry"
config_dir: "/etc/filebeat/conf.d"
shutdown_timeout: 0
output:
elasticsearch:
hosts:
- mslog-int.oscaddie.net:9200
- mslog.oscaddie.net:9200
protocol: https
username: username
password: Password
registry_file: "/var/lib/filebeat/registry"
ssl.certificate_authorities: "/path to ssl"
shipper: {}
logging: {}
runoptions: {}
processors: {}

---

filebeat:
prospectors:
- type: log
paths:
- /var/log/.log
- /var/log/
/*
- /var/log/*
encoding: plain
exclude_files:
- .dat$
- .gz$
- filebeat
- mslogprod.log
fields_under_root: false
document_type: syslog-beat
scan_frequency: 10s
harvester_buffer_size: 16384
max_bytes: 10485760

  tail_files: false

  # Experimental: If symlinks is enabled, symlinks are opened and harvested. The harvester is openening the
  # original for harvesting but will report the symlink name as source.
  #symlinks: false

  backoff: 1s
  max_backoff: 10s
  backoff_factor: 2

  # Experimental: Max number of harvesters that are started in parallel.
  # Default is 0 which means unlimited
  #harvester_limit: 0

  ### Harvester closing options

  # Close inactive closes the file handler after the predefined period.
  # The period starts when the last line of the file was, not the file ModTime.
  # Time strings like 2h (2 hours), 5m (5 minutes) can be used.
  close_inactive: 5m

  # Close renamed closes a file handler when the file is renamed or rotated.
  # Note: Potential data loss. Make sure to read and understand the docs for this option.
  close_renamed: false

  # When enabling this option, a file handler is closed immediately in case a file can't be found
  # any more. In case the file shows up again later, harvesting will continue at the last known position
  # after scan_frequency.
  close_removed: true

  # Closes the file handler as soon as the harvesters reaches the end of the file.
  # By default this option is disabled.
  # Note: Potential data loss. Make sure to read and understand the docs for this option.
  close_eof: false

  ### State options

  # Files for the modification data is older then clean_inactive the state from the registry is removed
  # By default this is disabled.
  clean_inactive: 0

  # Removes the state for file which cannot be found on disk anymore immediately
  clean_removed: true

  # Close timeout closes the harvester after the predefined time.
  # This is independent if the harvester did finish reading the file or not.
  # By default this option is disabled.
  # Note: Potential data loss. Make sure to read and understand the docs for this option.
  close_timeout: 0

GET /_template/filebeat-*
{
"filebeat-6.1.2": {
"order": 1,
"index_patterns": [
"filebeat-6.1.2-"
],
But no trace of filebeat-6.1.2-
in kibana dashboard When I try to define a index

GET /filebeat-*/_mapping I can't file any mapping with version 6.1.2

Centos 7.4
filebeat test config: Config OK

Could you please share the debug logs of Filebeat? Also, please format everything using </>.

Hello,

I realize that none of my filebeat 6.x client was able to send log to elasticsearch.

Thank you

filebeat -e -d "*"
2018-04-10T14:53:14.186Z        INFO    instance/beat.go:468    Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]
2018-04-10T14:53:14.187Z        DEBUG   [beat]  instance/beat.go:495    Beat metadata path: /var/lib/filebeat/meta.json
2018-04-10T14:53:14.187Z        INFO    instance/beat.go:475    Beat UUID: d5b83379-18aa-4e66-8efa-dc6fbab5e984
2018-04-10T14:53:14.187Z        INFO    instance/beat.go:213    Setup Beat: filebeat; Version: 6.2.3
2018-04-10T14:53:14.187Z        DEBUG   [beat]  instance/beat.go:230    Initializing output plugins
2018-04-10T14:53:14.187Z        DEBUG   [processors]    processors/processor.go:49      Processors:
2018-04-10T14:53:14.187Z        INFO    elasticsearch/client.go:145     Elasticsearch url: https://mslog-int.oscaddie.net:9200
2018-04-10T14:53:14.187Z        INFO    elasticsearch/client.go:145     Elasticsearch url: https://mslog.oscaddie.net:9200
2018-04-10T14:53:14.188Z        INFO    pipeline/module.go:76   Beat name: mslog.oscaddie.net
2018-04-10T14:53:14.188Z        WARN    [cfgwarn]       config/config.go:100    DEPRECATED: config_dir is deprecated. Use `filebeat.config.prospectors` instead. Will be removed in version: 7.0.0
2018-04-10T14:53:14.188Z        INFO    config/config.go:106    Additional config files are fetched from: /etc/filebeat/conf.d
2018-04-10T14:53:14.188Z        INFO    config/config.go:75     Additional configs loaded from: /etc/filebeat/conf.d/syslogs.yml
2018-04-10T14:53:14.188Z        INFO    instance/beat.go:301    filebeat start running.
2018-04-10T14:53:14.188Z        INFO    [monitoring]    log/log.go:97   Starting metrics logging every 30s
2018-04-10T14:53:14.189Z        DEBUG   [registrar]     registrar/registrar.go:88       Registry file set to: /var/lib/filebeat/registry
2018-04-10T14:53:14.189Z        INFO    registrar/registrar.go:108      Loading registrar data from /var/lib/filebeat/registry
2018-04-10T14:53:14.193Z        INFO    registrar/registrar.go:119      States Loaded from registrar: 357
2018-04-10T14:53:14.193Z        INFO    crawler/crawler.go:48   Loading Prospectors: 1
2018-04-10T14:53:14.193Z        DEBUG   [processors]    processors/processor.go:49      Processors:
2018-04-10T14:53:14.193Z        DEBUG   [registrar]     registrar/registrar.go:150      Starting Registrar
2018-04-10T14:53:14.194Z        DEBUG   [prospector]    log/config.go:178       recursive glob enabled
2018-04-10T14:53:14.194Z        DEBUG   [prospector]    log/prospector.go:120   exclude_files: [\.dat(?-m:$) \.gz(?-m:$) <substring 'filebeat'> mslogprod(?-s:.)log]. Number of stats: 357
2018-04-10T14:53:14.194Z        DEBUG   [prospector]    file/state.go:82        New state added for /var/log/boot.log
2018-04-10T14:53:14.194Z        DEBUG   [prospector]    file/state.go:82        New state added for /var/log/cloud-init-output.log
2018-04-10T14:53:14.194Z        DEBUG   [prospector]    file/state.go:82        New state added for /var/log/cloud-init.log
2018-04-10T14:53:14.194Z        DEBUG   [registrar]     registrar/registrar.go:200      Processing 1 events
2018-04-10T14:53:14.194Z        DEBUG   [registrar]     registrar/registrar.go:193      Registrar states cleaned up. Before: 357, After: 357
2018-04-10T14:53:14.194Z        DEBUG   [registrar]     registrar/registrar.go:228      Write registry file: /var/lib/filebeat/registry
2018-04-10T14:53:14.194Z        DEBUG   [prospector]    file/state.go:82        New state added for /var/log/yum.log-20180101
2018-04-10T14:53:14.194Z        DEBUG   [prospector]    file/state.go:82        New state added for /var/log/java_install.log
2018-04-10T14:53:14.198Z        DEBUG   [registrar]     registrar/registrar.go:253      Registry file updated. 357 states written.
2018-04-10T14:53:14.198Z        DEBUG   [registrar]     registrar/registrar.go:200      Processing 1 events
2018-04-10T14:53:14.198Z        DEBUG   [registrar]     registrar/registrar.go:193      Registrar states cleaned up. Before: 357, After: 357
2018-04-10T14:53:14.198Z        DEBUG   [registrar]     registrar/registrar.go:228      Write registry file: /var/lib/filebeat/registry
2018-04-10T14:53:14.198Z        DEBUG   [prospector]    file/state.go:82        New state added for /var/log/grubby
2018-04-10T14:53:14.202Z        DEBUG   [registrar]     registrar/registrar.go:253      Registry file updated. 357 states written.
2018-04-10T14:53:14.202Z        DEBUG   [registrar]     registrar/registrar.go:200      Processing 1 events
2018-04-10T14:53:14.202Z        DEBUG   [registrar]     registrar/registrar.go:193      Registrar states cleaned up. Before: 357, After: 357
2018-04-10T14:53:14.202Z        DEBUG   [registrar]     registrar/registrar.go:228      Write registry file: /var/lib/filebeat/registry

Hello,

Anyone have an idea why my filebeat client version 6.x are not able to send log to elasticsearch ?

Thank you

I found the solution. I had to update the index a second time for 6.2.3

Thank you

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.