I have upgraded my filebeat version from 6.7 to 7.0, here is my filebeat.yml
#=========================== Filebeat inputs =============================
filebeat.inputs:
- type: log
enabled: true
paths:
- /usr/local/app/trade-service/logs/*.json.log
exclude_files: ['\.gz$']
json.keys_under_root: true
json.add_error_key: true
json.message_key: message
fields: {service-instance: "01"}
fields_under_root: true
- type: log
enable: true
paths:
- /usr/local/app/trade-client/logs/*.json.log
exclude_files: ['\.gz$']
json.keys_under_root: true
json.add_error_key: true
json.message_key: message
#============================= Filebeat modules ===============================
filebeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: true
reload.period: 60s
#==================== Template ==========================
setup.template.enabled: true
setup.template.name: "app-trade-%{[agent.version]}"
setup.template.pattern: "app-trade-%{[agent.version]}-*"
setup.template.fields: "${path.config}/fields.yml"
setup.template.overwrite: true
setup.template.settings:
index.number_of_shards: 3
index.number_of_replicas: 0
#index.codec: best_compression
#_source.enabled: false
#================================ General =====================================
name: "app-trade-shipper"
fields:
firm: "XEBT"
env: "dev"
fields_under_root: true
#tags: ["service-X", "web-tier"]
#============================== Dashboards =====================================
#setup.dashboards.enabled: false
#setup.dashboards.url:
#============================== Kibana =====================================
setup.kibana:
host: "192.168.7.25:5601"
#space.id:
#============================= Elastic Cloud ==================================
#cloud.id:
#cloud.auth:
#================================ Outputs =====================================
#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
enable: true
hosts: ["192.168.7.25:9200"]
index: "app-trade-%{[agent.version]}-%{+yyyy.MM.dd}"
#protocol: "https"
#username: "elastic"
#password: "changeme"
#----------------------------- Logstash output --------------------------------
#output.logstash:
#hosts: ["localhost:5044"]
#ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]
#ssl.certificate: "/etc/pki/client/cert.pem"
#ssl.key: "/etc/pki/client/cert.key"
#================================ Processors =====================================
processors:
- add_host_metadata: ~
- add_cloud_metadata: ~
#================================ Logging =====================================
logging.level: info
#logging.selectors: ["*"]
logging.to_syslog: false
logging.to_eventlog: false
logging.metrics.enabled: true
logging.metrics.period: 60s
logging.to_files: true
logging.files:
path: /var/log/filebeat
name: filebeat
rotateeverybytes: 10485760
keepfiles: 7
permissions: 0600
logging.json: false
#============================== Xpack Monitoring ===============================
# reporting is disabled by default.
#xpack.monitoring.enabled: false
#xpack.monitoring.elasticsearch:
#================================= Migration ==================================
#migration.6_to_7.enabled: true
After adjusting the config-file "filebeat.yml" in directory "/etc/filebeat" the following problems occur.
-
The index "app-trade-%{[agent.version]}-%{+yyyy.MM.dd}" is not created but always the index "filebeat-7.0.0", but this worked fine in version 6.7.
-
Logging is always done into syslog
logs from syslog:
filebeat[3310]: 2019-04-12T12:25:37.734+0200#011INFO#011[index-management]#011idxmgmt/std.go:361#011Set setup.template.name to '{filebeat-7.0.0 {now/d}-000001}' as ILM is enabled.
filebeat[3310]: 2019-04-12T12:25:37.734+0200#011INFO#011[index-management]#011idxmgmt/std.go:366#011Set setup.template.pattern to 'filebeat-7.0.0-*' as ILM is enabled.
filebeat[3310]: 2019-04-12T12:25:37.734+0200#011INFO#011[index-management]#011idxmgmt/std.go:400#011Set settings.index.lifecycle.rollover_alias in template to {filebeat-7.0.0 {now/d}-000001} as ILM is enabled.
filebeat[3310]: 2019-04-12T12:25:37.734+0200#011INFO#011[index-management]#011idxmgmt/std.go:404#011Set settings.index.lifecycle.name in template to {filebeat-7.0.0 map[policy:{"phases":{"hot":{"actions":{"rollover":{"max_age":"30d","max_size":"50gb"}}}}}]} as ILM is enabled.
filebeat[3310]: 2019-04-12T12:25:37.736+0200#011INFO#011template/load.go:82#011Loading template for Elasticsearch version: 7.0.0
filebeat[3310]: 2019-04-12T12:25:37.737+0200#011INFO#011template/load.go:84#011Existing template will be overwritten, as overwrite is enabled.
filebeat[3310]: 2019-04-12T12:25:37.913+0200#011INFO#011template/load.go:145#011Elasticsearch template with name 'filebeat-7.0.0' loaded
filebeat[3310]: 2019-04-12T12:25:37.913+0200#011INFO#011[index-management]#011idxmgmt/std.go:272#011Loaded index template.
Thank you for your help and best regards, Peter.