# Upgrade Filebeat from 6.7 to 7.0

**URL:** <https://discuss.elastic.co/t/upgrade-filebeat-from-6-7-to-7-0/176603>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [April 12, 2019, 10:36am UTC](https://discuss.elastic.co/t/upgrade-filebeat-from-6-7-to-7-0/176603 "2019-04-12T10:36:38Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![PWIT](https://avatars.discourse-cdn.com/v4/letter/p/e68b1a/32.png) [@PWIT](https://discuss.elastic.co/u/PWIT)\
**Post date:** [April 12, 2019, 10:36am UTC](https://discuss.elastic.co/t/upgrade-filebeat-from-6-7-to-7-0/176603/1 "2019-04-12T10:36:38Z")

</div>

I have upgraded my filebeat version from 6.7 to 7.0, here is my filebeat.yml

```
#=========================== Filebeat inputs =============================
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /usr/local/app/trade-service/logs/*.json.log
  exclude_files: ['\.gz$']
  json.keys_under_root: true
  json.add_error_key: true
  json.message_key: message
  fields: {service-instance: "01"}
  fields_under_root: true

- type: log
  enable: true
  paths:
    - /usr/local/app/trade-client/logs/*.json.log
  exclude_files: ['\.gz$']
  json.keys_under_root: true
  json.add_error_key: true
  json.message_key: message
#============================= Filebeat modules ===============================
filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: true
  reload.period: 60s
#==================== Template ==========================
setup.template.enabled: true
setup.template.name: "app-trade-%{[agent.version]}"
setup.template.pattern: "app-trade-%{[agent.version]}-*"
setup.template.fields: "${path.config}/fields.yml"
setup.template.overwrite: true
setup.template.settings:
  index.number_of_shards: 3
  index.number_of_replicas: 0
  #index.codec: best_compression
  #_source.enabled: false
#================================ General =====================================
name: "app-trade-shipper"
fields:
  firm: "XEBT"
  env: "dev"
fields_under_root: true
#tags: ["service-X", "web-tier"]

#============================== Dashboards =====================================
#setup.dashboards.enabled: false
#setup.dashboards.url:

#============================== Kibana =====================================
setup.kibana:
  host: "192.168.7.25:5601"
  #space.id:
#============================= Elastic Cloud ==================================
#cloud.id:
#cloud.auth:

#================================ Outputs =====================================
#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
  enable: true
  hosts: ["192.168.7.25:9200"]
  index: "app-trade-%{[agent.version]}-%{+yyyy.MM.dd}"
  #protocol: "https"
  #username: "elastic"
  #password: "changeme"

#----------------------------- Logstash output --------------------------------
#output.logstash:
  #hosts: ["localhost:5044"]
  #ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]
  #ssl.certificate: "/etc/pki/client/cert.pem"
  #ssl.key: "/etc/pki/client/cert.key"
#================================ Processors =====================================
processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~
#================================ Logging =====================================
logging.level: info
#logging.selectors: ["*"]
logging.to_syslog: false
logging.to_eventlog: false
logging.metrics.enabled: true
logging.metrics.period: 60s
logging.to_files: true
logging.files:
  path: /var/log/filebeat
  name: filebeat
  rotateeverybytes: 10485760
  keepfiles: 7
  permissions: 0600

logging.json: false

#============================== Xpack Monitoring ===============================
# reporting is disabled by default.
#xpack.monitoring.enabled: false
#xpack.monitoring.elasticsearch:

#================================= Migration ==================================
#migration.6_to_7.enabled: true

```

After adjusting the config-file "filebeat.yml" in directory "/etc/filebeat" the following problems occur.

- The index "app-trade-%{[agent.version]}-%{+yyyy.MM.dd}" is not created but always the index "filebeat-7.0.0", but this worked fine in version 6.7.

- Logging is always done into syslog

logs from syslog:

```
filebeat[3310]: 2019-04-12T12:25:37.734+0200#011INFO#011[index-management]#011idxmgmt/std.go:361#011Set setup.template.name to '{filebeat-7.0.0 {now/d}-000001}' as ILM is enabled.
filebeat[3310]: 2019-04-12T12:25:37.734+0200#011INFO#011[index-management]#011idxmgmt/std.go:366#011Set setup.template.pattern to 'filebeat-7.0.0-*' as ILM is enabled.
filebeat[3310]: 2019-04-12T12:25:37.734+0200#011INFO#011[index-management]#011idxmgmt/std.go:400#011Set settings.index.lifecycle.rollover_alias in template to {filebeat-7.0.0 {now/d}-000001} as ILM is enabled.
filebeat[3310]: 2019-04-12T12:25:37.734+0200#011INFO#011[index-management]#011idxmgmt/std.go:404#011Set settings.index.lifecycle.name in template to {filebeat-7.0.0 map[policy:{"phases":{"hot":{"actions":{"rollover":{"max_age":"30d","max_size":"50gb"}}}}}]} as ILM is enabled.
filebeat[3310]: 2019-04-12T12:25:37.736+0200#011INFO#011template/load.go:82#011Loading template for Elasticsearch version: 7.0.0
filebeat[3310]: 2019-04-12T12:25:37.737+0200#011INFO#011template/load.go:84#011Existing template will be overwritten, as overwrite is enabled.
filebeat[3310]: 2019-04-12T12:25:37.913+0200#011INFO#011template/load.go:145#011Elasticsearch template with name 'filebeat-7.0.0' loaded
filebeat[3310]: 2019-04-12T12:25:37.913+0200#011INFO#011[index-management]#011idxmgmt/std.go:272#011Loaded index template.

```

Thank you for your help and best regards, Peter.

---

<div class="post-metadata">

**Author:** ![kierandelaney](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kierandelaney/32/44054_2.png) [@kierandelaney](https://discuss.elastic.co/u/kierandelaney)\
**Post date:** [April 12, 2019, 4:34pm UTC](https://discuss.elastic.co/t/upgrade-filebeat-from-6-7-to-7-0/176603/2 "2019-04-12T16:34:54Z")

</div>

You’ve got ILM turned on (see your log messages). You’ll need to specifically disable it in 7.0

---

<div class="post-metadata">

**Author:** ![suikast42](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/suikast42/32/29865_2.png) [@suikast42](https://discuss.elastic.co/u/suikast42)\
**Post date:** [April 15, 2019, 9:05am UTC](https://discuss.elastic.co/t/upgrade-filebeat-from-6-7-to-7-0/176603/3 "2019-04-15T09:05:36Z")

</div>

I am using the folowing output config, But still the same behaviour. Only filebeat-xx is created:

```
  output.elasticsearch:
hosts: ["my.cloud:9200"]
index: "es-logstream-%{+yyyy.MM.dd}"
pipeline: "ingest-logs"
worker: 3
  setup.template:
name: es-logstream-mapping
pattern: es-logstream*
enabled: true
overwrite: true
ilm.enabled: false
```

---

<div class="post-metadata">

**Author:** ![PWIT](https://avatars.discourse-cdn.com/v4/letter/p/e68b1a/32.png) [@PWIT](https://discuss.elastic.co/u/PWIT)\
**Post date:** [April 15, 2019, 9:27am UTC](https://discuss.elastic.co/t/upgrade-filebeat-from-6-7-to-7-0/176603/4 "2019-04-15T09:27:08Z")

</div>

it's works with (as in filebeat.refernces.yml in a standalone section not under output.elasticsearc)

```
#============================== Setup ILM =====================================
setup.ilm.enabled: false

```

best regards Peter

---

<div class="post-metadata">

**Author:** ![suikast42](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/suikast42/32/29865_2.png) [@suikast42](https://discuss.elastic.co/u/suikast42)\
**Post date:** [April 15, 2019, 1:24pm UTC](https://discuss.elastic.co/t/upgrade-filebeat-from-6-7-to-7-0/176603/5 "2019-04-15T13:24:28Z")

</div>

Ok. That works thanks.

That was ltlle confisuing to have two similar keys like

setup.template.ilm.enabled: false and  
setup.ilm.enabled: false

😉

---

<div class="post-metadata">

**Author:** ![kierandelaney](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kierandelaney/32/44054_2.png) [@kierandelaney](https://discuss.elastic.co/u/kierandelaney)\
**Post date:** [April 15, 2019, 4:19pm UTC](https://discuss.elastic.co/t/upgrade-filebeat-from-6-7-to-7-0/176603/6 "2019-04-15T16:19:03Z")

</div>

Yeah it does look a bit confusing. We had a very similar issue!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 13, 2019, 4:19pm UTC](https://discuss.elastic.co/t/upgrade-filebeat-from-6-7-to-7-0/176603/7 "2019-05-13T16:19:12Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.