# Filebeat setup creates more than one write index - ILM

**URL:** https://discuss.elastic.co/t/filebeat-setup-creates-more-than-one-write-index-ilm/290679
**Category:** Beats
**Tags:** ilm-index-lifecycle-management, filebeat
**Created:** [December 1, 2021, 2:16pm UTC](https://discuss.elastic.co/t/filebeat-setup-creates-more-than-one-write-index-ilm/290679 "2021-12-01T14:16:34Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![JGreene](https://avatars.discourse-cdn.com/v4/letter/j/c68b51/32.png) [@JGreene](https://discuss.elastic.co/u/JGreene)
#### Post date: [December 1, 2021, 2:16pm UTC](https://discuss.elastic.co/t/filebeat-setup-creates-more-than-one-write-index-ilm/290679/1 "2021-12-01T14:16:34Z")

</div>

Filebeat should not attempt to create / recreate write index if a write index exists. As indices age out with ILM policies and are deleted, subsequent runs of filebeat setup attempt to create the default index of `-000001` even when a later version of the index already exists.

```auto
[elasticsearch.server][WARN] path: /%3C888888-customerbeat-000001%3E, params: {index=<888888-customerbeat-000001>}
org.elasticsearch.transport.RemoteTransportException: [02-lab-es-03][172.19.18.17:9300][indices:admin/create]
Caused by: java.lang.IllegalStateException: alias [888888-customerbeat] has more than one write index [888888-customerbeat-000001,888888-customerbeat-000016]

```

Steps to reproduce:

1. Create index and ilm settings with Filebeat setup command.
2. Roll the index
3. Delete the initial index (-000001)
4. Run filebeat setup command.

---

<div class="post-metadata">

### Author: ![ChrsMark](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrsmark/32/55858_2.png) [@ChrsMark](https://discuss.elastic.co/u/ChrsMark)
#### Post date: [December 2, 2021, 8:14am UTC](https://discuss.elastic.co/t/filebeat-setup-creates-more-than-one-write-index-ilm/290679/2 "2021-12-02T08:14:51Z")

</div>

Hi @JGreene! Could you share Filebeat's configuration too?

@kvch any ideas here?

---

<div class="post-metadata">

### Author: ![JGreene](https://avatars.discourse-cdn.com/v4/letter/j/c68b51/32.png) [@JGreene](https://discuss.elastic.co/u/JGreene)
#### Post date: [December 2, 2021, 12:53pm UTC](https://discuss.elastic.co/t/filebeat-setup-creates-more-than-one-write-index-ilm/290679/3 "2021-12-02T12:53:23Z")

</div>

Sure, here it is. We have filebeat, metricbeat and a custom beat running.

Following the steps above, filebeat always tries to create the `-000001` for an index if it does not exist without reference to a later version of the index already present with a write alias. If ILM is enabled and indices are deleted, any new run of `filebeat setup` attempts to recreate the default index and write alias, resulting in the multiple write indices condition.

```auto
######################## Filebeat Configuration ############################

#========================== Modules configuration =============================
# Reference /etc/customerbeat/modules.d/*.yml

#========================= Filebeat global options ============================

filebeat.registry.path: /var/lib/customerbeat/registry
filebeat.registry.file_permissions: 0600
filebeat.registry.flush: 0s
filebeat.overwrite_pipelines: true
#filebeat.shutdown_timeout: 0
filebeat.config:
  inputs:
    enabled: true
    #path: inputs.d/*.yml
    #reload.enabled: true
    #reload.period: 10s
  modules:
    enabled: true
    path: /etc/customerbeat/modules.d/*.yml
    #reload.enabled: true
    #reload.period: 10s

#================================ General ======================================
name: "omitted"
#tags: ["888888"]

#-------------------------- Elasticsearch output -------------------------------
output.elasticsearch:
  enabled: true
  loadbalance: true
  hosts: ["https://omitted:9200", "https://omitted:9200", "https://omitted:9200"]
  #compression_level: 0
  #escape_html: false
  protocol: "https"
  username: "elastic"
  password: "changeme"
  worker: 4
  index: "888888-customerbeat"
  max_retries: 3
  bulk_max_size: 3200
  backoff.init: 1s
  backoff.max: 60s
  timeout: 90
  ssl.enabled: true
  ssl.verification_mode: full
  #ssl.supported_protocols: [TLSv1.3]
  ssl.certificate_authorities: ["/etc/filebeat/certs/ca/omittedmonca.crt"]
  ssl.certificate: "/etc/filebeat/certs/omitted.crt"
  ssl.key: "/etc/filebeat/certs/omitted.key"
  #ssl.key_passphrase: ''
  #ssl.cipher_suites: []
  #ssl.curve_types: []
  #ssl.renegotiation: never

#================================= Paths ======================================
path.home: /usr/share/filebeat/
path.config: /etc/customerbeat/
path.data: /var/lib/customerbeat/data
path.logs: /var/log/customerbeat/

#================================ Keystore ====================================
# Location of the Keystore containing the keys and their sensitive values.
keystore.path: "${path.config}/beats.keystore"

#================================ Logging ======================================
logging.level: info
#logging.selectors: []
logging.to_stderr: false
logging.to_syslog: false
logging.to_eventlog: false

logging.metrics.enabled: true
logging.metrics.period: 30s
logging.to_files: true
logging.files:
  path: /var/log/customerbeat/
  name: customerbeat.log
  rotateeverybytes: 10485760 # = 10MB
  keepfiles: 2
  permissions: 0600
  interval: 24h
  rotateonstartup: False
logging.json: true

# ====================== Index Lifecycle Management (ILM) ======================
setup.ilm.enabled: auto
setup.ilm.rollover_alias: "888888-customerbeat"
setup.ilm.pattern: "000001"
setup.ilm.policy_name: "888888-customerbeat_policy"
setup.ilm.policy_file: /etc/customerbeat/customerbeat_ilm.json
setup.template:
  name: "888888-customerbeat"
  pattern: "888888-*"
  enabled: true
setup.ilm.overwrite: true
setup.dashboards:
  enabled: true
  #index: "888888-*"
setup.kibana.space.id: "888888"

#============================== X-Pack Monitoring ===============================
monitoring.enabled: false
#monitoring.cluster_uuid:
monitoring.elasticsearch:
  hosts: ["https://omitted:9200", "https://omitted:9200", "https://omitted:9200"]
  compression_level: 1
  protocol: "https"
  username: "elastic"
  password: "changeme"
  max_retries: 3
  bulk_max_size: 50
  backoff.init: 1s
  backoff.max: 60s
  timeout: 90
  ssl.enabled: true
  ssl.verification_mode: full
  #ssl.supported_protocols: [TLSv1.3]
  ssl.certificate_authorities: ["/etc/filebeat/certs/ca/omittedmonca.crt"]
  ssl.certificate: "/etc/filebeat/certs/omitted.crt"
  ssl.key: "/etc/filebeat/certs/omitted.key"
  #ssl.key_passphrase: ''
  #ssl.cipher_suites: []
  #ssl.curve_types: []
  #ssl.renegotiation: never
  metrics.period: 1m
  state.period: 10m

# ================================= Dashboards =================================
setup.kibana:
  host: "https://omitted:5601"
  protocol: "https"
  username: "elastic"
  password: "changeme "
  path: ""
  ssl.enabled: true
  ssl.verification_mode: full
  #ssl.supported_protocols: [TLSv1.3]
  ssl.certificate_authorities: ["/etc/filebeat/certs/ca/omittedmonca.crt"]
  ssl.certificate: "/etc/filebeat/certs/omitted.crt"
  ssl.key: "/etc/filebeat/certs/omitted.key"

#================================ HTTP Endpoint ======================================
http.enabled: true
http.host: "omitted"
http.port: 5062

#============================= Process Security ================================

# Enable or disable seccomp system call filtering on Linux. Default is enabled.
seccomp.enabled: true

#================================= Migration ==================================

# This allows to enable 6.7 migration aliases
#migration.6_to_7.enabled: false

```

---

<div class="post-metadata">

### Author: ![JGreene](https://avatars.discourse-cdn.com/v4/letter/j/c68b51/32.png) [@JGreene](https://discuss.elastic.co/u/JGreene)
#### Post date: [December 2, 2021, 1:06pm UTC](https://discuss.elastic.co/t/filebeat-setup-creates-more-than-one-write-index-ilm/290679/4 "2021-12-02T13:06:04Z")

</div>

Additionally, we are on 7.15.2 (Beats, Elastic, & Kibana). However, this appears to happen for previous versions as well.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 30, 2021, 3:06pm UTC](https://discuss.elastic.co/t/filebeat-setup-creates-more-than-one-write-index-ilm/290679/5 "2021-12-30T15:06:22Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
