# Dynamic index name not working with Filebeat

**URL:** <https://discuss.elastic.co/t/dynamic-index-name-not-working-with-filebeat/251284>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [October 7, 2020, 2:13pm UTC](https://discuss.elastic.co/t/dynamic-index-name-not-working-with-filebeat/251284 "2020-10-07T14:13:05Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Pablo\_Albertengo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pablo_albertengo/32/47685_2.png) [@Pablo\_Albertengo](https://discuss.elastic.co/u/Pablo_Albertengo)\
**Post date:** [October 7, 2020, 2:13pm UTC](https://discuss.elastic.co/t/dynamic-index-name-not-working-with-filebeat/251284/1 "2020-10-07T14:13:06Z")

</div>

Hi guys!  
I am not being able to get Filebeat to write documents to indices based on a field.

This is my configuration:

```auto
filebeat.inputs:
- type: log

  # Change to true to enable this input configuration.
  enabled: true

  json.keys_under_root: true
  json.message_key: log

  # Paths that should be crawled and fetched. Glob based paths.
  paths:
    - /home/XXX/DEV/suppliers/2020-10/*/*.log
    - /home/XXX/DEV/suppliers/2020-11/*/*.log
    - /home/XXX/DEV/suppliers/2020-12/*/*.log

#============================= Filebeat modules ===============================

filebeat.config.modules:
  # Glob pattern for configuration loading
  path: ${path.config}/modules.d/*.yml

  # Set to true to enable config reloading
  reload.enabled: false

  # Period on which files under path should be checked for changes
  #reload.period: 10s

#==================== Elasticsearch template setting ==========================

setup.template.settings:
  index.number_of_shards: 1
  #index.codec: best_compression
  #_source.enabled: false

#================================ Outputs =====================================

# Configure what output to use when sending the data collected by the beat.

#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["xxx:9200"]

  # Protocol - either `http` (default) or `https`.
  #protocol: "https"

  # Authentication credentials - either API key or username/password.
  #api_key: "id:api_key"
  username: "xxx"
  password: "xxx"
  index: "supplierslog-%{[fields.operation]}-%{+yyyy.MM.dd}"

#================================ Processors =====================================

# Configure processors to enhance or manipulate events generated by the beat.

processors:
# - add_host_metadata: ~
# - add_cloud_metadata: ~
# - add_docker_metadata: ~
# - add_kubernetes_metadata: ~
# decode the log field (sub JSON document) if JSONencoded, then maps it's fields to elasticsearch fields
- decode_json_fields:
    fields: ["log"]
    target: ""
# overwrite existing target elasticsearch fields while decoding json fields    
    overwrite_keys: true

- drop_fields:
      fields: ["LOGS_HOME","level_value","agent","LOGS_SUPPLIER","level","host","thread_name","logger_name","input","ecs"]
      ignore_missing: true

setup.template.name: "supplierfields"
setup.template.fields: "supplierfields.yml"
setup.template.overwrite: true
setup.template.pattern: "supplierslog-*"
setup.ilm.rollover_alias: "supplierslog"
setup.ilm.enabled : false

```

This is an example line that is ingested by FileBeat:

```auto
{"@timestamp": "2020-10-07T07: 34: 14.120-03: 00", "@ version": "1", "login": "operator.esteban", "operation": "Search"}

```

This is the stack error:

```auto
2020-10-07T10:19:37.999-0300 INFO pipeline/output.go:105 Connection to backoff(elasticsearch(http://xxx:9200)) established
2020-10-07T10:19:45.889-0300 ERROR pipeline/output.go:121 Failed to publish events: temporary bulk send failure

```

But If I change the name of the index

```auto
index: "supplierslog-%{[fields.operation]}-%{+yyyy.MM.dd}

```

to

```auto
index: "supplierslog-%{+yyyy.MM.dd}"

```

works fine.

What am I doing wrong?

Thanks in advance,  
Pablo

**Filebeat version** : 7.6.2

---

<div class="post-metadata">

**Author:** ![Wolfram\_Haussig](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wolfram_haussig/32/70528_2.png) [@Wolfram\_Haussig](https://discuss.elastic.co/u/Wolfram_Haussig)\
**Post date:** [October 8, 2020, 5:20am UTC](https://discuss.elastic.co/t/dynamic-index-name-not-working-with-filebeat/251284/2 "2020-10-08T05:20:50Z")

</div>

Hello Pablo,

I am not sure but I think that the `fields` prefix is wrong - hav you tried it already with `index: "supplierslog-%{[operation]}-%{+yyyy.MM.dd}`?

Best regards  
Wolfram

---

<div class="post-metadata">

**Author:** ![Pablo\_Albertengo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pablo_albertengo/32/47685_2.png) [@Pablo\_Albertengo](https://discuss.elastic.co/u/Pablo_Albertengo)\
**Post date:** [October 14, 2020, 4:43pm UTC](https://discuss.elastic.co/t/dynamic-index-name-not-working-with-filebeat/251284/3 "2020-10-14T16:43:15Z")

</div>

> [@Wolfram\_Haussig](#):
>
> Wolfram

Thank you Wolfram!  
That's solve my problem.  
Maybe I misunderstood this doc:

> **[Configure the Elasticsearch output | Filebeat Reference \[7.6\] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/7.6/elasticsearch-output.html#index-option-es)**

---

<div class="post-metadata">

**Author:** ![Pablo\_Albertengo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pablo_albertengo/32/47685_2.png) [@Pablo\_Albertengo](https://discuss.elastic.co/u/Pablo_Albertengo)\
**Post date:** [October 14, 2020, 4:45pm UTC](https://discuss.elastic.co/t/dynamic-index-name-not-working-with-filebeat/251284/4 "2020-10-14T16:45:03Z")

</div>

> [@Pablo\_Albertengo](#):
>
> s doc:

> **[Configure the Elasticsearch output | Filebeat Reference \[7.6\] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/7.6/elasticsearch-output.html#index-option-es)**

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 11, 2020, 6:45pm UTC](https://discuss.elastic.co/t/dynamic-index-name-not-working-with-filebeat/251284/5 "2020-11-11T18:45:19Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.