# Filebeat eslog-%{\[elasticsearch.cluster.name\]}-%{\[fileset.name\]}-%{+yyyy.MM.dd} not work

**URL:** <https://discuss.elastic.co/t/filebeat-eslog-elasticsearch-cluster-name-fileset-name-yyyy-mm-dd-not-work/268861>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [March 31, 2021, 4:41am UTC](https://discuss.elastic.co/t/filebeat-eslog-elasticsearch-cluster-name-fileset-name-yyyy-mm-dd-not-work/268861 "2021-03-31T04:41:32Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![asasas234](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/asasas234/32/113152_2.png) [@asasas234](https://discuss.elastic.co/u/asasas234)\
**Post date:** [March 31, 2021, 4:41am UTC](https://discuss.elastic.co/t/filebeat-eslog-elasticsearch-cluster-name-fileset-name-yyyy-mm-dd-not-work/268861/1 "2021-03-31T04:41:32Z")

</div>

```
output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["localhost:9200"]
  index: "eslog-%{[elasticsearch.cluster.name]}-%{[fileset.name]}-%{+yyyy.MM.dd}"

```

I found that the above configuration will cause errors due to elasticsearch.cluster.name, how do I configure it?

---

<div class="post-metadata">

**Author:** ![asasas234](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/asasas234/32/113152_2.png) [@asasas234](https://discuss.elastic.co/u/asasas234)\
**Post date:** [March 31, 2021, 4:52am UTC](https://discuss.elastic.co/t/filebeat-eslog-elasticsearch-cluster-name-fileset-name-yyyy-mm-dd-not-work/268861/2 "2021-03-31T04:52:34Z")

</div>

The errors are as follows

`2021-03-31T12:51:21.609+0800	ERROR	[publisher_pipeline_output]	pipeline/output.go:180	failed to publish events: temporary bulk send failure`

---

<div class="post-metadata">

**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\
**Post date:** [March 31, 2021, 3:02pm UTC](https://discuss.elastic.co/t/filebeat-eslog-elasticsearch-cluster-name-fileset-name-yyyy-mm-dd-not-work/268861/3 "2021-03-31T15:02:01Z")

</div>

Hi @asasas234,

The variables in the index as `%{[elasticsearch.cluster.name]}` or `%{[fileset.name]}` need to exist in the published events.

Do your events contain these fields?

You can use the [`add_fields` processor](https://www.elastic.co/guide/en/beats/filebeat/current/add-fields.html) to add custom fields.

In any case default indexes use to be fine for most use cases, is there a reason why you are using custom fields?

---

<div class="post-metadata">

**Author:** ![asasas234](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/asasas234/32/113152_2.png) [@asasas234](https://discuss.elastic.co/u/asasas234)\
**Post date:** [April 3, 2021, 12:24pm UTC](https://discuss.elastic.co/t/filebeat-eslog-elasticsearch-cluster-name-fileset-name-yyyy-mm-dd-not-work/268861/4 "2021-04-03T12:24:11Z")

</div>

@jsoriano Thanks for your reply, I confirmed that elasticsearch.cluster.name exists as I was able to look it up on ES. I started with the index name without elasticsearch.cluster.name, and the insert was successful, and then when I looked up the inserted data on ES, it included elasticsearch.cluster.name

---

<div class="post-metadata">

**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\
**Post date:** [April 6, 2021, 11:09am UTC](https://discuss.elastic.co/t/filebeat-eslog-elasticsearch-cluster-name-fileset-name-yyyy-mm-dd-not-work/268861/5 "2021-04-06T11:09:46Z")

</div>

But do _all_ events contain these fields? It might happen that the errors are produced by events that don't contain these fields, even if other events contain it.

You can try to use `indeces`, and configure a different index for events that don't contain this field, something like this:

```auto
output.elasticsearch:
  hosts: ["localhost:9200"]
  indeces:
    - index: "eslog-%{[elasticsearch.cluster.name]}-%{[fileset.name]}-%{+yyyy.MM.dd}"
      when.has_fields: ['elasticsearch.cluster.name']
    - index: "otherlog-%{[fileset.name]}-%{+yyyy.MM.dd}"
      when.not.has_fields: ['elasticsearch.cluster.name']

```

---

<div class="post-metadata">

**Author:** ![asasas234](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/asasas234/32/113152_2.png) [@asasas234](https://discuss.elastic.co/u/asasas234)\
**Post date:** [April 7, 2021, 6:54am UTC](https://discuss.elastic.co/t/filebeat-eslog-elasticsearch-cluster-name-fileset-name-yyyy-mm-dd-not-work/268861/6 "2021-04-07T06:54:29Z")

</div>

> [@jsoriano](#):
>
> You can try to use `indeces` , and configure a different index for events that don't contain this field, something like this:

@jsoriano Thanks, I'll try it some time, theoretically this should not happen because I use the official elasticsearch module and I only enabled the server.log log collection, the rest I have disabled

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 5, 2021, 8:54am UTC](https://discuss.elastic.co/t/filebeat-eslog-elasticsearch-cluster-name-fileset-name-yyyy-mm-dd-not-work/268861/7 "2021-05-05T08:54:50Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.