# Trying to change filebeat index

**URL:** <https://discuss.elastic.co/t/trying-to-change-filebeat-index/317564>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [October 26, 2022, 11:24pm UTC](https://discuss.elastic.co/t/trying-to-change-filebeat-index/317564 "2022-10-26T23:24:26Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Russell\_Fulton](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/russell_fulton/32/62888_2.png) [@Russell\_Fulton](https://discuss.elastic.co/u/Russell_Fulton)\
**Post date:** [October 26, 2022, 11:24pm UTC](https://discuss.elastic.co/t/trying-to-change-filebeat-index/317564/1 "2022-10-26T23:24:26Z")

</div>

I am trying to change the index filebeat writes to from the default "filebeat..." to "suricata..."

I have the following in my `filebeat.yml`:

```auto
# ======================= Elasticsearch template setting =======================

setup.template.settings:
  index.number_of_shards: 1
  #index.codec: best_compression
  #_source.enabled: false
setup:template:
    enabled: true
    name: "suricata"
    pattern: "suricata-*"

```

running by hand (as nothing gets written to the log : ( )

```auto
rful011@secmonprd08:~$ sudo /usr/share/filebeat/bin/filebeat -c /etc/filebeat/filebeat.yml --path.home /usr/share/filebeat --path.config /etc/filebeat --path.data /var/lib/filebeat --path.logs /var/log/filebeat
Exiting: setup.template.name and setup.template.pattern have to be set if index name is modified

```

I can't see what is wrong.

At the moment I am doing this on just one machine but ultimately I want this deployed on several. Am I correct in assuming that I need to do this just once?

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [October 27, 2022, 12:06am UTC](https://discuss.elastic.co/t/trying-to-change-filebeat-index/317564/2 "2022-10-27T00:06:08Z")

</div>

> [@Russell\_Fulton](#):
>
> ```auto
> setup:template:
> enabled: true
> name: "suricata"
> pattern: "suricata-*"
> 
> ```

I think this is a typo, should be `setup.template` not `setup:template`.

---

<div class="post-metadata">

**Author:** ![Russell\_Fulton](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/russell_fulton/32/62888_2.png) [@Russell\_Fulton](https://discuss.elastic.co/u/Russell_Fulton)\
**Post date:** [October 27, 2022, 8:43pm UTC](https://discuss.elastic.co/t/trying-to-change-filebeat-index/317564/3 "2022-10-27T20:43:36Z")

</div>

doh! thanks!

filebeat now starts but does not create the "suricata" index. I also tried setting `setup.ilm.check_exists: true`

logs:

```auto
2022-10-28T09:34:34.561+1300 INFO instance/beat.go:328 Setup Beat: filebeat; Version: 7.17.1
2022-10-28T09:34:34.561+1300 INFO [index-management] idxmgmt/std.go:184 Set output.elasticsearch.index to 'filebeat-7.17.1' as ILM is enabled.
2022-10-28T09:34:34.561+1300 INFO [esclientleg] eslegclient/connection.go:105 elasticsearch url: https://secesprd02.its.auckland.ac.nz:9200
2022-10-28T09:34:34.561+1300 INFO [publisher] pipeline/module.go:113 Beat name: secmonprd08.its.auckland.ac.nz
2022-10-28T09:34:34.563+1300 INFO beater/filebeat.go:118 Enabled modules/filesets: (), suricata (eve)
2022-10-28T09:34:34.565+1300 INFO instance/beat.go:492 filebeat start running.
2022-10-28T09:34:34.565+1300 INFO [monitoring] log/log.go:142 Starting metrics logging every 30s
2022-10-28T09:34:34.565+1300 INFO memlog/store.go:119 Loading data file of '/var/lib/filebeat/registry/filebeat' succeeded. Active transaction id=34044
2022-10-28T09:34:34.983+1300 INFO memlog/store.go:124 Finished loading transaction log file for '/var/lib/filebeat/registry/filebeat'. Active transaction id=57700
2022-10-28T09:34:34.984+1300 INFO [registrar] registrar/registrar.go:109 States Loaded from registrar: 1
2022-10-28T09:34:34.984+1300 INFO [crawler] beater/crawler.go:71 Loading Inputs: 1
2022-10-28T09:34:34.984+1300 WARN [cfgwarn] registered_domain/registered_domain.go:61 BETA: The registered_domain processor is beta.
2022-10-28T09:34:34.984+1300 WARN [cfgwarn] log/input.go:89 DEPRECATED: Log input. Use Filestream input instead.
2022-10-28T09:34:34.985+1300 INFO [input] log/input.go:171 Configured paths: [/data/sensors/eve.json] {"input_id": "51dbcb5e-d306-43e3-9141-4238e8d9e083"}
2022-10-28T09:34:34.985+1300 INFO [crawler] beater/crawler.go:141 Starting input (ID: 3131651449671214263)
2022-10-28T09:34:34.985+1300 INFO [crawler] beater/crawler.go:108 Loading and starting Inputs completed. Enabled inputs: 1
2022-10-28T09:34:34.986+1300 INFO [input.harvester] log/harvester.go:309 Harvester started for file. {"input_id": "51dbcb5e-d306-43e3-9141-4238e8d9e083", "source": "/data/sensors/eve.json", "state_id": "native::131465221-2050", "finished": false, "os_id": "131465221-2050", "old_source": "/data/sensors/eve.json", "old_finished": true, "old_os_id": "131465221-2050", "harvester_id": "74e6c9a1-c305-4f90-8095-0702bd654655"}
2022-10-28T09:34:35.988+1300 INFO [publisher_pipeline_output] pipeline/output.go:143 Connecting to backoff(elasticsearch(https://secesprd02.its.auckland.ac.nz:9200))
2022-10-28T09:34:35.988+1300 INFO [publisher] pipeline/retry.go:219 retryer: send unwait signal to consumer
2022-10-28T09:34:35.988+1300 INFO [publisher] pipeline/retry.go:223 done
2022-10-28T09:34:36.055+1300 INFO [esclientleg] eslegclient/connection.go:284 Attempting to connect to Elasticsearch version 7.17.1
2022-10-28T09:34:36.084+1300 INFO [esclientleg] eslegclient/connection.go:284 Attempting to connect to Elasticsearch version 7.17.1
2022-10-28T09:34:36.140+1300 INFO [index-management] idxmgmt/std.go:261 Auto ILM enable success.
2022-10-28T09:34:36.150+1300 INFO [index-management.ilm] ilm/std.go:170 ILM policy filebeat exists already.
2022-10-28T09:34:36.150+1300 INFO [index-management] idxmgmt/std.go:397 Set setup.template.name to '{filebeat-7.17.1 {now/d}-000001}' as ILM is enabled.
2022-10-28T09:34:36.150+1300 INFO [index-management] idxmgmt/std.go:402 Set setup.template.pattern to 'filebeat-7.17.1-*' as ILM is enabled.
2022-10-28T09:34:36.150+1300 INFO [index-management] idxmgmt/std.go:436 Set settings.index.lifecycle.rollover_alias in template to {filebeat-7.17.1 {now/d}-000001} as ILM is enabled.
2022-10-28T09:34:36.150+1300 INFO [index-management] idxmgmt/std.go:440 Set settings.index.lifecycle.name in template to {filebeat {"policy":{"phases":{"hot":{"actions":{"rollover":{"max_age":"30d","max_size":"50gb"}}}}}}} as ILM is enabled.
2022-10-28T09:34:36.167+1300 INFO template/load.go:110 Template "filebeat-7.17.1" already exists and will not be overwritten.
2022-10-28T09:34:36.167+1300 INFO [index-management] idxmgmt/std.go:297 Loaded index template.
2022-10-28T09:34:36.171+1300 INFO [index-management.ilm] ilm/std.go:126 Index Alias filebeat-7.17.1 exists already.
2022-10-28T09:34:36.172+1300 INFO [publisher_pipeline_output] pipeline/output.go:151 Connection to backoff(elasticsearch(https://secesprd02.its.auckland.ac.nz:9200)) established
2022-10-28T09:35:04.574+1300 INFO [monitoring] log/log.go:184 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cgroup":{"cpu":{"cfs":{"period":{"us":100000}},"id":"user.slice"},"cpuacct":{"id":"user.slice","total":{"ns":407526084257}},"memory":{"id":"user.slice","mem":{"limit":{"bytes":9223372036854771712},"usage":{"bytes":495489024}}}},"cpu":{"system":{"ticks":100,"time":{"ms":107}},"total":{"ticks":860,"time":{"ms":871},"value":860},"user":{"ticks":760,"time":{"ms":764}}},"handles":{"limit":{"hard":1048576,"soft":1024},"open":13},"info":{"ephemeral_id":"bcbf09d7-8277-4b80-9aba-2266834f2af4","uptime":{"ms":30093},"version":"7.17.1"},"memstats":{"

```

output setup:

```auto
output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["secesprd02.its.auckland.ac.nz"]

  # Protocol - either `http` (default) or `https`.
  protocol: "https"

  # Authentication credentials - either API key or username/password.
  index: "suricata-%{[agent.version]}-%{+yyyy.MM.dd}"

```

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [October 27, 2022, 9:16pm UTC](https://discuss.elastic.co/t/trying-to-change-filebeat-index/317564/4 "2022-10-27T21:16:28Z")

</div>

> [@Russell\_Fulton](#):
>
> `setup:template`

`setup.template`

Huh did not see the updates...

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [October 27, 2022, 9:21pm UTC](https://discuss.elastic.co/t/trying-to-change-filebeat-index/317564/5 "2022-10-27T21:21:26Z")

</div>

Apologies, not sure what's going on. I'm not getting the full thread until I reply... 7.17.

Try adding

`setup.ilm.enabled: false`

I think I went through a couple different methods of this and why the certain settings work here

> [@Filebeat writing to its own index](https://discuss.elastic.co/t/filebeat-writing-to-its-own-index/310842/28):
>
> As explained in the post I linked, You have a moduled enabled in your configuration whether it is important now or not... It overrides the output settings in some cases but let's put that aside there are other issues... Also from the docs [here](https://www.elastic.co/guide/en/beats/filebeat/7.12/elasticsearch-output.html#index-option-es)... which is your key issue... When [index lifecycle management (ILM)](https://www.elastic.co/guide/en/beats/filebeat/7.12/ilm.html) is enabled, the default index is "filebeat-%{[agent.version]}-%{+yyyy.MM.dd}-%{index\_num}" , for example, "filebeat-7.12.1-2022-07-28-000001" . Custom index settings are ignored when…

so I think you need

```auto
setup.ilm.enabled: false
setup.template:
    enabled: true
    name: "suricata"
    pattern: "suricata-*"

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 24, 2022, 11:21pm UTC](https://discuss.elastic.co/t/trying-to-change-filebeat-index/317564/6 "2022-11-24T23:21:38Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.