# Filebeat writing to its own index

**URL:** <https://discuss.elastic.co/t/filebeat-writing-to-its-own-index/310842>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [July 28, 2022, 9:09am UTC](https://discuss.elastic.co/t/filebeat-writing-to-its-own-index/310842 "2022-07-28T09:09:54Z")\
**Posts on this page:** 1\
**Showing post:** 28

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [August 3, 2022, 5:25pm UTC](https://discuss.elastic.co/t/filebeat-writing-to-its-own-index/310842/28 "2022-08-03T17:25:17Z")

</div>

> [@tractor\_boy](#):
>
> What's the relevance of nginx here? I

As explained in the post I linked, You have a moduled enabled in your configuration whether it is important now or not... _It overrides the output settings in some cases_ but let's put that aside there are other issues...

Also from the docs [here](https://www.elastic.co/guide/en/beats/filebeat/7.12/elasticsearch-output.html#index-option-es)... **which is your key issue...**

> When [index lifecycle management (ILM)](https://www.elastic.co/guide/en/beats/filebeat/7.12/ilm.html) is enabled, the default `index` is `"filebeat-%{[agent.version]}-%{+yyyy.MM.dd}-%{index_num}"` , for example, `"filebeat-7.12.1-2022-07-28-000001"` . **Custom `index` settings are ignored when ILM is enabled.** If you’re sending events to a cluster that supports index lifecycle management, see [_Index lifecycle management (ILM)_](https://www.elastic.co/guide/en/beats/filebeat/7.12/ilm.html) to learn how to change the index name.

So without this setting... you will never change the output index.  
`setup.ilm.enabled: false`

OR you have to set ILM all up...

in the above combinations I don't see a valid combination.

So it seems like you are struggling a bit **I have a couple suggestions if you are open to it.**.. it is pretty much back to basics....

So two approaches...  
1 Set it all up in filebeat (when you do this create a default ILM policy for you, which you can later edit)  
2 setup your template, policy and rollover alias etc... etc. in elasticsearch then use minimal filebeat config

_ **To be clear these are NOT snippets of filebeat.yml they are fully functional without all the extra stuff** _

**Method 1**

This works this is minimal filebeat below.  
`filebeat setup -e`  
`filebeat-e`

```auto
filebeat.inputs:

- type: log
  enabled: true
  paths:
    - /var/log/myindex-app/*.log

# matching on this type 2022-07-20 10:56:29,393
  multiline:
    pattern: '^\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2},\d{3}'
    negate: true
    match: after

setup.template.enabled: false

setup.ilm:
  enabled: true
  policy_name: "myindex"
  overwrite: true
  rollover_alias: "myindex-%{[agent.version]}"
  pattern: "{now/d}-0000001"

output.elasticsearch:
  hosts: ["localhost:9200"]

```

```auto
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open myindex-7.12.0-2022.08.03-0000001 8rjPeIKlRuCD-BWMHRMQcw 1 1 7 0 14.3kb 14.3kb

```

**Method 2**

Setup your policy and and template directly in elasticsearch  
Then important you have to create a a boot strap index this is the alias from the write alias to the concrete index...

```auto
PUT myindex-7.12.0-2022.08.03-0000001?pretty
{
  "aliases": {
    "my-index-7.12.0": {
      "is_write_index": true
    }
  }
}

```

then this config will work

`filebeat setup -e`  
`filebeat-e`

```auto
filebeat.inputs:

- type: log
  enabled: true
  paths:
    - /var/log/myindex-app/*.log

# matching on this type 2022-07-20 10:56:29,393
  multiline:
    pattern: '^\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2},\d{3}'
    negate: true
    match: after

setup.ilm.enabled: false
setup.template.enabled: false

output.elasticsearch:
  hosts: ["localhost:9200"]
  index: "myindex-%{[agent.version]}"

```

results

```auto
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open myindex-7.12.0-2022.08.03-0000001 TsXkf6AkQGeVLs25Ico7iw 1 1 7 0 14.1kb 14.1kb

```

Now you have 2 working options....

---

_[View the full topic](https://discuss.elastic.co/t/filebeat-writing-to-its-own-index/310842)._