Filebeat writing to its own index

Elastic Stack Beats
28

As explained in the post I linked, You have a moduled enabled in your configuration whether it is important now or not... It overrides the output settings in some cases but let's put that aside there are other issues...

Also from the docs here... which is your key issue...

When index lifecycle management (ILM) is enabled, the default index is "filebeat-%{[agent.version]}-%{+yyyy.MM.dd}-%{index_num}" , for example, "filebeat-7.12.1-2022-07-28-000001" . Custom index settings are ignored when ILM is enabled. If you’re sending events to a cluster that supports index lifecycle management, see Index lifecycle management (ILM) to learn how to change the index name.

So without this setting... you will never change the output index.
setup.ilm.enabled: false

OR you have to set ILM all up...

in the above combinations I don't see a valid combination.

So it seems like you are struggling a bit I have a couple suggestions if you are open to it... it is pretty much back to basics....

So two approaches...
1 Set it all up in filebeat (when you do this create a default ILM policy for you, which you can later edit)
2 setup your template, policy and rollover alias etc... etc. in elasticsearch then use minimal filebeat config

To be clear these are NOT snippets of filebeat.yml they are fully functional without all the extra stuff

Method 1

This works this is minimal filebeat below.
filebeat setup -e
filebeat-e

filebeat.inputs:

- type: log
  enabled: true
  paths:
    - /var/log/myindex-app/*.log

# matching on this type 2022-07-20 10:56:29,393
  multiline:
    pattern: '^\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2},\d{3}'
    negate: true
    match: after

setup.template.enabled: false

setup.ilm:
  enabled: true
  policy_name: "myindex"
  overwrite: true
  rollover_alias: "myindex-%{[agent.version]}"
  pattern: "{now/d}-0000001"

output.elasticsearch:
  hosts: ["localhost:9200"]

health status index                             uuid                   pri rep docs.count docs.deleted store.size pri.store.size
yellow open   myindex-7.12.0-2022.08.03-0000001 8rjPeIKlRuCD-BWMHRMQcw   1   1          7            0     14.3kb         14.3kb

Method 2

Setup your policy and and template directly in elasticsearch
Then important you have to create a a boot strap index this is the alias from the write alias to the concrete index...

PUT myindex-7.12.0-2022.08.03-0000001?pretty
{
  "aliases": {
    "my-index-7.12.0": {
      "is_write_index": true
    }
  }
}

then this config will work

filebeat setup -e
filebeat-e

filebeat.inputs:

- type: log
  enabled: true
  paths:
    - /var/log/myindex-app/*.log

# matching on this type 2022-07-20 10:56:29,393
  multiline:
    pattern: '^\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2},\d{3}'
    negate: true
    match: after

setup.ilm.enabled: false
setup.template.enabled: false

output.elasticsearch:
  hosts: ["localhost:9200"]
  index: "myindex-%{[agent.version]}"

results

health status index                             uuid                   pri rep docs.count docs.deleted store.size pri.store.size
yellow open   myindex-7.12.0-2022.08.03-0000001 TsXkf6AkQGeVLs25Ico7iw   1   1          7            0     14.1kb         14.1kb

Now you have 2 working options....