Filebeat setup (error loading template: failed to put data stream. no matching index template found for data stream)

[ulisses]

I'm running a dev environment to understand how i can use filebeat and elasticsearch to store our application logs but i can't make it work with custom configurations.

I'm trying to change index name following this documentation: Change the index name | Filebeat Reference [8.2] | Elastic

Result from ./filebeat setup command:

Overwriting ILM policy is disabled. Set `setup.ilm.overwrite: true` for enabling.

Exiting: error loading template: failed to put data stream: could not put data stream: 400 Bad Request: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"no matching index template found for data stream [customname]"}],"type":"illegal_argument_exception","reason":"no matching index template found for data stream [customname]"},"status":400}. Response body: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"no matching index template found for data stream [customname]"}],"type":"illegal_argument_exception","reason":"no matching index template found for data stream [customname]"},"status":400}

My custom filebeat.yml:

setup.template:
  name: "customname"
  pattern: "customname-%{[agent.version]}"

filebeat.inputs:
- type: filestream
  enabled: true
  paths:
    - /a_path/app.log.json
  parsers:
    - ndjson:
        target: ""

output.elasticsearch:
  hosts: ["https://elastic-server:9200"]
  username: "elastic"
  password: "password"
  index: "customname-%{[agent.version]}"
  ssl:
    enabled: true
    ca_trusted_fingerprint: abigvalue

[ulisses]

My solution was to create the Index Template manually on Kibana interface (cloning the default Index template generated by filebeat), since filebeat could not do it as expected and documented.

[rugenl (Len Rugen)]

Try

pattern: "customname-%{[agent.version]}*"

I think there was a change around 8.1.x that caused this.

[rugenl (Len Rugen)]

filebeat was probably trying to create a data stream, did you create a non data stream index? Not that it matters for usage.

[ulisses]

Did not work.

[ulisses]

filebeat was probably trying to create a data stream, did you create a non data stream index? Not that it matters for usage.

No. I did not.