Filebeat setup (error loading template: failed to put data stream. no matching index template found for data stream)

I'm running a dev environment to understand how i can use filebeat and elasticsearch to store our application logs but i can't make it work with custom configurations.

I'm trying to change index name following this documentation: Change the index name | Filebeat Reference [8.2] | Elastic

Result from ./filebeat setup command:

Overwriting ILM policy is disabled. Set `setup.ilm.overwrite: true` for enabling.

Exiting: error loading template: failed to put data stream: could not put data stream: 400 Bad Request: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"no matching index template found for data stream [customname]"}],"type":"illegal_argument_exception","reason":"no matching index template found for data stream [customname]"},"status":400}. Response body: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"no matching index template found for data stream [customname]"}],"type":"illegal_argument_exception","reason":"no matching index template found for data stream [customname]"},"status":400}

My custom filebeat.yml:

setup.template:
  name: "customname"
  pattern: "customname-%{[agent.version]}"

filebeat.inputs:
- type: filestream
  enabled: true
  paths:
    - /a_path/app.log.json
  parsers:
    - ndjson:
        target: ""

output.elasticsearch:
  hosts: ["https://elastic-server:9200"]
  username: "elastic"
  password: "password"
  index: "customname-%{[agent.version]}"
  ssl:
    enabled: true
    ca_trusted_fingerprint: abigvalue

My solution was to create the Index Template manually on Kibana interface (cloning the default Index template generated by filebeat), since filebeat could not do it as expected and documented.

Try

pattern: "customname-%{[agent.version]}*"

I think there was a change around 8.1.x that caused this.

filebeat was probably trying to create a data stream, did you create a non data stream index? Not that it matters for usage.

Did not work.

filebeat was probably trying to create a data stream, did you create a non data stream index? Not that it matters for usage.

No. I did not.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.