Apparently I need to edit a Filebeat 'data stream template'. Such a thing does not exist!

Elastic Stack Elasticsearch
1

We're using Filebeat 8.14.3 to index network logs. We'd like to enable the _size field for all Filebeat data streams. (My previous thread on this topic).

Here's the attempt to enable the "_size" field:

PUT /_index_template/filebeat-8.14.3
{
  "index_patterns": ["filebeat-8.14.3-*"],
  "template": {
    "mappings": {
      "_size": {
        "enabled": true
      }
    }
  }
}

Here's the error message:

"composable template [filebeat-8.14.3] with index patterns [filebeat-8.14.3-*], priority [null] and no data stream configuration would cause data streams [filebeat-8.14.3] to no longer match a data stream template"

But what is a data stream template? The documentation explains the relationship between an index template and a data stream, but I can't find any clear reference to a 'data stream template'. As it stands, I'm editing the index template for a specific version of filebeat, so the existence of a data stream template would make sense; new data streams would inherit settings from that template. But there is no such entity to be found! Data streams: yes. Index templates: yes. Data stream templates: no.

What am I missing here?

2

A data stream template is an index template that includes the data_stream object.

3

OK that's a great start. Thank you.

4

Hi @artschooldropout, did you have any luck with fixing this error?

I originally had a data stream setup with filebeat, migrating from the default index to data stream.

However, when I reconfigured and setup my filebeat again, I got a similar error to you

composable template [filebeat-7.17.22] with index patterns [filebeat-7.17.22-*], priority [150] and no data stream configuration would cause data streams [filebeat-7.17.22] to no longer match a data stream

I assumed for me, this is caused because I use the default index template, but changed it in Kibana when I tried to migrate to a data stream. I might need to specify the index template to use

5

@Tom_N alas, no. I posted in Reddit, and was told that this is not possible: see here

6

I see. Thank you for the reply!