Does Filebeat plan to support Data Streams?

Elastic Stack Beats
1

Does Filebeat plan to support Data Streams, and if so, what is the expected development timeline?

Per docs,

Prior to Elasticsearch 7.9, you’d typically use an index alias with a write index to manage time series data. Data streams replace this functionality, require less maintenance, and automatically integrate with data tiers.

Yet as of 7.17.x, Filebeat still seems to only offer the option to use an index alias with daily indices.

(If I am misinterpreting this and there is some way to configure filebeat.yml for Data Streams, please let me know. Currently running filebeat setup --index-management does not seem to be configurable to support data_stream: {} in the resulting index template.)

Related: Filebeat and Data stream

2

From 8.0 all Beats send events to data streams.

If you want to use data streams in 7.17, you can edit the index template to add "data_stream": {} and load it manually. Then disable ILM and set output.elasticsearch.index to the name of the data stream.

3

To be clear do you mean a PUT /_template/<index-template> where the request body includes the entire existing index template plus data_stream: {} added? Or does PUT /_template/<index-template> support a partial-update (PATCH) where I can just add that single field?

4

Answering my own question regarding partial updates: appears the answer is no.

1 Like
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.