The first thing I learned was how to ingest logs to indexes, and I was asked to perform this process but already using datastreams and I set myself the task of doing it. I have already created the ILM (Index Lifecycle Management) and the index template. I have named the policy “tmes_policy” and th…

[image] juancamiloll: I understand the solution you propose would not be using datastream since clearly with your suggestion we would be working with normal indexes or am I wrong? No, you can still use datastreams, you just need to use a custom datastream name that does not collide with the …

It looks like you're matching one of the built-in templates. See the "Avoid index pattern collisions" section of Index templates | Elasticsearch Guide [8.14] | Elastic . You can either use a different index pattern, or use a higher priority on your template.

I think the issue goes this way, I don't know what @leandrojmp thinks. [image] Cannot create datastream under new index template Logstash Hi I am using logstash running Kubernetes under ECK. I am ingesting both logs (filebeat) and metrics (metricbeat). Now i want ingest d…

I would avoid any data stream naming scheme that could have any collision with any of the ones that elastic uses, which are logs-*, metrics-*, synthetics-* and traces-*. You could for example use just tmes as your datastream name. But this leads to another issue with is the fact that logstash does…

@leandrojmp thanks for your answer, according to what little I understand the solution you propose would not be using datastream since clearly with your suggestion we would be working with normal indexes or am I wrong?

Discuss the Elastic Stack

Help datastreams

Elastic Search

juancamiloll July 26, 2024, 5:08pm 3

I think the issue goes this way, I don't know what @leandrojmp thinks.

Topic		Replies	Views
Logstash + datastreams Elasticsearch	1	279	October 24, 2022
Logstash index template + data streams Logstash datastreams	1	8446	February 28, 2022
Cannot create datastream under new index template Logstash	6	1838	December 12, 2023
Issue with Logstash, ILM and Data stream Logstash ilm-index-lifecycle-management	5	1136	December 19, 2024
Not eligible for data streams because config contains one or more settings that are not compatible with data streams Logstash ilm-index-lifecycle-management	5	7696	August 2, 2023

Elasticsearch is a trademark of Elasticsearch BV, registered in the U.S. and in other countries
Trademarks
Terms
Privacy
Brand
Code of Conduct

Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.

Help datastreams

Related topics