Hi all,
I'm using ElasticSearch in AWS. It's the default cluster type that you get for free. It is almost entirely managed automatically. My logs go to cloudwatch, and then a lambda streams my log data into ES. I don't seem to have access to the url that is used to upload the data, so I can't define a pipeline to use in the url. So, I then set the default pipeline for the cloudwatch logs indices using curl. However, I just realized that this only works for indices that already exist. The way that cloudwatch streams its logging events into ES, you get a brand new index every day for the log events that took place that day. That means that even though I set the default pipeline, every day there is a new index, and it doesn't have a pipeline set. Is there a way I can specify that all new indices created with the form "cwl-*" should set their default pipeline to "my-ingest-pipeline"? For instance, is there a default setting I can set at the cluster level instead of the index level? Or maybe something else I'm missing?
I believe you can specify this through an index template.
That's it right there! Thanks so much Christian!