# Default ingest pipeline overwritten

**URL:** https://discuss.elastic.co/t/default-ingest-pipeline-overwritten/348640
**Category:** Elasticsearch
**Created:** [December 5, 2023, 12:54pm UTC](https://discuss.elastic.co/t/default-ingest-pipeline-overwritten/348640 "2023-12-05T12:54:52Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![Jasonespo](https://avatars.discourse-cdn.com/v4/letter/j/898d66/32.png) [@Jasonespo](https://discuss.elastic.co/u/Jasonespo)
#### Post date: [December 5, 2023, 12:54pm UTC](https://discuss.elastic.co/t/default-ingest-pipeline-overwritten/348640/1 "2023-12-05T12:54:53Z")

</div>

Hi,

I created an index template `logs-{dataset\_name}-default' as well as setting up a data stream. I also setup a default ingest pipeline for this index.

However after a number of days (and maybe coincidentally an Elastic cluster upgrade) my default ingest pipeline setting got overwritten from my own to  
`logs-dev-default@2.1.0 ` which appears to be a manged pipeline?

```auto
{
  "managed_by": "fleet",
  "managed": true,
  "package": {
    "name": "log"
  }
}

```

I was wondering if anyone knows a reason why this might be the case? This caused my logging flow to break as my logs weren't being processed by my custom ingest pipeline.

Thanks,

Jason

---

<div class="post-metadata">

### Author: ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [December 5, 2023, 5:00pm UTC](https://discuss.elastic.co/t/default-ingest-pipeline-overwritten/348640/2 "2023-12-05T17:00:40Z")

</div>

Hi @Jasonespo What version are you on?

---

<div class="post-metadata">

### Author: ![Jasonespo](https://avatars.discourse-cdn.com/v4/letter/j/898d66/32.png) [@Jasonespo](https://discuss.elastic.co/u/Jasonespo)
#### Post date: [December 5, 2023, 5:04pm UTC](https://discuss.elastic.co/t/default-ingest-pipeline-overwritten/348640/3 "2023-12-05T17:04:59Z")

</div>

We're on the latest version 8.11.1. I have since added the default ingest pipeline setting to my index template instead of direct to the indices that are created. I think this is the solution. Also forced my template to be used instead of logs-\* by setting the priority to 500.

Still would be good to know why it got overwritten. I think what happened is that my index was using the logs-\* index template and so adopted the lifecycle policy. After X days I believe the index rolled over and the new index created no longer used my settings?

---

<div class="post-metadata">

### Author: ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [December 5, 2023, 5:21pm UTC](https://discuss.elastic.co/t/default-ingest-pipeline-overwritten/348640/4 "2023-12-05T17:21:47Z")

</div>

Hi @Jasonespo So I will do my best to explain as I understand .....

So the default ingest pipeline IS managed by Fleet Integrations and that is because that is the pipeline that is used by integrations, say like nginx etc that have OOTB ingest pipelines ... so that base pipeline IS managed but that pipeline is basically empty for your custom logs

So, for your "Custom" Logs you should use the `@custom` pipeline that will NOT be overwritten

so here is my example Datastream  
`logs-mydataset-mynamespace`

in this case, you would use / create the `"logs-mydataset@custom",`

which would have your custom pipeline and would NOT be overwritten

Hope this makes sense...  
This is the  
`logs-mydataset-2.3.0`

```auto
[
  {
    "pipeline": {
      "name": "logs-mydataset@custom",
      "ignore_missing_pipeline": true
    }
  }
]

```

 ![Screenshot 2023-12-05 at 9.08.37 AM](https://us1.discourse-cdn.com/elastic/original/3X/5/8/58af0d81e561245aad326a22f83ddc94d1ae6727.png)

---

<div class="post-metadata">

### Author: ![Jasonespo](https://avatars.discourse-cdn.com/v4/letter/j/898d66/32.png) [@Jasonespo](https://discuss.elastic.co/u/Jasonespo)
#### Post date: [December 5, 2023, 5:28pm UTC](https://discuss.elastic.co/t/default-ingest-pipeline-overwritten/348640/5 "2023-12-05T17:28:50Z")

</div>

Ok that makes sense. Thank you! 🙂 What I have mentioned above does work for my use case at the moment, but I will keep what you have said in mind if it breaks.

I'm almost certain that my problem was that I was using the `logs-*` index template without realising and not my own one!

---

<div class="post-metadata">

### Author: ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [December 5, 2023, 6:05pm UTC](https://discuss.elastic.co/t/default-ingest-pipeline-overwritten/348640/6 "2023-12-05T18:05:17Z")

</div>

Right, but if you use custom logs integration with Elastic Agent it will always "Manage" parts of it... the key is your custom `dataset`

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 2, 2024, 6:05pm UTC](https://discuss.elastic.co/t/default-ingest-pipeline-overwritten/348640/7 "2024-01-02T18:05:28Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
