# Drop Events while using elastic kubernetes integration to collect logs

**URL:** <https://discuss.elastic.co/t/drop-events-while-using-elastic-kubernetes-integration-to-collect-logs/356083>\
**Category:** Elasticsearch\
**Created:** [March 25, 2024, 9:00am UTC](https://discuss.elastic.co/t/drop-events-while-using-elastic-kubernetes-integration-to-collect-logs/356083 "2024-03-25T09:00:04Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jobin\_James](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jobin_james/32/120704_2.png) [@Jobin\_James](https://discuss.elastic.co/u/Jobin_James)\
**Post date:** [March 25, 2024, 9:00am UTC](https://discuss.elastic.co/t/drop-events-while-using-elastic-kubernetes-integration-to-collect-logs/356083/1 "2024-03-25T09:00:04Z")

</div>

Hello,

Elastic search Version: 8.12.2  
ECK Operator Version: 2.11.0

I am using elastic Kubernetes integration for pushing logs from the k8s cluster using an elastic agent. I would like to know how can i drop the unwanted fields. I tried the processor, but it didn't work.

Thanks

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [March 25, 2024, 2:45pm UTC](https://discuss.elastic.co/t/drop-events-while-using-elastic-kubernetes-integration-to-collect-logs/356083/2 "2024-03-25T14:45:49Z")

</div>

Hi @Jobin_James

> [@Jobin\_James](#):
>
> I tried the processor, but it didn't work.

You'll have to show us what you tried and what did not work and what you're trying to accomplish.

Standalone agent of Fleet Managed?

Yes, you can drop fields but they need to be available and some of the fields like the agent field are appended after the processor run.

So show us tell us what you're trying to accomplish and what you tried. Perhaps we can help

---

<div class="post-metadata">

**Author:** ![Jobin\_James](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jobin_james/32/120704_2.png) [@Jobin\_James](https://discuss.elastic.co/u/Jobin_James)\
**Post date:** [March 25, 2024, 4:55pm UTC](https://discuss.elastic.co/t/drop-events-while-using-elastic-kubernetes-integration-to-collect-logs/356083/3 "2024-03-25T16:55:32Z")

</div>

Hello @stephenb,  
Thanks for the answer.

I am using the fleet managed elastic agent to ingest logs from the k8s cluster to the Elasticsearch.

I have attached a screenshot of the indexed document and I would like to remove most of the fields related to cloud and ecs as this is irrelevant to my developers. Can I do this using the processor or should I use the ingest pipeline?

 ![Screenshot 2024-03-25 at 5.49.10 PM](https://us1.discourse-cdn.com/elastic/original/3X/4/8/488294f520f42e590ea34059d8330da50f6f3e01.png)

Thanks

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [March 25, 2024, 5:19pm UTC](https://discuss.elastic.co/t/drop-events-while-using-elastic-kubernetes-integration-to-collect-logs/356083/4 "2024-03-25T17:19:04Z")

</div>

BTW to me it is not clear if you want to Drop Events... or Remove Fields...

Ok... can you share what you tried?

Can you share your config?

For most the cloud fields you can turn off the add metadata processor.  
Not sure Which ECS fields you are referring to?

Some of the fields (agent and host I believe) are added AFTER the integration processing but you can certainly drop them in an ingest pipeline.... in fact you can do that with any / all the fields and often users find that to be an easy way to centralized removing fields.

There is documentation on how to do that [here](https://www.elastic.co/guide/en/fleet/8.12/ingest-pipeline-kubernetes.html) and [here](https://www.elastic.co/guide/en/fleet/8.12/data-streams-pipeline-tutorial.html) but instead of adding fields you can remove them and

In short [Remove Processor](https://www.elastic.co/guide/en/elasticsearch/reference/current/remove-processor.html)

In Kibana Dev Tools

```auto
PUT /_ingest/pipeline/logs-kubernetes.container_logs@custom
{
  "processors": [
    {
      "remove": {
      "field": ["user_agent", "URL"] <<< Add Fields Here 
      }
    }
  ]
}

```

If you want actually to Drop Whole Events...

You can do that with the processors at the Agent Level (again you would need to show us what you tried otherwise we are just guessing)

Or use [Drop Event](https://www.elastic.co/guide/en/elasticsearch/reference/current/drop-processor.html) in the ingest Pipeline

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 22, 2024, 5:19pm UTC](https://discuss.elastic.co/t/drop-events-while-using-elastic-kubernetes-integration-to-collect-logs/356083/5 "2024-04-22T17:19:33Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.