# How to remove fields not required when sending logs via elastic agent

**URL:** <https://discuss.elastic.co/t/how-to-remove-fields-not-required-when-sending-logs-via-elastic-agent/346892>\
**Category:** Elastic Agent\
**Created:** [November 10, 2023, 6:23pm UTC](https://discuss.elastic.co/t/how-to-remove-fields-not-required-when-sending-logs-via-elastic-agent/346892 "2023-11-10T18:23:34Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![HHobeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hhobeck/32/121853_2.png) [@HHobeck](https://discuss.elastic.co/u/HHobeck)\
**Post date:** [November 10, 2023, 6:23pm UTC](https://discuss.elastic.co/t/how-to-remove-fields-not-required-when-sending-logs-via-elastic-agent/346892/1 "2023-11-10T18:23:34Z")

</div>

Dear community.

I have exactly the same issue like shi in Reference [1] but with the different that I'm using elastic agent with custom log integration. Under the surface I guess file beat will be used but I have no luck with the configuration.

The processors configuration looks like the following (yes it is json because to avoid whitespace hell):

```auto
[
  ...
  {
    "drop_fields": {
      "fields": ["agent.ephemeral_id", "agent.hostname", "agent.id", "agent.type", "agent.version", "ecs.version", "input.type", "log.offset", "version"]
    }
  }
]

```

I'm still getting all the meta fields in the document on the index e.g. `ecs.version`:

```auto
{
    ...
    "ecs": {
      "version": "8.0.0"
    },
    ...
}

```

Can anyone help me please?

Regards

HHobeck

Reference [1]: [How to remove fields not required while sending log data from file beat](https://discuss.elastic.co/t/how-to-remove-fields-not-required-while-sending-log-data-from-file-beat/290484)  
Reference [2]: [[SOLVED]How to remove agent.\* and ecs.version? - #3 by Duked](https://discuss.elastic.co/t/solved-how-to-remove-agent-and-ecs-version/183643/3)  
Reference [3: [Filebeat didn't drop some of the fields like agent.\*, ecs.\* etc](https://discuss.elastic.co/t/filebeat-didnt-drop-some-of-the-fields-like-agent-ecs-etc/243911)

---

<div class="post-metadata">

**Author:** ![HHobeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hhobeck/32/121853_2.png) [@HHobeck](https://discuss.elastic.co/u/HHobeck)\
**Post date:** [November 11, 2023, 7:21am UTC](https://discuss.elastic.co/t/how-to-remove-fields-not-required-when-sending-logs-via-elastic-agent/346892/2 "2023-11-11T07:21:44Z")

</div>

I have wrote a script processor step and print out the event object as JSON. It seems to be that the property `ecs.version` and others are not present in this object at this stage. I think it will be enriched later. That is the reason why the drop\_fields processor step doesn't work properly.

How to avoid enriching the event with such unneeded meta data?

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [November 11, 2023, 12:48pm UTC](https://discuss.elastic.co/t/how-to-remove-fields-not-required-when-sending-logs-via-elastic-agent/346892/3 "2023-11-11T12:48:23Z")

</div>

> [@HHobeck](#):
>
> How to avoid enriching the event with such unneeded meta data?

You will need to remove those using a custom Ingest Pipeline for your integration.

Also, If you are using Fleet I'm not sure you should remove the `agent` fields as those fields seems to be used by Fleet, but only someone from Elastic can confirm.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 9, 2023, 12:48pm UTC](https://discuss.elastic.co/t/how-to-remove-fields-not-required-when-sending-logs-via-elastic-agent/346892/4 "2023-12-09T12:48:39Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.