Remove unnecessary exported fields

Elastic Observability Metrics
1

Hello,
Is there any way how to remove unnecessary exported fields before indexing?
Using Elastic Agent integration, and along with it comes many exported fields. Tried Elastic Agent processor "drop_fields", but all listed fields for dropping still apears in document. Another option would be to use ingestion pipeline, but at this point it would be more costly and processing in the Agent itself is more preffered.

Just a short example used in configuration file:

processors:
  - drop_fields:
      fields:
        - "cloud.provider"
        - "cloud.region"
        - "host.mac"
        - "host.os.family"
        - "host.os.kernel"
        - "host.os.name"
        - "host.os.codename"
     ignore missing: true
2

Hi @s.buksa

What version are you using?

If I recall, unfortunately most of those fields, the host, and agent fields are actually added after The agent processors are run.

I agree, not ideal. So I am fairly sure you're going to have to drop them in an ingest pipeline after they arrive at Elasticsearch.

I'm not sure if the fix of this is on our roadmap, but you certainly can open an issue if you'd like.

3

Hi,
Thank you for your reply. I'm using version 8.9.2.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.