Large number of Empty Fields in Discover Tab

Hello all,

I am trying reduce the noise when using Kibana and I have noticed that within the discover tab and when trying to create a lens there are 5700+ fields which are always displayed as empty. I have tried to go through the process to build a custom fields.yml file, change the metricbeat.yml to allow this to be loaded in and then delete the old index however, I still see all these fields despite removing them from what I believe should have been what creates the template for the datastream/index.

In my metricbeat.yml I have added the following:
setup.template.enabled: false

My fields.yml file is down from 32k lines to about 16k lines I have primarily kept the following keys:

key: ecs
key: beat
key: docker
key: host
key: process
key: common
key: system

Is this a known bug I did find this: [UnifiedFieldList] Fields are not marked as Empty in Discover sidebar · Issue #147124 · elastic/kibana · GitHub

But it doesn't appear to be quite my issue. But I am getting a little close to the end of my rope for fixing this. How do I limit or stop these from being imported into the datastream/index template?

Hi

I'm new to using Elasticsearch and Kibana. But I have the same/similar issue. There are fields in my index that I know I will never need.

aws.billing-AmortizedCost.unit
aws.billing-BlendedCost.amount
aws.billing.BlendedCost.unit
aws.billingCurrency
aws.billing.end_date
aws.billing.EstimatedCharges
aws.billing-group_definition.key
aws.billing.group_definition.type
aws.billing.NormalizedUsageAmount-amount
aws.billing.NormalizedUsageAmount-unit
aws.billing.ServiceName
aws.billing-start_date
aws.billing.UnblendedCost.amount
aws.billing-lJnbIendedCost-unit
aws.billing-lJsageQuantity.amount
aws.billing.UsageQuantity.unit
aws.cloudwatch.namespace
aws.dynamodb.metrics.AccountMaxReads. max
aws.dynamodb.metrics.AccountMaxTabIeLevelReads. max
aws.dynamodb.metrics.AccountMaxTabIeLevelWrites.max
aws.dynamodb.metrics.AccountMaxWrites.max
aws.dynamodb.metrics.AccountProvisionedReadCapacitylJtiIization.avg
aws.dynamodb.metrics.AccountProvisionedWriteCapacityUtilization.av
aws.dynamodb.metrics.ConditionalCheckFailedRequests.sum
aws.dynamodb.metrics.ConsumedReadCapacityUnits.avg
aws.dynamodb.metrics.ConsumedReadCapacityUnits.sum

and a lot more (7000+). How can I get rid of those fields?

Thanks for your help