I've tried disabling all the processor metadata and somehow narrowed it down but I still can't get rid of agent.ephemeral_id, agent.hostname, agent.id, agent.type, agent.version and ecs.version and log.offset.
Is there a way to disable it or I have to manually specify them in the logstash config to make sure they don't get ingested/indexed ?
You can drop them from filebeat, logstash or configure the mapping/index template to change how they are treated: index or not, doc_values or not, etc.
If this feature exists, it's a hidden undocumented feature so I think it's safe to assume it doesn't exists.
Make sense, those fields are part of the basic functionality of the Beats. You'll break a bunch of stuff by dropping them. But I have to assume you have a custom setup already and are not using any of the features that use those fields.
All of it looks like crap to any reading the forums hehe
Always enclose the config you post in proper formatting:
The </> button in the forum post editor or triple back ticks. ```
I prefer the backticks, easier I think.
There is a post preview feature to check what it looks like and it is the only way to share config snippets or files correctly embedded. Or a github gist, etc.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.