# Help: How to filter unwanted metadata and fields from filebeats **URL:** \ **Category:** Kibana\ **Created:** [August 19, 2020, 11:01pm UTC](https://discuss.elastic.co/t/help-how-to-filter-unwanted-metadata-and-fields-from-filebeats/245685 "2020-08-19T23:01:46Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Moksha20](https://avatars.discourse-cdn.com/v4/letter/m/779978/32.png) [@Moksha20](https://discuss.elastic.co/u/Moksha20)\ **Post date:** [August 19, 2020, 11:01pm UTC](https://discuss.elastic.co/t/help-how-to-filter-unwanted-metadata-and-fields-from-filebeats/245685/1 "2020-08-19T23:01:46Z")
I am very new to this filebeat shipping thing. Once I ship the log to kibana I am getting so many meta data field both for kibana and filebeat...now I can filter it in kibana visualization but is there any way to filter out/exclude those field during log ingestion . Thanks in advance.. ---
**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\ **Post date:** [August 19, 2020, 11:56pm UTC](https://discuss.elastic.co/t/help-how-to-filter-unwanted-metadata-and-fields-from-filebeats/245685/2 "2020-08-19T23:56:10Z")
Believe you are looking to [drop fields](https://www.elastic.co/guide/en/beats/filebeat/current/drop-fields.html) in your filebeat config. ---
**Author:** ![Moksha20](https://avatars.discourse-cdn.com/v4/letter/m/779978/32.png) [@Moksha20](https://discuss.elastic.co/u/Moksha20)\ **Post date:** [September 3, 2020, 1:22pm UTC](https://discuss.elastic.co/t/help-how-to-filter-unwanted-metadata-and-fields-from-filebeats/245685/3 "2020-09-03T13:22:02Z")
Hi Aron. I was just trying to use remove\_field option... ```auto remove_field => ["message","log.file.path","agent.hostname","agent.id"] ``` But unfortunately it is removing the 'message' only not other fields. Could you please elaborate drop\_field with some example. Thanks in advance. Reagrds, Moksha ---
**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\ **Post date:** [September 3, 2020, 1:26pm UTC](https://discuss.elastic.co/t/help-how-to-filter-unwanted-metadata-and-fields-from-filebeats/245685/4 "2020-09-03T13:26:55Z")
Try this format instead of dot notation. ```auto remove_field => ["message","[log][file][path]","[agent][hostname]","[agent][id]" ] ``` ---
**Author:** ![Moksha20](https://avatars.discourse-cdn.com/v4/letter/m/779978/32.png) [@Moksha20](https://discuss.elastic.co/u/Moksha20)\ **Post date:** [September 3, 2020, 4:07pm UTC](https://discuss.elastic.co/t/help-how-to-filter-unwanted-metadata-and-fields-from-filebeats/245685/5 "2020-09-03T16:07:25Z")
Hi Aron, Thanks its working now. Another query ....I guess I can not exclude '\_id' from the log as its being a metadata?? If its possible do we have any separate syntax for that?? Regards , Moksha ---
**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\ **Post date:** [September 3, 2020, 4:13pm UTC](https://discuss.elastic.co/t/help-how-to-filter-unwanted-metadata-and-fields-from-filebeats/245685/6 "2020-09-03T16:13:16Z")
`_id` is [metadata](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-fields.html) and lies outside `_source` data. You can only remove `_source` data. ---
**Author:** ![Moksha20](https://avatars.discourse-cdn.com/v4/letter/m/779978/32.png) [@Moksha20](https://discuss.elastic.co/u/Moksha20)\ **Post date:** [September 10, 2020, 6:18am UTC](https://discuss.elastic.co/t/help-how-to-filter-unwanted-metadata-and-fields-from-filebeats/245685/7 "2020-09-10T06:18:10Z")
Hi Aaron, Thanks a lot for the help... Can I ask you for another help... with below topic > [@Filebeat log ingestion and scan frequency](https://discuss.elastic.co/t/filebeat-log-ingestion-and-scan-frequency/248113): > > Hi Expert and Team, Could you please help me with the log ingestion from filebeat.There are two scenarios 1)My logs are created in 15 minutes frequency in the same name and replace the older one.what should my .yml file should be ?? as the older one gets deleted with new one what time setting should I use?? 2)To avoid the above setting I am renaming the file with timestamp and keeping files of 1 day(which costing me disk spaces though)..but problem whenever a new file is getting creating it … Thanks Again, Moksha ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [October 8, 2020, 6:18am UTC](https://discuss.elastic.co/t/help-how-to-filter-unwanted-metadata-and-fields-from-filebeats/245685/8 "2020-10-08T06:18:12Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.