# Removing fields from logstash **URL:** \ **Category:** Logstash\ **Created:** [August 28, 2023, 7:16am UTC](https://discuss.elastic.co/t/removing-fields-from-logstash/341782 "2023-08-28T07:16:58Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![bharti](https://avatars.discourse-cdn.com/v4/letter/b/73ab20/32.png) [@bharti](https://discuss.elastic.co/u/bharti)\ **Post date:** [August 28, 2023, 7:16am UTC](https://discuss.elastic.co/t/removing-fields-from-logstash/341782/1 "2023-08-28T07:16:58Z")
```auto input { file { path => "/var/log/abc.log" } beats { port => 5044 } } filter { mutate { remove_field => ["agent.version.keyword"] } } ``` ---
**Author:** ![bharti](https://avatars.discourse-cdn.com/v4/letter/b/73ab20/32.png) [@bharti](https://discuss.elastic.co/u/bharti)\ **Post date:** [August 28, 2023, 7:17am UTC](https://discuss.elastic.co/t/removing-fields-from-logstash/341782/2 "2023-08-28T07:17:48Z")
Hello I need a help with the configuration of logstash I want to remove some fields from logstash I read that which fields I can remove , so am removing the above field,but its not working ,can you plz help me out . Am new on elastic and its a bit urgent ---
**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\ **Post date:** [August 28, 2023, 8:04am UTC](https://discuss.elastic.co/t/removing-fields-from-logstash/341782/3 "2023-08-28T08:04:57Z")
You should use the nested fields. ```auto mutate { remove_field => ["[agent][version][keyword]" ] # or just set "agent" and remove all nested fields related to the agent field } ``` ---
**Author:** ![bharti](https://avatars.discourse-cdn.com/v4/letter/b/73ab20/32.png) [@bharti](https://discuss.elastic.co/u/bharti)\ **Post date:** [August 28, 2023, 10:04am UTC](https://discuss.elastic.co/t/removing-fields-from-logstash/341782/4 "2023-08-28T10:04:53Z")
any other way?....its is not working ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [August 28, 2023, 10:40am UTC](https://discuss.elastic.co/t/removing-fields-from-logstash/341782/5 "2023-08-28T10:40:33Z")
Is the `agent.version` field mapped as `text` with a `keyword` subfield? If so you can not remove a subfield as it does not exist in the document, just in the mapping. You should however be able to remove the full `agent.version` field if you want. ---
**Author:** ![bharti](https://avatars.discourse-cdn.com/v4/letter/b/73ab20/32.png) [@bharti](https://discuss.elastic.co/u/bharti)\ **Post date:** [August 28, 2023, 11:07am UTC](https://discuss.elastic.co/t/removing-fields-from-logstash/341782/7 "2023-08-28T11:07:59Z")
```auto input { file { path => "/var/log/abc.log" start_position => "beginning" sincedb_path => "NULL" } beats { port => 5044 } } filter { mutate { remove_field => ["agent.version"] } } ``` ---
**Author:** ![bharti](https://avatars.discourse-cdn.com/v4/letter/b/73ab20/32.png) [@bharti](https://discuss.elastic.co/u/bharti)\ **Post date:** [August 28, 2023, 11:09am UTC](https://discuss.elastic.co/t/removing-fields-from-logstash/341782/8 "2023-08-28T11:09:22Z")
it's still there ...doesn't remove I checked the field, it is not a subfield ---
**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\ **Post date:** [August 28, 2023, 11:43am UTC](https://discuss.elastic.co/t/removing-fields-from-logstash/341782/9 "2023-08-28T11:43:26Z")
Christian has right, you are trying to remove the field from Kibana. My mistake `remove_field => ["[agent][version]" ]` or full agent `remove_field => ["agent"]` ---
**Author:** ![bharti](https://avatars.discourse-cdn.com/v4/letter/b/73ab20/32.png) [@bharti](https://discuss.elastic.co/u/bharti)\ **Post date:** [August 29, 2023, 4:34am UTC](https://discuss.elastic.co/t/removing-fields-from-logstash/341782/10 "2023-08-29T04:34:27Z")
tried both ways ....nothing is working ---
**Author:** ![bharti](https://avatars.discourse-cdn.com/v4/letter/b/73ab20/32.png) [@bharti](https://discuss.elastic.co/u/bharti)\ **Post date:** [August 29, 2023, 4:35am UTC](https://discuss.elastic.co/t/removing-fields-from-logstash/341782/11 "2023-08-29T04:35:11Z")
```auto input { file { path => "/var/log/abc.log" start_position => "beginning" sincedb_path => "NULL" } beats { port => 5044 } } filter { mutate { remove_field => ["agent"] } } ``` ---
**Author:** ![bharti](https://avatars.discourse-cdn.com/v4/letter/b/73ab20/32.png) [@bharti](https://discuss.elastic.co/u/bharti)\ **Post date:** [August 29, 2023, 4:36am UTC](https://discuss.elastic.co/t/removing-fields-from-logstash/341782/12 "2023-08-29T04:36:21Z")
this configuration is in '02-beats-input.conf' file. This is okay, right? ---
**Author:** ![bharti](https://avatars.discourse-cdn.com/v4/letter/b/73ab20/32.png) [@bharti](https://discuss.elastic.co/u/bharti)\ **Post date:** [August 29, 2023, 4:39am UTC](https://discuss.elastic.co/t/removing-fields-from-logstash/341782/13 "2023-08-29T04:39:30Z")
![image](https://us1.discourse-cdn.com/elastic/original/3X/d/f/df9593d2739d6417a888324484fa73e2085ced5f.jpeg) ---
**Author:** ![bharti](https://avatars.discourse-cdn.com/v4/letter/b/73ab20/32.png) [@bharti](https://discuss.elastic.co/u/bharti)\ **Post date:** [August 30, 2023, 4:48am UTC](https://discuss.elastic.co/t/removing-fields-from-logstash/341782/14 "2023-08-30T04:48:56Z")
Hello, thanks for your help, it is working now. Actually, kibana is taking time to implement the logstash/conf.d file. ---
**Author:** ![bharti](https://avatars.discourse-cdn.com/v4/letter/b/73ab20/32.png) [@bharti](https://discuss.elastic.co/u/bharti)\ **Post date:** [August 30, 2023, 4:55am UTC](https://discuss.elastic.co/t/removing-fields-from-logstash/341782/15 "2023-08-30T04:55:07Z")
One more question : I have more than one log file in my config file and I want to remove the same fields in each file . So how can I achieve this ---
**Author:** ![bharti](https://avatars.discourse-cdn.com/v4/letter/b/73ab20/32.png) [@bharti](https://discuss.elastic.co/u/bharti)\ **Post date:** [August 30, 2023, 4:55am UTC](https://discuss.elastic.co/t/removing-fields-from-logstash/341782/16 "2023-08-30T04:55:45Z")
```auto input { file { path => "/var/log/abc.log" start_position => "beginning" sincedb_path => "NULL" } beats { port => 5044 } } filter { mutate { remove_field => ["[agent][version]","[agent][type]" ] } } ``` ---
**Author:** ![chouben](https://avatars.discourse-cdn.com/v4/letter/c/e495f1/32.png) [@chouben](https://discuss.elastic.co/u/chouben)\ **Post date:** [August 30, 2023, 7:21am UTC](https://discuss.elastic.co/t/removing-fields-from-logstash/341782/17 "2023-08-30T07:21:32Z")
I'm not sure if there is a similar way to achieve the below approach on Filebeat itself. Logstash has the ability to combine multiple files (config for pipelines.yml under /etc/logstash): ```auto - pipeline.id: pingPoller path.config: "/etc/logstash/conf.d/{Ping_dns-input.conf,Ping_server1-input.conf,Ping_server2-input.conf,Ping-filter_output.conf}" queue.type: persisted ``` So if you route your filebeats over Logstash: 1. It would be quite easy to achieve the removal for all Filebeats in 1 place 2. If you require specific actions per Harvester in Filebeat, you could combine actions via a shared file for all pipelines. ---
**Author:** ![bharti](https://avatars.discourse-cdn.com/v4/letter/b/73ab20/32.png) [@bharti](https://discuss.elastic.co/u/bharti)\ **Post date:** [August 30, 2023, 9:44am UTC](https://discuss.elastic.co/t/removing-fields-from-logstash/341782/19 "2023-08-30T09:44:42Z")
And for adding field? My requirement is to add some fields in my logs file .....in each file, it will be different ---
**Author:** ![chouben](https://avatars.discourse-cdn.com/v4/letter/c/e495f1/32.png) [@chouben](https://discuss.elastic.co/u/chouben)\ **Post date:** [August 30, 2023, 11:17am UTC](https://discuss.elastic.co/t/removing-fields-from-logstash/341782/20 "2023-08-30T11:17:23Z")
My processor for one of my Filebeat instances looks like: ```auto - type: log processors: - dissect: #2021-12-08T08:34:04.370+0100 INFO [monitoring] log/log.go:144 Non-zero metrics in the last 30s {"monitoring": {"metrics #datatype: string to integer, long, float, double, boolean or ip tokenizer: "%{date}\t%{event.type}\t%{class}\t%{script}\t%{messageCut}" field: "message" target_prefix: "" - timestamp: field: date layouts: - '2006-01-02T15:04:05.999Z07:00' - '2006-01-02T15:04:05.999Z0700' - '2006-01-02T15:04:05.999999999Z07:00' #- '2006-01-02T15:04:05.999-07:00' test: - '2021-12-08T08:34:04.370+0100' - drop_fields: fields: ["date", "class", "script", "message"] - rename: fields: - from: "messageCut" to: "message" # Change to true to enable this input configuration. enabled: true # Paths that should be crawled and fetched. Glob based paths. paths: - /var/data/log/heartbeat/heartbeat.log fields: service.type: heartbeat event.module: heartbeat event.dataset: heartbeat.beat fields_under_root: true ``` We mostly do processing in logstash, which was built for this purpose. You could easily add a different processor in filebeat to add fields. ---
**Author:** ![bharti](https://avatars.discourse-cdn.com/v4/letter/b/73ab20/32.png) [@bharti](https://discuss.elastic.co/u/bharti)\ **Post date:** [September 4, 2023, 6:38am UTC](https://discuss.elastic.co/t/removing-fields-from-logstash/341782/21 "2023-09-04T06:38:10Z")
Is there any other way to add fields? ---
**Author:** ![bharti](https://avatars.discourse-cdn.com/v4/letter/b/73ab20/32.png) [@bharti](https://discuss.elastic.co/u/bharti)\ **Post date:** [September 4, 2023, 6:38am UTC](https://discuss.elastic.co/t/removing-fields-from-logstash/341782/22 "2023-09-04T06:38:40Z")
Am confused about the processor part [Next page](https://discuss.elastic.co/t/removing-fields-from-logstash/341782.md?page=2)