# Wildcards in logstash remove\_field
**URL:** https://discuss.elastic.co/t/wildcards-in-logstash-remove-field/241644
**Category:** Logstash
**Created:** [July 17, 2020, 12:45pm UTC](https://discuss.elastic.co/t/wildcards-in-logstash-remove-field/241644 "2020-07-17T12:45:46Z")
**Posts on this page:** 13
**Page:** 1
### Author:  [@Gauti](https://discuss.elastic.co/u/Gauti)
#### Post date: [July 17, 2020, 12:45pm UTC](https://discuss.elastic.co/t/wildcards-in-logstash-remove-field/241644/1 "2020-07-17T12:45:46Z")
Hi All,
I have a data source with almost 692 fields, out of which only 200 fields are valid, i want to remove those fields , i tried using below one, but no luck
mutate { remove\_field =\> [".value"] }
all the unwanted fields are ending with ".value" filed , is there any way i can achieve this?
Few example fields FYI..
inc\_active.value
inc\_additional\_assignee\_list.value
inc\_approval.value
inc\_assigned\_to.value
inc\_close\_notes.value
Any advice please.
Thanks
Gautham
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [July 17, 2020, 4:14pm UTC](https://discuss.elastic.co/t/wildcards-in-logstash-remove-field/241644/2 "2020-07-17T16:14:39Z")
If they are top level fields you can use a prune filter with a wildcard in blacklist\_names. If those examples are actually top-level objects that contain a value field then you will have to do it in ruby.
---
### Author:  [@Gauti](https://discuss.elastic.co/u/Gauti)
#### Post date: [July 17, 2020, 4:52pm UTC](https://discuss.elastic.co/t/wildcards-in-logstash-remove-field/241644/3 "2020-07-17T16:52:51Z")
@Badger i tried to replicate the below ruby code,
> [@How to remove subfield using ruby](https://discuss.elastic.co/t/how-to-remove-subfield-using-ruby/68072):
>
> Hi , I am using ruby filter to remove fields that start with 'ID'. In the same way how can i remove subfield of a field called 'outputdata'. Below, code couldn't remove. ruby { code =\> " event.to\_hash.keys.each { |k| event.remove(k) if k.start\_with?('[outputdata.ID](http://outputdata.ID):') } Thank you
I used something like below and was getting error,
`ruby { code => 'event['result'].keys.each { |k| event['result'].remove(k) if k.end_with?('.value')' }`
can you provide some ruby code example for my scenario.
Thanks
Gautham
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [July 17, 2020, 5:13pm UTC](https://discuss.elastic.co/t/wildcards-in-logstash-remove-field/241644/4 "2020-07-17T17:13:01Z")
You need to explain exactly what you want to do. When you say inc\_active.value, is inc\_active a top level object that contains a field called value? If so, do you want to remove the value field or the inc\_active object? Or is inc\_active.value a top level field that has a period in its name?
---
### Author:  [@Gauti](https://discuss.elastic.co/u/Gauti)
#### Post date: [July 17, 2020, 5:43pm UTC](https://discuss.elastic.co/t/wildcards-in-logstash-remove-field/241644/5 "2020-07-17T17:43:51Z")
@Badger I'm actually pulling data from servicenow this has duplicate fields like
```
inc_active.value : true
inc_active.display_value : true
inc_additional_assignee_list.value : windows
inc_additional_assignee_list.display_value : windows
inc_approval.value : 1
inc_approval.display_value : 1
inc_assigned_to.value : 0
inc_assigned_to.display_value : 0
inc_close_notes.value : NA
inc_close_notes.display_value : NA
```
From these documents i need to remove all fields which is ending with ".value", so that i my documents in elk will have only display\_value fields
```
inc_active.display_value : true
inc_additional_assignee_list.display_value : windows
inc_approval.display_value : 1
inc_assigned_to.display_value : 0
inc_close_notes.display_value : NA
```
These are few fields, like this i have a total of 200 fields to be removed. Instead of adding all these 200 fields in prune blacklist, was checking if we can put a wildcard
Thanks
Gautham
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [July 17, 2020, 5:47pm UTC](https://discuss.elastic.co/t/wildcards-in-logstash-remove-field/241644/6 "2020-07-17T17:47:11Z")
Once again, is inc\_approval.value a top level field, or is inc\_approval an object that contains fields called value and display\_value? If you do not understand the difference then add
```
output { stdout { codec => rubydebug } }
```
to your configuration and show us what the inc\_approval fields look like.
---
### Author:  [@Gauti](https://discuss.elastic.co/u/Gauti)
#### Post date: [July 17, 2020, 6:08pm UTC](https://discuss.elastic.co/t/wildcards-in-logstash-remove-field/241644/7 "2020-07-17T18:08:23Z")
@Badger Its an object that contains the fields.
here is the rubydebug output
```
{
"@timestamp" => 2020-07-17T18:29:01.629Z,
"@version" => "1",
"result" => {
"inc_active" => {
"display_value" => "true",
"value" => "true"
},
"inc_additional_assignee_list" => {
"display_value" => "windows",
"value" => "windows"
},
"inc_approval.display_value" => {
"display_value" => "1",
"value" => "1"
},
"inc_due_date" => {
"display_value" => "0",
"value" => "0"
},
"inc_close_notes" => {
"display_value" => " NA",
"link" => "https://prod.service-now.com/api/now/table/7bd1900ae45",
"value" => "NA"
},
"inc_u_remove_device_from_local" => {
"display_value" => nil,
"value" => ""
},
"inc_u_subject" => {
"display_value" => "High Visibility Notification OPEN Problem 663: Failure rate increase on Database ser",
"value" => "High Visibility Notification OPEN Problem 663: Failure rate increase on Database ser"
},
"inc_approval_set" => {
"display_value" => "",
"value" => ""
},
```
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [July 17, 2020, 7:39pm UTC](https://discuss.elastic.co/t/wildcards-in-logstash-remove-field/241644/8 "2020-07-17T19:39:49Z")
OK, so you will need ruby. I have not tested it, but try
```
ruby {
code => '
event.to_hash.each { |k, v|
if v.is_a? Hash and v.key? "value"
event.remove("[#{k}][value]")
end
}
'
}
```
---
### Author:  [@Gauti](https://discuss.elastic.co/u/Gauti)
#### Post date: [July 17, 2020, 8:01pm UTC](https://discuss.elastic.co/t/wildcards-in-logstash-remove-field/241644/9 "2020-07-17T20:01:21Z")
@Badger It didnt work ☹ still getting the same output.
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [July 17, 2020, 8:03pm UTC](https://discuss.elastic.co/t/wildcards-in-logstash-remove-field/241644/10 "2020-07-17T20:03:38Z")
I misread the rubydebug. How about this?
```auto
ruby {
code => '
event.get("result").each { |k, v|
if v.is_a? Hash and v.key? "value"
event.remove("[result][#{k}][value]")
end
}
'
}
```
---
### Author:  [@Gauti](https://discuss.elastic.co/u/Gauti)
#### Post date: [July 17, 2020, 8:25pm UTC](https://discuss.elastic.co/t/wildcards-in-logstash-remove-field/241644/11 "2020-07-17T20:25:23Z")
@Badger Bingo.....It worked... Thank you very much.
A small query, can i add any exception to this, like whitelisting one or two fields without removing?
Thanks
Gautham
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [July 17, 2020, 8:40pm UTC](https://discuss.elastic.co/t/wildcards-in-logstash-remove-field/241644/12 "2020-07-17T20:40:27Z")
> [@Gauti](#):
>
> can i add any exception to this, like whitelisting one or two fields without removing?
You could do something like
```
unless ["someKey", "anotherKey"].include? k
event.remove("[result][#{k}][value]")
end
```
If you have more than a couple of values to whitelist you might want to use a hash rather than an array for performance reasons.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 14, 2020, 8:40pm UTC](https://discuss.elastic.co/t/wildcards-in-logstash-remove-field/241644/13 "2020-08-14T20:40:35Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.