# Using wildcard to update/remove fields inside logstash filter

**URL:** <https://discuss.elastic.co/t/using-wildcard-to-update-remove-fields-inside-logstash-filter/104460>\
**Category:** Logstash\
**Created:** [October 18, 2017, 10:22pm UTC](https://discuss.elastic.co/t/using-wildcard-to-update-remove-fields-inside-logstash-filter/104460 "2017-10-18T22:22:08Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![bishplz](https://avatars.discourse-cdn.com/v4/letter/b/7bcc69/32.png) [@bishplz](https://discuss.elastic.co/u/bishplz)\
**Post date:** [October 18, 2017, 10:22pm UTC](https://discuss.elastic.co/t/using-wildcard-to-update-remove-fields-inside-logstash-filter/104460/1 "2017-10-18T22:22:08Z")

</div>

Hi  
I have a data structure like this. This is an stdout after parsing with XML filter.

{  
"offset" =\> 6859148,  
"input\_type" =\> "log",  
"source" =\> "requests.log.6",  
"type" =\> "log",  
"datestamp" =\> "08-09-17 03:03:07.401",  
"@timestamp" =\> 2017-10-17T18:32:04.491Z,  
"@version" =\> "1",  
"msg\_type" =\> "REQUEST",  
"parsed\_msg" =\> {  
"Body" =\> [  
[0] {  
"getIndContents" =\> [  
[0] {  
"returnXML" =\> [  
[0] "true"  
],  
"indexName" =\> [  
[0] "Some Index"  
],  
"fieldNames" =\> [  
[0] {  
"JavaLangstring" =\> [  
[0] "Out$float",  
[1] "Mark$float",  
[2] "Pct$float",  
[3] "Description$String",  
[4] "Is$String",  
[5] "Preferred\_Id$String",  
[6] "Co$float",  
[7] "Ma$date",  
[8] "Pr$float",  
[9] "Yi$float",  
[10] "OA$float",  
[11] "Dur$float",  
[12] "Mo$String@CODE",  
[13] "SP$String@CODE",  
[14] "Mar$float",  
[15] "Pct$float"  
]  
}  
],  
"login" =\> [  
[0] "index\_website"  
],  
"userName" =\> [  
[0] "System"  
],  
"version" =\> [  
[0] "1.1"  
],  
"xmlns" =\> "[http://somewebsite.com](http://somewebsite.com)",  
"site" =\> [  
[0] "abcsite"  
],  
"universe" =\> [  
[0] "1"  
],  
"currency" =\> [  
[0] "USD"  
],  
"pwd" =\> [  
[0] "pass123"  
],  
"asOfDate" =\> [  
[0] "07/31/2017"  
]  
}  
]  
}  
]  
},  
"request\_id" =\> "7771f2ca-7e45-4479-b7c7-91566b0f89ee"  
},  
{  
"offset" =\> 1275245,  
"input\_type" =\> "log",  
"source" =\> "requests.log.8",  
"type" =\> "log",  
"datestamp" =\> "08-08-17 17:58:17.405",  
"@timestamp" =\> 2017-10-17T18:32:01.769Z,  
"@version" =\> "1",  
"msg\_type" =\> "REQUEST",  
"parsed\_msg" =\> {  
"Body" =\> [  
[0] {  
"getBiginfo" =\> [  
[0] {  
"xmlns" =\> "[somewebsite.com](http://somewebsite.com)",  
"site" =\> [  
[0] "sometest"  
],  
"ownerOnly" =\> [  
[0] "true"  
],  
"login" =\> [  
[0] "in\_web"  
],  
"pwd" =\> [  
[0] "Pass123"  
],  
"userName" =\> [  
[0] "CHECK1"  
],  
"version" =\> [  
[0] "1.1"  
],  
"typeCode" =\> [  
[0] "LL\_DEFINITION"  
]  
}  
]  
}  
]  
},  
{  
"offset" =\> 12233404,  
"input\_type" =\> "log",  
"source" =\> "requests.log.8",  
"type" =\> "log",  
"datestamp" =\> "08-08-17 00:21:43.031",  
"@timestamp" =\> 2017-10-17T18:32:04.528Z,  
"@version" =\> "1",  
"msg\_type" =\> "REQUEST",  
"parsed\_msg" =\> {  
"Body" =\> [  
[0] {  
"getAllForUsr" =\> [  
[0] {  
"xmlns" =\> "[somewebsite.com](http://somewebsite.com)",  
"site" =\> [  
[0] {  
"xsi:nil" =\> "true"  
}  
],  
"login" =\> [  
[0] "in\_web"  
],  
"pwd" =\> [  
[0] "BbBbBbB"  
],  
"userName" =\> [  
[0] "UMEMA"  
],  
"version" =\> [  
[0] "1.1"  
]  
}  
]  
}  
]  
},  
"request\_id" =\> "c62a2214-e111-48a9-b9f9-0e28791eb7e2"  
}

I am able to do this :  
mutate{  
update =\> { "[parsed\_msg][Body][0]" =\> "XXXXXXXXX" }  
}

which replaces the contents of entire Body[0] with "XXXXXXX"  
But the further inside Body[0] here "getIndContents" or "GetAllForUsr" etc is a method name which can vary based on the method being called by the application so i wanna use a wildcard to traverse down to pwd field and mask it out which doesn't seem to work.  
I tried all possible combinations like Body[0][\*][0][pwd][0] but later found out that wildcards are not supported.  
How can I either remove the pwd field or update it to have XXXXXX ?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [October 19, 2017, 5:49am UTC](https://discuss.elastic.co/t/using-wildcard-to-update-remove-fields-inside-logstash-filter/104460/2 "2017-10-19T05:49:18Z")

</div>

Wildcards aren't supported. You need to use a ruby filter to traverse the datastructures with code. Examples of this have been posted in the past.

---

<div class="post-metadata">

**Author:** ![bishplz](https://avatars.discourse-cdn.com/v4/letter/b/7bcc69/32.png) [@bishplz](https://discuss.elastic.co/u/bishplz)\
**Post date:** [October 20, 2017, 5:50pm UTC](https://discuss.elastic.co/t/using-wildcard-to-update-remove-fields-inside-logstash-filter/104460/4 "2017-10-20T17:50:18Z")

</div>

I tried to look around and found out what i'm trying to achieve is possible through a recursive function.  
However i cannot find any examples on ruby functions inside a logstash config.  
Would you be able to point me to an example where it is possible to reach a node for example ("pwd") where the path to it can vary across events ?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 21, 2017, 8:34pm UTC](https://discuss.elastic.co/t/using-wildcard-to-update-remove-fields-inside-logstash-filter/104460/6 "2017-11-21T20:34:21Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.