# Logstash xml filter ruby cleanup code help
**URL:** https://discuss.elastic.co/t/logstash-xml-filter-ruby-cleanup-code-help/279099
**Category:** Logstash
**Created:** [July 19, 2021, 5:48pm UTC](https://discuss.elastic.co/t/logstash-xml-filter-ruby-cleanup-code-help/279099 "2021-07-19T17:48:49Z")
**Posts on this page:** 5
**Page:** 1
### Author:  [@grants](https://discuss.elastic.co/u/grants)
#### Post date: [July 19, 2021, 5:48pm UTC](https://discuss.elastic.co/t/logstash-xml-filter-ruby-cleanup-code-help/279099/1 "2021-07-19T17:48:49Z")
I'm using the xml logstash filter to parse xml values into json, but I'm having issues indexing them into Elasticsearch due to an issue where a null string gets parsed into an empty object.
Source String Snip:
```auto
%localappdata%\Microsoft\OneDrive\OneDriveStandaloneUpdater.exe
```
Argument here is a string when populated but the xml filter plugin treats this an an empty object. When sending this data to elastic search the `ignore_malformed` index mapping setting does not work to ignore objects when the type expected is a string / keyword.
My current work around is to strip out nested empty objects from my parsed xml with some ruby code. I can't really figure out how to make this a function that can be called to loop through all the key value pairs in the hash so if anyone has some advice on how to better write this code, it would be greatly appreciated.
Ruby Code:
```ruby
xml = event.get("[winlog][event_data][TaskContentXml]")
key_path = "[winlog][event_data][TaskContentXml]"
xml.each do |key,value|
key_path_1 = key_path + "[" + key + "]"
## loop 1
if !value.nil? && value.is_a?(Hash)
if value.empty?
logger.warn("TaskContentXml1: Empty Hash value at key: " + key_path_1)
event.remove(key_path_1)
else
## loop 2
value.each do |key,value|
key_path_2 = key_path_1 + "[" + key + "]"
if !value.nil? && value.is_a?(Hash)
if value.empty?
logger.warn("TaskContentXml2: Empty Hash value at key: " + key_path_2)
event.remove(key_path_2)
else
## loop 3
value.each do |key,value|
key_path_3 = key_path_2 + "[" + key + "]"
if !value.nil? && value.is_a?(Hash)
if value.empty?
logger.warn("TaskContentXml3: Empty Hash value at key: " + key_path_3)
event.remove(key_path_3)
else
## loop 4
value.each do |key,value|
key_path_4 = key_path_3 + "[" + key + "]"
if !value.nil? && value.is_a?(Hash)
if value.empty?
logger.warn("TaskContentXml4: Empty Hash value at key: " + key_path_4)
event.remove(key_path_4)
else
## loop again
end
end
end
end
end
end
end
end
end
end
end
end
```
Related issue: [Invalid mapping case not handled by index.mapping.ignore\_malformed · Issue #12366 · elastic/elasticsearch · GitHub](https://github.com/elastic/elasticsearch/issues/12366)
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [July 19, 2021, 10:50pm UTC](https://discuss.elastic.co/t/logstash-xml-filter-ruby-cleanup-code-help/279099/2 "2021-07-19T22:50:30Z")
Take a look at [this](https://discuss.elastic.co/t/how-to-exclude-xml-json-key-value-if-key-length-is-greater-than-15-char-and-value-length-is-greater-than-100-char/270248/8) which recursively descends into hashes and arrays in the parsed XML and deletes items that match a condition (in that case based on the length of the element name, but you can change that to be based on the value).
---
### Author:  [@grants](https://discuss.elastic.co/u/grants)
#### Post date: [July 20, 2021, 1:15pm UTC](https://discuss.elastic.co/t/logstash-xml-filter-ruby-cleanup-code-help/279099/3 "2021-07-20T13:15:40Z")
Thanks for the reply.
This looks really close to what I'd need, but I'm not too familiar with ruby so when I see this code block:
> event.to\_hash.each { |k, v|
> removeBigThings(v, "[#{k}]", event)
> }
I don't really know what this does. I know `k, v` are just the key value pairs on the object that are going to be iterated over. but why is `event` in there and also why are you passing `[#{k}]` into the function vs just `k` ?
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [July 20, 2021, 5:16pm UTC](https://discuss.elastic.co/t/logstash-xml-filter-ruby-cleanup-code-help/279099/4 "2021-07-20T17:16:48Z")
> [@grants](#):
>
> why is `event` in there and also why are you passing `[#{k}]` into the function vs just `k` ?
event is there because inside the function I call event.remove. If it were not passed to the function then event would not be in scope.
I use `"[#{k}]"` so that a field called "foo" results in the value `"[foo]"`. Inside the function I use `"#{name}[#{k}]"`. If [foo] is a hash with a field called "bar" this will result in the recursive call being passed `"[foo][bar]"`
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 17, 2021, 5:17pm UTC](https://discuss.elastic.co/t/logstash-xml-filter-ruby-cleanup-code-help/279099/5 "2021-08-17T17:17:13Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.