# Global field gsub filter?

**URL:** <https://discuss.elastic.co/t/global-field-gsub-filter/83243>\
**Category:** Logstash\
**Created:** [April 21, 2017, 6:07pm UTC](https://discuss.elastic.co/t/global-field-gsub-filter/83243 "2017-04-21T18:07:23Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![\_Ian](https://avatars.discourse-cdn.com/v4/letter/_/ccd318/32.png) [@\_Ian](https://discuss.elastic.co/u/_Ian)\
**Post date:** [April 21, 2017, 6:07pm UTC](https://discuss.elastic.co/t/global-field-gsub-filter/83243/1 "2017-04-21T18:07:23Z")

</div>

Is there a way to apply a gsub action to every field during filtering?

I'm currently using dissect to split apart a message and want to do a more global replace of a null character to an actual null ES is happy with. Currently I can apply this iteratively inside a large subset of IF statemetents, but only while calling each defined field directly by name. I'd like to catch every field at once as it leaves the subset of IF dissect statements and sub out exact full field matches with a null.

I have a feeling I may end up needing ruby code to do so?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [April 24, 2017, 5:28am UTC](https://discuss.elastic.co/t/global-field-gsub-filter/83243/2 "2017-04-24T05:28:50Z")

</div>

> Is there a way to apply a gsub action to every field during filtering?

Not without a ruby filter.

---

<div class="post-metadata">

**Author:** ![\_Ian](https://avatars.discourse-cdn.com/v4/letter/_/ccd318/32.png) [@\_Ian](https://discuss.elastic.co/u/_Ian)\
**Post date:** [April 24, 2017, 2:14pm UTC](https://discuss.elastic.co/t/global-field-gsub-filter/83243/3 "2017-04-24T14:14:13Z")

</div>

I seem to be getting caught up in this 5.0 change: [https://www.elastic.co/guide/en/logstash/5.3/event-api.html](https://www.elastic.co/guide/en/logstash/5.3/event-api.html)

```
ruby { code => "event.to_hash.each { |k, v|
event[k] = v.gsub!('-', '') } "
}

```

Results in:

```
[ERROR][logstash.filters.ruby] Ruby exception occurred: Direct event field references (i.e. event['field'] = 'value') have been disabled in favor of using event get and set methods (e.g. event.set('field', 'value')). Please consult the Logstash 5.0 breaking changes documentation for more details.

```

What would be the best way to accomplish this?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [April 24, 2017, 2:48pm UTC](https://discuss.elastic.co/t/global-field-gsub-filter/83243/4 "2017-04-24T14:48:33Z")

</div>

Just replace

```
event[k] = v.gsub!('-', '')

```

with

```
event.set(k, v.gsub!('-', ''))
```

---

<div class="post-metadata">

**Author:** ![\_Ian](https://avatars.discourse-cdn.com/v4/letter/_/ccd318/32.png) [@\_Ian](https://discuss.elastic.co/u/_Ian)\
**Post date:** [April 24, 2017, 3:27pm UTC](https://discuss.elastic.co/t/global-field-gsub-filter/83243/5 "2017-04-24T15:27:41Z")

</div>

Getting undefined method gsub! now. Is there a way to iterate through each field value with event.get?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [April 24, 2017, 3:29pm UTC](https://discuss.elastic.co/t/global-field-gsub-filter/83243/6 "2017-04-24T15:29:37Z")

</div>

> Getting undefined method gsub! now.

Please always quote complete error message.

> Is there a way to iterate through each field value with event.get?

event.get returns whatever object the field contains. gsub! will only work for string values.

---

<div class="post-metadata">

**Author:** ![\_Ian](https://avatars.discourse-cdn.com/v4/letter/_/ccd318/32.png) [@\_Ian](https://discuss.elastic.co/u/_Ian)\
**Post date:** [April 24, 2017, 6:52pm UTC](https://discuss.elastic.co/t/global-field-gsub-filter/83243/7 "2017-04-24T18:52:58Z")

</div>

For anyone else looking to iterate through every field this is what we ended up using:

```
ruby {
   code => "
      hash = event.to_hash
      hash.each do |k,v|
        if v == nil or v == '' or v == '-' or v == ' '
          event.remove(k)
        end
   "}
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 22, 2017, 6:55pm UTC](https://discuss.elastic.co/t/global-field-gsub-filter/83243/8 "2017-05-22T18:55:13Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.