# Remove wildcard fields in logstash

**URL:** https://discuss.elastic.co/t/remove-wildcard-fields-in-logstash/248173
**Category:** Logstash
**Created:** [September 10, 2020, 1:22pm UTC](https://discuss.elastic.co/t/remove-wildcard-fields-in-logstash/248173 "2020-09-10T13:22:22Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![Yanick\_Quirion](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yanick_quirion/32/58795_2.png) [@Yanick\_Quirion](https://discuss.elastic.co/u/Yanick_Quirion)
#### Post date: [September 10, 2020, 1:22pm UTC](https://discuss.elastic.co/t/remove-wildcard-fields-in-logstash/248173/1 "2020-09-10T13:22:23Z")

</div>

Hello,

I have this data:

```auto
"winlog" => {
    "channel" => "Security",
    "record_id" => 1401770357,
    "event_data" => {
        "param14" => "-",
        "param9" => "-",
        "param21" => "-",
        "param13" => "-",
        "param19" => "-",
        "param15" => "-",
        "param3" => "coded\",\"Accept\":\"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8\",\"Accept-Encoding\":\"gzip, deflate, br\",\"Accept-Language\":\"fr,fr-FR;q=0.8,en-US;q=0.5,en;q=0.3\",\"Cookie\":\"_ga=GA1.2.1774089451.1521394333; _fbp=fb.1.1551118745254.676256511; _fbc=fb.1.1592585214629.IwAR0dWXKxMpI_oreMHAmH__1b7QWeoVc-Uwob6Bo_dLQN2ohFNUsC6IgrzuA; _hjid=a468c35d-cc62-463c-9768-f6a0b16ba200; ",
        "param18" => "-",
        "param16" => "-",
        "param12" => "-",
        "param5" => "-",
        "param4" => "_gcl_au=1.1.2121093392.1596402799\",\"Expect\":\"100-continue\",\"Host\":\"authentification.usherbrooke.ca\",\"Referer\":\"https://cas.usherbrooke.ca/login\",\"User-Agent\":\"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:80.0) Gecko/20100101 Firefox/80.0\"}",
        "param17" => "-",
        "param8" => "-",
        "param10" => "-",
        "param6" => "-",
        "param11" => "-",
        "param20" => "-",
        "param2" => "{\"Origin\":\"https://cas.usherbrooke.ca\",\"Upgrade-Insecure-Requests\":\"1\",\"X-Forwarded-For\":\"70.83.116.71\",\"X-MS-Forwarded-Client-IP\":\"70.83.116.71,132.210.7.250\",\"X-MS-ADFS-Proxy-Client-IP\":\"132.210.7.250\",\"client-request-id\":\"2225ca06-d83e-4867-52e8-018000080032\",\"X-MS-Proxy\":\"adfsproxy41\",\"X-MS-Endpoint-Absolute-Path\":\"/adfs/ls/\",\"Content-Length\":\"8304\",\"Content-Type\":\"application/x-www-form-urlen",
        "param7" => "-"
     }

```

I want to remove all field ending by param1, param2, etc.. (param\d+).

I'm not very good with ruby code. I've found this code on this side:

```auto
ruby {
    code => '
        event.get("winlog").each { |k, v|
            if v.is_a? Hash and v.key? "param1"
                event.remove("[winlog][#{k}][param1]")
            end
        }
    '
  }

```

I can remove "param1" but I would like to remove all field ending param\d+.

The code I found comes from this article: [Wildcards in logstash remove\_field](https://discuss.elastic.co/t/wildcards-in-logstash-remove-field/241644/7)

Can someone can help me out with this ruby problem?

Thank you all and best regards,  
Yanick

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [September 10, 2020, 2:54pm UTC](https://discuss.elastic.co/t/remove-wildcard-fields-in-logstash/248173/2 "2020-09-10T14:54:51Z")

</div>

Why not just remove the [winlog][event\_data] field, which you could do using a mutate filter?

---

<div class="post-metadata">

### Author: ![Yanick\_Quirion](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yanick_quirion/32/58795_2.png) [@Yanick\_Quirion](https://discuss.elastic.co/u/Yanick_Quirion)
#### Post date: [September 10, 2020, 3:05pm UTC](https://discuss.elastic.co/t/remove-wildcard-fields-in-logstash/248173/3 "2020-09-10T15:05:56Z")

</div>

Hi Badger,

That's a good idea, but sometimes there are some pertinent information such as [winlog][event\_data][hostname]. So I can't remove this entire branch, just the [winlog][event\_data][param\d+].

Thank you for you answer and best regards,  
Yanick

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [September 10, 2020, 3:44pm UTC](https://discuss.elastic.co/t/remove-wildcard-fields-in-logstash/248173/4 "2020-09-10T15:44:05Z")

</div>

OK, so try something like this (which I have not tested)

```
ruby {
    code => '
        winlog = event.get("winlog")
        if winlog.is_a?(Hash) and winlog.key?("event_data")
            if winlog["event_data"].is_a?(Hash)
                winlog["event_data"].each { k, v }
                if /^param\d+/.match(k)
                    event.remove("[winlog][event_data][#{k}]")
                end
            end
        end
    '
}
```

---

<div class="post-metadata">

### Author: ![Yanick\_Quirion](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yanick_quirion/32/58795_2.png) [@Yanick\_Quirion](https://discuss.elastic.co/u/Yanick_Quirion)
#### Post date: [September 10, 2020, 5:40pm UTC](https://discuss.elastic.co/t/remove-wildcard-fields-in-logstash/248173/5 "2020-09-10T17:40:10Z")

</div>

Hi Badger!

Thank you so much for your help. After doing a slight change, it worked like a charm!

I would like to be smart as you!! 🙂

Here is the final code:

```auto
  ruby {
    code => '
      winlog = event.get("winlog")
        if winlog.is_a?(Hash) and winlog.key?("event_data")
          if winlog["event_data"].is_a?(Hash)
            winlog["event_data"].each { |k, v| 
              if /^param\d+/.match(k)
                event.remove("[winlog][event_data][#{k}]")
              end
            }
          end
      end
    '
  }

```

Thank you so much! If you have a good place to learn ruby coding into logstash, I will be happy to know it!

Thanks again and best regards!

Yanick

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [October 8, 2020, 5:40pm UTC](https://discuss.elastic.co/t/remove-wildcard-fields-in-logstash/248173/6 "2020-10-08T17:40:17Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
