# Drop field based on pattern

**URL:** <https://discuss.elastic.co/t/drop-field-based-on-pattern/276443>\
**Category:** Logstash\
**Created:** [June 19, 2021, 9:57pm UTC](https://discuss.elastic.co/t/drop-field-based-on-pattern/276443 "2021-06-19T21:57:10Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![mohsin106](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mohsin106/32/65203_2.png) [@mohsin106](https://discuss.elastic.co/u/mohsin106)\
**Post date:** [June 19, 2021, 9:57pm UTC](https://discuss.elastic.co/t/drop-field-based-on-pattern/276443/1 "2021-06-19T21:57:10Z")

</div>

Hi, I have some field names that begin with the string "sub" that I want Logstash to drop.

Can I do something like:

```auto
if [sub*] {
    drop {}
}

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 19, 2021, 9:58pm UTC](https://discuss.elastic.co/t/drop-field-based-on-pattern/276443/2 "2021-06-19T21:58:22Z")

</div>

Use a prune filter with the blacklist\_names option.

---

<div class="post-metadata">

**Author:** ![mohsin106](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mohsin106/32/65203_2.png) [@mohsin106](https://discuss.elastic.co/u/mohsin106)\
**Post date:** [June 22, 2021, 9:25pm UTC](https://discuss.elastic.co/t/drop-field-based-on-pattern/276443/3 "2021-06-22T21:25:21Z")

</div>

I tried doing the following to drop the "last-clear" field and any field that is blank:

```auto
prune {
    blacklist_names => ["last-clear", " "]
}

```

The "last-clear" has dropped, but I keep getting this WARN message:

> [WARN] 2021-06-22 20:58:20.397 [[qa-mx-oc]\>worker0] elasticsearch - Could not index event to Elasticsearch. {:status=\>400, :action=\>["index", {:\_id=\>nil, :\_index=\>"qa-mx-2021.06.22", :routing=\>nil, :\_type=\>"\_doc"}, #LogStash::Event:0x2d94acac], :response=\>{"index"=\>{"\_index"=\>"qa-mx-2021.06.22", "\_type"=\>"\_doc", "\_id"=\>"m6WENXoBVd9j9yI8SwBf", "status"=\>400, "error"=\>{"type"=\>"mapper\_parsing\_exception", "reason"=\>"failed to parse", "caused\_by"=\>{"type"=\>"illegal\_argument\_exception", "reason"=\>"field name cannot be an empty string"}}}}}

I tried enabling debug but can't correlate this error message to the debug messages. I tried grepping the \_id value from the debug logs but didn't get anything.

How can I find out what field is causing the WARN message?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 22, 2021, 9:49pm UTC](https://discuss.elastic.co/t/drop-field-based-on-pattern/276443/4 "2021-06-22T21:49:51Z")

</div>

`""` is a valid field name in JSON, but a lot of things object to empty names. It looks like elasticsearch is one of them.

```
prune { blacklist_names => [""] }

```

does not work. I think it is deleting every field on the event and then the empty event is discarded.

```
mutate { remove_field => [""] }

```

does work.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 20, 2021, 9:50pm UTC](https://discuss.elastic.co/t/drop-field-based-on-pattern/276443/5 "2021-07-20T21:50:31Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.