# Logstash blocked when using Regex filter

**URL:** <https://discuss.elastic.co/t/logstash-blocked-when-using-regex-filter/299313>\
**Category:** Logstash\
**Created:** [March 10, 2022, 9:39am UTC](https://discuss.elastic.co/t/logstash-blocked-when-using-regex-filter/299313 "2022-03-10T09:39:50Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![valm-dili13](https://avatars.discourse-cdn.com/v4/letter/v/a9a28c/32.png) [@valm-dili13](https://discuss.elastic.co/u/valm-dili13)\
**Post date:** [March 10, 2022, 9:39am UTC](https://discuss.elastic.co/t/logstash-blocked-when-using-regex-filter/299313/1 "2022-03-10T09:39:50Z")

</div>

Dear community,

When I add a specific filter in my pipeline, few Kb of data are processed then everything is blocked. (No more output when debugging logstash).

When I check my beats logs, it says that the remote port is closed :

```auto
2022-03-08T14:01:27.848+0100	ERROR	[publisher_pipeline_output]	pipeline/output.go:154	Failed to connect to backoff(async(tcp://server1:5044)): dial tcp 10.X.X.X:5044: connectex: No connection could be made because the target machine actively refused it.

```

Here are the problematic filters :

```auto
 filter {
      if [log][file][path] =~ /^E:.*Dirianance.*_Logs_hrep.*\.log$/ {
          mutate {
              add_field => {
                  "classification" => "debug"
                  "service" => "guiv"
                  "es_index" => "guiv-%{+yyyy.MM.dd}"
              }
          }
      }
  }

```

&

```auto
filter {
    if [log][file][path] =~ /^C:.*bpubnet.*logs.*LogFiles.*ZXYS4.*\.log$/ {
        mutate {
            add_field => {
                "classification" => "security"
                "service" => "ffz"
                "es_index" => "ffz-%{+yyyy.MM.dd}"
            }
        }
    }
}

```

And here is the ouput :

```auto
output {
    elasticsearch {
        hosts => ["https://server1:9200"]
        ssl => true
        ssl_certificate_verification => true
        cacert => "/etc/logstash/certs/elasticsearch-ca.pem"
        user => "dt_logstash_writer"
        password => "XXXXXXXXXXXXXXXXX"
        ilm_enabled => "false"
        manage_template => false
        index => "%{es_index}"
        ecs_compatibility => "disabled"
    }
}

```

I first thought it was a queue problem and try to change the value of `queue.max_bytes` from 1gb to 64mo. But it did not fix the problem.

I've found a simple fix which consist of setting these variables directly from the beats config, but I want to understand why the Regex are blocking the software.

Any Idea why these regex blocks logstash when trying to process inputs ?

Thanks for your help.

Best regards,

Valentin Magnan

---

<div class="post-metadata">

**Author:** ![valm-dili13](https://avatars.discourse-cdn.com/v4/letter/v/a9a28c/32.png) [@valm-dili13](https://discuss.elastic.co/u/valm-dili13)\
**Post date:** [March 10, 2022, 10:18am UTC](https://discuss.elastic.co/t/logstash-blocked-when-using-regex-filter/299313/2 "2022-03-10T10:18:44Z")

</div>

Actually, I think it is caused by my variable "es\_index".

Pretty sure about it since this filter also blocks logstash :

```auto
 filter {
  2 if [service] == "ffz" {
  3 mutate {
  4 add_field => {
  5 "es_index" => "ffz-%{+yyyy.MM.dd}"
  6 }
  7 }
  8 }
  9 }

```

But it does not make sense for me because since it is a simple text variable in my index-pattern and when I create the field, it is also text.

---

<div class="post-metadata">

**Author:** ![valm-dili13](https://avatars.discourse-cdn.com/v4/letter/v/a9a28c/32.png) [@valm-dili13](https://discuss.elastic.co/u/valm-dili13)\
**Post date:** [March 17, 2022, 5:55pm UTC](https://discuss.elastic.co/t/logstash-blocked-when-using-regex-filter/299313/3 "2022-03-17T17:55:45Z")

</div>

Problem was the index used was not existing in Elasticsearch.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 14, 2022, 5:56pm UTC](https://discuss.elastic.co/t/logstash-blocked-when-using-regex-filter/299313/4 "2022-04-14T17:56:15Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.