# Logstash goes wrong after add filter

**URL:** <https://discuss.elastic.co/t/logstash-goes-wrong-after-add-filter/278730>\
**Category:** Logstash\
**Created:** [July 15, 2021, 3:01am UTC](https://discuss.elastic.co/t/logstash-goes-wrong-after-add-filter/278730 "2021-07-15T03:01:56Z")\
**Posts on this page:** 8\
**Page:** 2

<div class="post-metadata">

**Author:** ![quyennguyen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/quyennguyen/32/91626_2.png) [@quyennguyen](https://discuss.elastic.co/u/quyennguyen)\
**Post date:** [July 16, 2021, 10:52am UTC](https://discuss.elastic.co/t/logstash-goes-wrong-after-add-filter/278730/21 "2021-07-16T10:52:36Z")

</div>

I change my filter again  
Update 10-syslog-filter.conf

```auto
filter {
  
    if [fileset][name] == "auth" {
      grok {
        match => { "message" => ["%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostname]} sshd(?:\[%{POSINT:[system][auth][pid]}\])?: %{DATA:[system][auth][ssh][event]} %{DATA:[system][auth][ssh][method]} for (invalid user )?%{DATA:[system][auth][user]} from %{IPORHOST:[system][auth][ssh][ip]} port %{NUMBER:[system][auth][ssh][port]} ssh2(: %{GREEDYDATA:[system][auth][ssh][signature]})?",
               "%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostname]} sshd(?:\[%{POSINT:[system][auth][pid]}\])?: %{DATA:[system][auth][ssh][event]} user %{DATA:[system][auth][user]} from %{IPORHOST:[system][auth][ssh][ip]}",
               "%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostname]} sshd(?:\[%{POSINT:[system][auth][pid]}\])?: Did not receive identification string from %{IPORHOST:[system][auth][ssh][dropped_ip]}",
               "%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostname]} sudo(?:\[%{POSINT:[system][auth][pid]}\])?: \s*%{DATA:[system][auth][user]} :( %{DATA:[system][auth][sudo][error]} ;)? TTY=%{DATA:[system][auth][sudo][tty]} ; PWD=%{DATA:[system][auth][sudo][pwd]} ; USER=%{DATA:[system][auth][sudo][user]} ; COMMAND=%{GREEDYDATA:[system][auth][sudo][command]}",
               "%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostname]} groupadd(?:\[%{POSINT:[system][auth][pid]}\])?: new group: name=%{DATA:system.auth.groupadd.name}, GID=%{NUMBER:system.auth.groupadd.gid}",
               "%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostname]} useradd(?:\[%{POSINT:[system][auth][pid]}\])?: new user: name=%{DATA:[system][auth][user][add][name]}, UID=%{NUMBER:[system][auth][user][add][uid]}, GID=%{NUMBER:[system][auth][user][add][gid]}, home=%{DATA:[system][auth][user][add][home]}, shell=%{DATA:[system][auth][user][add][shell]}$"]}

        add_field => ["activity", "SSH Login"]
	add_tag => ["linux_auth"]
        remove_field => "message"
      }
      date {
        match => ["[system][auth][timestamp]", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
      }
      geoip {
        source => "[system][auth][ssh][ip]"
        target => "[system][auth][ssh][geoip]"
      }
    }
    else if [fileset][name] == "syslog" {
      grok {
	      match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
	      add_field => ["received_at", "%{@timestamp}"]
	      add_field => ["received_from", "%{host}"]
       }
      date {
      match => ["syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"]
      }
    }
  
}

```

---

<div class="post-metadata">

**Author:** ![rojin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rojin/32/97697_2.png) [@rojin](https://discuss.elastic.co/u/rojin)\
**Post date:** [July 16, 2021, 10:55am UTC](https://discuss.elastic.co/t/logstash-goes-wrong-after-add-filter/278730/22 "2021-07-16T10:55:40Z")

</div>

I saw the custom pattern defined in the link you provided:

```auto
pattern_definitions => { "GREEDYMULTILINE" => "(.|\n)*" }

```

I saw the Error at line 423 of the file link (logstash-logs.txt) you provided.

---

<div class="post-metadata">

**Author:** ![quyennguyen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/quyennguyen/32/91626_2.png) [@quyennguyen](https://discuss.elastic.co/u/quyennguyen)\
**Post date:** [July 16, 2021, 11:13am UTC](https://discuss.elastic.co/t/logstash-goes-wrong-after-add-filter/278730/23 "2021-07-16T11:13:27Z")

</div>

The `pattern_definitions` didn't work at all, and I'm tired of that for 2 months, so I delete it.

And the error isn't my log (time isn't correct), you mistake with someone.

Have you seen my new filter file? I just remove it and everything worked fine!

---

<div class="post-metadata">

**Author:** ![rojin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rojin/32/97697_2.png) [@rojin](https://discuss.elastic.co/u/rojin)\
**Post date:** [July 16, 2021, 11:21am UTC](https://discuss.elastic.co/t/logstash-goes-wrong-after-add-filter/278730/24 "2021-07-16T11:21:11Z")

</div>

That's weird... I tested the link again and it gave me the file I mentioned! Well, anyway glad to hear it worked!

---

<div class="post-metadata">

**Author:** ![quyennguyen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/quyennguyen/32/91626_2.png) [@quyennguyen](https://discuss.elastic.co/u/quyennguyen)\
**Post date:** [July 16, 2021, 12:12pm UTC](https://discuss.elastic.co/t/logstash-goes-wrong-after-add-filter/278730/25 "2021-07-16T12:12:06Z")

</div>

Thanks a lot for helping me! 🥰

---

<div class="post-metadata">

**Author:** ![rojin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rojin/32/97697_2.png) [@rojin](https://discuss.elastic.co/u/rojin)\
**Post date:** [July 16, 2021, 12:18pm UTC](https://discuss.elastic.co/t/logstash-goes-wrong-after-add-filter/278730/26 "2021-07-16T12:18:14Z")

</div>

You're welcome! 😃

---

<div class="post-metadata">

**Author:** ![mutt13y](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mutt13y/32/74962_2.png) [@mutt13y](https://discuss.elastic.co/u/mutt13y)\
**Post date:** [July 21, 2021, 10:20am UTC](https://discuss.elastic.co/t/logstash-goes-wrong-after-add-filter/278730/27 "2021-07-21T10:20:07Z")

</div>

As an addendum. Instead of GREEDYMULTILINE you could add the (m?) flag to your pattern. This causes dot and \s To match newline and so DATA / GREEDYDATA  
would work

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 18, 2021, 10:20am UTC](https://discuss.elastic.co/t/logstash-goes-wrong-after-add-filter/278730/28 "2021-08-18T10:20:18Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

[Previous page](https://discuss.elastic.co/t/logstash-goes-wrong-after-add-filter/278730.md?page=1)
