# \[Solved\] Logstash pattern issue

**URL:** <https://discuss.elastic.co/t/solved-logstash-pattern-issue/77772>\
**Category:** Logstash\
**Created:** [March 8, 2017, 6:57am UTC](https://discuss.elastic.co/t/solved-logstash-pattern-issue/77772 "2017-03-08T06:57:40Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![raamee](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/raamee/32/5110_2.png) [@raamee](https://discuss.elastic.co/u/raamee)\
**Post date:** [March 8, 2017, 6:57am UTC](https://discuss.elastic.co/t/solved-logstash-pattern-issue/77772/1 "2017-03-08T06:57:40Z")

</div>

Hi,

I came across a weird issue.

I made a grok pattern that works well and matches my log in [http://grokdebug.herokuapp.com](http://grokdebug.herokuapp.com), but when I use the same pattern in my logstash, its not working properly. I get a field mismatch. Please check below:

This is the log:

`Tue Mar 7 12:23:41 2017 : Auth: (49087) Login incorrect (eap_peap: TLS Alert write:fatal:decode error): [johndoe] (from client WIFI-Control-Dev port 13 cli bb-41-e2-1c-12-12)`

Grok pattern:

> (%{NOTSPACE:removed}) %{DATA:AUTHWAY} (?:(%{DATA:removed}))?: (?:[%{DATA:username}]) (from client %{NOTSPACE:radius\_client} port %{INT:radius\_port}(?: cli %{NOTSPACE:radius\_cli})?(?: via %{DATA:radius\_via})?)

In Grok debugger, its working but not in my server. In my server, the logstash config read as:

> filter {  
> if [type] == "radius-auth" {  
> grok {  
> patterns\_dir =\> "/etc/logstash/patterns/"  
> match =\> ["message", "%{FREERADIUS\_LINE}"]  
> }  
> if [fr\_logclass] == "Auth" {  
> grok {  
> patterns\_dir =\> "/etc/logstash/patterns/"  
> match =\> ["fr\_message", "%{FREERADIUS\_AUTH}"]  
> }  
> }  
> date {  
> match =\> [ "timestamp" , "EEE MMM dd HH:mm:ss YYYY",  
> "EEE MMM d HH:mm:ss YYYY"]  
> }  
> }

/etc/logstash/patterns/radius-auth.pattern contains

```
FREERADIUS_DATE %{DAY} %{MONTH} ?%{MONTHDAY} %{TIME} %{YEAR}
FREERADIUS_LOGTYPE Auth|Info|Error|Proxy
FREERADIUS_CLIENT %{NOTSPACE:radius_client}
FREERADIUS_PORT %{INT:radius_port}
FREERADIUS_CLI %{NOTSPACE:radius_cli}
FREERADIUS_VIA %{DATA:radius_via}
FREERADIUS_FROM \(from client %{FREERADIUS_CLIENT} port %{FREERADIUS_PORT}(?: cli %{FREERADIUS_CLI})?(?: via %{FREERADIUS_VIA})?\)
FREERADIUS_USERNAME %{DATA:username}
FREERADIUS_REASON (?:\(%{DATA:radius_reason}\))?:
FREERADIUS_MODULE \brlm_[a-z]+\b

# Auth log lines
FREERADIUS_LOGINOK \(%{NOTSPACE:removed}\) %{GREEDYDATA:AUTHWAY}: \[%{FREERADIUS_USERNAME}\] %{FREERADIUS_FROM}
FREERADIUS_LOGININCORRECT \(%{NOTSPACE:removed}\) %{DATA:AUTHWAY} (?:\(%{DATA:radius_reason}\))?: (?:\[%{DATA:username}\]) \(from client %{NOTSPACE:radius_client} port %{INT:radius_port}(?: cli %{NOTSPACE:radius_cli})?(?: via %{DATA:radius_via})?\)
FREERADIUS_INVALIDUSER \(%{NOTSPACE:removed}\) %{GREEDYDATA:AUTHWAY}: \[%{FREERADIUS_USERNAME}\] %{FREERADIUS_FROM}
FREERADIUS_AUTH (?:%{FREERADIUS_LOGINOK}|%{FREERADIUS_LOGININCORRECT}|%{FREERADIUS_INVALIDUSER})

# Main match on whole log line:
FREERADIUS_LINE %{FREERADIUS_DATE:timestamp} : %{FREERADIUS_LOGTYPE:fr_logclass}: +%{GREEDYDATA:fr_message}

```

But in my server, the radius\_reason field is not getting separated and its merging with AUTHWAY field.

**Ruby debug:**

```
{
       "fr_message" => "(49087) Login incorrect (eap_peap: TLS Alert write:fatal:decode error): [johndoe] (from client WIFI-Control-Dev port 13 cli bb-41-e2-1c-12-12)",
           "offset" => 185,
          "AUTHWAY" => "Login incorrect (eap_peap: TLS Alert write:fatal:decode error)",
       "input_type" => "log",
      "radius_port" => "13",
           "source" => "/var/log/radius/radius.log",
          "message" => "Tue Mar 7 12:23:41 2017 : Auth: (49087) Login incorrect (eap_peap: TLS Alert write:fatal:decode error): [johndoe] (from client WIFI-Control-Dev port 13 cli bb-41-e2-1c-12-12)",
             "type" => "radius-auth",
             "tags" => [
        [0] "beats_input_codec_plain_applied"
    ],
       "@timestamp" => 2017-03-07T12:23:41.000Z,
      "fr_logclass" => "Auth",
          "removed" => "49087",
    "radius_client" => "WIFI-Control-Dev",
         "@version" => "1",
             "beat" => {
        "hostname" => "localhost.localdomain",
            "name" => "localhost.localdomain",
         "version" => "5.1.1"
    },
             "host" => "localhost.localdomain",
       "radius_cli" => "bb-41-e2-1c-12-12",
        "timestamp" => "Tue Mar 7 12:23:41 2017",
         "username" => "johndoe"
}

```

The AUTHWAY field should only contain "Login Incorrect" . But it is also accompanied by "(eap\_peap: TLS Alert write:fatal:decode error)" which should be in separate field as "radius\_reason". In online grok debugger, both displayed as separate field, but not in my server.

Any idea how to resolve?

Thank you.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [March 9, 2017, 7:00pm UTC](https://discuss.elastic.co/t/solved-logstash-pattern-issue/77772/2 "2017-03-09T19:00:17Z")

</div>

I don't have time to dig into all the details, but I strongly suggest that you reduce your use of GREEDYDATA and DATA, especially since you have optional matches (the `?` operator). Try to make more exact expressions.

---

<div class="post-metadata">

**Author:** ![raamee](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/raamee/32/5110_2.png) [@raamee](https://discuss.elastic.co/u/raamee)\
**Post date:** [March 10, 2017, 4:28am UTC](https://discuss.elastic.co/t/solved-logstash-pattern-issue/77772/3 "2017-03-10T04:28:38Z")

</div>

Ok, thank you . I will check further. I'm with this issue since 5 days 😉 and couldn't figure out why.

---

<div class="post-metadata">

**Author:** ![raamee](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/raamee/32/5110_2.png) [@raamee](https://discuss.elastic.co/u/raamee)\
**Post date:** [March 15, 2017, 6:51am UTC](https://discuss.elastic.co/t/solved-logstash-pattern-issue/77772/4 "2017-03-15T06:51:07Z")

</div>

So after thinking further, I found a way to make it work.

This is the grok pattern I used and it works both in grok debugger and in server.

> (%{DATA:removed}) %{DATA:AUTHWAY}(?::)? (?:((.\*?)):)?%{SPACE}[%{DATA:username}] (from client %{NOTSPACE:radius\_client} port %{INT:radius\_port}(?: cli %{NOTSPACE:radius\_cli})?(?: via %{DATA:radius\_via})?)

Here I used `(?::)? (?:\((.*?)\):)?%{SPACE}` instead of `(?:\(%{DATA:radius_reason}\))?:` which was in the grok pattern earlier I posted in first post.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 12, 2017, 6:51am UTC](https://discuss.elastic.co/t/solved-logstash-pattern-issue/77772/5 "2017-04-12T06:51:20Z")

</div>

Closed on April 12, 2017, 6:51am UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.