# Grok debugger says good but logstash doesn't match

**URL:** <https://discuss.elastic.co/t/grok-debugger-says-good-but-logstash-doesnt-match/213620>\
**Category:** Logstash\
**Created:** [January 2, 2020, 7:55pm UTC](https://discuss.elastic.co/t/grok-debugger-says-good-but-logstash-doesnt-match/213620 "2020-01-02T19:55:54Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Willl](https://avatars.discourse-cdn.com/v4/letter/w/8491ac/32.png) [@Willl](https://discuss.elastic.co/u/Willl)\
**Post date:** [January 2, 2020, 7:55pm UTC](https://discuss.elastic.co/t/grok-debugger-says-good-but-logstash-doesnt-match/213620/1 "2020-01-02T19:55:54Z")

</div>

Hi,

Running filebeat with system module on /var/log/message. I tested the following in Kibana Grok Debugger,

Log:

> Jan 2 14:42:52 dev10 sudo: pam\_unix(sudo:session): session opened for user root by root(uid=0)

with Pattern:

> %{SYSLOGBASE2} %{GREEDYDATA:message}

Debugger returns Structured data:

> {  
> "program": "sudo",  
> "logsource": "dev10",  
> "message": "pam\_unix(sudo:session): session opened for user root by root(uid=0)",  
> "timestamp": "Jan 2 14:42:52"  
> }

In Logstash, I have filter:

> input {  
> beats {  
> port =\> 5044  
> }  
> }  
> filter {  
> if [dataset] == "system.syslog" {  
> grok {  
> match =\> { "message" =\> "%{SYSLOGBASE} %{GREEDYDATA:message}" }  
> overwrite =\> ["message"]  
> }  
> }  
> }  
> output {  
> stdout { codec =\> rubydebug }  
> }

I kicked off Logstash and I don't get the message I expected, i.e. just the greedy data part.

> {  
> "event" =\> {  
> "module" =\> "system",  
> "dataset" =\> "system.syslog",  
> "timezone" =\> "-05:00"  
> },  
> "message" =\> "Jan 2 14:42:52 dev10 sudo: pam\_unix(sudo:session): session opened for user root by root(uid=0)",  
> "log" =\> {  
> "offset" =\> 3529252,  
> "file" =\> {  
> "path" =\> "/var/log/messages"  
> }  
> },  
> "fileset" =\> {  
> "name" =\> "syslog"  
> },  
> "input" =\> {  
> "type" =\> "log"  
> },  
> "ecs" =\> {  
> "version" =\> "1.1.0"  
> },  
> "@version" =\> "1",  
> "host" =\> {  
> "hostname" =\> "dev10",  
> "os" =\> {  
> "name" =\> "SLES",  
> "platform" =\> "sles",  
> "family" =\> "suse",  
> "version" =\> "12-SP4",  
> "kernel" =\> "4.12.14-94.41-default"  
> },  
> "name" =\> "dev10",  
> "architecture" =\> "x86\_64",  
> "id" =\> "d976befa1d66e6bd04192c375dcc7785",  
> "containerized" =\> false  
> },  
> "service" =\> {  
> "type" =\> "system"  
> },  
> "@timestamp" =\> 2020-01-02T19:42:54.599Z,  
> "agent" =\> {  
> "hostname" =\> "dev10",  
> "ephemeral\_id" =\> "c5e4d281-0cc8-43b2-a25a-c9831c0e9d25",  
> "version" =\> "7.5.1",  
> "type" =\> "filebeat",  
> "id" =\> "1bbe9716-9685-4270-be07-c32d0b1776c2"  
> },  
> "tags" =\> [  
> [0] "beats\_input\_codec\_plain\_applied"  
> ]  
> }

---

<div class="post-metadata">

**Author:** ![Willl](https://avatars.discourse-cdn.com/v4/letter/w/8491ac/32.png) [@Willl](https://discuss.elastic.co/u/Willl)\
**Post date:** [January 2, 2020, 8:50pm UTC](https://discuss.elastic.co/t/grok-debugger-says-good-but-logstash-doesnt-match/213620/2 "2020-01-02T20:50:45Z")

</div>

> [@Willl](#):
>
> overwrite =\> ["message"]

Removed the "overwrite" and the output looks better. Now when I go to Kibana/syslog dashboard. I get shards failed.

"reason": {  
"type": "illegal\_argument\_exception",  
"reason": "Fielddata is disabled on text fields by default. Set fielddata=true on [host.hostname] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."  
}

---

<div class="post-metadata">

**Author:** ![Willl](https://avatars.discourse-cdn.com/v4/letter/w/8491ac/32.png) [@Willl](https://discuss.elastic.co/u/Willl)\
**Post date:** [January 3, 2020, 4:33pm UTC](https://discuss.elastic.co/t/grok-debugger-says-good-but-logstash-doesnt-match/213620/3 "2020-01-03T16:33:02Z")

</div>

I sorted this out. Thanks.  
I ran curl to enabled field data.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 31, 2020, 4:33pm UTC](https://discuss.elastic.co/t/grok-debugger-says-good-but-logstash-doesnt-match/213620/4 "2020-01-31T16:33:05Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.