# Need logstash grok help
**URL:** https://discuss.elastic.co/t/need-logstash-grok-help/113268
**Category:** Logstash
**Created:** [December 27, 2017, 6:25am UTC](https://discuss.elastic.co/t/need-logstash-grok-help/113268 "2017-12-27T06:25:41Z")
**Posts on this page:** 7
**Page:** 1
### Author:  [@subin](https://discuss.elastic.co/u/subin)
#### Post date: [December 27, 2017, 6:25am UTC](https://discuss.elastic.co/t/need-logstash-grok-help/113268/1 "2017-12-27T06:25:41Z")
Hi,
My pattern works well in grok-debugger, but fails in Logstash's pipeline. I've been trying for hours; no luck. Here are the logs that I'm trying to parse:
1. `Nov 10 20:06:05 mx1 postfix/qmgr[4016]: D134620D03: from=, size=6638, nrcpt=1 (queue active)`
2. `Nov 10 20:05:49 mx2 postfix/qmgr[4016]: 64FBB20D03: from=<>, size=5733, nrcpt=1 (queue active)`
3. `Nov 10 20:06:05 mx1 postfix/qmgr[4016]: 38A6B2090E: removed`
EIther of the filters work for all three log-patterns, in grok-debugger:
`POST_QMGR %{SYSLOGBASE} %{QUEUEID:qid}: (?:removed|from=<(?:%{FROM_EMAIL:from})?>(?:, size=%{NUMBER:size}, nrcpt=%{NUMBER:nrcpt} \(%{DATA:queuestatus}\))?)`
`POST_QMGR %{SYSLOGBASE} %{QUEUEID:qid}: (removed|from=<(%{FROM_EMAIL:from})?>(, size=%{NUMBER:size}, nrcpt=%{NUMBER:nrcpt} \(%{DATA:queuestatus}\))?)`
where:
`FROM_EMAIL %{USERNAME:[from][emailname]}@%{HOSTNAME:[from][domain]}`
`QUEUEID (?:[A-F0-9]+|NOQUEUE)`
But, neither of them work for log no.1, in the pipeline. What modifications should be done?
Thanks.
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [December 27, 2017, 7:44am UTC](https://discuss.elastic.co/t/need-logstash-grok-help/113268/2 "2017-12-27T07:44:39Z")
Please show your entire config file.
---
### Author:  [@subin](https://discuss.elastic.co/u/subin)
#### Post date: [December 27, 2017, 7:58am UTC](https://discuss.elastic.co/t/need-logstash-grok-help/113268/3 "2017-12-27T07:58:44Z")
Please note that I've taken off %{SYSLOGBASE} from the aforementioned POST\_QMGR syntax. So, now, my config file is:
> input {
> stdin {}
> }
>
> filter {
>
> `if [message] =~ /postfix\/qmgr\[\d+\]:/` {
> grok {
> patterns\_dir =\> "/usr/share/logstash/patterns"
> `match => { "message" => "%{SYSLOGBASE} %{POST_QMGR}" }`
> tag\_on\_failure =\> ["\_grok\_qmgr\_fail"]
> add\_tag =\> ["\_grok\_postfix\_success"]
> }
> }
>
> else {
> mutate {
> add\_tag =\> ["\_no\_match"]
> }
> }
>
> }
>
> output {
> elasticsearch {
> hosts =\> ["elk00.vpn:9200", "elk01.vpn:9200", "elk02.vpn:9200"]
> user =\> "logstash\_logs"
> password =\> "logstash\_logs."
> ssl =\> true
> cacert =\> "/etc/logstash/ca.pem"
> index =\> "logstash-postfix-%{+YYYY.MM.dd.HH.mm}"
> }
> stdout { codec =\> rubydebug }
This fails only for log no.1
Thanks.
---
### Author:  [@subin](https://discuss.elastic.co/u/subin)
#### Post date: [December 27, 2017, 10:19am UTC](https://discuss.elastic.co/t/need-logstash-grok-help/113268/4 "2017-12-27T10:19:06Z")
I've found the pipeline failing (but successful in debugger) for another log and filter. The log-line is:
`Nov 11 19:17:33 ezm06-pco postfix/pickup[7864]: 3364E62F1D: uid=5000 from=`
And the filter is:
`POST_PICKUP %{QUEUEID:qid}: uid=%{NUMBER:uid} from=<(%{FROM_EMAIL:from})?>`
The common thing here is the field `from=` that is contributing to the failure. If it's just `from=<>`, the filter works well in pipeline.
Hope this helps.
---
### Author:  [@subin](https://discuss.elastic.co/u/subin)
#### Post date: [December 27, 2017, 11:40am UTC](https://discuss.elastic.co/t/need-logstash-grok-help/113268/5 "2017-12-27T11:40:56Z")
Solved.
I replaced the initial syntax `FROM_EMAIL` with a simpler new syntax `EMAIL`:
`EMAIL %{USERNAME:emailname}@%{HOSTNAME:domain}`
It now works both in debugger and pipeline.
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [December 27, 2017, 6:46pm UTC](https://discuss.elastic.co/t/need-logstash-grok-help/113268/6 "2017-12-27T18:46:26Z")
Thanks for sharing your solution!
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 24, 2018, 6:46pm UTC](https://discuss.elastic.co/t/need-logstash-grok-help/113268/7 "2018-01-24T18:46:41Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.