# Logstash output not the same as grok debugger

**URL:** <https://discuss.elastic.co/t/logstash-output-not-the-same-as-grok-debugger/175258>\
**Category:** Logstash\
**Created:** [April 3, 2019, 6:09pm UTC](https://discuss.elastic.co/t/logstash-output-not-the-same-as-grok-debugger/175258 "2019-04-03T18:09:21Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![markov](https://avatars.discourse-cdn.com/v4/letter/m/f1d935/32.png) [@markov](https://discuss.elastic.co/u/markov)\
**Post date:** [April 3, 2019, 6:09pm UTC](https://discuss.elastic.co/t/logstash-output-not-the-same-as-grok-debugger/175258/1 "2019-04-03T18:09:21Z")

</div>

hello guys,  
my grok debugger pattern output doesn't look like the logstash output in kibana!! any idea !

 ![Selection_007](https://us1.discourse-cdn.com/elastic/original/3X/6/1/6155894852d9882d139127479d39a4dc7c615848.png)  
here is in kibana :

 ![Selection_009](https://us1.discourse-cdn.com/elastic/original/3X/4/1/41081b3fc4b9dc615b8432f0a1c679e90ab7c640.jpeg)

---

<div class="post-metadata">

**Author:** ![markov](https://avatars.discourse-cdn.com/v4/letter/m/f1d935/32.png) [@markov](https://discuss.elastic.co/u/markov)\
**Post date:** [April 7, 2019, 9:23pm UTC](https://discuss.elastic.co/t/logstash-output-not-the-same-as-grok-debugger/175258/2 "2019-04-07T21:23:54Z")

</div>

any help!

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 7, 2019, 11:09pm UTC](https://discuss.elastic.co/t/logstash-output-not-the-same-as-grok-debugger/175258/3 "2019-04-07T23:09:00Z")

</div>

Please do not post pictures of text, just post the text itself. What does your pattern look like and what do the lines of text you are processing look like?

---

<div class="post-metadata">

**Author:** ![markov](https://avatars.discourse-cdn.com/v4/letter/m/f1d935/32.png) [@markov](https://discuss.elastic.co/u/markov)\
**Post date:** [April 8, 2019, 9:18am UTC](https://discuss.elastic.co/t/logstash-output-not-the-same-as-grok-debugger/175258/4 "2019-04-08T09:18:55Z")

</div>

```
2019-03-11 11:12:40,670 ERROR [org.hibernate.util.JDBCExceptionReporter] ORA-28144: Echec de l'exécution du gestionnaire d'audit détaillé
ORA-20417: ERROR SECURITY DATA
ORA-06512: à "PRODUCTION_AUDIT", ligne 39
ORA-06512: à ligne 1

```

and this is the pattern used that correctly match in grok debugger:

```
%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:Loglevel} \[(?<classname>[^\]]+)\] %{GREEDYDATA:Error}(?<msg>[^\)]+)
```

---

<div class="post-metadata">

**Author:** ![markov](https://avatars.discourse-cdn.com/v4/letter/m/f1d935/32.png) [@markov](https://discuss.elastic.co/u/markov)\
**Post date:** [April 8, 2019, 10:08am UTC](https://discuss.elastic.co/t/logstash-output-not-the-same-as-grok-debugger/175258/5 "2019-04-08T10:08:24Z")

</div>

and also it regroup all other log events in one record!!

```
{
  "_index": "filebeat-2019.04.08",
  "_type": "doc",
  "_id": "HQJe_GkBGmfQmneQJD3j",
  "_version": 1,
  "_score": null,
  "_source": {
    "number": "0",
    "n": "null",
    "prospector": {},
    "classname": "impl.SensusAnalyzeLogger",
    "Url": "business.service.interfaces.ICroneJobsService",
    "beat": {},
    "@version": "1",
    "Loglevel": "INFO",
    "timestamp": "2019-03-11 00:00:00,013",
    "msg": "getMapAllCronedJobsParams | null | >> entry >> | null\n2019-03-11 00:00:00,014 INFO [impl.SensusAnalyzeLogger] null | 0 | null | persistence.manager.interfaces.ICroneEntityManager | getMapJobsCronPramas | null | >> entry >> | null\n2019-03-11 00:00:00,016 INFO [com.sensus.persistence.commons.query.impl.SensusHibernateQueryFacade] @PARAMETER_BY_ID@|query time :2 ms |result size : 6|{paramIds=[7, 8, 6, 11, 9, 10]}\n2019-03-11 00:00:00,016 INFO [log.impl.SensusAnalyzeLogger] null | 0 | null | persistence.manager.interfaces.ICroneEntityManager | getMapJobsCronPramas | null | << exit << | 2 ms\n2019-03-11 00:00:00,016 INFO [impl.SensusAnalyzeLogger] null | 0 | null | service.interfaces.ICroneJobsService | getMapAllCronedJobsParams | null | << exit << | 3 ms\n2019-03-11 00:01:00,014 INFO [impl.SensusAnalyzeLogger] null | 0 | null | service.interfaces.ICroneJobsService | getMapAllCronedJobsParams | null | >> entry >> | null\n
```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 8, 2019, 11:21am UTC](https://discuss.elastic.co/t/logstash-output-not-the-same-as-grok-debugger/175258/6 "2019-04-08T11:21:25Z")

</div>

> [@markov](#):
>
> %{GREEDYDATA:Error}(?\<msg\>[^)]+)

OK, so your pattern says to collect everything into Error, followed by one of more characters that are not close-parenthesis grouped into msg. And that's what you get -- one character in msg.

If you want to store everything up to the first newline in Error and all the rest in msg, then tell grok that.

```
    grok { match => { "message" => "%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:Loglevel} \[(?<classname>[^\]]+)\] (?<Error>[^
]+)%{GREEDYDATA:msg}" } }

```

---

<div class="post-metadata">

**Author:** ![markov](https://avatars.discourse-cdn.com/v4/letter/m/f1d935/32.png) [@markov](https://discuss.elastic.co/u/markov)\
**Post date:** [April 8, 2019, 11:43am UTC](https://discuss.elastic.co/t/logstash-output-not-the-same-as-grok-debugger/175258/8 "2019-04-08T11:43:32Z")

</div>

`2019-03-11 09:11:42,233 ERROR [impl.CommonModificationBudgetaireBusinessServiceImpl] DISPONIBLE_MC_NON com.sensus.common.exception.SensusFunctionalException: DISPONIBLE_MC_NON at verifierDisponibleGlobaleMC(CommonModificationBudgetaireBusinessServiceImpl.java:8474)`

my pattern is working with java log too so i have to stop on ")" character for that i put `(?<msg>[^)]+)` at the end, and i guess `%{GREEDYDATA:msg}` can't do this also i tried the pattern you suggest in grok debugger and it shows Compile ERROR

---

<div class="post-metadata">

**Author:** ![markov](https://avatars.discourse-cdn.com/v4/letter/m/f1d935/32.png) [@markov](https://discuss.elastic.co/u/markov)\
**Post date:** [April 9, 2019, 8:58am UTC](https://discuss.elastic.co/t/logstash-output-not-the-same-as-grok-debugger/175258/9 "2019-04-09T08:58:11Z")

</div>

what do you see 🤔

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 9, 2019, 11:02am UTC](https://discuss.elastic.co/t/logstash-output-not-the-same-as-grok-debugger/175258/10 "2019-04-09T11:02:13Z")

</div>

With the grok filter I showed I get

```
       "msg" => "\nORA-20417: ERROR SECURITY DATA\nORA-06512: à \"PRODUCTION_AUDIT\", ligne 39\nORA-06512: à ligne 1",
     "Error" => "ORA-28144: Echec de l\\'exécution du gestionnaire d\\'audit détaillé",
```

---

<div class="post-metadata">

**Author:** ![markov](https://avatars.discourse-cdn.com/v4/letter/m/f1d935/32.png) [@markov](https://discuss.elastic.co/u/markov)\
**Post date:** [April 9, 2019, 11:32am UTC](https://discuss.elastic.co/t/logstash-output-not-the-same-as-grok-debugger/175258/11 "2019-04-09T11:32:52Z")

</div>

> [@Badger](#):
>
> %{TIMESTAMP\_ISO8601:timestamp} %{LOGLEVEL:Loglevel} [(?\<classname\>[^]]+)] (?\<Error\>[^]+)%{GREEDYDATA:msg}

and i get this !!!

`[parse_exception] [patterns] Invalid regex pattern found in: [%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:Loglevel} \[(?<classname>[^\]]+)\] (?<Error>[^]+)%{GREEDYDATA:msg}]. empty char-class, with { header={ processor_type="grok" & property_name="patterns" } }`

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 9, 2019, 11:38am UTC](https://discuss.elastic.co/t/logstash-output-not-the-same-as-grok-debugger/175258/12 "2019-04-09T11:38:31Z")

</div>

> [@markov](#):
>
> empty char-class

That would suggest you are trying to use

```
 grok { match => { "message" => "%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:Loglevel} \[(?<classname>[^\]]+)\] (?<Error>[^]+)%{GREEDYDATA:msg}" } }

```

rather than

```
        grok { match => { "message" => "%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:Loglevel} \[(?<classname>[^\]]+)\] (?<Error>[^
]+)%{GREEDYDATA:msg}" } }

```

No need to quote the newline, just put a literal newline in the char class.

---

<div class="post-metadata">

**Author:** ![markov](https://avatars.discourse-cdn.com/v4/letter/m/f1d935/32.png) [@markov](https://discuss.elastic.co/u/markov)\
**Post date:** [April 9, 2019, 12:00pm UTC](https://discuss.elastic.co/t/logstash-output-not-the-same-as-grok-debugger/175258/13 "2019-04-09T12:00:05Z")

</div>

sorry i didn't get it unfortunately i have to post a picture again so i can understand you.  
this is what i'm getting

 ![Selection_011](https://us1.discourse-cdn.com/elastic/original/3X/0/9/095486995322d0b4ff302ecb7b42d2686eb782d5.png)

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 9, 2019, 12:24pm UTC](https://discuss.elastic.co/t/logstash-output-not-the-same-as-grok-debugger/175258/14 "2019-04-09T12:24:35Z")

</div>

I cannot speak to the grok debugger, only to the grok filter.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 7, 2019, 12:32pm UTC](https://discuss.elastic.co/t/logstash-output-not-the-same-as-grok-debugger/175258/15 "2019-05-07T12:32:24Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.