# Grok parsing issue on Bitbucket/Sonar API Logs **URL:** \ **Category:** Logstash\ **Created:** [January 2, 2018, 7:43am UTC](https://discuss.elastic.co/t/grok-parsing-issue-on-bitbucket-sonar-api-logs/113737 "2018-01-02T07:43:50Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![karanarora](https://avatars.discourse-cdn.com/v4/letter/k/e19b73/32.png) [@karanarora](https://discuss.elastic.co/u/karanarora)\ **Post date:** [January 2, 2018, 7:43am UTC](https://discuss.elastic.co/t/grok-parsing-issue-on-bitbucket-sonar-api-logs/113737/1 "2018-01-02T07:43:50Z")
Hi, In my logstash pipeline, I'm trying to use grok to segregate/filter out meaningful bits of information from the API call output logs to Tools like Bitbucket/SonarQube etc. to fetch metrics data into Elasticsearch The pattern works fine on the [grokdebug.herokuapp.com](http://grokdebug.herokuapp.com) when I test it over here, however I get \_grokparsefailures under tags on Kibana dashboard for this particular index: **API Call Output --\>** { "committer": { "emailAddress": "xyz@aon.com", "name": "XYZ" }, "committerTimestamp": 1512496703000, "author": { "emailAddress": "xyz@aon.com", "name": "XYZ" }, "authorTimestamp": 1512456913000, "id": "86620bfe84d0cf250a86189c7bac7c0433a9a056", "displayId": "86620bfe84d", "message": "Create", "parents": [ { "id": "a3a100c5ea865a813275759377f1b1d07e4f7db7", "displayId": "a3a100c5ea8" } ] }, **grok regex pattern for Above Log Output --\>** {\n%{SPACE}%{SPACE}"%{GREEDYDATA:details}":%{SPACE}{\n%{SPACE}%{SPACE}%{SPACE}%{SPACE}"%{GREEDYDATA:Email}":%{SPACE}"%{GREEDYDATA:CommitterEmailAddress}",\n%{SPACE}%{SPACE}%{SPACE}%{SPACE}"%{GREEDYDATA:Name}":%{SPACE}"%{GREEDYDATA:CommitterName}"\n%{SPACE}%{SPACE}},\n%{SPACE}%{SPACE}"%{GREEDYDATA:timestamp}":%{SPACE}%{GREEDYDATA:CommitTimestamp},\n%{SPACE}%{SPACE}"%{GREEDYDATA:Author}":%{SPACE}{\n%{SPACE}%{SPACE}%{SPACE}%{SPACE}"%{GREEDYDATA:Email}":%{SPACE}"%{GREEDYDATA:AuthorEmailAddress}",\n%{SPACE}%{SPACE}%{SPACE}%{SPACE}"%{GREEDYDATA:AuthorName}":%{SPACE}"%{GREEDYDATA:AuthorsName}"\n%{SPACE}%{SPACE}},\n%{SPACE}%{SPACE}"%{GREEDYDATA:CommitDetails}":%{SPACE}%{GREEDYDATA:CommitTimestamp},\n%{SPACE}%{SPACE}"%{GREEDYDATA:CommitID}":%{SPACE}"%{GREEDYDATA:id},\n%{SPACE}%{SPACE}"%{GREEDYDATA:ShortHashID}":%{SPACE}%{GREEDYDATA:DisplayHash},\n%{SPACE}%{SPACE}"%{GREEDYDATA:CommitMessage}":%{SPACE}"%{GREEDYDATA:Comment}",\n%{SPACE}%{SPACE}"%{GREEDYDATA:CommitParent}":%{SPACE}[\n%{SPACE}%{SPACE}{\n%{SPACE}%{SPACE}%{SPACE}%{SPACE}%{SPACE}%{SPACE}"%{GREEDYDATA:ParentCommitdetails}":%{SPACE}"%{GREEDYDATA:ParentCommitID}",\n%{SPACE}%{SPACE}%{SPACE}%{SPACE}%{SPACE}%{SPACE}"%{GREEDYDATA:ParentCommitHashShort}":%{SPACE}"%{GREEDYDATA:ParentCommitShortHash}"\n%{SPACE}%{SPACE}%{SPACE}%{SPACE}}\n%{SPACE}%{SPACE}]\n}, Can someone help me if the grok pattern above can be optimized or something is amiss here. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [January 2, 2018, 8:44am UTC](https://discuss.elastic.co/t/grok-parsing-issue-on-bitbucket-sonar-api-logs/113737/2 "2018-01-02T08:44:24Z")
You should always try to use as specific patters as possible as using multiple DATA and/or GREEDYDATA can be very inefficient. If space is used as a separator, NOTSPACE might be a good substitute. ---
**Author:** ![karanarora](https://avatars.discourse-cdn.com/v4/letter/k/e19b73/32.png) [@karanarora](https://discuss.elastic.co/u/karanarora)\ **Post date:** [January 2, 2018, 10:40am UTC](https://discuss.elastic.co/t/grok-parsing-issue-on-bitbucket-sonar-api-logs/113737/3 "2018-01-02T10:40:42Z")
I'm new to groking...what specific patterns can I use here instead of the GREEDYDATA...the data that is fetched is via API calls pertaining to specific queries I made on Bitbucket I've tried to achieve the same for other tools' API logs as well but to no avail, Fields break up properly in constructor but not when the pipelines execute. **Sonar API Logs --\>** { "metric": "duplicated\_lines", "periods": [ { "index": 1, "value": "0" } ], "value": "38404" }, **grok pattern for Above --\>** {\n%{SPACE}%{SPACE}"%{GREEDYDATA:metric1\_sonar},\n%{SPACE}%{SPACE}"periods":%{SPACE}[\n%{SPACE}%{SPACE}%{SPACE}%{SPACE}{\n%{SPACE}%{SPACE}%{SPACE}%{SPACE}%{SPACE}%{SPACE}"index":%{SPACE}1,\n%{SPACE}%{SPACE}%{SPACE}%{SPACE}%{SPACE}%{SPACE}"value":%{SPACE}"0"\n%{SPACE}%{SPACE}%{SPACE}%{SPACE}}\n%{SPACE}%{SPACE}],\n%{SPACE}%{SPACE}"value":%{SPACE}"%{GREEDYDATA:metric1\_count}"\n}, ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [January 2, 2018, 11:39am UTC](https://discuss.elastic.co/t/grok-parsing-issue-on-bitbucket-sonar-api-logs/113737/4 "2018-01-02T11:39:44Z")
I am not clear on what the data looks like. If the data is in JSON format, use a JSON filter instead of grok. Grok is very flexible, but not the ideal choice for all types of data. ---
**Author:** ![karanarora](https://avatars.discourse-cdn.com/v4/letter/k/e19b73/32.png) [@karanarora](https://discuss.elastic.co/u/karanarora)\ **Post date:** [January 2, 2018, 3:17pm UTC](https://discuss.elastic.co/t/grok-parsing-issue-on-bitbucket-sonar-api-logs/113737/5 "2018-01-02T15:17:28Z")
Yes. The API Calls output is in JSON format. Are there any specific patterns which can match to the above log output type w.r.t json filters in the pipelines & can I build the same in the grok debug/constructor utility ? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [January 2, 2018, 6:04pm UTC](https://discuss.elastic.co/t/grok-parsing-issue-on-bitbucket-sonar-api-logs/113737/6 "2018-01-02T18:04:58Z")
If the data is in JSON, use the [json filter](https://www.elastic.co/guide/en/logstash/current/plugins-filters-json.html), not grok. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [January 30, 2018, 6:05pm UTC](https://discuss.elastic.co/t/grok-parsing-issue-on-bitbucket-sonar-api-logs/113737/7 "2018-01-30T18:05:13Z")
Closed on January 30, 2018, 6:05pm UTC This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.