# Parsing multiline unstructured log..Please give grok for this below log\\

**URL:** <https://discuss.elastic.co/t/parsing-multiline-unstructured-log-please-give-grok-for-this-below-log/203224>\
**Category:** Logstash\
**Created:** [October 11, 2019, 11:20am UTC](https://discuss.elastic.co/t/parsing-multiline-unstructured-log-please-give-grok-for-this-below-log/203224 "2019-10-11T11:20:30Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![smr](https://avatars.discourse-cdn.com/v4/letter/s/ebca7d/32.png) [@smr](https://discuss.elastic.co/u/smr)\
**Post date:** [October 11, 2019, 11:20am UTC](https://discuss.elastic.co/t/parsing-multiline-unstructured-log-please-give-grok-for-this-below-log/203224/1 "2019-10-11T11:20:31Z")

</div>

INFO 2019-04-23 01:57:00,258 [AsyncHttpClient-Callback] org.mule.runtime.core.internal.util.queue.QueueXaResourceManager: Starting ResourceManager  
INFO 2019-04-23 01:57:00,258 [AsyncHttpClient-Callback] org.mule.runtime.core.internal.util.queue.QueueXaResourceManager: Started ResourceManager  
INFO 2019-04-23 01:57:00,259 [AsyncHttpClient-Callback] org.mule.runtime.core.internal.construct.FlowConstructLifecycleManager: Starting flow: request-routerFlow-Sapi  
INFO 2019-04-23 01:57:00,299 [AsyncHttpClient-Callback] org.mule.runtime.core.internal.construct.FlowConstructLifecycleManager: Starting flow: custom-transactionid-Flow  
INFO 2019-04-23 01:57:00,323 [AsyncHttpClient-Callback] org.mule.runtime.core.internal.construct.FlowConstructLifecycleManager: Starting flow: hannaford-togo-xapi-main  
INFO 2019-04-23 01:57:00,353 [AsyncHttpClient-Callback] org.mule.runtime.core.internal.construct.FlowConstructLifecycleManager: Starting flow: hannaford-togo-xapi-console  
INFO 2019-04-23 01:57:00,361 [AsyncHttpClient-Callback] org.mule.runtime.core.internal.construct.FlowConstructLifecycleManager: Starting flow: post:\qbusterFTE:application\json:hannaford-togo-xapi-config  
INFO 2019-04-23 01:57:00,385 [AsyncHttpClient-Callback] org.mule.runtime.core.internal.context.DefaultMuleContext:

* * *

- Application: hannaford-togo-xapi-group2 \*
- OS encoding: UTF-8, Mule encoding: UTF-8 \*
- 

```
                                                               *

```

* * *

---

<div class="post-metadata">

**Author:** ![TechGeekNZ](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/techgeeknz/32/54697_2.png) [@TechGeekNZ](https://discuss.elastic.co/u/TechGeekNZ)\
**Post date:** [October 14, 2019, 1:55am UTC](https://discuss.elastic.co/t/parsing-multiline-unstructured-log-please-give-grok-for-this-below-log/203224/2 "2019-10-14T01:55:26Z")

</div>

That log file looks pretty structured. Have a look at [http://grokdebug.herokuapp.com/](http://grokdebug.herokuapp.com/) or Kibana's inbuilt debugger (Dev Tools -\> Grok Debugger) to try out your grok filters

I can see from the logs that you can use LOGLEVEL and TIMESTAMP\_ISO8601 patterns ([http://grokdebug.herokuapp.com/patterns](http://grokdebug.herokuapp.com/patterns)). In your beats configuration, you can tell it that the log is multiline and split new log records when a line hits LOGLEVEL

---

<div class="post-metadata">

**Author:** ![smr](https://avatars.discourse-cdn.com/v4/letter/s/ebca7d/32.png) [@smr](https://discuss.elastic.co/u/smr)\
**Post date:** [October 14, 2019, 5:38am UTC](https://discuss.elastic.co/t/parsing-multiline-unstructured-log-please-give-grok-for-this-below-log/203224/3 "2019-10-14T05:38:07Z")

</div>

Thank you for the reply  
But Here i have the file which contains some lines which are as below and i really need Help in parsing this multi line pattern..TIA

---

<div class="post-metadata">

**Author:** ![smr](https://avatars.discourse-cdn.com/v4/letter/s/ebca7d/32.png) [@smr](https://discuss.elastic.co/u/smr)\
**Post date:** [October 14, 2019, 5:39am UTC](https://discuss.elastic.co/t/parsing-multiline-unstructured-log-please-give-grok-for-this-below-log/203224/4 "2019-10-14T05:39:40Z")

</div>

INFO 2019-05-03 13:22:16,015 [[MuleRuntime].cpuLight.03: [hford-togo-xapi-group2].hford-togo-xapi-main.CPU\_LITE @48cef31a] 0-ff240161-6dc7-11e9-bcae-0242ac120002org.mule.runtime.core.internal.processor.LoggerMessageProcessor: {  
"QBusterHeader": {  
"LineNo": 1,  
"StartTime": 142825,  
"EndTime": 0,  
"NumberOfItems": 5,  
"UserName": "HTG-8345",  
"ReferenceNumber": "5019406"  
},  
"QBusterDetail": [{  
"LineNo": 2,  
"PluID": 5923,  
"Qty": 1,  
"RandomWeight\_FG": "N"  
}, {  
"LineNo": 3,  
"PluID": 4460030768,  
"Qty": 1,  
"RandomWeight\_FG": "N"  
}, {  
"LineNo": 4,  
"PluID": 4460030768,  
"Qty": 1,  
"RandomWeight\_FG": "N"  
}, {  
"LineNo": 5,  
"PluID": 2100065883,  
"Qty": 1,  
"RandomWeight\_FG": "N"  
}, {  
"LineNo": 6,  
"PluID": 5480002001,  
"Qty": 1,  
"RandomWeight\_FG": "N"  
}  
]  
}  
org.mule.runtime.core.internal.message.DefaultMessageBuilder$MessageImplementation  
{  
payload=org.mule.runtime.core.internal.streaming.bytes.ManagedCursorStreamProvider$ManagedCursorDecorator@65d5ed8e  
mediaType=application/json; charset=UTF-8  
attributes=org.mule.extension.http.api.HttpRequestAttributes  
{  
Request path=/htg/v1/qbusterFTE  
Raw request path=/htg/v1/qbusterFTE  
Method=POST  
Listener path=/htg/v1/\*  
Local Address=172.18.0.2:8081  
Query String=  
Relative Path=/htg/v1/qbusterFTE  
Masked Request Path=/qbusterFTE  
Remote Address=172.18.0.1:58994  
Request Uri=/htg/v1/qbusterFTE  
Raw request Uri=/htg/v1/qbusterFTE  
Scheme=http  
Version=HTTP/1.1  
Headers=[  
user-agent=curl/7.29.0  
[host=api.aaahl.com](http://host=api.aaahl.com)  
accept=application/json  
content-type=application/json  
postman-token=cd3aeb5f-8450-4877-a872-b2eb06004c4a,ba8ec422-2382-4e65-aa83-c41374ea1b73  
x-m-client-id=b5c8f47b98a740afad9687235bea4ad3  
x-m-client-secret=\*\*\*\*  
cache-control=no-cache,no-cache  
content-length=1774  
expect=100-continue  
x-forwarded-proto=https  
x-forwarded-for=10.129.42.146  
connection=close  
]  
Query Parameters=  
URI Parameters=  
}  
attributesMediaType=_/_  
}

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [October 14, 2019, 12:05pm UTC](https://discuss.elastic.co/t/parsing-multiline-unstructured-log-please-give-grok-for-this-below-log/203224/5 "2019-10-14T12:05:19Z")

</div>

Please format your posts using [markdown](https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet).

---

<div class="post-metadata">

**Author:** ![TechGeekNZ](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/techgeeknz/32/54697_2.png) [@TechGeekNZ](https://discuss.elastic.co/u/TechGeekNZ)\
**Post date:** [October 15, 2019, 9:58pm UTC](https://discuss.elastic.co/t/parsing-multiline-unstructured-log-please-give-grok-for-this-below-log/203224/6 "2019-10-15T21:58:00Z")

</div>

If you can use a grok filter to identify which log entries contain the JSON information, you could then pull it out to another field and then apply another filter which will deal with JSON and also nested as well.

If anyone else has a more efficient way to do this, feel free to weigh in 🙂

I will be looking at doing something similar down the track and researching how this can be done efficiently

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 12, 2019, 9:58pm UTC](https://discuss.elastic.co/t/parsing-multiline-unstructured-log-please-give-grok-for-this-below-log/203224/7 "2019-11-12T21:58:01Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
