# Read logs stored using Enterprise Semantic Logging format
**URL:** https://discuss.elastic.co/t/read-logs-stored-using-enterprise-semantic-logging-format/46449
**Category:** Logstash
**Created:** [April 5, 2016, 8:20pm UTC](https://discuss.elastic.co/t/read-logs-stored-using-enterprise-semantic-logging-format/46449 "2016-04-05T20:20:47Z")
**Posts on this page:** 11
**Page:** 1
### Author:  [@Sameer\_Panicker](https://discuss.elastic.co/u/Sameer_Panicker)
#### Post date: [April 5, 2016, 8:20pm UTC](https://discuss.elastic.co/t/read-logs-stored-using-enterprise-semantic-logging-format/46449/1 "2016-04-05T20:20:47Z")
We are using Semantic Logging mechanism to log our service request and response in physical files.
Is there a existing GROK Pattern to pull my logs in a good way ?
OR
Any other efficient way to read these logs?
EventId : 1, Level : Informational, Message : Request: 128.194.38.33 GET [http://server001/TokenService/api/auth/token](http://server001/TokenService/api/auth/token), Payload : [id : 4aeeed26-2f5e-4a7b-b4a3-6c35bbad678d] [ipAddress : 168.194.81.321] [method : GET] [requestUri : [http://server001/TokenService/api/auth](http://server001/TokenService/api/auth)] [content :] , EventName : RequestInfo, Timestamp : 2016-03-29T02:34:21.7249900Z, ProcessId : 8052, ThreadId : 16104
EventId : 2, Level : Informational, Message : Response: OK, Payload : [id : 4aeeed26-2f5e-4a7b-b4a3-6c35bbad678d] [reasonPhrase : OK] [content : {"Token":"eyJ0eXAiOiJKV12NoZW1hcy54bWxzb2FwLm9yZy93cy8yMDA1LzA1L2lkZW50aXR5L2NsYWltcy9uYW1lIjoiYTExMTM3NzEifQ.YpAKLuFeBtOjm9e4kwNxcdsnJlkd9OH3fXJP2\_olzAc","refrToken":"829314e36f933f5f910cf6c"}] , EventName : ResponseInfo, Timestamp : 2016-03-29T02:34:29.5154635Z, ProcessId : 8052, ThreadId : 16104
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [April 5, 2016, 10:32pm UTC](https://discuss.elastic.co/t/read-logs-stored-using-enterprise-semantic-logging-format/46449/2 "2016-04-05T22:32:53Z")
> [@Sameer\_Panicker](#):
>
> Is there a existing GROK Pattern to pull my logs in a good way ?
No, but it's not hard to build one using [http://grokdebug.herokuapp.com/](http://grokdebug.herokuapp.com/)
---
### Author:  [@Sameer\_Panicker](https://discuss.elastic.co/u/Sameer_Panicker)
#### Post date: [April 5, 2016, 10:35pm UTC](https://discuss.elastic.co/t/read-logs-stored-using-enterprise-semantic-logging-format/46449/3 "2016-04-05T22:35:21Z")
I tried it. Even had posted this on the forum, but NO reply ☹
> [@Optimizing my current GROK Filter](https://discuss.elastic.co/t/optimizing-my-current-grok-filter/45289):
>
> I am new to GROK work. As of now, I have created a filter which extracts all the data. But I need to extract only the SessionID, Method, RequestURI and Content values only. This is my request log EventId : 1, Level : Informational, Message : Request, Payload : [sessionID : 43a7be8b-3222-4958-9082-ecc4428f0037] [ipAddress : 172.16.22.88] [method : POST] [requestUri : [https://myserver.my](https://myserver.my),com:4433/MyService/api/click2mob] [content : {"callType":"Sales\_Appl","customer":{"firstName":"Service","las…
This is my request log
EventId : 1, Level : Informational, Message : Request, Payload : [**sessionID** : 43a7be8b-3222-4958-9082-ecc4428f0037] [ipAddress : 172.16.22.88] [method : **POST**] [requestUri : **[https://myserver.my](https://myserver.my),com:4433/MyService/api/click2mob**] [content : {"callType":"Sales\_Appl","customer":{"firstName":"Service","lastName":"Interceptor","phones":[{"countryCode":"1","areaCode":"612","number":"2221213"}],"emailAddresses":[{"[email":"myinterceptor@bestbuy.com](mailto:email%22:%22myinterceptor@bestbuy.com)"}]}}] , EventName : RequestInfo, Timestamp : 2016-03-22T04:06:01.4157082Z, ProcessId : 9784, ThreadId : 18768
My Current GROK Filter in Logstash -
%{WORD:Event} %{NOTSPACE:Colons} %{WORD:EventID}%{NOTSPACE:Colons} %{WORD:Level} %{NOTSPACE:Colons} %{WORD:LevelDetail}%{NOTSPACE:Colons} %{WORD:Message} %{NOTSPACE:Colons} %{WORD:Request}%{NOTSPACE:Colons} %{WORD:Payload} %{NOTSPACE:Colons} %{NOTSPACE:Colons}%{WORD: Session} %{NOTSPACE:Colons} %{UUID:SessionID}%{NOTSPACE:Colons} %{NOTSPACE:Colons}%{WORD: IPAddress} %{NOTSPACE:Colons} %{IPORHOST:IP}%{NOTSPACE:Colons} %{NOTSPACE:Colons}%{WORD: Method} %{NOTSPACE:Colons} %{WORD:HTTP\_VERB}%{NOTSPACE:Colons} %{NOTSPACE:Colons}%{WORD: RequestURI} %{NOTSPACE:Colons} %{URI:URL}%{NOTSPACE:Colons}%{WORD: content} %{NOTSPACE:Colons} %{GREEDYDATA:keyvalues}%{NOTSPACE:Colons} %{WORD: EventName} %{NOTSPACE:Colons} %{WORD:RequestInfo}%{NOTSPACE:Colons} %{WORD:IIS\_TimeStamp} %{NOTSPACE:Colons} %{TIMESTAMP\_ISO8601:IIS\_TimeStamp}%{NOTSPACE:Colons} %{WORD: ProcessID} %{NOTSPACE:Colons} %{WORD:Process\_ID}%{NOTSPACE:Colons} %{WORD: ThreadID} %{NOTSPACE:Colons} %{WORD:Thread\_ID}
How can I optimize this query to get only the req. values?
Is there anything that can help me here ?
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [April 5, 2016, 10:40pm UTC](https://discuss.elastic.co/t/read-logs-stored-using-enterprise-semantic-logging-format/46449/4 "2016-04-05T22:40:05Z")
I'd start by putting the pattern in [http://grokdebug.herokuapp.com/](http://grokdebug.herokuapp.com/) and the building the pattern piece by piece. You have a lot of `NOTSPACE:Colons` there, which I assume you are trying to use to represent the `:`, which is not necessary.
eg `EventId : 1, Level : Informational` should be `%{WORD:Event} : %{NUMBER:EventID}, %{WORD:Level} : %{WORD:EventLevel}`.
---
### Author:  [@Sameer\_Panicker](https://discuss.elastic.co/u/Sameer_Panicker)
#### Post date: [April 5, 2016, 10:46pm UTC](https://discuss.elastic.co/t/read-logs-stored-using-enterprise-semantic-logging-format/46449/5 "2016-04-05T22:46:24Z")
Thanks Mark. That shortens my pattern.
One more thing, in logstash when I add this...it is not captured in ES. Is there anything that I need to add for multiple line search. Since, these logs can span upto multiple lines. And my current logstash conf reads line by line only.
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [April 5, 2016, 10:49pm UTC](https://discuss.elastic.co/t/read-logs-stored-using-enterprise-semantic-logging-format/46449/6 "2016-04-05T22:49:26Z")
You need to look at this then - [https://www.elastic.co/guide/en/logstash/current/plugins-filters-multiline.html](https://www.elastic.co/guide/en/logstash/current/plugins-filters-multiline.html)
---
### Author:  [@Sameer\_Panicker](https://discuss.elastic.co/u/Sameer_Panicker)
#### Post date: [April 5, 2016, 10:53pm UTC](https://discuss.elastic.co/t/read-logs-stored-using-enterprise-semantic-logging-format/46449/7 "2016-04-05T22:53:07Z")
Will this work for multiline grok pattern. Bcoz it is mentioned for regex only.
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [April 5, 2016, 11:16pm UTC](https://discuss.elastic.co/t/read-logs-stored-using-enterprise-semantic-logging-format/46449/8 "2016-04-05T23:16:44Z")
Ahh sorry I forgot that we moved that to [https://www.elastic.co/guide/en/logstash/current/plugins-codecs-multiline.html](https://www.elastic.co/guide/en/logstash/current/plugins-codecs-multiline.html)
---
### Author:  [@Sameer\_Panicker](https://discuss.elastic.co/u/Sameer_Panicker)
#### Post date: [April 6, 2016, 6:26pm UTC](https://discuss.elastic.co/t/read-logs-stored-using-enterprise-semantic-logging-format/46449/9 "2016-04-06T18:26:31Z")
Mark - Can you please help me here. How can I get the entire message. I tried GREEDYDATA, DATA, etc. but couldnt get the entire message value. Is there anything which matches IN BETWEEN -\> "Message and EventName : " or a Start and End tag ?
Payload : [sessionID : 7e9c103d-92a8-48f9-a8b9-e1489r7ebeff] [message : **Value cannot be null.**
**Parameter name: phoneObj] [paramName : phoneObj] [source : MySol.API.Common] [stackTrace : at MySol.API.Common.Filters.EmptyParameterFilterAttribute.OnActionExecuting(HttpActionContext actionContext) in e:\Builds\16\IVR\Current.DEV\src\Nightingale\MySol.API.Common\Filters\EmptyParameterFilterAttribute.cs:line 30**
\*\* at System.Web.Http.Filters.ActionFilterAttribute.OnActionExecutingAsync(HttpActionContext actionContext, CancellationToken cancellationToken)\*\*
**--- End of stack trace from previous location where exception was thrown ---**
\*\* at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\*\*
\*\* at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\*\*
\*\* at System.Web.Http.Filters.ActionFilterAttribute.d\_\_0.MoveNext()\*\*
**--- End of stack trace from previous location where exception was thrown ---**
\*\* at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\*\*
\*\* at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\*\*
\*\* at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult() **** at System.Web.Http.Controllers.ActionFilterResult.d__2.MoveNext() **** --- End of stack trace from previous location where exception was thrown --- **** at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) **** at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) **** at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()\*\*
\*\* at System.Web.Http.Controllers.ExceptionFilterResult.d\_\_0.MoveNext()\*\*]
EventName : ResponseInfo
---
### Author:  [@Sameer\_Panicker](https://discuss.elastic.co/u/Sameer_Panicker)
#### Post date: [April 7, 2016, 5:45am UTC](https://discuss.elastic.co/t/read-logs-stored-using-enterprise-semantic-logging-format/46449/10 "2016-04-07T05:45:14Z")
Any update ??
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 5:03am UTC](https://discuss.elastic.co/t/read-logs-stored-using-enterprise-semantic-logging-format/46449/11 "2017-07-06T05:03:23Z")