# Extract log patterns

**URL:** <https://discuss.elastic.co/t/extract-log-patterns/169316>\
**Category:** Kibana\
**Created:** [February 21, 2019, 2:43am UTC](https://discuss.elastic.co/t/extract-log-patterns/169316 "2019-02-21T02:43:48Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![anandsunderraman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anandsunderraman/32/45504_2.png) [@anandsunderraman](https://discuss.elastic.co/u/anandsunderraman)\
**Post date:** [February 21, 2019, 2:43am UTC](https://discuss.elastic.co/t/extract-log-patterns/169316/1 "2019-02-21T02:43:48Z")

</div>

I am a past life SUMO logic user.  
There are times when you want to analyze the log and extract / identify patterns rather than manually identifying them.  
I am looking for something similar to the SUMO logic log reduce operator: [https://help.sumologic.com/05Search/LogReduce/Detect-Patterns-with-LogReduce](https://help.sumologic.com/05Search/LogReduce/Detect-Patterns-with-LogReduce).

I have searched the ELK documentation and have not been able to find anything that matches that functionality.

Any help would be appreciated.

PS: Splunk has the pattern job functionality

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [February 21, 2019, 2:49am UTC](https://discuss.elastic.co/t/extract-log-patterns/169316/2 "2019-02-21T02:49:25Z")

</div>

In the Elastic Stack, pattern extraction happens before sending data to Elasticsearch and viewing in Kibana. It's handled by Logstash with grok, or by the ingest API in Elasticsearch with grok.

---

<div class="post-metadata">

**Author:** ![anandsunderraman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anandsunderraman/32/45504_2.png) [@anandsunderraman](https://discuss.elastic.co/u/anandsunderraman)\
**Post date:** [February 21, 2019, 1:06pm UTC](https://discuss.elastic.co/t/extract-log-patterns/169316/3 "2019-02-21T13:06:30Z")

</div>

I think I did not state my problem right.

We do use grok patterns to extract messages and store it in elastic.

For example my application logs something like:  
"Saved record with id: 1234"  
"Saved record with id: 9870"  
"Saved record with id: 4523"

it also logs  
"Error: connection timeout"  
"Error: database restarted"

So I have all these logs.

What SUMO and Splunk offer is way to extract these patterns from the messages and give a tabular output something to the following effect

Message pattern Frequency  
Saved record with id: \<\> . 80%  
Error: connection timeout 10%  
Error: database restarted 10%

If this can be achieved using grok patterns, I would be interested to understand it.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [February 21, 2019, 10:48pm UTC](https://discuss.elastic.co/t/extract-log-patterns/169316/4 "2019-02-21T22:48:31Z")

</div>

What does your config look like?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 21, 2019, 10:48pm UTC](https://discuss.elastic.co/t/extract-log-patterns/169316/5 "2019-03-21T22:48:48Z")

</div>

Closed on March 21, 2019, 10:48pm UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.