# Grok pattern to match different log formats in same log file

**URL:** <https://discuss.elastic.co/t/grok-pattern-to-match-different-log-formats-in-same-log-file/140155>\
**Category:** Logstash\
**Created:** [July 16, 2018, 1:46pm UTC](https://discuss.elastic.co/t/grok-pattern-to-match-different-log-formats-in-same-log-file/140155 "2018-07-16T13:46:00Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Shweta\_Priyadarshani](https://avatars.discourse-cdn.com/v4/letter/s/ecae2f/32.png) [@Shweta\_Priyadarshani](https://discuss.elastic.co/u/Shweta_Priyadarshani)\
**Post date:** [July 16, 2018, 1:46pm UTC](https://discuss.elastic.co/t/grok-pattern-to-match-different-log-formats-in-same-log-file/140155/1 "2018-07-16T13:46:01Z")

</div>

Hi Everyone,

I am trying to apply filter in logstash conf file by giving grok pattern . But the challenge here is that my log file thats the input for logstash , has different formats. Ex:

1.) 27.06.2018 00:00:00.009 _INFO_ [sling-default-5236-com.adobe.granite.threaddump.impl.BackupCleaner] com.adobe.granite.threaddump.impl.BackupCleaner File /AEM/prod/primary/author/crx-quickstart/threaddumps/20180620/threaddump.124633.txt.gz successfully deleted

2.) 27.06.2018 00:35:58.952 _INFO_ [172.31.87.111 [1530059758951] GET /libs/granite/core/content/login.html HTTP/1.1] com.merckgroup.aem.healthcare.biopharma.neurology.merckneurology.filters.ValidateLicenseFilter path information/libs/granite/core/content/login.html

What can be done so that i can put grok pattern for all the different log formats .

Also , is there a way to determine how many different log formats i have in my log file , because my log file is more than 1lac lines so looking for different formats manually is a very tough job

Thanks in advance

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [July 16, 2018, 2:31pm UTC](https://discuss.elastic.co/t/grok-pattern-to-match-different-log-formats-in-same-log-file/140155/2 "2018-07-16T14:31:14Z")

</div>

The grok filter can take an [array of patterns](https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html#plugins-filters-grok-match) to match against. Be sure to [anchor](https://www.elastic.co/blog/do-you-grok-grok) your patterns.

If you need many different patterns you are going to have to construct those patterns. There is no tool to do it for you. Take a log, index it into a disposable index, go see which events have a \_grokparsefailure tag and construct patterns that match them. Delete the index and do it again.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 13, 2018, 2:44pm UTC](https://discuss.elastic.co/t/grok-pattern-to-match-different-log-formats-in-same-log-file/140155/3 "2018-08-13T14:44:36Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.