# Help with using Grok to parse these three log formats **URL:** \ **Category:** Logstash\ **Created:** [April 1, 2023, 9:54pm UTC](https://discuss.elastic.co/t/help-with-using-grok-to-parse-these-three-log-formats/329107 "2023-04-01T21:54:48Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![roman-tasi](https://avatars.discourse-cdn.com/v4/letter/r/7ea924/32.png) [@roman-tasi](https://discuss.elastic.co/u/roman-tasi)\ **Post date:** [April 1, 2023, 9:54pm UTC](https://discuss.elastic.co/t/help-with-using-grok-to-parse-these-three-log-formats/329107/1 "2023-04-01T21:54:48Z")
Hi I am experienced with Dissect but not Grok. I think I need to use Grok here because the log format varies between the logs, so dissect will only work on one format, not all three: **Case 1:** `2023-03-31 00:01:24,366 INFO [qtp509886383-311239:https://128.171.103.69:443/Microsoft-Server-ActiveSync?User=jonathan&DeviceId=LMJ0U2G6OL6HN3T5A5N89GC5QG&DeviceType=iPhone&Cmd=Sync] [ip=47.173.247.240;] security - cmd=Auth; account=jonathan@whtasi.org; protocol=zsync; ` **Case 2:** `2023-03-31 00:00:13,319 INFO [qtp509886383-311183:http://127.0.0.1:88/service/soap/AuthRequest] [name=beau@whtasi.org;oip=66.95.159.176;ua=zclient/8.5.0_GA_3042;] security - cmd=Auth; account=beau@whtasi.org; protocol=soap; ` **Case 3:** `2023-03-31 19:20:53,401 INFO [ImapSSLServer-1368] [ip=167.170.57.114;] security - cmd=Auth; account=patricia@uhtasi.org; protocol=imap;` If you can give me a pattern that would work, and explain it, thank you! Oh I forgot that I want to parse out the **ip** (for the Case 1 and Case 3 logs), **DeviceType** (for the Case 1 log), **oip** (for the Case 2 log), **account** (all Cases), and **protocol** (all Cases) fields. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [April 1, 2023, 11:29pm UTC](https://discuss.elastic.co/t/help-with-using-grok-to-parse-these-three-log-formats/329107/2 "2023-04-01T23:29:00Z")
> [@roman-tasi](#): > > If you can give me a pattern that would work Don't try to match the entire line with a pattern. Just pull out the fields you need ``` grok { break_on_match => false match => { "message" => [ "account=(?[^;]*);", "DeviceType=%{WORD:deviceType}", "\[ip=%{IPV4:ip}", "oip=%{IPV4:oip}", "protocol=%{WORD:protocol}" ] } } ``` ---
**Author:** ![roman-tasi](https://avatars.discourse-cdn.com/v4/letter/r/7ea924/32.png) [@roman-tasi](https://discuss.elastic.co/u/roman-tasi)\ **Post date:** [April 2, 2023, 4:04am UTC](https://discuss.elastic.co/t/help-with-using-grok-to-parse-these-three-log-formats/329107/3 "2023-04-02T04:04:20Z")
Hi that solved it. But can you also help with this: **Case 1** `Apr 1 15:45:09 mail postfix/smtps/smtpd[28960]: warning: unknown[113.160.218.194]: SASL LOGIN authentication failed: authentication failure` **Case 2** `Apr 1 15:44:51 mail postfix/smtps/smtpd[28960]: warning: cpe-172-91-158-20.socal.res.rr.com[172.91.158.20]: SASL LOGIN authentication failed: authentication failure` **Case 3** `Apr 1 15:43:24 mail postfix/smtps/smtpd[28960]: warning: 94-26-146-179.arbital.ru[94.26.146.179]: SASL LOGIN authentication failed: authentication failure` I want to pull out the **ip** field as well as the entire " **SASL LOGIN authentication failed: authentication failure**" Thanks! ---
**Author:** ![Sunile\_Manjee](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sunile_manjee/32/111461_2.png) [@Sunile\_Manjee](https://discuss.elastic.co/u/Sunile_Manjee)\ **Post date:** [April 2, 2023, 12:44pm UTC](https://discuss.elastic.co/t/help-with-using-grok-to-parse-these-three-log-formats/329107/4 "2023-04-02T12:44:25Z")
If you are interested in Elastic providing you a grok pattern based on your logs, take a look at this. [Using Elastic ML to generate Grok patterns for log sources - YouTube](https://youtu.be/KfBSqnHMEko) ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [April 2, 2023, 2:02pm UTC](https://discuss.elastic.co/t/help-with-using-grok-to-parse-these-three-log-formats/329107/5 "2023-04-02T14:02:52Z")
If you want to unconditionally pull out an IP address if present then use `"%{IPV4:ip}",`. ---
**Author:** ![roman-tasi](https://avatars.discourse-cdn.com/v4/letter/r/7ea924/32.png) [@roman-tasi](https://discuss.elastic.co/u/roman-tasi)\ **Post date:** [April 2, 2023, 5:09pm UTC](https://discuss.elastic.co/t/help-with-using-grok-to-parse-these-three-log-formats/329107/6 "2023-04-02T17:09:15Z")
But then how do I pull out the " **SASL LOGIN authentication failed: authentication failure**" ?? ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [April 2, 2023, 5:15pm UTC](https://discuss.elastic.co/t/help-with-using-grok-to-parse-these-three-log-formats/329107/7 "2023-04-02T17:15:16Z")
I don't know. What about those lines tells you that you want to capture everything after the `[172.91.158.20]: `? ---
**Author:** ![roman-tasi](https://avatars.discourse-cdn.com/v4/letter/r/7ea924/32.png) [@roman-tasi](https://discuss.elastic.co/u/roman-tasi)\ **Post date:** [April 2, 2023, 6:35pm UTC](https://discuss.elastic.co/t/help-with-using-grok-to-parse-these-three-log-formats/329107/8 "2023-04-02T18:35:21Z")
Would it be this? `"]: =(?[^:]*):"` This is basically only getting the `SASL LOGIN authentication failed` and cutting out the `: authentication failure` I'm searching for the `]: ` after the IP which leads into the `SASL LOGIN authentication failed` ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [April 2, 2023, 10:37pm UTC](https://discuss.elastic.co/t/help-with-using-grok-to-parse-these-three-log-formats/329107/9 "2023-04-02T22:37:55Z")
If you want the whole of the rest of the line then use `]: %{GREEDYDATA:error}`, but that is going to match any line that contains `]: `. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 30, 2023, 10:38pm UTC](https://discuss.elastic.co/t/help-with-using-grok-to-parse-these-three-log-formats/329107/10 "2023-04-30T22:38:09Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.