# Log Parsing **URL:** \ **Category:** Logstash\ **Created:** [December 3, 2018, 6:52am UTC](https://discuss.elastic.co/t/log-parsing/159098 "2018-12-03T06:52:29Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![Vineeth\_Ananthula](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vineeth_ananthula/32/42757_2.png) [@Vineeth\_Ananthula](https://discuss.elastic.co/u/Vineeth_Ananthula)\ **Post date:** [December 3, 2018, 6:52am UTC](https://discuss.elastic.co/t/log-parsing/159098/1 "2018-12-03T06:52:30Z")
I am collecting hundred types of router logs from a syslog server . I was able to write grok pattern for few of them and feed them into elasticsearch and kibana. But its becoming a tedious job to write for each type of log. Can anyone please tell me a better way to parse logs in logstash? Below are few types of logs:- (There are many other types of logs) Ex:- 200.65.181.67 RP/0/RSP0/CPU0:Dec 3 10:51:15 IST: ntpd[266]: %IP-IP\_NTP-5-HP\_CONN\_RECOVERED : High priority NTP peer connection recovered - Stratum 15-\>2. Ex:-212.65.181.67 LC/0/2/CPU0:Dec 3 11:07:38 IST: bfd\_agent[128]: %L2-BFD-6-SESSION\_STATE\_UP : BFD session to neighbor 10.141.0.2 on interface TenGigE0/2/0/0.141142 is up. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [December 3, 2018, 9:08am UTC](https://discuss.elastic.co/t/log-parsing/159098/2 "2018-12-03T09:08:50Z")
What do your patterns look like for these? ---
**Author:** ![Vineeth\_Ananthula](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vineeth_ananthula/32/42757_2.png) [@Vineeth\_Ananthula](https://discuss.elastic.co/u/Vineeth_Ananthula)\ **Post date:** [December 3, 2018, 9:19am UTC](https://discuss.elastic.co/t/log-parsing/159098/4 "2018-12-03T09:19:31Z")
1.100.65.181.67 RP/0/RSP0/CPU0:Nov 22 17:58:59 IST: SSHD\_[65794]: %SECURITY-SSHD-6-INFO\_SUCCESS\_COMMAND : Successfully authenticated user 'nieg' from '100.65.181.42' on 'vty0'(cipher 'aes128-ctr', mac 'hmac-sha1', command 'show mpls forwarding prefix 6.6.6.6/32') GROK PATTERN:%{IPV4:ROUTERIP}\s%{WIN3}[:]%{MONTH}\s%{MONTHDAY}\s%{TIME}\s%{TZ1}[:]\s%{PROCESSNAME}\S[%{POSINT:PROCESSID}][:]\s[%]%{WORD:MESSAGECATEGORY}%{WORD-GROUP}%{POSINT:SEVERITYLEVEL}%{DATA:MESSAGECODE}\s[:]%{DATA:MESSAGE}[\s][']%{WORD:USER}[']\s.\*\s[']%{IPV4:SOURCEIP}['] 2.100.65.181.67 LC/0/0/CPU0:Nov 23 17:40:15 IST: arp[120]: %IP-ARP-6-INFO\_DUPADDR : 'Duplicate IP address 10.0.10.13 on TenGigE0/0/0/1.2083, sourced by 000c.2920.1d8c' GROK PATTERN:%{IPV4:ROUTERIP}\s%{WIN3}[:]%{MONTH}\s%{MONTHDAY}\s%{TIME}\s%{TZ1}[:][\s]%{PROCESSNAME}[%{POSINT:PROCESSID}][:]\s[%]%{WORD:MESSAGECATEGORY}%{WORD:GROUPID}%{POSINT:SEVERITYLEVEL}%{DATA:MESSAGECODE}\s[:]\s([']%{GREEDYDATA:MESSAGE}\s%{IPV4:DUPLICATEIP}\s%{NOTSPACE}\s%{DATA:INTERFACE}?%{NOTSPACE}\s%{MAC:SOURCEMAC}['])? 1. 100.65.181.67 RP/0/RSP0/CPU0:Nov 28 12:33:01 IST: SSHD\_[65793]: %SECURITY-SSHD-4-WARNING\_LOGIN : The requested term-type 'dumb' is not supported GROK PATTERN:%{IPV4:ROUTERIP}\s%{WIN3}[:]%{MONTH}\s%{MONTHDAY}\s%{TIME}\s%{TZ1}[:]\s%{PROCESSNAME}\S[%{POSINT:PROCESSID}][:]\s[%]%{WORD:MESSAGECATEGORY}%{WORD:GROUPID}%{POSINT:SEVERITYLEVEL}%{DATA:MESSAGECODE}\s%{NOTSPACE}%{GREEDYDATA:MESSAGE}\s PS1: WIN3,TZ1,PROCESSNAME ARE THE CUSTOM PATTERNS PS: I HAVE WRITTEN THE PATTERN FOR THESE FOLLOWING TYPES OF LOGS. I DONT HAVE ANY PROBLEM IN WRITING GROK PATTERN FOR THE LOGS IN MY QUEST'N. I GET MANY TYPES OF LOGS FROM MY ROUTER, SO SHOULD I KEEP WRITING GROK PATTERNS OR IS THERE ANY OTHER WAY ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [December 3, 2018, 9:21am UTC](https://discuss.elastic.co/t/log-parsing/159098/5 "2018-12-03T09:21:00Z")
Please format your code/logs/config using the `` button, or markdown style back ticks. It helps to make things easy to read which helps us help you 🙂 There's also no need to use all caps in your text. It looks like most of the patterns share the same common elements. Why not create some custom patterns and leverage those? ---
**Author:** ![Vineeth\_Ananthula](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vineeth_ananthula/32/42757_2.png) [@Vineeth\_Ananthula](https://discuss.elastic.co/u/Vineeth_Ananthula)\ **Post date:** [December 3, 2018, 9:23am UTC](https://discuss.elastic.co/t/log-parsing/159098/6 "2018-12-03T09:23:50Z")
Sorry for that I will do it from next time Okay @warkolm so your point is write custom patterns . There is no other way. Am I right?? ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [December 3, 2018, 9:25am UTC](https://discuss.elastic.co/t/log-parsing/159098/7 "2018-12-03T09:25:28Z")
We are working on automated grok discovery with ML. I am not sure of the time line though. ---
**Author:** ![Vineeth\_Ananthula](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vineeth_ananthula/32/42757_2.png) [@Vineeth\_Ananthula](https://discuss.elastic.co/u/Vineeth_Ananthula)\ **Post date:** [December 3, 2018, 9:26am UTC](https://discuss.elastic.co/t/log-parsing/159098/8 "2018-12-03T09:26:29Z")
Thank you @warkolm ---
**Author:** ![StephenGoodall](https://avatars.discourse-cdn.com/v4/letter/s/bb73d2/32.png) [@StephenGoodall](https://discuss.elastic.co/u/StephenGoodall)\ **Post date:** [December 17, 2018, 3:44am UTC](https://discuss.elastic.co/t/log-parsing/159098/9 "2018-12-17T03:44:12Z")
This site sometimes comes in handy to get a baseline to work from: [https://grokdebug.herokuapp.com/discover](https://grokdebug.herokuapp.com/discover) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [January 14, 2019, 3:44am UTC](https://discuss.elastic.co/t/log-parsing/159098/10 "2019-01-14T03:44:23Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.