# Parsing repeated patterns

**URL:** <https://discuss.elastic.co/t/parsing-repeated-patterns/29287>\
**Category:** Logstash\
**Created:** [September 15, 2015, 1:26am UTC](https://discuss.elastic.co/t/parsing-repeated-patterns/29287 "2015-09-15T01:26:14Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![mparthas](https://avatars.discourse-cdn.com/v4/letter/m/d78d45/32.png) [@mparthas](https://discuss.elastic.co/u/mparthas)\
**Post date:** [September 15, 2015, 1:26am UTC](https://discuss.elastic.co/t/parsing-repeated-patterns/29287/1 "2015-09-15T01:26:14Z")

</div>

Hi,

I have a log line that has patterns repeating multiple times and the number of times could be varying from line to line. Is there a recipe for this ?

For example,

[abc.com](http://abc.com) 300 IN CNAME [xyz.com](http://xyz.com)  
[xyz.com](http://xyz.com) 300 IN CNAME [dgh.com](http://dgh.com)  
[dgh.com](http://dgh.com) 300 IN CNAME [jkl.com](http://jkl.com)  
[jkl.com](http://jkl.com) 300 IN A 1.2.3.4

The number of CNAME lines is arbitrary. But the structure is same.

thanks  
mohan

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [September 15, 2015, 3:47am UTC](https://discuss.elastic.co/t/parsing-repeated-patterns/29287/2 "2015-09-15T03:47:54Z")

</div>

Do you want to parse those four lines as a single message, or what are you trying to do? What is the desired output from Logstash given that input?

---

<div class="post-metadata">

**Author:** ![mparthas](https://avatars.discourse-cdn.com/v4/letter/m/d78d45/32.png) [@mparthas](https://discuss.elastic.co/u/mparthas)\
**Post date:** [September 15, 2015, 4:27am UTC](https://discuss.elastic.co/t/parsing-repeated-patterns/29287/3 "2015-09-15T04:27:00Z")

</div>

Hi,

Actually it appears like this..in one line like this:

10-Jan-2011 12:30:42.462 client 1.2.3.4#12345: view 1: UDP: query: [abc.com](http://abc.com) IN A response: NOERROR +E [abc.com](http://abc.com) 300 IN CNAME [xyz.com](http://xyz.com); [xyz.com](http://xyz.com) 300 IN CNAME [dgh.com](http://dgh.com); [dgh.com](http://dgh.com) 300 IN CNAME [jkl.com](http://jkl.com); [kl.com](http://kl.com) 300 IN A 1.2.3.4

I am trying to parse the individual components so that I can reconstruct this to a different format using ruby. In the 'query' above, I need to extract "[abc.com](http://abc.com)" and the query type (A) and in the 'response' NOERROR, and then the series of fields broken down by . I guess this will go into an array so that I can extract them and mutate appropriately.

thanks  
mohan

---

<div class="post-metadata">

**Author:** ![mparthas](https://avatars.discourse-cdn.com/v4/letter/m/d78d45/32.png) [@mparthas](https://discuss.elastic.co/u/mparthas)\
**Post date:** [September 15, 2015, 4:29am UTC](https://discuss.elastic.co/t/parsing-repeated-patterns/29287/4 "2015-09-15T04:29:37Z")

</div>

Sorry.. the line got cut..I meant "line broken down by . Note that there is no limit for the repetition. Any help would be appreciated ..

thanks  
mohan

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [September 15, 2015, 5:48am UTC](https://discuss.elastic.co/t/parsing-repeated-patterns/29287/5 "2015-09-15T05:48:58Z")

</div>

Use grok to extract the input query, the result of the query (e.g. NOERROR), and the long string with the resulting records into discrete fields. Then use the mutate filter's split option to split the list of records into an array. Each element of that array can then be processed further but I suspect you'll need a ruby filter for that.

---

<div class="post-metadata">

**Author:** ![gringo](https://avatars.discourse-cdn.com/v4/letter/g/f19dbf/32.png) [@gringo](https://discuss.elastic.co/u/gringo)\
**Post date:** [December 17, 2015, 2:09am UTC](https://discuss.elastic.co/t/parsing-repeated-patterns/29287/6 "2015-12-17T02:09:17Z")

</div>

Would you be able to provide an example on how to capture the multiple occurrences of URL in every line of a log.

---

<div class="post-metadata">

**Author:** ![Mayank\_Agrawal](https://avatars.discourse-cdn.com/v4/letter/m/3ab097/32.png) [@Mayank\_Agrawal](https://discuss.elastic.co/u/Mayank_Agrawal)\
**Post date:** [September 18, 2016, 7:07pm UTC](https://discuss.elastic.co/t/parsing-repeated-patterns/29287/7 "2016-09-18T19:07:31Z")

</div>

@mparthas Were you able to solve the issue ??? I am also stuck on the similar issue.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:38am UTC](https://discuss.elastic.co/t/parsing-repeated-patterns/29287/8 "2017-07-06T04:38:03Z")

</div>


