# How to get multiple entries of similar pattern from the same input line using grok pattern?

**URL:** <https://discuss.elastic.co/t/how-to-get-multiple-entries-of-similar-pattern-from-the-same-input-line-using-grok-pattern/245208>\
**Category:** Logstash\
**Created:** [August 17, 2020, 9:41am UTC](https://discuss.elastic.co/t/how-to-get-multiple-entries-of-similar-pattern-from-the-same-input-line-using-grok-pattern/245208 "2020-08-17T09:41:18Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![venkata.kodapaka](https://avatars.discourse-cdn.com/v4/letter/v/48db29/32.png) [@venkata.kodapaka](https://discuss.elastic.co/u/venkata.kodapaka)\
**Post date:** [August 17, 2020, 9:41am UTC](https://discuss.elastic.co/t/how-to-get-multiple-entries-of-similar-pattern-from-the-same-input-line-using-grok-pattern/245208/1 "2020-08-17T09:41:18Z")

</div>

Hi Team,

I'm aware of grok pattern and able to parse and store the data into elastic search using logstash configuration file with _filter_ and _grok_ patterns.

For example:  
If the data input line is:

**Start-field1|field2|field3**

then  
field1, field2, field3 are being parsed and getting stored into elastic search successfully without any problem.

But now I have a input line like below:

**Start-field1|field2|field3#Start-field1|field2|field3#Start-field1|field2|field3**

means there are multiple occurrences of required pattern in the same input line, with **Start** as starting of pattern and **#** among all the required patterns.

Is there any way to fetch all such fields and store into elastic search?

Kindly help me and let me know in case of any further information is required.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 17, 2020, 2:31pm UTC](https://discuss.elastic.co/t/how-to-get-multiple-entries-of-similar-pattern-from-the-same-input-line-using-grok-pattern/245208/2 "2020-08-17T14:31:44Z")

</div>

You could use mutate+split to convert that to an array using # as the delimiter, then pass the array to grok, which will iterate over the entries.

---

<div class="post-metadata">

**Author:** ![venkata.kodapaka](https://avatars.discourse-cdn.com/v4/letter/v/48db29/32.png) [@venkata.kodapaka](https://discuss.elastic.co/u/venkata.kodapaka)\
**Post date:** [August 18, 2020, 5:43am UTC](https://discuss.elastic.co/t/how-to-get-multiple-entries-of-similar-pattern-from-the-same-input-line-using-grok-pattern/245208/3 "2020-08-18T05:43:07Z")

</div>

Thanks for the reply.

Could you please help me any reference links. or configuration. I'm not aware of mutate and split.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 18, 2020, 12:37pm UTC](https://discuss.elastic.co/t/how-to-get-multiple-entries-of-similar-pattern-from-the-same-input-line-using-grok-pattern/245208/4 "2020-08-18T12:37:54Z")

</div>

mutate+split is documented [here](https://www.elastic.co/guide/en/logstash/current/plugins-filters-mutate.html#plugins-filters-mutate-split). grok [here](https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html).

---

<div class="post-metadata">

**Author:** ![venkata.kodapaka](https://avatars.discourse-cdn.com/v4/letter/v/48db29/32.png) [@venkata.kodapaka](https://discuss.elastic.co/u/venkata.kodapaka)\
**Post date:** [August 18, 2020, 1:23pm UTC](https://discuss.elastic.co/t/how-to-get-multiple-entries-of-similar-pattern-from-the-same-input-line-using-grok-pattern/245208/5 "2020-08-18T13:23:34Z")

</div>

Hi @Badger,

I've gotten the documentation links, but not getting how to use both mutate+split which will parse the data and to pass that array to grok to process further.

Kind help.

---

<div class="post-metadata">

**Author:** ![venkata.kodapaka](https://avatars.discourse-cdn.com/v4/letter/v/48db29/32.png) [@venkata.kodapaka](https://discuss.elastic.co/u/venkata.kodapaka)\
**Post date:** [August 19, 2020, 8:04am UTC](https://discuss.elastic.co/t/how-to-get-multiple-entries-of-similar-pattern-from-the-same-input-line-using-grok-pattern/245208/6 "2020-08-19T08:04:17Z")

</div>

Hi Team,

I've tried to use mutate + split and grok pattern in logstash configuration file as below:

filter{  
mutate {  
split =\> { "message" =\> "#" }  
}  
grok {  
match =\> { "message" =\> ["Detailed\_Dashboard-%{USERNAME:ThreadId}|%{WORD:FileName}|%{DATA:FilePath}|%{DATA:TableName}|%{DATA:User}|%{DATA:Class}|%{DATA:Method}|%{DATA:Server}|%{DATA:FromTime}|%{DATA:ToTime}|%{NUMBER:ResponseTime:int}|%{GREEDYDATA:IsError}"]}  
remove\_field =\> ["message"]  
add\_field =\> { "pattern\_type" =\> "Detailed\_Dashboard" }  
}  
}

It's not saving the 2 entries into elastic search, instead it's storing the data as 2 elements in each field Eg: TableName field contains data **dsta, dsta**

Could you please help me how to use these correctly in order to save those entries as separate rows.

Any help would be highly appreciated.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 19, 2020, 1:15pm UTC](https://discuss.elastic.co/t/how-to-get-multiple-entries-of-similar-pattern-from-the-same-input-line-using-grok-pattern/245208/7 "2020-08-19T13:15:07Z")

</div>

> [@venkata.kodapaka](#):
>
> Could you please help me how to use these correctly in order to save those entries as separate rows.

If your data has arrays and you want to save them as different documents then you can use a [split](https://www.elastic.co/guide/en/logstash/current/plugins-filters-split.html) filter.

---

<div class="post-metadata">

**Author:** ![venkata.kodapaka](https://avatars.discourse-cdn.com/v4/letter/v/48db29/32.png) [@venkata.kodapaka](https://discuss.elastic.co/u/venkata.kodapaka)\
**Post date:** [August 19, 2020, 2:37pm UTC](https://discuss.elastic.co/t/how-to-get-multiple-entries-of-similar-pattern-from-the-same-input-line-using-grok-pattern/245208/8 "2020-08-19T14:37:56Z")

</div>

Hi @Badger,

I want to store each pattern as it like in each individual line.

For example take below scenario

Input:  
**Start-Elasticsearch|100|Database#Start-Logstash|200|Parser#Start-Kibana|300|UI**

then I want to store 3 records into elastic search like

**filed1 field2 field3**  
Elasticsearch 100 Database  
Logstash 200 Parser  
Kibana 300 UI

Is it do-able or not using logstash pipeline configuration or not?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 19, 2020, 2:56pm UTC](https://discuss.elastic.co/t/how-to-get-multiple-entries-of-similar-pattern-from-the-same-input-line-using-grok-pattern/245208/9 "2020-08-19T14:56:17Z")

</div>

Yes, use mutate+split to divide the string into an array using # as a delimiter. Use a split filter to separate the array into three events. Use a csv filter to parse each event into separate fields.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 16, 2020, 2:56pm UTC](https://discuss.elastic.co/t/how-to-get-multiple-entries-of-similar-pattern-from-the-same-input-line-using-grok-pattern/245208/10 "2020-09-16T14:56:19Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
