# How can merge logs based on the multi parameters

**URL:** https://discuss.elastic.co/t/how-can-merge-logs-based-on-the-multi-parameters/198494
**Category:** Logstash
**Created:** [September 7, 2019, 6:32am UTC](https://discuss.elastic.co/t/how-can-merge-logs-based-on-the-multi-parameters/198494 "2019-09-07T06:32:03Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![sahere37](https://avatars.discourse-cdn.com/v4/letter/s/b2d939/32.png) [@sahere37](https://discuss.elastic.co/u/sahere37)
#### Post date: [September 7, 2019, 6:32am UTC](https://discuss.elastic.co/t/how-can-merge-logs-based-on-the-multi-parameters/198494/1 "2019-09-07T06:32:04Z")

</div>

hi all  
I have a question, how can merge multi lines of logs based on the multi parameters into one event?. for example, log is as following:

```
IT=1 AC=12345 p1=12345
IW=1 AC=12345 A1=check1 
IW=1 E1=0
IW=2 AC=12345 A1=check2
IW=2 E1=4
IT=1 p2=ok
IT=2 AC=7894 p1=245
IW=4 AC=7894 A1=get 
IW=4 E1=2
IT=2 p2=failed

```

now the expected result is as following:

```
   IT=1 AC=12345 p1=12345 IW={1,2} A1={check1,check2} E1={0,4} p2=ok
   IT=2 AC=7894 p1=245 IW=4 A1=get E1=2 p2=failed

```

actually i want to aggregate events first based on the IW parameter:

```
IW=1 AC=12345 A1=check1 E1=0
IW=2 AC=12345 A1=check2 E1=4
IW=4 AC=7894 A1=get E1=2

```

and then aggregate based on the IT parameter

```
IT=1 AC=12345 p1=12345 p2=ok
IT=2 AC=7894 p1=245 p2=failed

```

and at the end aggregate based on the AC parameter in one event

```
  AC=12345 IT=1 p1=12345 IW={1,2} A1={check1,check2} E1={0,4} p2=ok
  AC=7894 IT=2 p1=245 IW=4 A1=get E1=2 p2=failed

```

any advise will be so appreciated

---

<div class="post-metadata">

### Author: ![sahere37](https://avatars.discourse-cdn.com/v4/letter/s/b2d939/32.png) [@sahere37](https://discuss.elastic.co/u/sahere37)
#### Post date: [September 7, 2019, 11:24am UTC](https://discuss.elastic.co/t/how-can-merge-logs-based-on-the-multi-parameters/198494/2 "2019-09-07T11:24:28Z")

</div>

when i used three aggregate filter to aggregate first based on IW and then based on IT and at the end based on the AC, the output is multiple events where it is expected to be just one

---

<div class="post-metadata">

### Author: ![pjanzen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pjanzen/32/13756_2.png) [@pjanzen](https://discuss.elastic.co/u/pjanzen)
#### Post date: [September 7, 2019, 7:42pm UTC](https://discuss.elastic.co/t/how-can-merge-logs-based-on-the-multi-parameters/198494/3 "2019-09-07T19:42:37Z")

</div>

Hi,

You have in your example 10 log lines which are processed as 10 separate events in Elasticsearch. But you expect the result to be 2 log events before you index them to Elasticsearch? And also the order if where you put the fields are on a different place then where the event original sees it.

Unless I totally miss the point here, I do not see how you could solve that in Logstash. If you can guaranty the order the events are coming in you could do something with the multiline codec but I highly doubt that is feasible in a production environment.

---

<div class="post-metadata">

### Author: ![sahere37](https://avatars.discourse-cdn.com/v4/letter/s/b2d939/32.png) [@sahere37](https://discuss.elastic.co/u/sahere37)
#### Post date: [September 8, 2019, 5:20am UTC](https://discuss.elastic.co/t/how-can-merge-logs-based-on-the-multi-parameters/198494/4 "2019-09-08T05:20:07Z")

</div>

`You have in your example 10 log lines which are processed as 10 separate events in Elasticsearch. But you expect the result to be 2 log events before you index them to Elasticsearch?`  
hi, in this example yes, number of lines may be different.

`And also the order if where you put the fields are on a different place then where the event original sees it`

Could you please explain more. i cannot get it. if you mean the place of fields as AC, p1, p2 ,...; their location is fixed and i can extract these fields using grok filter

`If you can guaranty the order the events are coming in you could do something with the multiline codec but I highly doubt that is feasible in a production environment.`

actually the order of lines may differ. the point is that the lines can be linked based on the multi parameter. first based on the IW and IT parameter and then based on the AC parameter

---

<div class="post-metadata">

### Author: ![sahere37](https://avatars.discourse-cdn.com/v4/letter/s/b2d939/32.png) [@sahere37](https://discuss.elastic.co/u/sahere37)
#### Post date: [September 14, 2019, 4:19am UTC](https://discuss.elastic.co/t/how-can-merge-logs-based-on-the-multi-parameters/198494/5 "2019-09-14T04:19:43Z")

</div>

any update and solution? any advise will be so appreciated

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [October 12, 2019, 4:19am UTC](https://discuss.elastic.co/t/how-can-merge-logs-based-on-the-multi-parameters/198494/6 "2019-10-12T04:19:47Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
