Merge multiple lines with a specific condition

Elastic Stack Logstash
1

Hello,

Is there any way to group by 5 lines using the multiline codec plugin or with another tool?

My file log look like this :

2018-11-23 07:55:52 Regulatory [INFO] Account : A,B,100002506,23/11/2018,23/11/2018 07:55:50:517,23/11/2018 07:55:51:295,Account Movement
2018-11-23 07:56:10 Regulatory [INFO] Account : A,B,100002506,23/11/2018,23/11/2018 07:56:08:906,23/11/2018 07:56:09:674,Account Movement
2018-11-23 07:58:55 Regulatory [INFO] Account : A,B,100002506,23/11/2018,23/11/2018 07:58:53:164,23/11/2018 07:58:54:438,Account Movement
2018-11-23 08:16:20 Regulatory [INFO] Account : A,B,100002506,23/11/2018,23/11/2018 08:16:18:296,23/11/2018 08:16:19:183,Account Movement
2018-11-23 08:17:03 Regulatory [INFO] Transaction : VALIDATE,A,B,100002506,23/11/2018,23/11/2018 08:17:02:606,23/11/2018 08:17:03:621,Payment Order
2018-11-23 08:17:14 Regulatory [INFO] Transaction : PROCESS,A,B,100002506,23/11/2018,23/11/2018 08:17:13:964,23/11/2018 08:17:14:920,Payment Order
2018-11-23 08:19:27 Regulatory [INFO] Transaction : VALIDATE,A,B,100002506,23/11/2018,23/11/2018 08:19:27:011,23/11/2018 08:19:27:509,Payment Order
2018-11-23 08:19:31 Regulatory [INFO] Transaction : PROCESS,A,B,100002506,23/11/2018,23/11/2018 08:19:31:104,23/11/2018 08:19:31:767,Payment Order
2018-11-23 08:20:27 Regulatory [INFO] Transaction : VALIDATE,A,B,100002506,23/11/2018,23/11/2018 08:20:26:726,23/11/2018 08:20:27:158,Payment Order
2018-11-23 08:20:28 Regulatory [INFO] Transaction : PROCESS,A,B,100002506,23/11/2018,23/11/2018 08:20:28:545,23/11/2018 08:20:28:929,Payment Order
2018-11-23 08:26:04 Regulatory [INFO] Transaction : VALIDATE,A,B,100002506,23/11/2018,23/11/2018 08:26:03:757,23/11/2018 08:26:04:477,Payment Order
2018-11-23 08:26:07 Regulatory [INFO] Transaction : PROCESS,A,B,100002506,23/11/2018,23/11/2018 08:26:06:812,23/11/2018 08:26:07:225,Payment Order
2018-11-23 08:27:28 Regulatory [INFO] Transaction : VALIDATE,A,B,100002506,23/11/2018,23/11/2018 08:27:28:358,23/11/2018 08:27:28:965,Payment Order
2018-11-23 08:27:31 Regulatory [INFO] Transaction : PROCESS,A,B,100002506,23/11/2018,23/11/2018 08:27:30:466,23/11/2018 08:27:31:192,Payment Order

I want to group by 5 lines and then calculate the difference in seconds between the start_date of the first line "23/11/2018 07:55:50:517" and the end_date of the 5th line "23/11/2018 08:17:03:621"

Did someone have an idea about this use case?

Thank you in advance.

2

Hi @magnusbaeck, what do you think about this use case?

Thank you in advance

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.