Wondering what is the best approach to take when dealing with input log files, where the files contain more than one multiline pattern of interest, and where the patterns are of quite different formats?
For example, multiline pattern 1:
Model Metrics Type: Regression Description: Metrics reported on training frame MSE: 0.030414708 RMSE: 0.17439812
Scoring History: Timestamp Duration Number of Trees Training RMSE Training MAE Training Deviance 2018-09-10 22:08:46 0.309 sec 0 0.49995 0.49994 0.24995 2018-09-10 22:08:46 0.548 sec 1 0.44996 0.44995 0.20246 2018-09-10 22:08:46 0.588 sec 2 0.40496 0.40495 0.16399
(The 2 patterns in this example are edited for brevity. Generally speaking, they are longer than shown, but are of a fixed number of lines)
Given that the different patterns are in the same input file, are we realistically needing more than one instance of filebeat to process them? From what I've read, using 2 prospectors on the same input file can lead to incomplete results. Or would we be better off with a custom module to do some pre/post processing of the input file?
Appreciate any guidance. Many thanks.