Optimal way to handle log with multiple format?

I try to parse this dataset: https://github.com/logpai/loghub/tree/master/Android using logstash.
I have tried using grok filter but some parts of the log contains multiple templates.

example of log:

03-17 16:13:38.811  1702  2395 D WindowManager: printFreezingDisplayLogsopening app wtoken = AppWindowToken{9f4ef63 token=Token{a64f992 ActivityRecord{de9231d u0 com.tencent.qt.qtl/.activity.info.NewsDetailXmlActivity t761}}}, allDrawn= false, startingDisplayed =  false, startingMoved =  false, isRelaunching =  false
03-17 16:13:38.819  1702  8671 D PowerManagerService: acquire lock=233570404, flags=0x1, tag="View Lock", name=com.android.systemui, ws=null, uid=10037, pid=2227
03-17 16:13:38.820  1702  8671 D PowerManagerService: ready=true,policy=3,wakefulness=1,wksummary=0x23,uasummary=0x1,bootcompleted=true,boostinprogress=false,waitmodeenable=false,mode=false,manual=38,auto=-1,adj=0.0userId=0
03-17 16:13:38.839  1702  2113 V WindowManager: Skipping AppWindowToken{df0798e token=Token{78af589 ActivityRecord{3b04890 u0 com.tencent.qt.qtl/com.tencent.video.player.activity.PlayerActivity t761}}} -- going to hide
03-17 16:13:38.859  2227  2227 D TextView: visible is system.time.showampm

My grok filter:

filter {
	grok {
	patterns_dir => ["./patterns"]
	match => { "message" => "%{MONTHANDDAY:date}%{SPACE}%{TIME:time}%{SPACE}%{NUMBER:pid}%{SPACE}\
	%{NUMBER:tid}%{SPACE}%{WORD:level}%{SPACE}%{WORD:App}:%{SPACE}%{GREEDYDATA:INFO}" }
	}
}

Result: Logstash can handle log with same structure. How about the postfix with 100+ templates? Do I need to manually define grok pattern for each template or what is the correct tool for this?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.