How to perform concatenation of values from the same field in multiple lines?


(Ivan Klekovic) #1

Hello,

This is the excerpt from the MS SharePoint log file:

Timestamp              	Process                                 	TID   	Area                          	Category                      	EventID	Level     	Message 	Correlation
03/04/2015 15:49:43.01 	w3wp.exe (0x1B48)                       	0x1654	SharePoint Foundation         	Files                         	ak8dj	High    	UserAgent not available, file operations may not be optimized...	0577ef9c-e7bf-402c-ea87-f8ab50bf959f
03/04/2015 15:49:43.01*	w3wp.exe (0x1B48)                       	0x1654	SharePoint Foundation         	Files                         	ak8dj	High    	...)     at Microsoft.SharePoint.Library...	0577ef9c-e7bf-402c-ea87-f8ab50bf959f

I'm trying to detect multiline message (e.g. by matching asterisk (*) in the Timestamp field and specifying it as a new field) and then somehow appending value of the log message field from the current line to the previous one.
Looking at multiline codec plugin, much in the same way, except I need to extract and concatenate only one field, not the whole line.

Grok-ing results with:

> {"message":"03/04/2015 15:49:43.01 \tw3wp.exe (0x1B48)                       \t0x1654\tSharePoint Foundation         \tFiles                         \tak8dj\tHigh    \tUserAgent not available, file operations may not be optimized...\t0577ef9c-e7bf-402c-ea87-f8ab50bf959f\r","@version":"1","@timestamp":"2015-08-03T14:14:51.994Z","host":"","path":"C:\\temp\\cAll\\SharePoint\\SP2013FOUND-20150304-1549_regular - Copy.log","tags":[],"parsedtime":"03/04/2015 15:49:43.01","process":"w3wp.exe","processcode":"0x1B48","tid":"0x1654","area":"SharePoint Foundation         ","category":"Files                         ","eventID":"ak8dj","level":"High","eventmessage":"UserAgent not available, file operations may not be optimized...","CorrelationID":"0577ef9c-e7bf-402c-ea87-f8ab50bf959f"}
> {"message":"03/04/2015 15:49:43.01*\tw3wp.exe (0x1B48)                       \t0x1654\tSharePoint Foundation         \tFiles                         \tak8dj\tHigh    \t...)     at Microsoft.SharePoint.Library...\t0577ef9c-e7bf-402c-ea87-f8ab50bf959f\r","@version":"1","@timestamp":"2015-08-03T14:14:51.994Z","host":"","path":"C:\\temp\\cAll\\SharePoint\\SP2013FOUND-20150304-1549_regular - Copy.log","tags":[],"parsedtime":"03/04/2015 15:49:43.01","multiline":"*","process":"w3wp.exe","processcode":"0x1B48","tid":"0x1654","area":"SharePoint Foundation         ","category":"Files                         ","eventID":"ak8dj","level":"High","eventmessage":"...)     at Microsoft.SharePoint.Library...","CorrelationID":"0577ef9c-e7bf-402c-ea87-f8ab50bf959f"}

Any idea how to isolate values from the field EventMessage and perform the requested operation, resulting with a single line with concatenated EventMessage field?

Thanks!


(system) #2