My log looks like this:
User Name,Transaction,Target,Transaction Time,Total Time
PCGennes,POLICY LEVEL To INTEREST,PCG - TBA,01/09/2015 16:13:02:316 - 01/09/2015 16:13:04:100,1.78
,,,Request Time:,0.39
,,,Server Time:,0.92
,,,Response Time:,0.48
PCGennes,GENERIC To POLICY LEVEL,PCG - TBA,01/09/2015 16:12:33:725 - 01/09/2015 16:12:41:842,8.117
,,,Request Time:,6.518
,,,Server Time:,1.36
,,,Response Time:,0.239
PCGennes,HOME To GENERIC,PCG,01/09/2015 16:11:47:801 - 01/09/2015 16:11:50:723,2.922
,,,Request Time:,0.416
,,,Server Time:,2.141
,,,Response Time:,0.365
Config looks like this:
input {
stdin {
codec => multiline {
pattern => "^,"
what => "previous"
}
}
}
filter {
csv {
separator => ","
}
if ([column1] == "User Name") {
drop {}
}
}
output {
stdout { codec => json }
}
Here is the output. Notice the \n newline characters are still there, which means the CSV filter does not pick up the columns correctly:
{"@timestamp":"2015-09-04T08:21:06.045Z","message":["PCGennes,POLICY LEVEL To INTEREST,PCG - TBA,01/09/2015 16:13:02:316 - 01/09/2015 16:13:04:100,1.78\n,,,Request Time:,0.39\n,,,Server Time:,0.92\n,,,Response Time:,0.48"],"@version":"1","tags":["multiline"],"host":"bradschulzMBR.local","column1":"PCGennes","column2":"POLICY LEVEL To INTEREST","column3":"PCG - TBA","column4":"01/09/2015 16:13:02:316 - 01/09/2015 16:13:04:100","column5":"1.78"}{"@timestamp":"2015-09-04T08:21:06.046Z","message":["PCGennes,GENERIC To POLICY LEVEL,PCG - TBA,01/09/2015 16:12:33:725 - 01/09/2015 16:12:41:842,8.117\n,,,Request Time:,6.518\n,,,Server Time:,1.36\n,,,Response Time:,0.239"],"@version":"1","tags":["multiline"],"host":"bradschulzMBR.local","column1":"PCGennes","column2":"GENERIC To POLICY LEVEL","column3":"PCG - TBA","column4":"01/09/2015 16:12:33:725 - 01/09/2015 16:12:41:842","column5":"8.117"}