Goal is: for each line in the log, there should be a document in elastic containing the 'message' (text after time stamp). Each document should also contain fields for the the project name, plan name, and build #. <--this is where I'm getting stuck
example log snippet (first two lines, labeled line 1 and 2):
line 1: simple 01-Jan-2016 14:26:01 Build TestProj - Framework Code - Build #25 (TST-FC-25) started building on agent .NET Core 2
line 2: simple 01-Jan-2016 14:26:01 .NET-related builds, tests and publishing.
I have a Grok to get and create the fields I want - build name, build number, and project name (and have them as fields in elastic):
Is my end goal achievable with logstash and the Grok plugin alone? Or do I need ruby here ?
***NOTE: I'm using filebeat for shipping logs, and elastic does not recommend the multiline codec , so I'm curious what my other options are (assuming the multiline codec is even a viable option).
Going the ruby route assuming that is the best way...here is what I have which creates the field in elastic however it is coming up as "-" (I assume that means null or empty):
code => " @@projName = 'lineBelowSeemsToPopulateWithEmptyStr' @@projName = event['BamProjName'] event['BamProjectName'] = @@projName if @@projName.nil? || @@projName.empty? "