Hello everyone out there!
I recently started using ELK so sorry if my question silly.
I need help with modifying my grok filter to achieve particular set of data. Tried lot but could not so looking for help.
Log file sample:--
2017-08-23 16:00:02,106 INFO : Test run configuration: RunId: a6e5a151-197a-48d7-86c5-6714ced296b7, Team: Learning, FrameWork:CTFXUnit, Branch: release, Environment:QA05, Portal:QA052_AUTO_LMS_SIGNOFF, Browser:Chrome, EnableDatabaseLog:True, EnableTestRailLog:True, TestRailConfiguration:Milestone: , TestRunType: TestPlan, TestPlan:LMS_Regression_Xunit, UpdateExistingTestRailRun: True., DebugLogLevel:2, TraitFilters:Name:Category, Operator:Equal, Value:Dothraki,
2017-08-23 16:00:02,121 INFO : Test Run:a6e5a151-197a-48d7-86c5-6714ced296b7, Status: Ready.
2017-08-23 16:00:05,377 INFO : Downloading items from TFS please wait...
2017-08-23 16:01:15,357 INFO : Runing batch file:'RestoreNuGetPackages.bat' started.
2017-08-23 16:01:18,693 INFO : Runing batch file:'RestoreNuGetPackages.bat' completed
2017-08-23 16:01:18,693 INFO : Test Run:a6e5a151-197a-48d7-86c5-6714ced296b7, Status: SourceControlFilesCopied.
2017-08-23 16:01:18,946 INFO : Runing batch file:'C:\temp\a6e5a151-197a-48d7-86c5-6714ced296b7\TestScript\LMS.XUnit.ObjectModels\AutoItX\RegisterAutoIt.bat' started.
2017-08-23 16:01:19,143 INFO : Runing batch file:'C:\temp\a6e5a151-197a-48d7-86c5-6714ced296b7\TestScript\LMS.XUnit.ObjectModels\AutoItX\RegisterAutoIt.bat' completed
2017-08-23 16:01:19,329 INFO : MSBuild in progress...
2017-08-23 16:01:39,354 INFO : MSBuild completed...
2017-08-23 16:01:39,354 INFO : Test Run:a6e5a151-197a-48d7-86c5-6714ced296b7, Status: MSBuildCompleted.
2017-08-23 16:01:39,594 INFO : Test Run:a6e5a151-197a-48d7-86c5-6714ced296b7, Status: UpdateTestRunConfigCompleted.
2017-08-23 16:01:39,598 INFO : Test Run:a6e5a151-197a-48d7-86c5-6714ced296b7, Status: Running.
2017-08-23 16:01:40,267 INFO : Discovering tests...
2017-08-23 16:01:40,268 INFO : Apply test case filter: Name:Category, Operator:Equal, Value:Dothraki.
2017-08-23 16:01:43,370 INFO : Discovered 12 of 337 tests...
2017-08-23 16:01:50,528 INFO : Running: LMS.XUnit.Tests.CreateRelativeCertTest.CreateRelativeCertification.
2017-08-23 16:17:18,856 ERROR: [FAIL] LMS.XUnit.Tests.CreateRelativeCertTest.CreateRelativeCertification: Object reference not set to an instance of an object..
2017-08-23 16:17:18,856 ERROR: at CSOD.Automation.Selenium.WebControl.get_Enabled() in C:\TeamCity\buildAgent\work\4c2b8c219adc5e13\CSOD.Automation.Selenium\WebControl.cs:line 134
at LMS.XUnit.ObjectModels.LMSWebControl.get_Enabled()
at LMS.XUnit.ObjectModels.LMSWebControl.VerifyAndExecute(Action action, Boolean checkEnabled, Boolean checkDisplayed)
at LMS.XUnit.ObjectModels.LMSWebControl.Click()
at LMS.XUnit.Tests.CreateCertificationSD.CreateRelativeScheduling()
at LMS.XUnit.Tests.CreateRelativeCertTest.CreateRelativeCertification()
2017-08-23 16:17:18,869 INFO : Running: LMS.XUnit.Tests.CreateOneTimeCertTest.CreateOneTimeCertification.
2017-08-23 16:24:23,726 INFO : [PASS] LMS.XUnit.Tests.CreateOneTimeCertTest.CreateOneTimeCertification
2017-08-23 16:24:23,735 INFO : Running: LMS.XUnit.Tests.CreateOneTimeCertTest.UserCompletesOneTimeCert.
2017-08-23 16:29:19,516 INFO : [PASS] LMS.XUnit.Tests.CreateOneTimeCertTest.UserCompletesOneTimeCert
Grok filter for logstash config:
filter {
grok {
match => { "message" => "%{TIMESTAMP_ISO8601:timestamp} INFO : Test run configuration: RunId: (?<runid>[^\s]*), Team: (?<Team>[^\s]*), FrameWork:CTFXUnit, Branch: (?<Branch>[^\s]*), Environment:(?<Environment>[^\s]*), Portal:(?<Portal>[^\s]*), Browser:(?<Browser_Type>[^\s]*), EnableDatabaseLog:True, EnableTestRailLog:True, TestRailConfiguration:Milestone: , TestRunType: TestPlan, TestPlan:(?<Test_Plan>[^\s]*), UpdateExistingTestRailRun: True., DebugLogLevel:2, TraitFilters:Name:Category, Operator:Equal, Value:(?<Trait_Value>[^\s]*),"}
match => { "message" => "%{TIMESTAMP_ISO8601:timestamp} ERROR: \[(?<Test_Run_Status>[^\s]*)\] LMS.XUnit.Tests.(?<Test_Name>[^\s]*)"}
match => { "message" => "%{TIMESTAMP_ISO8601:timestamp} INFO : \[(?<Test_Run_Status>[^\s]*)\] LMS.XUnit.Tests.(?<Test_Name>[^\s]*)"}
}
date {
match => [ "timestamp", "ISO8601"]
remove_field => ["timestamp"]
}
if "_grokparsefailure" in [tags]
{ drop {} }
}
Output in Kibana:--
The issue i have is, as can be seen from output in kibana, many of my rows does not have value. One of the row has values which i want to get populated for all the rows depending on condition. Can someone please help me here.
Extreme thanks in advance.