Grok jenkins console logs

[chaymaa]

Hi All,
I have a problem on groking jenkins console log, i have tested the filter instructions using input type file and it's OK but when i implement this configuration using a tcp input type the results is not the same.

the content of input file is:
002_SeL_Scenario_Realisation_Circuit_Depuis_Modele : Action1 [Connexion] Summary | Status : Passed
002_SeL_Scenario_Realisation_Circuit_Depuis_Modele : Action1 [Creation_Modele] Summary | Status : Passed
002_SeL_Scenario_Realisation_Circuit_Depuis_Modele : Action1 [Ajout_Document_Errone_Circuit] Summary | Status : Passed

logstash.conf:
input {
file {
path => ["c:/aaa.txt"]
start_position => "beginning"
}
}

filter {
grok {
match => {
"message" => "%{WORD:scenario} : %{WORD:action} %{DATA:feature} %{DATA:action} : %{WORD:status}"
}
remove_field => ["action"]
add_field => {"date" => "%{+YYYY.MM.dd}"}
}
}
Kibana results OK:
{ "path" => "c:/aaa.txt",
"status" => "Passed",
"feature" => "[connexion]",
"date" => "2019.08.08",
"message" => ""002_SeL_Scenario_Realisation_Circuit_Depuis_Modele : Action1 [Connexion] Summary | Status : Passed",\r",
"scenario" => "002_SeL_Scenario_Realisation_Circuit_Depuis_Modele",
}
{
"path" => "c:/aaa.txt",
"status" => "Passed",
"feature" => "[Creation_Modele]",
"date" => "2019.08.08",
"message" => ""002_SeL_Scenario_Realisation_Circuit_Depuis_Modele : Action1 [Creation_Modele] Summary | Status : Passed",\r",
"scenario" => "002_SeL_Scenario_Realisation_Circuit_Depuis_Modele",
}
The problem is there, when i implement that using tcp input, i expect the same result but is not true:
jenkins console logs:
Démarré par l'utilisateur Jenkins
Construction à distance sur windowsServer2 in workspace C:\Automatisation\workspace\Report
[WS-CLEANUP] Deleting project workspace...
C:\Automatisation\workspace\Report>exit 0
[Report] $ cmd /c call C:\Users\ADMINI~1.WIN\AppData\Local\Temp\2\jenkins8691771370815427256.bat
C:\Automatisation\workspace\Report>C:/Automatisation/affichage.bat
002_SeL_Scenario_Realisation_Circuit_Depuis_Modele : Action1 [Connexion] Summary | Status : Passed
002_SeL_Scenario_Realisation_Circuit_Depuis_Modele : Action1 [Creation_Modele] Summary | Status : Passed

logstash conf:
input { tcp{port =xxxx }}
filter {
json{
sources => "message"}
grok {
match => {
"message" => "%{WORD:scenario} : %{WORD:action} %{DATA:feature} %{DATA:action} : %{WORD:status}"
}
remove_field => ["action"]
}
}

kibana result:
{
message => "..... Démarré par l'utilisateur ...."
"status" => [
[ 0] "Passed",
[ 1] "Passed",
],
"feature" => [
[ 0] "[Connexion]",
[ 1] "[Creation_Modele]",
],
"scenario" => [
[ 0] "002_SeL_Scenario_Realisation_Circuit_Depuis_Modele",
[ 1] "002_SeL_Scenario_Realisation_Circuit_Depuis_Modele",
],
}
i dont have scenario , feature and status in separated document, i have one document containing table of values.
When i remove json{ ..} part i have one document result contains the grok result of the first line (002_SeL_Scenario_Realisation_Circuit_Depuis_Modele : Action1 [Connexion] Summary | Status : Passed) the following lines ares not goked.

can someone help me please, thnx in advance.

[chaymaa]

Hi all,

The problem is resolved, i had grok my log as following:

filter {
json{source => "message"}
split{ field => "message"}
grok {
match => {
"message" => "%{WORD:scenario} : %{WORD:action} %{DATA:feature} %{DATA:action} : %{WORD:status}"
}
remove_field => ["action"]
add_field => {"date" => "%{+YYYY.MM.dd}"}
remove_field => ["data"]
}