Kv filter doubts

can you please provide me with the link for logstash documentation ?

Seriously? Googling "logstash upgrade plugin" doesn't result in relevant hits?

will check it

@magnusbaeck assume if the field is status=0, i can give the given the condition if ["status"]="0", but what if there is no field status at all. how do i give a "if" for that

See https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html#conditionals, specifically the section that starts with "You can check for the existence of a specific field".

Thanks it worked, if (![fieldname).

if a string is empty, how to check that? i tried using null and nil, it did not work

if [field] == "" {

It did not work. No error was thrown but it tok the following characters after the "" as the value.

e.g in the input log file -{"status": ""}{"userid":"tom"}
after parsing -"status"=>{"userid":"tom"}
but the expected output is not this

I don't understand. Please show

  • an example line of input,
  • your Logstash configuration,
  • what Logstash produces with a stdout { codec => rubydebug } output, and
  • what you expected to get instead.

Input Logfile:

CRM-WEB [INFO] 2018/03/22 00:03:33.717 Interface userInfo InputData:{"username":"bdh","errorcode":"0020"},OutputData:{"usernm":"aj","ido":"2125"} [Locale:English][URL:webpage] [c.c..ghij][webc:100]
CRM-WEB [INFO] 2018/03/22 00:03:33.717 Interface userInfo InputData:{"username":"bdh","errorcode":""}OutputData:{"usernm":"aj","ido":"2125"} [Locale:English][URL:webpage] [c.c..ghij][web:010]

Config file:

input
{
file
{
path => "/home/murugar/Documents/checkpilot2.log"
type => "file"
start_position => "beginning"
sincedb_path=> "/dev/null"

    }

}
filter
{

grok
    {
    match => ["message","%{DATA:applicationtype}\s\[%{LOGLEVEL:loglevel}\]\s\s%{DATA:logdate}\s\s\s%{GREEDYDATA:msg}"]
    }

    if "_grokparsefailure" in [tags]
    {
    drop{}

    }

 kv
    {
    source => "msg"
    transform_key => "lowercase"
    field_split => "\]\[,"
    value_split => ":="
    }

date
{
match => ["logdate","yyyy/MM/dd HH:mm:ss.SSS"]
target => "@timestamp"

    }

if ["errorcode"] == ""
{
mutate
{
add_field => {"ErrorDescription" => " No errorcode is present"}
}

}
else if ["errorcode"] = "0020"
{
mutate
{
add_field => {"ErrorDescription" => The error in networks"}
}

}

else if ["errorcode"] == "003"
{
mutate
{
add_field => {"ErrorDescription" => "Time is blank or wrong format"}
}
}
(exceptions go on)
}
output
{
stdout
{
codec => rubydebug
}
}

The output in the console

"message" => "CRM-WEB [INFO] 2018/03/22 00:03:33.717 Interface userInfo InputData:{"username":"bdh","errorcode":"0020"},OutputData:{"usernm":"aj","ido":"2125"} [Locale:English][URL:webpage] [c.c..ghij][webc:100]",
"@version" => "1",
"@timestamp" => "2018-03-22T04:03:33.717Z",
"path" => "/home/murugar/Documents/checkpilot2.log",
"host" => "0.0.0.0",
"type" => "file",
"applicationtype" => "CRM-WEB",
"loglevel" => "INFO",
"logdate" => "2018/03/22 00:03:33.717",
"msg" => "Interface userInfo InputData:{"username":"bdh","errorcode":"0020"},OutputData:{"usernm":"aj","ido":"2125"} [Locale:English][URL:webpage] [c.c..ghij][webc:100]",
"Interface userInfo InputData" => "{"username":"bdh"",
""errorcode"" => "0020",
"OutputData" => "{"usernm":"aj"",
""ido"" => "2125",
"Locale" => "English",
"URL" => "webpage",
"webc" => "100",
"ErrorDescription" => "Error in networks"

"message" => "RM-WEB [INFO] 2018/03/22 00:03:33.717 Interface userInfo InputData:{"username":"bdh","errorcode":""}OutputData:{"usernm":"aj","ido":"2125"} [Locale:English][URL:webpage] [c.c..ghij][web:010]",
"@version" => "1",
"@timestamp" => "2018-03-22T04:03:33.717Z",
"path" => "/home/murugar/Documents/checkpilot2.log",
"host" => "0.0.0.0",
"type" => "file",
"applicationtype" => "RM-WEB",
"loglevel" => "INFO",
"logdate" => "2018/03/22 00:03:33.717",
"msg" => "Interface userInfo InputData:{"username":"bdh","errorcode":""}OutputData:{"usernm":"aj","ido":"2125"} [Locale:English][URL:webpage] [c.c..ghij][web:010]",
"Interface userInfo InputData" => "{"username":"bdh"",
""errorcode"" => """}OutputData:{"usernm":"aj"",
""ido"" => "2125",
"Locale" => "English",
"URL" => "webpage",
"web" => "010"
Expected Output:

the field "ErrorDescrition" => "No errorcode is present" should be in the second snippet but that field is not added. rest of the output is fine

the field "ErrorDescrition" => "No errorcode is present" should be in the second snippet but that field is not added.

Right, because msg contains "errorcode":"" instead of "errorcode":"0020". Do you expect error code to propagate from one message to the next? Or why would you expect the second event to get the same errorcode value as the first one?

Magnus it is not about the rror code vale. Error code values are already present in the given input log file. If errorcode:"", then that corressponding if condition should be executed which is the addition of error description: no errorcode is present. Why isnt that if condition is executed.

There is no control over error values. Those are predefined in the input log files. But the corresponding "if conditions" for the error values should be executed accordingly. why is it not happening in the second snippet case?

For errorcode:0020, i got the field error description field which reads "errror in networks" as shown is rubydebug output but why not for the second case which should read as " no errorcode is present"

if ["errorcode"] == ""

That's the wrong syntax. Change to:

if [errorcode] == ""

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.