Add new field from message

Hi,

I wonder if someone can help. I have looked online and I haven't found a solution. We have an event that passes a message like "error detected in request error code: [error code] please check"

Is there a way to be able to tell logstash to look for error detected and create a new field and populate with the name error code and the value of [error code].

Any assistance is appreciated

Hello Peter,

That is definately possible by using the Translate Filter: https://www.elastic.co/guide/en/logstash/current/plugins-filters-translate.html

filter {
  translate {
    field => "[error_code]"
    destination => "[error_description]"
    dictionary => {
      "100" => "Continue"
      "101" => "Switching Protocols"
      "200" => "OK"
      "500" => "Server Error"
    }
    fallback => "I'm a teapot"
  }
}

You can also use a dictionary file. i.e.:

dictionary_path => "/etc/logstash/dicts/translate-errorcode.yaml"

Probably the most challenging is the extraction of the error code from your source data. I would suggest some testing with a good grok match pattern.

grok { match => { "message" => "\[%{DATA:errorCode}\]" } }

I have used the Dissect filter to split the message into different fields which has worked however the way the message has been written it reads

Error : Client Id - 1 : Error Code - 2 : Description : Error has occurred : Employee ID - 1

When using the dissect I have the following mapping
"%{error}: %{company id} : %{Error Code} : %{Error Description}: %{Employee Id}"
is there a way to say for the company ID and error code I want the value after the - but in the Description i would like the value after the first : then a new field for employee id?

dissect { mapping => { "message" => "%{f0} : %{f1} - %{f2} : %{f3} - %{f4} : %{f5} : %{f6} : %{f7} - %{f8}" } }

will get you

        "f0" => "Error",
        "f1" => "Client Id",
        "f2" => "1",
        "f3" => "Error Code",
        "f4" => "2",
        "f5" => "Description",
        "f6" => "Error has occurred",
        "f7" => "Employee ID",
        "f8" => "1",

You can merge fields and substitute constant strings however suites you.

That worked Badger thanks.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.