Add new field from message

Elastic Stack Logstash
1

Hi,

I wonder if someone can help. I have looked online and I haven't found a solution. We have an event that passes a message like "error detected in request error code: [error code] please check"

Is there a way to be able to tell logstash to look for error detected and create a new field and populate with the name error code and the value of [error code].

Any assistance is appreciated

2

Hello Peter,

That is definately possible by using the Translate Filter: https://www.elastic.co/guide/en/logstash/current/plugins-filters-translate.html

filter {
  translate {
    field => "[error_code]"
    destination => "[error_description]"
    dictionary => {
      "100" => "Continue"
      "101" => "Switching Protocols"
      "200" => "OK"
      "500" => "Server Error"
    }
    fallback => "I'm a teapot"
  }
}

You can also use a dictionary file. i.e.:

dictionary_path => "/etc/logstash/dicts/translate-errorcode.yaml"

Probably the most challenging is the extraction of the error code from your source data. I would suggest some testing with a good grok match pattern.

3 
grok { match => { "message" => "\[%{DATA:errorCode}\]" } }
4

I have used the Dissect filter to split the message into different fields which has worked however the way the message has been written it reads

Error : Client Id - 1 : Error Code - 2 : Description : Error has occurred : Employee ID - 1

When using the dissect I have the following mapping
"%{error}: %{company id} : %{Error Code} : %{Error Description}: %{Employee Id}"
is there a way to say for the company ID and error code I want the value after the - but in the Description i would like the value after the first : then a new field for employee id?

5 
dissect { mapping => { "message" => "%{f0} : %{f1} - %{f2} : %{f3} - %{f4} : %{f5} : %{f6} : %{f7} - %{f8}" } }

will get you

        "f0" => "Error",
        "f1" => "Client Id",
        "f2" => "1",
        "f3" => "Error Code",
        "f4" => "2",
        "f5" => "Description",
        "f6" => "Error has occurred",
        "f7" => "Employee ID",
        "f8" => "1",

You can merge fields and substitute constant strings however suites you.

6

That worked Badger thanks.