How to extract an Information from an attribute?

Elastic Stack Logstash
1

Hello elastic community,

I have a problem with one of my Logstash parsing configurations.

One customer want to use a country field. The information to fill the country field I can extract from the path, but I don't know exactly how to do this. All my tries weren't successfull to realize it.

my Code look like:

input {
    file {
            path => [ "/was/log//1/mip/vn/1/server.log" ]
            type => "serverlog"
            codec => multiline {
                    pattern => "^%{TIME}"
                    negate => true
                    what => previous
            }
    }

}

filter {
    if [type] == "serverlog" {
            mutate {
                    add_field => { "stage" => "PROD" }
            }
            grok {
                    match => [ "message", "%{TIME:log_timestamp} %{DATA:log_level}%{SPACE}\[%{DATA:loggername}\] \(%{DATA:log_info}\)%{SPACE}%{GREEDYDATA:message}" ]
                    match => [ "path", "/was/log/1/mip/%{DATA:Country}/%{GREEDYDATA}" ]
                    overwrite => [ "message" ]
            }
            mutate {
                    uppercase => [ "Country" ]
            }
    }

}

I expected by using this configuration, that the Country field will be filled, but this do not work.

I hope that somebody have an idea how I can realize the customers request.

Greetz

2

By the default the grok filter breaks after the first successful match, so as long as your first expression matches it'll never look at your path field. Split the filter in two.

grok {
  match => [ "message", "%{TIME:log_timestamp} %{DATA:log_level}%{SPACE}\[%{DATA:loggername}\] \(%{DATA:log_info}\)%{SPACE}%{GREEDYDATA:message}" ]
  overwrite => [ "message" ]
}
grok {
  match => [ "path", "/was/log/1/mip/%{WORD:Country}/" ]
}

Additional comments:

  • To avoid surprises one should try hard to avoid more than one DATA or GREEDYDATA pattern in a single expression. In this case DATA is unnecessarily broad.
  • A GREEDYDATA pattern at the end of an expression serves no purpose.
1 Like
3

Hello Magnus,

thank you very much for you help. I could realize the request with the help of your solution.

Greetz