How does Logstash determine that a field matches a string regularly？

Elastic Stack Logstash
1

I am a newcomer to ELK, and now I encounter a problem as shown in the title: How does Logstash determine that a field matches a string regularly?
such as whether the [path] field contains the "err" string? (the path field means the filename such as "/var/log/game_err.log")
Is this how it is configured in logstash.conf?

Sincerely hope to get an answer, thank you!

2

Hello @wentao_Xiong

You are trying to get the file path if that is the case then use filebeat. To get the substring from the filename "err" then use grok pattern.

input
{
<Your-filebeats-configuration>
}

filter
{
grok
{
match => 
{
"message" => '/%{GREEDYDATA:path}/%{GREEDYDATA:filename}'
}
}

}
output
{
if [filename] == "game_err"
{
stdout{codec => jsondebug}
}
}
4

Hello~ @sudhagar_ramesh
Thanks a lot for your answer! I would like to elaborate on my situation:

The data I collect has the format of "/var/log/game_err.log" or "/xx/xx/xx_errxx.log" in the field [path] value, I need to judge whether the field [path] contains Some string "err"?

My logstash.conf is

input {
  beats {
    port => 5044
  }
}

filter {
  if [path] in "err" {
    mutate {
      add_field => {
        "log_type" => "err"
      }
    }
  }
  if [path] in "oss" {
    mutate {
      add_field => {
        "log_type" => "oss"
      }
    }
  }
}

output {
  if [log_type] == "err" {
    elasticsearch {
      hosts => ["http://******:9200"]
      index => "log-err-%{+YYYY.MM.dd}"
      user => "****"
      password => "****"
    }
  }else if [log_type] == "oss" {
    redis {
      host => "******"
      password => "****"
      port => 6379
      data_type => list
      db => 0
      key => "log-oss"
    }
  }
}

Why if [path] in "err" doesn't work?

Thanks again and looking forward to your reply!

5

I solved this problem，thanks ！

input {
  beats {
    port => 5044
  }
}

filter {
  if [log][file][path] =~ /err/ {
    mutate {
      add_field => {
        "log_type" => "err"
      }
    }
  }
  if [log][file][path] =~ /oss/ {
    mutate {
      add_field => {
        "log_type" => "oss"
      }
    }
  }
}

output {
  if [log_type] == "err" {
    elasticsearch {
      hosts => ["http://xxxxx:9200"]
      index => "log-err-%{+YYYY.MM.dd}"
      user => "xxxxxxxxx"
      password => "xxxx"
    }
  }else if [log_type] == "oss" {
    redis {
      host => "xxxxxxxx"
      password => "xxxx"
      port => 6379
      data_type => list
      db => 0
      key => "log-oss"
    }
  }

  stdout { codec => rubydebug }
}