Logstash if statement for program

Elastic Stack Logstash
1

hi
i know its elementary but i am stuck at an if statement

with if statement I get grok failure
and without it every thing is fine
I'm using logstash 7.7

indent preformatted text by 4 spaces
```
input {stdin{}}
filter {
    if [program] == "sshd" {
        grok {
            patterns_dir   => "${LL_PATTERN_DIR:/etc/logstash/patterns.d}"
            match => { "message" => "%{SSH_AUTHFAIL_WRONGCREDS}" }
            add_field => { "ssh_authresult" => "fail" "ssh_failreason" => 
"wrong_credentials" }
            add_tag => [ "_grok_sshd_success", "matched" ]
        }
        grok {
            patterns_dir   => "${LL_PATTERN_DIR:/etc/logstash/patterns.d}"
            match => { "message" => "%{SSH_AUTHFAIL_WRONGUSER}" }
            add_field => { "ssh_authresult" => "fail" "ssh_failreason" => 
"unknown_user" }
            add_tag => [ "_grok_sshd_success", "matched" ]
        }
        grok {
            patterns_dir   => "${LL_PATTERN_DIR:/etc/logstash/patterns.d}"
            match => { "message" => "%{SSH_AUTH_SUCCESS}" }
            add_field => { "ssh_authresult" => "success" }
            add_tag => [ "_grok_sshd_success", "matched" ]
        }
        grok {
            patterns_dir   => "${LL_PATTERN_DIR:/etc/logstash/patterns.d}"
            match => { "message" => "%{SSH_DISCONNECT}" }
            add_tag => [ "_grok_sshd_success", "matched", "ssh_disconnect" ]
        }
        mutate {
            remove_tag => [ "matched", "_grokparsefailure" ]
        }
        geoip {
            source => "ssh_client_ip"
        }
    }
}
output{stdout{codec => rubydebug}}
```

log sample

```
2020-05-24T01:11:11+04:30 ldap sshd[29859]: Accepted publickey for cadmin from 192.168.2.32 port 44192 ssh2: RSA SHA256:ORw82lHe311CAgeD08StZzO31tlRZJcddddddaGx6Kg
```
2

Could you please provide your custom grok pattern for below variables ?

SSH_AUTHFAIL_WRONGCREDS
SSH_AUTHFAIL_WRONGUSER
SSH_AUTH_SUCCESS
SSH_DISCONNECT

I would like to use it for testing what you are facing.

3

thanks for the response
i suspect some requirement doesn't meet or something

its from this source and I have modified groks to my need

indent preformatted text by 4 spaces
SSH_AUTHFAIL_WRONGUSER Failed %{WORD:ssh_authmethod} for invalid user %{USERNAME:ssh_user} from %{IP:ssh_client_ip} port %{NUMBER:ssh_client_port} (?<ssh_protocol>\w+\d+)
SSH_AUTHFAIL_WRONGCREDS Failed %{WORD:ssh_authmethod} for %{USERNAME:ssh_user} from %{IP:ssh_client_ip} port %{NUMBER:ssh_client_port} (?<ssh_protocol>\w+\d+)
SSH_AUTH_SUCCESS Accepted %{WORD:ssh_authmethod} for %{USERNAME:ssh_user} from %{IP:ssh_client_ip} port %{NUMBER:ssh_client_port} (?<ssh_protocol>\w+\d+)(?:: %{WORD:ssh_pubkey_type} %{GREEDYDATA:ssh_pubkey_fingerprint})?
SSH_DISCONNECT Received disconnect from %{IP:ssh_client_ip} port %{INT:ssh_client_port}.*?:\s+%{GREEDYDATA:ssh_disconnect_reason}
4

Thanks for the reply. I have tested on logstash 7.7.0 .

logstash_1 | [2020-05-24T23:19:20,436][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"7.7.0"}

Hmm.

From the log line you have provided , it looks filter behaves totally normal to me.

logstash_1  | /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/awesome_print-1.7.0/lib/awesome_print/formatters/base_formatter.rb:31: warning: constant ::Fixnum is deprecated
logstash_1  | {
logstash_1  |                    "message" => "2020-05-24T01:11:11+04:30 ldap sshd[29859]: Accepted publickey for cadmin from 192.168.2.32 port 44192 ssh2: RSA SHA256:ORw82lHe311CAgeD08StZzO31tlRZJcddddddaGx6Kg",
logstash_1  |               "ssh_protocol" => "ssh2",
logstash_1  |     "ssh_pubkey_fingerprint" => "SHA256:ORw82lHe311CAgeD08StZzO31tlRZJcddddddaGx6Kg",
logstash_1  |            "ssh_pubkey_type" => "RSA",
logstash_1  |                 "@timestamp" => 2020-05-24T23:19:45.942Z,
logstash_1  |                      "geoip" => {},
logstash_1  |            "ssh_client_port" => "44192",
logstash_1  |                       "tags" => [
logstash_1  |         [0] "_grok_sshd_success",
logstash_1  |         [1] "_geoip_lookup_failure"
logstash_1  |     ],
logstash_1  |             "ssh_authresult" => "success",
logstash_1  |                    "program" => "sshd",
logstash_1  |                   "ssh_user" => "cadmin",
logstash_1  |                   "@version" => "1",
logstash_1  |                       "host" => "logstash",
logstash_1  |                       "path" => "/mnt/logs/sshd.log",
logstash_1  |             "ssh_authmethod" => "publickey",
logstash_1  |              "ssh_client_ip" => "192.168.2.32"
logstash_1  | }
logstash_1  | [2020-05-24T23:19:46,834][INFO ][logstash.runner          ] Logstash shut down.

It matches 3rd grok filter.

        grok {
            patterns_dir   => "${LL_PATTERN_DIR:/etc/logstash/patterns.d}"
            match => { "message" => "%{SSH_AUTH_SUCCESS}" }
            add_field => { "ssh_authresult" => "success" }
            add_tag => [ "_grok_sshd_success", "matched" ]
        }

geo_ip fails because 192.168.2.32 is not a global ip .

Could you provide the ruby debug output you are facing ?

Below is the configuration I have used for testing .

input {                                                                                                                                                                                       
  file {                                                                                                                                                                                      
    path => [                                                                                                                                                                                 
      "/mnt/logs/sshd.log"                                                                                                                                                                    
    ]                                                                                                                                                                                         
    mode => "read"                                                                                                                                                                            
    start_position => "beginning"                                                                                                                                                             
    exit_after_read => "true"                                                                                                                                                                 
    file_completed_action => "log"                                                                                                                                                            
    file_completed_log_path => "/dev/null"                                                                                                                                                    
    sincedb_path => "/dev/null"
  }
}
filter {
    # For testing
    mutate { add_field => { "program" => "sshd"} }

    if [program] == "sshd" {
        grok {
            patterns_dir   => "${LL_PATTERN_DIR:/etc/logstash/patterns.d}"
            match => { "message" => "%{SSH_AUTHFAIL_WRONGCREDS}" }
            add_field => { "ssh_authresult" => "fail" "ssh_failreason" => "wrong_credentials" }
            add_tag => [ "_grok_sshd_success", "matched" ]
        }
        grok {
            patterns_dir   => "${LL_PATTERN_DIR:/etc/logstash/patterns.d}"
            match => { "message" => "%{SSH_AUTHFAIL_WRONGUSER}" }
            add_field => { "ssh_authresult" => "fail" "ssh_failreason" => "unknown_user" }
            add_tag => [ "_grok_sshd_success", "matched" ]
        }
        grok {
            patterns_dir   => "${LL_PATTERN_DIR:/etc/logstash/patterns.d}"
            match => { "message" => "%{SSH_AUTH_SUCCESS}" }
            add_field => { "ssh_authresult" => "success" }
            add_tag => [ "_grok_sshd_success", "matched" ]
        }
        grok {
            patterns_dir   => "${LL_PATTERN_DIR:/etc/logstash/patterns.d}"
            match => { "message" => "%{SSH_DISCONNECT}" }
            add_tag => [ "_grok_sshd_success", "matched", "ssh_disconnect" ]
        }
        mutate {
            remove_tag => [ "matched", "_grokparsefailure" ]
        }
        geoip {
            source => "ssh_client_ip"
        }
    }
}

output {
  stdout { }
}
6

i have found out if i do import filter and output in the same file and run it with logstash -f it does everything right but when it runs from systemctl it wond work

7

I am glad that if statement has worked out.

For your information , if you have installed logstash from package manager and using systemd ,
mostly you have to place logstash configuration file under /etc/logstash/conf.d .

8

thanks alot
the if statement works fine there is something seriusly worng and i cant tshoot it alone
i will appreciate if you could check this post out

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.