How to parse IP address from file /var/log/syslog in Logstash 10-syslog-conf


(Lewis Litchfield) #1

Time file syslog_message syslog_program received_from
November 26th 2015, 18:00:49.000 /var/log/syslog Invalid response packet from host 131.247.30.106.
avahi-daemon xen3ubut14dt-0


(Mark Walkom) #2

Some more information would be useful.

What is your current configuration? What version are you on?


(Lewis Litchfield) #3

I am as bad as my users, not supplying enough information
Ubuntu 14.04
Elasticsearch 2.0.0
Logstash 2.0.0
Kibana 4.2.0

The Kibana Discover columns
1.) Time November 26th 2015, 18:00:49.000
2.) file /var/log/syslog
3.) syslog_message Invalid response packet from host 131.247.30.106
4.) syslog_program avahi-daemon
5.) received_from xen3ubut14dt-0

None of the Available Fields have anything close to an IP Address only from the message field.


(Lewis Litchfield) #4

Logstash config

filter {
  if [type] == "syslog" {
    grok {
      match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{HOSTNAME:host_target} logname=%{USERNAME:logname}  sshd\[BASE10NUM}\]:  Failed password for (invalid user |)%{USERNAME:username} from %{IP:src_ip} port %{BASE10NUM:port} ssh2 %{SYSLOGHOST:syslog_hostname} tty=%{TTY:tty} ruser=%{USERNAME:ruser} rhost=(?:%{HOSTNAME:remote_host}|\s*) user=%{USERNAME:user} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
      add_field => [ "received_at", "%{@timestamp}" ]
      add_field => [ "received_from", "%{host}" ]
   add_field => [ "src_ip", "%{clientip}" ]
    }

filter {

if [type] == "syslog" {

grok {

}

grok {

type => "syslog"

match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{HOSTNAME:host_target} logname=%{USERNAME:logname} uid=%{BASE10NUM:uid} euid=%{BASE10NUM:euid} tty=%{TTY:tty} ruser=%{USERNAME:ruser} rhost=(?:%{HOSTNAME:remote_host}|\s*) user=%{USERNAME:user}" }

add_tag => "sudo_auth_failure"

#}

grok {

type => "syslog"

pattern => "%{SYSLOGTIMESTAMP:timestamp} %{HOSTNAME:host_target} sshd[%{BASE10NUM}]: Failed password for %{USERNAME:username} from %{IP:src_ip} port %{BASE10NUM:port} ssh2"

add_tag => "ssh_failed_login"

#}

grok {

type => "syslog"

pattern => "%{SYSLOGTIMESTAMP:timestamp} %{HOSTNAME:host_target} sshd[%{BASE10NUM}]: Accepted password for %{USERNAME:username} from %{IP:src_ip} port %{BASE10NUM:port} ssh2"

add_tag => "ssh_sucessful_login"

#}

    syslog_pri { }
    date {
      match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
    }
  }
}

(Mark Walkom) #5

That's really difficult to read, can you please edit and use the code formatting on your config.


(Lewis Litchfield) #6

I am a novice at this I do not know how to use code formatting. I found formatting is color coding with a special character created by alt+21 and a letter for the color, ┬že for yellow. Can you point me to an example?

Thanks.

Lewis Litchfield
Technology Services - Viking Center Lab
University of South Florida Sarasota-Manatee
941-359-4231


(Mark Walkom) #7

You can format highlighted text by using the buttons at the top of the text box.


(Lewis Litchfield) #8

Is this what you are looking for? 10-syslog.conf follows:
filter {

  •  if [type] == "syslog" {*
    
  •    grok {*
    
  •      match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{HOSTNAME:host_target} logname=%{USERNAME:logname}  sshd\[BASE10NUM}\]:  Failed password for (invalid user |)%{USERNAME:username} from %{IP:src_ip} port %{BASE10NUM:port} ssh2 %{SYSLOGHOST:syslog_hostname} tty=%{TTY:tty} ruser=%{USERNAME:ruser} rhost=(?:%{HOSTNAME:remote_host}|\s*) user=%{USERNAME:user} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }*
    
  •      add_field => [ "received_at", "%{@timestamp}" ]*
    
  •      add_field => [ "received_from", "%{host}" ]*
    
  •   add_field => [ "src_ip", "%{clientip}" ]*
    
  •    }*
    

}

  •    syslog_pri { }*
    
  •    date {*
    
  •      match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]*
    
  •    }*
    
  •  }*
    
  • }*

strong text


(system) #9