I currently have logstash sending to Elastic cloud - I'd like to get that message section parsed out a bit so that its not one giant section. Could someone help me with that?
Here's my filter section - it is just the "out of the box" code
filter {
if [type] == "syslog" {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
add_field => [ "received_at", "%{@timestamp}" ]
add_field => [ "received_from", "%{host}" ]
}
date {
match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
}
Here's the syslog as parsed by logstash:
Mar 25 12:14:54 mid-mon1 logstash[938]: {
Mar 25 12:14:54 mid-mon1 logstash[938]: "host" => "10.X.XX.XXX",
Mar 25 12:14:54 mid-mon1 logstash[938]: "@version" => "1",
Mar 25 12:14:54 mid-mon1 logstash[938]: "@timestamp" => 2021-03-25T17:14:53.793Z,
Mar 25 12:14:54 mid-mon1 logstash[938]: "type" => "syslog",
Mar 25 12:14:54 mid-mon1 logstash[938]: "message" => "<163>1 2021-03-25T12:14:53-05:00 mid-XXX wafd - - [meta sequenceId=\"155\"] [client 94.XXX.176.XXX] ModSecurity: Warning. String match \"multipart\" at REQUEST_HEADERS:Content-Type. [file \"/tmp/waf/10/modsecurity_slr_46_known_vulns.conf\"] [line \"720\"] [id \"2170100\"] [rev \"11202017\"] [msg \"SLR: Apache Struts (Body inspection Enabled)\"] [tag \"application-struts\"] [tag \"language-java\"] [tag \"platform-multi\"] [tag \"attack-injection\"] [hostname \"172.XX.X.XX (HTTP_redirect_no_cert)\"] [uri \"/\"] [unique_id \"3f77d992-0563-42a1-b62c-fc739ba7a124\"]\n",
Mar 25 12:14:54 mid-mon1 logstash[938]: "tags" => [
Mar 25 12:14:54 mid-mon1 logstash[938]: [0] "_grokparsefailure"
Mar 25 12:14:54 mid-mon1 logstash[938]: ]
Mar 25 12:14:54 mid-mon1 logstash[938]: }
Any help in parsing the message section would be much appreciated.
