I am having difficulty parsing the below.
Actual rsyslog line.
2018-06-18T12:36:55.588494-05:00 lb-workers.awsprod.com haproxy[2639]: 172.0.0.1:27947 [18/Jun/2018:12:36:55.587] application~ payment_backend/<NOSRV> 0/-1/-1/-1/0 5031593 - - SCNN 27/27/0/0/0 0/0 {50.0.0.0} "GET /v2/ping HTTP/1.1"
What's received by Logstash.
{
"message":"<13>Jun 18 12:36:55 10.0.0.0 {\"@timestamp\": \"2018-06-18T12:36:55.588494-05:00\",\"message\":\" 172.0.0.1:27947 [18\\/Jun\\/2018:12:36:55.587] application~ payment_backend\\/<NOSRV> 0\\/-1\\/-1\\/-1\\/0 503 1593 - - SCNN 27\\/27\\/0\\/0\\/0 0\\/0 {50.0.0.0} \\\"GET \\/v2\\/ping HTTP\\/1.1\\\"\",\"@fields\":{\"host\":\"lb-workers.awsprod.com\",\"syslog_facility\":\"local0\",\"syslog_facility_code\":\"16\",\"syslog_severity\":\"info\",\"syslog_severity_code\":\"6\",\"program\":\"haproxy\",\"pid\":\"2639\" }}"
}
Unescaped version of above.
{
"message":"<13>Jun 18 12:36:55 10.0.0.0 {"@timestamp": "2018-06-18T12:36:55.588494-05:00","message":" 172.0.0.1:27947 [18/Jun/2018:12:36:55.587] application~ payment_backend/<NOSRV> 0/-1/-1/-1/0 503 1593 - - SCNN 27/27/0/0/0 0/0 {50.0.0.0} "GET /v2/ping HTTP/1.1"","@fields":{"host":"lb-workers.awsprod.com","syslog_facility":"local0","syslog_facility_code":"16","syslog_severity":"info","syslog_severity_code":"6","program":"haproxy","pid":"2639" }}"
}
What do I need to do and in what order?
- Parse "message" as JSON
- Unescape log line
- Apply GROK filter.
I believe the fields should correspond to a SYSLOG grok pattern but I"m having difficulty with the whole thing.