Filebeat Apache Module Ingest Pipeline

I am currently working on formatting the @timestamp in access logs for the apache2 module in Filebeat. Currently getting the following error message on kibana: "Provided Grok expressions do not match field value:"

My current ingest pipeline is the following:
{
"description": "Pipeline for parsing Apache2 access logs. Requires the geoip and user_agent plugins.",
"processors": [{
"grok": {
"field": "message",
"patterns":[
"%{IPORHOST:apache2.access.remote_ip} - %{DATA:apache2.access.user_name} \[%{HTTPDATE:apache2.access.time}\] "(?:%{WORD:apache2.access.method} %{DATA:apache2.access.url} HTTP/%{NUMBER:apache2.access.http_version}|-)?" %{NUMBER:apache2.access.response_code} (?:%{NUMBER:apache2.access.body_sent.bytes}|-)( "%{DATA:apache2.access.referrer}")?( "%{DATA:apache2.access.agent}")?",
"%{IPORHOST:apache2.access.remote_ip} - %{DATA:apache2.access.user_name} \[%{HTTPDATE:apache2.access.time}\] "-" %{NUMBER:apache2.access.response_code} -",
"\[%{HTTPDATE:apache2.access.time}\] %{IPORHOST:apache2.access.remote_ip} %{DATA:apache2.access.ssl.protocol} %{DATA:apache2.access.ssl.cipher} "%{WORD:http.request.method} %{DATA:apache2.access.url} HTTP/%{NUMBER:apache2.access.http_version}" %{NUMBER:apache2.access.body_sent.bytes}"
],
"ignore_missing": true
}
},{
"remove":{
"field": "message"
}
}, {
"rename": {
"field": "@timestamp",
"target_field": "read_timestamp"
}
}, {
"date": {
"field": "apache2.access.time",
"target_field": "@timestamp",
"formats": ["yyyy-MM-dd'T'HH:mm:ss.SSS'Z'"],
"ignore_failure": false
}
}, {
"remove": {
"field": "apache2.access.time",
"ignore_failure": true
}
}, {
"rename": {
"field": "apache2.access.agent",
"target_field": "apache2.access.user_agent.original",
"ignore_failure": true
}
}],
"on_failure" : [{
"set" : {
"field" : "error.message",
"value" : "{{ _ingest.on_failure_message }}"
}
}]
}

It was working earlier before i added the milliseconds to the format. I am trying to format the @timestamp to be something like 2019-08-20T00:02:15.000Z. Is there are problem with the time format or something else in the pipeline.

Best Regards,
Juan Vicente

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.