Apache2 SSL log Parsing Issue

[AlexB]

I'm using Filebeat to push Apache2 logs directly to Elasticsearch but I'm getting an error "Provided Grok expressions do not match field value:" when I try to add ssl_request_log file with two newly defined fields.

I've added the Grok pattern to the default.json injest in the apache2 module. I've added the two new SSL fields that didn't exist to the index. The expression works inside the Grok Debugger.

The json pattern is:

%{IPORHOST:apache2.access.remote_ip} \\[%{HTTPDATE:apache2.access.time}\\] %{DATA:apache2.access.ssl_protocol} %{DATA:apache2.access.ssl_cipher} %{WORD:apache2.access.method} %{DATA:apache2.access.url} HTTP/%{NUMBER:apache2.access.http_version} (?:%{NUMBER:apache2.access.body_sent.bytes}|-)

The sample data is:

10.10.75.120 [15/Aug/2018:16:16:06 -0400] TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256 GET /wp-content/plugins/awesome-weather/awesome-weather.css?ver=4.9.8 HTTP/1.1 -

But it's not parsing the message. The only difference between the \[ \] in the json and single backslash in the Grok Pattern. The default Apache2 json patterns are almost identical except for the two new fields I added to the index apache2.access.ssl_cipher and apache2.access.ssl_protocol.

I'm guessing I didn't add the fields properly to the index?

Edit: I tried removing the two custom ssl fields from the log and still get the same error.

Thanks for any suggestions!

[kvch (Noémi Ványi)]

Are you sure Apache pipeline is updated on Elasticsearch? By default Filebeat does not update Ingest pipelines if they already exist. You can force reloading the pipeline by running ./filebeat setup --pipelines -modules=apache2.

[AlexB]

Thank you!! That solved it. Once the pipeline was updated it parsed on the first try.