Apache Access logs not appearing in Elasticsearch/Kibana

Elastic Stack Logstash
1

Hi,

We’ve been successfully ingesting our Apache logs however as of around 2 weeks ago the access logs have stopped appearing however the error logs are still showing up in ES/Kibana. We did upgrade from 8.19.2 to 8.19.3 around that time but the timing doesn’t 100% correlate.

There are no error logs appearing in either logstash or ES and if I create an output file filter in logstash I see entries for the Apache access logs in there:

{"message":"10.65.193.100 - - [19/Sep/2025:14:44:13 +0100] \"GET /status/api/server_logs/status HTTP/1.1\" 401 802 \"https://server.foo/status/app/server_logs\" \"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:142.0) Gecko/20100101 Firefox/142.0\"","user_agent":{"name":"Firefox","device":{"name":"{\"name\":\"Other\"}"},"original":"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:142.0) Gecko/20100101 Firefox/142.0","os":{"name":"%{[ua_tmp][os_name]}"}},"@version":"1","host":{"name":"node.foo"},"apache":{"access":{}},"agent":{"hostname":"node.foo","version":"7.17.22","ephemeral_id":"ba05901f-49f8-4c38-91bd-f2f44acec47e","name":"node.foo","type":"filebeat","id":"8cc85ed2-6fec-4c43-833a-0c51d16ae3dc"},"source":{"address":"10.65.193.100","ip":["10.65.193.100","10.65.193.100"],"as":{},"geo":{}},"http":{"response":{"body":{"bytes":"802"},"status_code":"401"},"request":{"method":"get","referrer":"https://server.foo/status/app/server_logs"},"version":"1.1"},"labels":{"tier":"test"},"url":{"original":"/status/api/server_logs/status"},"event":{"outcome":"failure","created":"2025-09-19T13:44:13.046Z","category":"web","kind":"event","original":"10.65.193.100 - - [19/Sep/2025:14:44:13 +0100] \"GET /status/api/server_logs/status HTTP/1.1\" 401 802 \"https://server.foo/status/app/server_logs\" \"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:142.0) Gecko/20100101 Firefox/142.0\"","module":"apache","dataset":"apache.access","type":"access"},"ecs":{"version":"1.12.0"},"fileset":{"name":"access"},"service":{"name":"service","type":"class"},"log":{"file":{"path":"/var/log/apache2/moo_access_log"},"offset":23417700},"input":{"type":"log"},"tags":["beats","beats_input_codec_plain_applied","_dateparsefailure","_geoip_lookup_failure"]}

(We were seeing dateparsefailures previously and in the successfully-stored error logs.

We’re using the following grok rules to capture + tokenise our Apache access logs (this hasn’t changed).

      grok {
        match => {
          "message" => [
            "%{IPORHOST:[destination][domain]} %{IPORHOST:[source][ip]} - (-|%{D
ATA:[user][name]}) \[%{HTTPDATE:[apache][access][time]}\] \"(?:%{WORD:[http][req
uest][method]} %{DATA:[url][original]} HTTP/%{NUMBER:[http][version}|-)?\" %{NUM
BER:[http][response][status_code]:long} (?:%{NUMBER:[http][response][body][bytes
]:long}|-)( \"%{DATA:[http][request][referrer]}\")?( \"%{DATA:[user_agent][origi
nal]}\")?",
            "%{IPORHOST:[source][address]} - (-|%{DATA:[user][name]}) \[%{HTTPDA
TE:[apache][access][time]}\] \"(?:%{WORD:[http][request][method]} %{DATA:[url][o
riginal]} HTTP/%{NUMBER:[http][version]}|-)?\" %{NUMBER:[http][response][status_
code]:long} (?:%{NUMBER:[http][response][body][bytes]:long}|-)( \"%{DATA:[http][
request][referrer]}\")?( \"%{DATA:[user_agent][original]}\")?",
            "%{IPORHOST:[source][address]} - (-|%{DATA:[user][name]}) \[%{HTTPDA
TE:[apache][access][time]}\] \"-\" %{NUMBER:[http][response][status_code]:long} 
-",
            "\[%{HTTPDATE:[apache][access][time]}\] %{IPORHOST:[source][address]
} %{DATA:[apache][access][ssl][protocol]} %{DATA:[apache][access][ssl][cipher]} 
\"%{WORD:[http][request][method]} %{DATA:[url][original]} HTTP/%{NUMBER:[http][v
ersion]}\" (-|%{NUMBER:[http][response][body][bytes]:long})"
          ]
        }
        tag_on_failure => "_apache_access_grokparsefailure"
      }

Given the lack of error output and that the data appears to be getting sent via Logstash we’re at a bit of a loss as to what’s causing the issue or where to look to diagnose it. I’ve had a look at the index templates but all the fields appear to be present and correct.