Apache, Filebeat, Kibana 8.3.3

[Gunnar]

Hello, I'm new to Filebeat and Kibana.

I'm having a strange issue, I'm sending apache logfiles to Elasticsearch using the apache module. On Kibana logfile shows like this:

13:06:23.462
www.xxx.de 185.191.171.22 - - [23/Aug/2022:13:06:21 +0200] "GET /de/.html HTTP/1.1" 200 19761 "-" "Mozilla/5.0 (compatible; SemrushBot/7~bl; +http://www.semrush.com/bot.html)"
13:06:23.462
www.xxx.de 66.249.64.64 - - [23/Aug/2022:13:06:22 +0200] "GET /de/93.html HTTP/1.1" 200 12127 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.5112.79 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
13:06:23.462
www.xxx.de 10.xxx.xxx.xxx - - [23/Aug/2022:13:06:22 +0200] "GET /apiv2/metadata/ HTTP/1.1" 200 52 "-" "Lucee (CFML Engine)"
13:06:24.000
apache.access
[apache][access] 40.77.167.xxx - "GET /de/xxx.html? HTTP/1.1" 200 14655
13:06:24.000
apache.access
[apache][access] 10.xxx.xxx.xxx - "GET /apiv2/xxx

My issue is that event.dataset isn't tagged correctly all entries for one timestamp are either matched correctly or not. (Matching alternates, so first entries match, next don't, next do... etc)

Any ideas what to look for? Thanks for your help.

[jsoriano (Jaime Soriano)]

Hi,

I think I don't understand the problem, could you please share an screenshot of Kibana that illustrates the problem?

[Gunnar]

Hi, thanks for taking a look, here is a screenshot from Observability>Logs>Stream I hope it makes the issue more clear.

[jsoriano (Jaime Soriano)]

Is it possible that you have also some filebeat input enabled? event.dataset is only included in events coming from modules, not for events coming from inputs.

[stephenb (Stephen Brown)]

Can you confirm whether or not you are using the Apache module?

Also I'm a bit confused. Some of those Do not look like the common Apache log format.

You would have to provide samples of your Apache logs.

[Gunnar]

Thanks a lot, you are right, the apache module and as well filebeat input was enabled. After disabling filebeat, everything looks correct!