# Tomcat filebeat module expected log format? **URL:** \ **Category:** Beats\ **Tags:** beats-module, filebeat\ **Created:** [May 7, 2021, 9:43pm UTC](https://discuss.elastic.co/t/tomcat-filebeat-module-expected-log-format/272450 "2021-05-07T21:43:47Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![sarahw](https://avatars.discourse-cdn.com/v4/letter/s/a8b319/32.png) [@sarahw](https://discuss.elastic.co/u/sarahw)\ **Post date:** [May 7, 2021, 9:43pm UTC](https://discuss.elastic.co/t/tomcat-filebeat-module-expected-log-format/272450/1 "2021-05-07T21:43:47Z")
I have been trying to use the Tomcat module for filebeat 7.9 ([Tomcat module | Filebeat Reference [7.9] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/7.9/filebeat-module-tomcat.html)) to ingest tomcat access logs from files but my log files are producing errors. I'd like to know what format the module expects the file-based logs to be in. Tomcat log format is configured in server.xml and the default format out of the box is `"%h %l %u %t "%r" %s %b"`, with fields defined as - **`%h`** - Remote IP address - **`%l`** - Remote logical username from identd (always returns '-') - **`%u`** - Remote user that was authenticated (if any), else '-' (escaped if required) - **`%t`** - Date and time, in Common Log Format - **`%r`** - First line of the request (method and request URI) - **`%s`** - HTTP status code of the response - **`%b`** - Bytes sent, excluding HTTP headers, or '-' if zero [https://tomcat.apache.org/tomcat-9.0-doc/config/valve.html#Access\_Log\_Valve](https://tomcat.apache.org/tomcat-9.0-doc/config/valve.html#Access_Log_Valve) (This is version 9, but the format appears to be the same in Tomcat 10.) This results in logs that look like: ```auto 111.22.0.333 - - [07/May/2021:17:08:49 -0400] "GET /app-path?name=john HTTP/1.1" 302 - 111.33.0.111 - - [07/May/2021:17:08:49 -0400] "GET /app-path/ HTTP/1.1" 200 14291 111.44.0.111 - - [07/May/2021:17:08:50 -0400] "GET /favicon.ico HTTP/1.1" 404 762 ``` My module config looks like: ```auto - module: tomcat log: enabled: true # Set which input to use between udp (default), tcp or file. var.input: file # Set paths for the log files when file input is used. var.paths: - /usr/local/tomcat/logs/localhost_access_log.* ``` When the logs are ingested into elastic they clearly have not been properly parsed and have a `dissect_parsing_error` flag, e.g.: ```auto { "_index": "filebeat-7.9.2-default-2021.03.20-000004", "_type": "_doc", "_id": "CWKpSHkBuDbbQBpRlerr", "_score": 1, "_source": { "agent": { "hostname": "myserver.domain.com", "name": "myserver.domain.com", "id": "", "type": "filebeat", "ephemeral_id": "", "version": "7.9.2" }, "log": { "file": { "path": "/usr/local/tomcat/logs/localhost_access_log.2021-05-07.txt" }, "offset": 243, "flags": [ "dissect_parsing_error" ] }, "fileset": { "name": "log" }, "tags": [ "tomcat.log", "forwarded" ], "observer": { "product": "TomCat", "vendor": "Apache", "type": "Web" }, "input": { "type": "log" }, "@timestamp": "2021-05-07T21:09:03.105Z", "ecs": { "version": "1.5.0" }, "service": { "type": "tomcat" }, "event": { "original": "111.12.0.333 - - [07/May/2021:17:08:57 -0400] \"GET /app-path?name=john HTTP/1.1\" 200 527", "module": "tomcat", "dataset": "tomcat.log" } }, "fields": { "@timestamp": [ "2021-05-07T21:09:03.105Z" ] } } ``` I tried turning on debug in the Tomcat module config and I get the following messages in the filebeat log: ```auto 2021-05-06T21:49:47.306-0400 WARN [processor.javascript] console/console.go:52 msgid_select: no messageid captured! 2021-05-06T21:49:47.307-0400 WARN [processor.javascript] console/console.go:52 linear_select trying entry 0 2021-05-06T21:49:47.307-0400 WARN [processor.javascript] console/console.go:52 linear_select failed entry 0 2021-05-06T21:49:47.307-0400 WARN [processor.javascript] console/console.go:52 linear_select trying entry 1 2021-05-06T21:49:47.307-0400 WARN [processor.javascript] console/console.go:52 linear_select failed entry 1 2021-05-06T21:49:47.307-0400 WARN [processor.javascript] console/console.go:52 linear_select didn't match ``` Any help would be appreciated. If someone could point me to an example of a working file-based config that would be great. ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [May 13, 2021, 2:52pm UTC](https://discuss.elastic.co/t/tomcat-filebeat-module-expected-log-format/272450/2 "2021-05-13T14:52:50Z")
Hi @sarahw Sorry for the long response time. @Marius_Iversen perhaps you could take a look, I have repeated the same issue with 7.12.1. His logs looks valid to me, using the tomcat default/common format, and I don't know how to debug the pipeline.js pipeline. Perhaps you could take a look? Maybe we are missing something. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 10, 2021, 4:52pm UTC](https://discuss.elastic.co/t/tomcat-filebeat-module-expected-log-format/272450/3 "2021-06-10T16:52:53Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed. ---
**Author:** ![Guillaume\_Dufrenne](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guillaume_dufrenne/32/103641_2.png) [@Guillaume\_Dufrenne](https://discuss.elastic.co/u/Guillaume_Dufrenne)\ **Post date:** [August 17, 2022, 1:20pm UTC](https://discuss.elastic.co/t/tomcat-filebeat-module-expected-log-format/272450/5 "2022-08-17T13:20:24Z")
Hi, @stephenb , @Marius_Iversen @sarahw These logs are access logs They are correctly handled by the apache access logs module ​​​​​​​[Apache module | Filebeat Reference [8.3] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-apache.html) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 4, 2022, 8:30am UTC](https://discuss.elastic.co/t/tomcat-filebeat-module-expected-log-format/272450/6 "2022-11-04T08:30:28Z")