# Filebeat Apache Log fields (remote\_ip, geo etc) **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [February 9, 2021, 3:56pm UTC](https://discuss.elastic.co/t/filebeat-apache-log-fields-remote-ip-geo-etc/263760 "2021-02-09T15:56:33Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![radu990](https://avatars.discourse-cdn.com/v4/letter/r/c57346/32.png) [@radu990](https://discuss.elastic.co/u/radu990)\ **Post date:** [February 9, 2021, 3:56pm UTC](https://discuss.elastic.co/t/filebeat-apache-log-fields-remote-ip-geo-etc/263760/1 "2021-02-09T15:56:34Z")
Hello, Sorry for bothering. I'm a new user for ELK. I have managed to enable Apache module for Filebeat and I'm seeing the Apache logs coming in Discover Section in Kibana. However the remote\_ip and other fields from Apache are being containted in the message field. How can I get the remote\_ip and other fields mapped correctly? Mainly remote\_ip, User, URL (GET /mail) ... Thank you in advance! ============================== Example of a Log in Discover in Kibana @timestamp Feb 9, 2021 @ 16:46:36.153 \_id Xhx5h3cB1OZ-J5vvq\_rA \_index filebeat-7.10.2-2021.02.09-000001 ## \_score \_type \_doc agent.ephemeral\_id c75f9d91-47af-49eb-8797-79920a1308a7 agent.hostname Server agent.id 77a89e44-7cec-4d01-8b8f-8195dd8c3679 agent.name Server agent.type filebeat agent.version 7.10.2 ecs.version 1.5.0 **error.message** **Provided Grok expressions do not match field value: [85.85.85.85 [webmail.company.com](http://webmail.company.com) "CN=User/O=Company/C=CZ" [14/Dec/2020:05:58:18 +0100] "GET /mail/User.nsf/iNotes/Proxy/?OpenDocument&Form=s\_ReadViewEntries&PresetFields=DBQuotaInfo;1,FolderName;($Inbox),UnreadCountInfo;1,SearchSort;DateD,s\_UsingHttps;1,noPI;1&TZType=UTC&Start=1&Count=23&resortdescending=6 HTTP/1.1" 200 2054 "[https://webmail.company.com/mail/User.nsf/iNotes/Proxy/?OpenDocument&Form=l\_ScriptFrame&l=en&gz&CR&MX&TSF=20170318T181650,92Z&TSX=20180206T185427,18Z&EFF=%2FiNotes%2FForms9\_x&charset=UTF-8&charset=UTF-8&KIC&ua=safari&pt](https://webmail.company.com/mail/User.nsf/iNotes/Proxy/?OpenDocument&Form=l_ScriptFrame&l=en&gz&CR&MX&TSF=20170318T181650,92Z&TSX=20180206T185427,18Z&EFF=%2FiNotes%2FForms9_x&charset=UTF-8&charset=UTF-8&KIC&ua=safari&pt)" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36" 31** event.dataset apache.access event.ingested Feb 9, 2021 @ 16:47:13.932 event.module apache fileset.name access host.architecture x86\_64 host.hostname Server host.id fec37629-61fa-466c-b662-9537045df62a host.ip fe80::c573:3ba4:33e1:4ede, 192.168.1.13, fe80::5efe:c0a8:10d, fe80:💯7f:fffe host.mac 00:15:5d:01:19:1c, 00:00:00:00:00:00:00:e0, 00:00:00:00:00:00:00:e0 host.name Server host.os.build 7601.24546 host.os.family windows host.os.kernel 6.1.7601.24545 (win7sp1\_ldr\_escrow.200102-1707) host.os.name Windows Server 2008 R2 Enterprise host.os.platform windows host.os.version 6.1 input.type log log.file.path D:\Lotus\Domino\data\weblogs\access12142020.log log.offset 8,054,853 **message** **85.85.85.85 [webmail.company.com](http://webmail.company.com) "CN=User/O=Company/C=CZ" [14/Dec/2020:05:58:18 +0100] "GET /mail/User.nsf/iNotes/Proxy/?OpenDocument&Form=s\_ReadViewEntries&PresetFields=DBQuotaInfo;1,FolderName;($Inbox),UnreadCountInfo;1,SearchSort;DateD,s\_UsingHttps;1,noPI;1&TZType=UTC&Start=1&Count=23&resortdescending=6 HTTP/1.1" 200 2054 "[https://webmail.company.com/mail/User.nsf/iNotes/Proxy/?OpenDocument&Form=l\_ScriptFrame&l=en&gz&CR&MX&TSF=20170318T181650,92Z&TSX=20180206T185427,18Z&EFF=%2FiNotes%2FForms9\_x&charset=UTF-8&charset=UTF-8&KIC&ua=safari&pt](https://webmail.company.com/mail/User.nsf/iNotes/Proxy/?OpenDocument&Form=l_ScriptFrame&l=en&gz&CR&MX&TSF=20170318T181650,92Z&TSX=20180206T185427,18Z&EFF=%2FiNotes%2FForms9_x&charset=UTF-8&charset=UTF-8&KIC&ua=safari&pt)" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36" 31** service.type apache suricata.eve.timestamp Feb 9, 2021 @ 16:46:36.153 ---
**Author:** ![radu990](https://avatars.discourse-cdn.com/v4/letter/r/c57346/32.png) [@radu990](https://discuss.elastic.co/u/radu990)\ **Post date:** [February 10, 2021, 10:59am UTC](https://discuss.elastic.co/t/filebeat-apache-log-fields-remote-ip-geo-etc/263760/2 "2021-02-10T10:59:33Z")
Looks Like I need a grok pattern Can anybody please help with the grok pattern for below example of logs? > **85.85.85.85 [webmail.company.com](http://webmail.company.com) "CN=First Last/O=Company/C=CZ" [14/Dec/2020:05:58:18 +0100] "GET /mail/User.nsf/iNotes/Proxy/?OpenDocument&Form=s\_ReadViewEntries&PresetFields=DBQuotaInfo;1,FolderName;($Inbox),UnreadCountInfo;1,SearchSort;DateD,s\_UsingHttps;1,noPI;1&TZType=UTC&Start=1&Count=23&resortdescending=6 HTTP/1.1" 200 2054 "[https://webmail.company.com/mail/User.nsf/iNotes/Proxy/?OpenDocument&Form=l\_ScriptFrame&l=en&gz&CR&MX&TSF=20170318T181650,92Z&TSX=20180206T185427,18Z&EFF=%2FiNotes%2FForms9\_x&charset=UTF-8&charset=UTF-8&KIC&ua=safari&pt](https://webmail.company.com/mail/User.nsf/iNotes/Proxy/?OpenDocument&Form=l_ScriptFrame&l=en&gz&CR&MX&TSF=20170318T181650,92Z&TSX=20180206T185427,18Z&EFF=%2FiNotes%2FForms9_x&charset=UTF-8&charset=UTF-8&KIC&ua=safari&pt)" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36" 125 INOTES\_LOGIN\_ID=First%20Last; Shimmer=SI\_TLM:20210209T072811%2C40Z&ST\_Counter:3&LAO:mail&SAB:1&CS\_TLM:20210209T072831%2C15Z&V\_TLM:20210210T080147%2C82Z&DMS:5&ui:X&MOTLM:20210129T113159%2C00Z&DBQS:1503571%2C%207168000%2C%206963200%2C%200%2C%201503571&SPRKL:1&KOSCZ:GTB&FISD:1; INOTES\_LOGIN\_ID=First%20Last; DWAShared=0; DWAMode=0; INOTES\_LOGIN\_ID=First%20Last; DWAShared=0; DWAMode=0; LtpaToken2=FpoGJJz33bYLI+CtWy6OlIgoTJouNGEiduvxvQbcN8HRI7K6LThCsb1Dl8CzN72Zi05RGOUmQRMiOQcTk1norKHi6SbkEGI6GlXzjSIweBRSc8c+XPyAwA44PKPbu3WzrPfR0+uoC0sgTPvochvQ/VfPL/sSaqUFoRswRwyI+UeaOwTs/DvKiWLCpiKrVkFk3SmDjrxPBHb/WiL5nDkpp8Dsjjxnlo4vpx7BdOoVNai1jybvHkW28KXxkb21o8SSpmU7ZFdHyZFjDWCYuuCVOx7asV/q4a3lWdxlPfWdPcUguHML+xDmsrMPm6fTUSKeKIKdQEPr6VDmitBi7Z5URIlkRrUyslkTcc28y6fQir3Y20Hc9TmOvwaBlG/ehnpv; LtpaToken=0x4JJ4oWKojdqoz08Ng+MRUkkJq2vYGLGN9lp8HL8FxbD+xnivE7qzCzf92Q6x5OAPOBFRNgxd3Qg225zLwnJFWO0lGeIweH8VDgyWOMImNe6E9z9HBnQAN43vQ2uwtpv3X5E5DN0oLIPKLxAkqsHUDJqJ0SE6NZ6UnfLoR82JyjZVC/s6QEov5DNdpAY/o2Gxh0vWmE+wuQGuCh4mVCIP9KU/dbX4F0Ld9JEExzIpkdzKELibU2Akov0Krv0eWADSV++m/5ECLpaf6N6/VzkZEkt5XoOoL6OD/6ni4zojvo3O+X9Bn7Mdk2MnsQ1AccIohj5eN8Oi81QbD0a9b7jw==; ShimmerS=ET:20210210T114045%2c00Z&R:0&AT:M" "D:/Lotus/Domino/Data/mail/User.nsf"** What I would need is **Client IP** (85.85.85.85), **VirtualHostname** (webmail / [webmail.company.com](http://webmail.company.com)) , **User** (part after CN=, First Last), **Time** (14/Dec/2020:05:58:18), **URL** (GET /mail/User.nsf/iNotes/Proxy/?OpenDocument&Form=s\_ReadViewEntries&PresetFields=DBQuotaInfo;1,FolderName; ... ) and the **Device Info** ( "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36" ) I know it should start with below, however I can't get anyhow the User name to proceed with [%{HTTPDATE:timestamp}] and possible next would be "(?:%{WORD:verb} %{NOTSPACE:request} and not sure how to get the Device info. Any help would be appreciated! > %{IPORHOST:clientip} %{WORD:VirtualHost} ??? ---
**Author:** ![ChrsMark](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrsmark/32/55858_2.png) [@ChrsMark](https://discuss.elastic.co/u/ChrsMark)\ **Post date:** [February 10, 2021, 12:02pm UTC](https://discuss.elastic.co/t/filebeat-apache-log-fields-remote-ip-geo-etc/263760/3 "2021-02-10T12:02:56Z")
Hi! You can find the grok pattern which Filbeat apache module uses at [beats/pipeline.yml at 829c3b7dcc6365161d83a3b10f05a9f9990f36c3 · elastic/beats · GitHub](https://github.com/elastic/beats/blob/829c3b7dcc6365161d83a3b10f05a9f9990f36c3/filebeat/module/apache/access/ingest/pipeline.yml#L9). Also have a look into the sample logs that are used for testing at [beats/filebeat/module/apache/access/test at 1ce20cbe17269eafc8dbb0d859413643e9a10886 · elastic/beats · GitHub](https://github.com/elastic/beats/tree/1ce20cbe17269eafc8dbb0d859413643e9a10886/filebeat/module/apache/access/test). If in your case, the logs are different and cannot be parsed by the pipeline then you might need to create custom grok pattern from scratch or update the one of the apache's module so as to cover your case. C. ---
**Author:** ![radu990](https://avatars.discourse-cdn.com/v4/letter/r/c57346/32.png) [@radu990](https://discuss.elastic.co/u/radu990)\ **Post date:** [February 10, 2021, 1:04pm UTC](https://discuss.elastic.co/t/filebeat-apache-log-fields-remote-ip-geo-etc/263760/4 "2021-02-10T13:04:08Z")
Thank you very much Chris. Very useful! Using the grok patterns from the link you've provided I have managed to get my right grok pattern. > %{IPORHOST:clientip} %{IPORHOST:destination.domain} "CN=%{DATA:username}" [%{HTTPDATE:apache.access.time}] "(?:%{WORD:http.request.method} %{DATA:url.original} HTTP/%{NUMBER:http.version}|-)?" %{NUMBER:http.response.status\_code:long} (?:%{NUMBER:http.response.body.bytes:long}|-) ("%{DATA:http.request.referrer}") ("%{DATA:user\_agent.original}") ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [March 10, 2021, 3:04pm UTC](https://discuss.elastic.co/t/filebeat-apache-log-fields-remote-ip-geo-etc/263760/5 "2021-03-10T15:04:59Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.