# How to filter my logs **URL:** \ **Category:** Logstash\ **Created:** [May 25, 2018, 9:19am UTC](https://discuss.elastic.co/t/how-to-filter-my-logs/133292 "2018-05-25T09:19:43Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![kimfut](https://avatars.discourse-cdn.com/v4/letter/k/b9e5f3/32.png) [@kimfut](https://discuss.elastic.co/u/kimfut)\ **Post date:** [May 25, 2018, 9:19am UTC](https://discuss.elastic.co/t/how-to-filter-my-logs/133292/1 "2018-05-25T09:19:43Z")
hello everyone , i am trying to filter my logs to something more structured , mostly the message with url part , i tried a lot of grok filter that i found online but everytime i got " \_grokparsefailure" in my logs and nothing wad filtered. there is example of my logs : ![Screenshot%20from%202018-05-25%2011-17-10](https://us1.discourse-cdn.com/elastic/original/3X/c/6/c666da85dc81244eac307796374695a4fb014c1d.png) can you help me please to define which filter can do the job thank you ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [May 25, 2018, 9:25am UTC](https://discuss.elastic.co/t/how-to-filter-my-logs/133292/2 "2018-05-25T09:25:46Z")
Have a look at [this blog post](https://www.elastic.co/blog/a-practical-introduction-to-logstash). It will show you how to work with Logstash and go about creating grok configuration. ---
**Author:** ![MrNerd](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mrnerd/32/30763_2.png) [@MrNerd](https://discuss.elastic.co/u/MrNerd)\ **Post date:** [May 25, 2018, 10:00am UTC](https://discuss.elastic.co/t/how-to-filter-my-logs/133292/3 "2018-05-25T10:00:07Z")
@kimfut I rode this guide and for me was the best one to understand how to filer > [Do you grok Grok? | Elastic Blog](https://www.elastic.co/blog/do-you-grok-grok) Regards MrNerd ---
**Author:** ![inhinyera16](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/inhinyera16/32/61625_2.png) [@inhinyera16](https://discuss.elastic.co/u/inhinyera16)\ **Post date:** [May 25, 2018, 10:36am UTC](https://discuss.elastic.co/t/how-to-filter-my-logs/133292/4 "2018-05-25T10:36:43Z")
Maybe your grok pattern doesnt match at all and so it fails. You need to atleast make sure that the logs are uniform in pattern. Also could you provide us your grok pattern? ---
**Author:** ![kimfut](https://avatars.discourse-cdn.com/v4/letter/k/b9e5f3/32.png) [@kimfut](https://discuss.elastic.co/u/kimfut)\ **Post date:** [May 25, 2018, 12:58pm UTC](https://discuss.elastic.co/t/how-to-filter-my-logs/133292/5 "2018-05-25T12:58:03Z")
that's my grok pattern : ``` filter { grok { match => {"message" => "%{IP:client} %{WORD:method} %{URIPATHPARAM:request} %{NUMBER:bytes} %{NUMBER:duration} "} } } ``` ---
**Author:** ![arkady\_renko](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/arkady_renko/32/30160_2.png) [@arkady\_renko](https://discuss.elastic.co/u/arkady_renko)\ **Post date:** [May 25, 2018, 2:12pm UTC](https://discuss.elastic.co/t/how-to-filter-my-logs/133292/6 "2018-05-25T14:12:24Z")
Message field in attached image has date string between ip address and method. But your grok filter doesnt have any date field. Your log in message looks like apache log. İf it is, you can use built-in COMBINEDAPACHELOG pattern. ---
**Author:** ![kimfut](https://avatars.discourse-cdn.com/v4/letter/k/b9e5f3/32.png) [@kimfut](https://discuss.elastic.co/u/kimfut)\ **Post date:** [May 25, 2018, 2:27pm UTC](https://discuss.elastic.co/t/how-to-filter-my-logs/133292/7 "2018-05-25T14:27:35Z")
> [@arkady\_renko](#): > > use built-in COMBINEDAPACHELOG pattern. thanx , but when i use : ``` grok { match => { "message" => "%{COMBINEDAPACHELOG}"} } ``` it's the same problem : ` tags: beats_input_codec_plain_applied, _grokparsefailure` that what displayed on kibana ---
**Author:** ![kimfut](https://avatars.discourse-cdn.com/v4/letter/k/b9e5f3/32.png) [@kimfut](https://discuss.elastic.co/u/kimfut)\ **Post date:** [May 28, 2018, 8:25am UTC](https://discuss.elastic.co/t/how-to-filter-my-logs/133292/8 "2018-05-28T08:25:39Z")
hello everyone , I was finally able to get a filter for my logs , this is an example in Json format : > { > "\_index": "beat-test", > "\_type": "doc", > "_id": "s9jhl2MBN2K6P6dYqwl_", > "\_version": 1, > "\_score": null, > "\_source": { > "verb": "GET", > "host": "[monetoring.com](http://monetoring.com)", > "bytes": 184693, > "message": "11.115.98.127 - - [04/May/2018:16:39:20 +0200] "GET /WebServiceRequestProxyCenter/api/request-url?service=density\_analyze%2C+siniat%2C+isolation+phonique&url=https%3A%2F%[2Fwww.google.fr](http://2Fwww.google.fr)%2Fsearch%3Fq%3Disolation%2Bphonique%26ie%3DUTF-8%26oe%3DUTF8%26hl%3Dfr%26start%3D0%26num%3D100&country=fr HTTP/1.1" 200 184693", > "ident": "-", > "clientip": "11.115.98.127", > "@timestamp": "2018-05-04T14:39:20.000Z", > "geoip": { > "ip": "11.115.98.127", > "latitude": 48.8582, > "timezone": "Europe/Paris", > "country\_name": "France", > "continent\_code": "EU", > "country\_code2": "FR", > "country\_code3": "FR", > "location": { > "lon": 2.3387000000000002, > "lat": 48.8582 > }, > "longitude": 2.3387000000000002 > }, > "httpversion": "1.1", > "beat": { > "version": "6.2.4", > "name": "[monetoring.com](http://monetoring.com)", > "hostname": "[monetoring.com](http://monetoring.com)" > }, > "@version": "1", > "offset": 15692839, > "auth": "-", > "source": "/home/hakim/monetoring.com.access.2018-05-04.log", > "timestamp": "04/May/2018:16:39:20 +0200", > "response": 200, > "tags": [ > "beats\_input\_codec\_plain\_applied" > ], > "request": "/WebServiceRequestProxyCenter/api/request-url?service=density\_analyze%2C+siniat%2C+isolation+phonique&url=https%3A%2F%[2Fwww.google.fr](http://2Fwww.google.fr)%2Fsearch%3Fq%3Disolation%2Bphonique%26ie%3DUTF-8%26oe%3DUTF8%26hl%3Dfr%26start%3D0%26num%3D100&country=fr" > }, > "fields": { > "@timestamp": [ > "2018-05-04T14:39:20.000Z" > ] > }, > "sort": [ > 1525444760000 > ] > } but it's not very interesting for me , because what I would like to have , it's this part : > "request": "/WebServiceRequestProxyCenter/api/request-url?service=density\_analyze%2C+siniat%2C+isolation+phonique&url=https%3A%2F%[2Fwww.google.fr](http://2Fwww.google.fr)%2Fsearch%3Fq%3Disolation%2Bphonique%26ie%3DUTF-8%26oe%3DUTF8%26hl%3Dfr%26start%3D0%26num%3D100&country=fr" more split to extract "/WebServiceRequestProxyCenter/api/request-url" apart also to delete all special caracters like we see here : % & so if you have suggest , help me please ---
**Author:** ![sana1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sana1/32/46439_2.png) [@sana1](https://discuss.elastic.co/u/sana1)\ **Post date:** [June 1, 2018, 8:53am UTC](https://discuss.elastic.co/t/how-to-filter-my-logs/133292/9 "2018-06-01T08:53:13Z")
Please give me grok filter for my log pattern, I have a log directory belong to same log server and each file has different data and format but the values are in key value pair. Please gimme way out to parse these logs right now my filter is filter { grok { match =\> { "message" =\> "%{COMBINEDAPACHELOG}" } } and my data is "GET / HTTP/1.1 Host: [yahoo.com](http://yahoo.com) User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86\_64; rv:59.0) Gecko/20100101 Firefox/59.0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8 Accept-Language: en-US,en;q=0.5 Accept-Encoding: gzip, deflate Cookie: B=fh4crrpd9sf94&b=3&s=e1; ucs=lnct=1525329090; HP=1 Connection: keep-alive Upgrade-Insecure-Requests: 1 HTTP/1.1 301 Moved Permanently Date: Thu, 07 May 2017 09:48:52 GMT Connection: keep-alive Via: http/1.1 [media-router-fp1012.prod.media.gq1.yahoo.com](http://media-router-fp1012.prod.media.gq1.yahoo.com) (ApacheTrafficServer [c s f]) Server: ATS Cache-Control: no-store, no-cache Content-Type: text/html Content-Language: en X-Frame-Options: SAMEORIGIN Strict-Transport-Security: max-age=2592000 Location: [https://www.yahoo.com/](https://www.yahoo.com/) Content-Length: 8" ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 29, 2018, 8:53am UTC](https://discuss.elastic.co/t/how-to-filter-my-logs/133292/10 "2018-06-29T08:53:21Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.