# Parse a log line and send only 1st 2 levels **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [November 2, 2017, 6:51pm UTC](https://discuss.elastic.co/t/parse-a-log-line-and-send-only-1st-2-levels/106238 "2017-11-02T18:51:09Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![pgokhale10](https://avatars.discourse-cdn.com/v4/letter/p/a587f6/32.png) [@pgokhale10](https://discuss.elastic.co/u/pgokhale10)\ **Post date:** [November 2, 2017, 6:51pm UTC](https://discuss.elastic.co/t/parse-a-log-line-and-send-only-1st-2-levels/106238/1 "2017-11-02T18:51:09Z")
Hi, Lets says my log looks like this: IP - - [24/Oct/2017:08:11:46 +0000] "GET **/X/Y/Z/ABCD** HTTP/1.1" 200 1225 "[www.x.com?query=&abcd](http://www.x.com?query=&abcd)" "Mozilla/5.0 (X11; Linux x86\_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.90 Safari/537.36" 0.012 0.012 . Now when the filebeat sees this line, it should just send the 1st 2 levels: **/X/Y** See the bold part of the log Thanks~ ---
**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\ **Post date:** [November 2, 2017, 8:56pm UTC](https://discuss.elastic.co/t/parse-a-log-line-and-send-only-1st-2-levels/106238/2 "2017-11-02T20:56:45Z")
Are you asking how to make Filebeat send only the first two levels of the path? If so, what is your current setup? Are you using Filebeat modules? ---
**Author:** ![pgokhale10](https://avatars.discourse-cdn.com/v4/letter/p/a587f6/32.png) [@pgokhale10](https://discuss.elastic.co/u/pgokhale10)\ **Post date:** [November 2, 2017, 8:58pm UTC](https://discuss.elastic.co/t/parse-a-log-line-and-send-only-1st-2-levels/106238/3 "2017-11-02T20:58:45Z")
Hi, Yes we have filebeat as a container sitting inside a pod. I am pointing filebeat to get data from ngnix access log. ilebeat: prospectors: # Each - is a prospector. Below are the prospector specific configurations - paths: - /var/nginx/_.log exclude\_lines: ['^._\b(itfm-cloud-dc-transformer)\b.\*$'] input\_type: log # You usually want tail\_files: true. Wavefront shows the ingestion timestamp to be when a log line # was received, not when it was written to your logfile. For this reason, ingesting back-in-time # data will surprise you unless you are expecting that. tail\_files: true # This is important if you have some kind of log rotator. Filebeat won't let go of an open FD # of a rotated file without this option. close\_inactive: 5m registry\_file: /var/lib/filebeat/registry ---
**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\ **Post date:** [November 2, 2017, 9:12pm UTC](https://discuss.elastic.co/t/parse-a-log-line-and-send-only-1st-2-levels/106238/4 "2017-11-02T21:12:39Z")
You can either add Logstash into the mix or use an Ingest Node pipeline to parse the path. In either case you will use grok to break the message into fields and this can include parsing the path down to two levels. If you use Ingest Node you will add a [`pipeline`](https://www.elastic.co/guide/en/beats/filebeat/current/configuration-filebeat-options.html#_pipeline) option to your config the is the name of the pipeline that [you `PUT` into ES](https://www.elastic.co/guide/en/elasticsearch/reference/5.6/put-pipeline-api.html). ---
**Author:** ![pgokhale10](https://avatars.discourse-cdn.com/v4/letter/p/a587f6/32.png) [@pgokhale10](https://discuss.elastic.co/u/pgokhale10)\ **Post date:** [November 2, 2017, 9:31pm UTC](https://discuss.elastic.co/t/parse-a-log-line-and-send-only-1st-2-levels/106238/5 "2017-11-02T21:31:47Z")
Thanks Andrew, I send logs from filebeat to wavefront. In this case wavefront is a place where i put the logs into different field using something like: pattern: '%{COMBINEDAPACHELOG} %{NUMBER:response}' So if I were to only send the urls 2 level deep I need to do something at the filebeat level. Any option for that? ---
**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\ **Post date:** [November 2, 2017, 11:08pm UTC](https://discuss.elastic.co/t/parse-a-log-line-and-send-only-1st-2-levels/106238/6 "2017-11-02T23:08:49Z")
Can you customize the Wavefront grok pattern? Filebeat does not have parsing capability. If you can add a second grok filter that operates on the `request` field with a pattern like `(?(/[^/]+){2})` this should create a `path` field containing `/X/Y`. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 30, 2017, 11:09pm UTC](https://discuss.elastic.co/t/parse-a-log-line-and-send-only-1st-2-levels/106238/7 "2017-11-30T23:09:18Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.