# N2disk uburst log parsing **URL:** \ **Category:** Kibana\ **Created:** [July 26, 2018, 2:20am UTC](https://discuss.elastic.co/t/n2disk-uburst-log-parsing/141680 "2018-07-26T02:20:49Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![layer8](https://avatars.discourse-cdn.com/v4/letter/l/f0a364/32.png) [@layer8](https://discuss.elastic.co/u/layer8)\ **Post date:** [July 26, 2018, 2:20am UTC](https://discuss.elastic.co/t/n2disk-uburst-log-parsing/141680/1 "2018-07-26T02:20:49Z")
Could use some advice\> Having difficulty with a fresh install (6.3). Built a conf file with a grok filter that checked out fine in debugger, but does not appear to be parsed by logstash/kibana. We see our log information in the message field only. Also tried the sample apache conf setup, same result, Kibana shows the test data strings access\_log but only in the message section. Must be something fundamental in the configuration that is overriding my conf file filter. ELK is new to me. My conf file setup: ################### /etc/logstash/conf.d/uburst.conf input { file { path =\> "/var/tmp/n2disk/uburst.log" start\_position =\> "beginning" } } filter { if [path] == "/var/tmp/n2disk/" { grok { match =\> { "message" =\> "%{NUMBER:start} %{NUMBER:end} %{NUMBER:duration} %{NUMBER:kbps} %{NUMBER:PeakMbps}" } remove\_field =\> ["message"] } } } output { elasticsearch { hosts =\> ["localhost:9200"] } stdout { codec =\> rubydebug } } ##################### The goal is to pull in the 5 field values below, graph the peak mbit/s rate over time etc. We are reading a file from localhost /var/tmp/n2disk/uburst.log, the tab separated file is as follows. ############# Start End Duration Kbit Peak-Mbit/s 1477408107.351533705 1477408107.366453800 0.014920095 1378 95.971 1477408110.330377529 1477408110.341709397 0.011331868 1058 94.741 ############# The Kibana discover messages shows the following: @timestamp July 23rd 2018, 16:16:36.766 t \_id AWTIyV\_kAC53BgREZmRx t \_index filebeat-2018.07.23 # \_score - t \_type doc t beat.hostname ubuntu-18 t beat.name ubuntu-18 t beat.version 5.6.10 t input\_type log t message 1532376992.306766204 1532376992.336646903 0.029880699 2788166 100000.192 # offset 3,476 t source /var/tmp/n2disk/uburst.log t type log ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [July 26, 2018, 2:54am UTC](https://discuss.elastic.co/t/n2disk-uburst-log-parsing/141680/2 "2018-07-26T02:54:08Z")
What does the stdout output show? ---
**Author:** ![layer8](https://avatars.discourse-cdn.com/v4/letter/l/f0a364/32.png) [@layer8](https://discuss.elastic.co/u/layer8)\ **Post date:** [July 26, 2018, 11:31am UTC](https://discuss.elastic.co/t/n2disk-uburst-log-parsing/141680/3 "2018-07-26T11:31:20Z")
hi Mark, nothing untoward that I can pickup. ############################ cat /var/log/logstash/logstash-plain.log [2018-07-25T15:20:00,641][INFO][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck\_url=\>[http://localhost:9200/](http://localhost:9200/), :path=\>"/"} [2018-07-25T15:20:01,726][WARN][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=\>"[http://localhost:9200/](http://localhost:9200/)"} [2018-07-25T15:20:02,338][INFO][logstash.outputs.elasticsearch] Using mapping template from {:path=\>nil} [2018-07-25T15:20:02,371][INFO][logstash.outputs.elasticsearch] Attempting to install template {:manage\_template=\>{"template"=\>"logstash-_", "version"=\>50001, "settings"=\>{"index.refresh\_interval"=\>"5s"}, "mappings"=\>{"default"=\>{"\_all"=\>{"enabled"=\>true, "norms"=\>false}, "dynamic\_templates"=\>[{"message\_field"=\>{"path\_match"=\>"message", "match\_mapping\_type"=\>"string", "mapping"=\>{"type"=\>"text", "norms"=\>false}}}, {"string\_fields"=\>{"match"=\>"_", "match\_mapping\_type"=\>"string", "mapping"=\>{"type"=\>"text", "norms"=\>false, "fields"=\>{"keyword"=\>{"type"=\>"keyword", "ignore\_above"=\>256}}}}}], "properties"=\>{"@timestamp"=\>{"type"=\>"date", "include\_in\_all"=\>false}, "@version"=\>{"type"=\>"keyword", "include\_in\_all"=\>false}, "geoip"=\>{"dynamic"=\>true, "properties"=\>{"ip"=\>{"type"=\>"ip"}, "location"=\>{"type"=\>"geo\_point"}, "latitude"=\>{"type"=\>"half\_float"}, "longitude"=\>{"type"=\>"half\_float"}}}}}}}} [2018-07-25T15:20:02,437][INFO][logstash.outputs.elasticsearch] New Elasticsearch output {:class=\>"LogStash::Outputs::ElasticSearch", :hosts=\>["[//localhost:9200](https://localhost:9200)"]} [2018-07-25T15:20:02,998][INFO][logstash.pipeline] Starting pipeline {"id"=\>"main", "pipeline.workers"=\>1, "pipeline.batch.size"=\>125, "pipeline.batch.delay"=\>5, "pipeline.max\_inflight"=\>125} [2018-07-25T15:20:05,156][INFO][logstash.pipeline] Pipeline main started [2018-07-25T15:20:05,439][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=\>9600} admin@ubuntu-18:~$ ############################ ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [July 27, 2018, 9:07am UTC](https://discuss.elastic.co/t/n2disk-uburst-log-parsing/141680/4 "2018-07-27T09:07:42Z")
It should show the documents being processed though, so perhaps it's a sincedb issue. Try using cat + a stdin to process things. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 24, 2018, 9:07am UTC](https://discuss.elastic.co/t/n2disk-uburst-log-parsing/141680/5 "2018-08-24T09:07:46Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.