# Unable to extract the message filed from the logstash filter **URL:** \ **Category:** Logstash\ **Created:** [August 19, 2019, 8:47pm UTC](https://discuss.elastic.co/t/unable-to-extract-the-message-filed-from-the-logstash-filter/195807 "2019-08-19T20:47:12Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Arun\_Soman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/arun_soman/32/47218_2.png) [@Arun\_Soman](https://discuss.elastic.co/u/Arun_Soman)\ **Post date:** [August 19, 2019, 8:47pm UTC](https://discuss.elastic.co/t/unable-to-extract-the-message-filed-from-the-logstash-filter/195807/1 "2019-08-19T20:47:12Z")
I am trying to split my message into different fields . I used Grok filer on logstash but I am still getting the same message content in the Kibana discover dash board. My message as follows. 2019-8-16T18:37:16.45 CT3 45.282 Want to extract CT3 and 45.282 dont want time stamp. ` input { beats { port =\> 5044 } } filter { grok { match =\> { "message" =\> "%{USERNAME:user} %{NUMBER:duration}" } remove\_field =\> ["message"] } } output { elasticsearch { hosts =\> ["[http://localhost:9200](http://localhost:9200)"] index =\> "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}" } }` I already tested tested with grove debugger , I got the result as follows ``` %{USERNAME:user} %{NUMBER:duration} ``` { "user": [ [ "CT3" ] ], "duration": [ [ "45.282" ] ], "BASE10NUM": [ [ "45.282" ] ] } But in Kibana I am getting the same message as follows 2019-8-16T18:37:16.45 CT3 45.282Preformatted text ---
**Author:** ![kumarabhi](https://avatars.discourse-cdn.com/v4/letter/k/6a8cbe/32.png) [@kumarabhi](https://discuss.elastic.co/u/kumarabhi)\ **Post date:** [August 19, 2019, 9:29pm UTC](https://discuss.elastic.co/t/unable-to-extract-the-message-filed-from-the-logstash-filter/195807/2 "2019-08-19T21:29:49Z")
If your message is always a fixed format better use dissect for extracting values. It's much simpler Also, for debugging purpose please send Logstash output to stdout rather than ES ``` output { stdout { codec => rubydebug } } ``` ---
**Author:** ![Arun\_Soman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/arun_soman/32/47218_2.png) [@Arun\_Soman](https://discuss.elastic.co/u/Arun_Soman)\ **Post date:** [August 19, 2019, 10:12pm UTC](https://discuss.elastic.co/t/unable-to-extract-the-message-filed-from-the-logstash-filter/195807/3 "2019-08-19T22:12:06Z")
Could you please explain little bit about dissect for extracting values. I want to extract values and plot the graph in kibana. ---
**Author:** ![kumarabhi](https://avatars.discourse-cdn.com/v4/letter/k/6a8cbe/32.png) [@kumarabhi](https://discuss.elastic.co/u/kumarabhi)\ **Post date:** [August 19, 2019, 10:46pm UTC](https://discuss.elastic.co/t/unable-to-extract-the-message-filed-from-the-logstash-filter/195807/4 "2019-08-19T22:46:45Z")
Hi Arun, i was suggesting output to stdout for convenience. Your final config after the fix should output to ES and then you can plot graph in Kibana ``` input { beats { port => 5044 } } filter { if "django" not in [log][file][path] { dissect { mapping => { "[log][file][path]" => "/%{}/%{}/%{}/%{}/%{}/%{task_log_folder}/%{}" } } mutate { split => { "task_log_folder" => "_" } add_field => { "jobID" => "%{[task_log_folder][0]}" } add_field => { "taskID" => "%{[task_log_folder][1]}" } add_field => { "taskVersion" => "%{[task_log_folder][2]}" } } mutate { convert => { "jobID" => "integer" } convert => { "taskID" => "integer" } convert => { "taskVersion" => "integer" } remove_field => ["task_log_folder"] } } } output { stdout { codec => rubydebug { } } } ``` This is an example that i created where i was extracting values from folder structure in file path. The mapping in my case can also be done as mapping =\> { "[log][file][path]" =\> "/%{}/%{}/%{}/%{}/%{}/%{jobId} **\_** %{taskId} **\_** %{taskversion}/%{}" } I was experimenting with split. In your case, you will be mapping "message" field. {} is used when you are not interested in the field. Try out an example with simple messages ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [August 20, 2019, 12:07pm UTC](https://discuss.elastic.co/t/unable-to-extract-the-message-filed-from-the-logstash-filter/195807/5 "2019-08-20T12:07:09Z")
That grok works for me ``` "duration" => "45.282", "user" => "CT3" ``` Are you sure there is a single space between the two fields? Spacing matters in both grok and dissect. ---
**Author:** ![Arun\_Soman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/arun_soman/32/47218_2.png) [@Arun\_Soman](https://discuss.elastic.co/u/Arun_Soman)\ **Post date:** [August 20, 2019, 3:25pm UTC](https://discuss.elastic.co/t/unable-to-extract-the-message-filed-from-the-logstash-filter/195807/6 "2019-08-20T15:25:37Z")
Yes , single space between two fileds. How do you get the value for duration and user \< "duration" =\> "45.282", "user" =\> "CT3" \> ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [August 20, 2019, 7:02pm UTC](https://discuss.elastic.co/t/unable-to-extract-the-message-filed-from-the-logstash-filter/195807/7 "2019-08-20T19:02:54Z")
> [@Arun\_Soman](#): > > How do you get the value for duration and user ``` input { generator { count => 1 lines => ['2019-8-16T18:37:16.45 CT3 45.282'] } } filter { grok { match => { "message" => "%{USERNAME:user} %{NUMBER:duration}" } remove_field => ["message"] } } ``` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [September 17, 2019, 7:02pm UTC](https://discuss.elastic.co/t/unable-to-extract-the-message-filed-from-the-logstash-filter/195807/8 "2019-09-17T19:02:56Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.