# How do I extract a part of a string in the message field in Kibana and then plot it on graph?

**URL:** <https://discuss.elastic.co/t/how-do-i-extract-a-part-of-a-string-in-the-message-field-in-kibana-and-then-plot-it-on-graph/97717>\
**Category:** Kibana\
**Created:** [August 21, 2017, 7:27am UTC](https://discuss.elastic.co/t/how-do-i-extract-a-part-of-a-string-in-the-message-field-in-kibana-and-then-plot-it-on-graph/97717 "2017-08-21T07:27:03Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![chandramohan](https://avatars.discourse-cdn.com/v4/letter/c/ea5d25/32.png) [@chandramohan](https://discuss.elastic.co/u/chandramohan)\
**Post date:** [August 21, 2017, 7:27am UTC](https://discuss.elastic.co/t/how-do-i-extract-a-part-of-a-string-in-the-message-field-in-kibana-and-then-plot-it-on-graph/97717/1 "2017-08-21T07:27:03Z")

</div>

![Screenshot from 2017-08-16 14_31_33](https://us1.discourse-cdn.com/elastic/original/3X/6/a/6a3f52744717a743979a647152db67ee1eed39a3.png) ![Screenshot from 2017-08-16 14_31_33](https://us1.discourse-cdn.com/elastic/original/3X/6/a/6a3f52744717a743979a647152db67ee1eed39a3.png)

Here, I want to extract the values of each lineNumber string(i.e, 26,121,18...) in the message field and then plot it on a graph. I am just able to plot the number of occurrences of string "lineNumber", but I need to plot n number of lineNumber values on a graph. So how do I do it? Please help me out.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [August 21, 2017, 7:37am UTC](https://discuss.elastic.co/t/how-do-i-extract-a-part-of-a-string-in-the-message-field-in-kibana-and-then-plot-it-on-graph/97717/2 "2017-08-21T07:37:56Z")

</div>

The most efficient and scalable way to do this is to parse the `message` field at ingest time and extract the fields you want to run analysis on. You can do this using a [grok processor](https://www.elastic.co/guide/en/elasticsearch/reference/current/grok-processor.html) within an [ingest pipeline](https://www.elastic.co/guide/en/elasticsearch/reference/current/ingest.html).

---

<div class="post-metadata">

**Author:** ![chandramohan](https://avatars.discourse-cdn.com/v4/letter/c/ea5d25/32.png) [@chandramohan](https://discuss.elastic.co/u/chandramohan)\
**Post date:** [August 21, 2017, 8:34am UTC](https://discuss.elastic.co/t/how-do-i-extract-a-part-of-a-string-in-the-message-field-in-kibana-and-then-plot-it-on-graph/97717/3 "2017-08-21T08:34:00Z")

</div>

Hello @Christian_Dahlqvist, I hadn't setup the grok filter while I was setting up my whole ELK server. But then I realized that to get the lineNumber as a new field, I have to setup the grok filter. Can you please elucidate more upon how to setup the grok filter in the logstash configuration file and then how to extract the lineNumber string's values and then plot the graph?  
I had used this link to setup the ELK server : [https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-elk-stack-on-ubuntu-14-04#configure-logstash](https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-elk-stack-on-ubuntu-14-04#configure-logstash)

But the config for grok didnt work. SO, please explain more on this.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [August 21, 2017, 8:43am UTC](https://discuss.elastic.co/t/how-do-i-extract-a-part-of-a-string-in-the-message-field-in-kibana-and-then-plot-it-on-graph/97717/4 "2017-08-21T08:43:28Z")

</div>

If all your log lines have a similar structure, it may be easier to instead use the [dissect filter](https://www.elastic.co/guide/en/logstash/current/plugins-filters-dissect.html) instead of grok. [This blog post](https://www.elastic.co/blog/logstash-dude-wheres-my-chainsaw-i-need-to-dissect-my-logs) provides a good introduction.

---

<div class="post-metadata">

**Author:** ![chandramohan](https://avatars.discourse-cdn.com/v4/letter/c/ea5d25/32.png) [@chandramohan](https://discuss.elastic.co/u/chandramohan)\
**Post date:** [August 21, 2017, 10:32am UTC](https://discuss.elastic.co/t/how-do-i-extract-a-part-of-a-string-in-the-message-field-in-kibana-and-then-plot-it-on-graph/97717/5 "2017-08-21T10:32:13Z")

</div>

Yes, the log lines have a similar structure as you can see in the above attached screenshot. Can you give an example filter to extract the value of "lineNumber" string? Because I included a filter in /etc/logstash/conf.d/10-syslog-filter.conf . But then the logs were not transferring through filebeat. So, give an example which I can directly deploy.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [August 21, 2017, 10:51am UTC](https://discuss.elastic.co/t/how-do-i-extract-a-part-of-a-string-in-the-message-field-in-kibana-and-then-plot-it-on-graph/97717/6 "2017-08-21T10:51:40Z")

</div>

I do currently not have time to write it for you, but if you show what you have tried so far we might be able to help you find what is wrong.

---

<div class="post-metadata">

**Author:** ![chandramohan](https://avatars.discourse-cdn.com/v4/letter/c/ea5d25/32.png) [@chandramohan](https://discuss.elastic.co/u/chandramohan)\
**Post date:** [August 21, 2017, 11:03am UTC](https://discuss.elastic.co/t/how-do-i-extract-a-part-of-a-string-in-the-message-field-in-kibana-and-then-plot-it-on-graph/97717/7 "2017-08-21T11:03:27Z")

</div>

filter {  
if [type] == "syslog" {  
grok {  
match =\> { "message" =\> "%{SYSLOGTIMESTAMP:syslog\_timestamp} %{SYSLOGHOST:syslog\_hostname} %{DATA:syslog\_program}(?:[%{POSINT:syslog\_pid}])?: %{GREEDYDATA:syslog\_message}" }  
add\_field =\> ["received\_at", "%{@timestamp}"]  
add\_field =\> ["received\_from", "%{host}"]  
}  
syslog\_pri { }  
date {  
match =\> ["syslog\_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"]  
}  
}  
}

This was one of the filter that was given in [https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-elk-stack-on-ubuntu-14-04#configure-logstash](https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-elk-stack-on-ubuntu-14-04#configure-logstash)

I replaced the syslog by log, as I had configured it on Filebeat. Then replaced SYSLOGTIMESTAMP:syslog\_timestamp by lineNumber:number. But then the logs were not flowing from Filebeat to logstash. So, how can I modify to highlight lineNumber as a field? and have to extract that lineNumber value in n number of lines to plot a graph. Thanks in advance.

---

<div class="post-metadata">

**Author:** ![chandramohan](https://avatars.discourse-cdn.com/v4/letter/c/ea5d25/32.png) [@chandramohan](https://discuss.elastic.co/u/chandramohan)\
**Post date:** [August 22, 2017, 6:51am UTC](https://discuss.elastic.co/t/how-do-i-extract-a-part-of-a-string-in-the-message-field-in-kibana-and-then-plot-it-on-graph/97717/8 "2017-08-22T06:51:00Z")

</div>

@Christian_Dahlqvist, any inputs?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [August 22, 2017, 7:03am UTC](https://discuss.elastic.co/t/how-do-i-extract-a-part-of-a-string-in-the-message-field-in-kibana-and-then-plot-it-on-graph/97717/9 "2017-08-22T07:03:45Z")

</div>

That pattern does not at all match your log format. Have you looked through the blog post and documentation around the dissect filter I provided? Have you tried using it?

---

<div class="post-metadata">

**Author:** ![chandramohan](https://avatars.discourse-cdn.com/v4/letter/c/ea5d25/32.png) [@chandramohan](https://discuss.elastic.co/u/chandramohan)\
**Post date:** [August 22, 2017, 8:35am UTC](https://discuss.elastic.co/t/how-do-i-extract-a-part-of-a-string-in-the-message-field-in-kibana-and-then-plot-it-on-graph/97717/10 "2017-08-22T08:35:04Z")

</div>

I just showed an example, I had modified according to my requirement. Dissect filter, I'll try it.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [August 22, 2017, 8:36am UTC](https://discuss.elastic.co/t/how-do-i-extract-a-part-of-a-string-in-the-message-field-in-kibana-and-then-plot-it-on-graph/97717/11 "2017-08-22T08:36:18Z")

</div>

Let us know if you have any problems with it and we'll try to help.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 19, 2017, 8:36am UTC](https://discuss.elastic.co/t/how-do-i-extract-a-part-of-a-string-in-the-message-field-in-kibana-and-then-plot-it-on-graph/97717/12 "2017-09-19T08:36:20Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.