Want to Create Pie Chart Visualization (Showing Most Popular Browsers)

Elastic Stack Kibana
1

Windows 7
elasticsearch v1.6.0
logstash v1.5.2
kibana v4.1.0
Apache Access Log file (access_log)

Background
I have successfully used logstash to import an Apache Access log with the Grok Fitler "COMBINEDAPACHELOG"

I need help creating a simple visualization. I need to create a pie chart that shows the top 5 browser used (agents) with user friendly names. Showing browser name and version. Google Chrome XX.X, Firefox vXX.X, IE9, IE10, IE11, Safari vXX.X

My pie chart visualization criteria
split slices
metric tab --> slice size --> count
buckets tab
aggregation --> Terms
field --> agent
order --> top
size --> 5
order by --> metric: Count

My pie chart legend shows agent names "mozilla", "nt", "windows", "6.1" "trident". I really want more user friendly browser names and browser versions. I am not sure what browser names and versions correspond with Example Google Chrome vXX.X, Firefox vXX.X, Safari, vXX.X, IE9, IE10.

Question
Can someone help me understand how to create my pie chart visualization?

Here is my logstash configuration file that I used to import data from my Apache Access File to elasticsearch.

input {
file {
path => "c:\websites\elkstack\logs\combinedtuesday"
type => "apache-access"
start_position => "beginning"
}
}

filter {
grok {
match => [ "message", "%{COMBINEDAPACHELOG}" ]
}

date {
locale => "en"
match => [ "timestamp" , "dd/MMM/YYYY:HH:mm:ss Z" ]
}
}

output {
elasticsearch {
host => "localhost"
cluster => "bauer"
index => "cxp"
}

stdout {
codec => rubydebug
}
}

2

Check out https://www.elastic.co/blog/kibana-4-video-tutorials-part-1 for some example videos.

3

Thank you Mark for your reply. I have watched the videos that you mentioned. I guess my challenge is not understanding how to get the Kibana Pie Chart "Legend" to display browser names and versions that as user-friendly names.

Good news is that I have the "agent" field populated in elasticsearch (from my Apache Access Log FIle), but the agent field does not contain browser names and versions like Google Chrome vXX.X, Firefox vXX.X, Safari vXX.X, or IE9, IE10, IE11.

How can I display more user-friendly browser names, versions in my Kibana Pie Chart "Legend"?

4

This is not currently possible, but is on the roadmap: https://github.com/elastic/kibana/issues/1896

5

Good to know. Thank you for taking the time to reply.

6

Given you are using ELK, use one of the .raw fields, it should help.

7

@bhutchinson - What does the agent field contain?

We use ELK to monitor our web traffic as you can see here and we do extract the browser and the version being used and graph it.

-- Asaf.

8

@asafyigal - Thank you for your reply and article. I will check it out.

My pie chart legend shows agent names "mozilla", "nt", "windows", "6.1" "trident". I really want more user friendly browser names and browser versions. I am not sure what browser names and versions correspond with Example Google Chrome vXX.X, Firefox vXX.X, Safari, vXX.X, IE9, IE10. Any idea what I can do to have my legend display more user friendly names like the visualizations that display in the link you provided me?

Here are a couple of lines from my Apache Access Log file. The "agent" field value is the last field value on the line.
Example "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0)"

75.78.177.126 - cthibodeaux [21/Jun/2015:00:17:32 -0400] "GET /wps/myportal/WestInteractivePortalDemo/site/HostedIVR HTTP/1.1" 302 - "-" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0)"
75.78.177.126 - cthibodeaux [21/Jun/2015:00:17:35 -0400] "GET /wps/myportal/WestInteractivePortalDemo/site/HostedIVR HTTP/1.1" 200 81441 "-" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0)"

9

@bhutchinson - I am not sure that your data is parsed correctly when entered. This is an example of the data in our system for "agent" and we then break it further - "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.132 Safari/537.36"

This is the config that we're using for apache logs and you can send me a sample raw line and I can take a look - you can email me at asaf@logz.io

grok {
match => [ "message" , "%{COMBINEDAPACHELOG}+%{GREEDYDATA:extra_fields}"]
overwrite => [ "message" ]
}

mutate {
  convert => ["response", "integer"]
  convert => ["bytes", "integer"]
  convert => ["responsetime", "float"]
}

useragent {
  source => "agent"
}

-- Asaf.

10

Edit: Understood and got the resolution :smile:

Sorry for bumping old thread, but I think this is nice place to ask rather than creating a new thread.

@asafyigal Could you please explain more? I need to show the browser versions in pie chart and OS version and I don't understand your last reply.

My current filter is

grok {
      match => [ "message", "(%{IPORHOST:clientip})? %{USER:ident} %{USER:auth} \[%{HTTPDATE:timestamp}\] \"(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})\" %{NUMBER:response} (?:%{NUMBER:bytes:int}|-) %{NUMBER:response_time:float} %{QS:referrer} %{QS:agent}" ]
11

@raamee - You would need to add the useragent filter in order to extract the browser version and the OS.

-- Asaf.

2 Likes