Help needed to create VISUALIZATION

[chandukreddi]

Hello Experts,

I am trying to create Visualization for my Elasticsearch data using Kibana.

My Grok Pattern:

JAVALOGBACKTIMESTAMP %{YEAR}-%{MONTHNUM}-%{MONTHDAY} %{HOUR}:%{MINUTE}:%{SECOND},%{INT:msec}

CASS_BASE ^%{LOGLEVEL:level}\s*[(%{DATA:process}:%{INT:threadId}|%{DATA:process})]\s*%{JAVALOGBACKTIMESTAMP:timestamp}\s*%{DATA:java_file}.java:%{INT:line_number}\s*-

CASS_GC_GRACE %{CASS_BASE} %{DATA:msg} in %{INT:gc_duration}ms. %{DATA}:\s*%{INT:eden_orig_bytes}\s*->\s*%{INT:eden_new_bytes}; %{DATA}:\s*%{INT:oldgen_orig_bytes}\s*->\s*%{INT:oldgen_new_bytes};(\s*%{DATA}:\s*%{INT:surv_org_bytes}\s*->\s*%{INT:surv_new_bytes};|)

After Parsing my log I can see the data from KIbana as I was expecting but I need some help in creating Visualization.

As you can see below Image is having my parsed data, I just want to visualize GC_DURATION field data as a timechart like 2 am gc_duration was 300 msec, 2:01 am 350 msec, 3 am 1000 msec not like count (when I was trying to create visualize it was giving only count) to understand the gc pause time.

Below is Image from Kibana

I appreciate your help!

Thanks
Chandra

[bhavyarm]

@thomasneirynck help? Thank you!

[thomasneirynck]

hi @chandukreddi ,

you probably want to create a line chart with "average gc_duration" as the metric. Go to "Visualize" and select a line-chart. Select as "metric" "average gc_duration", and then under "buckets", select "Date histogram".

[chandukreddi]

Thanks @thomasneirynck for your reply.. but I don't find anything name with metric

Went to Visualization.

Choosen Line chart

Choosed prd-log* index, this is where i have my data.

But I don't see any metric as you said.

Thanks
Chandra

[thomasneirynck]

hi @chandukreddi If you expand

you'll be able to configure the metric you want ES to compute.

[chandukreddi]

@thomasneirynck , Yes your correct I was able to choose Average but my field gc_duration was not showing up in that drop down list.

But I could see in Discover page.

I just want to let you know gc_duration field value is time and it's in milliseconds.

Thanks
Chandra

[thomasneirynck]

Can you index it as a number?

[chandukreddi]

I didn't get it! I believe it's already indexed.

I am converting that as INT, below is my grok pattern.

WARN [Service Thread] 2016-06-30 14:42:41,538 GCInspector.java:256 - G1 Young Generation GC in 211ms. G1 Eden Space: 16894656512 -> 0; G1 Old Gen: 13259744800 -> 13267292200;
CASS_GC_GRACE %{CASS_BASE} %{DATA:msg} in %{INT:gc_duration}ms. %{DATA}:\s*%{INT:eden_orig_bytes}\s*->\s*%{INT:eden_new_bytes}; %{DATA}:\s*%{INT:oldgen_orig_bytes}\s*->\s*%{INT:oldgen_new_bytes};(\s*%{DATA}:\s*%{INT:surv_org_bytes}\s*->\s*%{INT:surv_new_bytes};|)

Thanks
Chandra

[suripaleru (surendra)]

Hi, I tried to apply above concepts on pie chart but I count see more fields under metrics. I could see only two metric under metrics , those are level order and totlaexectioninmiili . How I can get all fields under metric dropdown ?

[chandukreddi]

Hi @thomasneirynck,

Now I could see my filed in Kibana, but Y-axis you asked me to choose average right? can I take Y-axis my gc_duration field (I see only aggregation on Y-axis) and x-axis timestamp?

I really appreciate your help!

Thanks
Chandra