How to visualize statistics on time series data in Kibana

Elastic Stack Elasticsearch
1

Howdy everyone,
I have events with the following structure in ES:

{
"_index": "logstash-2014.02.25",
"_type": "symantecav-logs",
"_id": "_5Hig6lPTUi2p-palnuplA",
"_score": null,
"_source": {
"message": [
"1393368016|0|2|5|3|69.16.1.13/UMTL300X.rtf|4|UMTL300X.rtf|39|192.168.23.7|17|0.167|18|0.232|43|192.168.25.22|44|9003|45|12133924"
],
"@version": 1,
"@timestamp": "2014-02-25T22:40:16.000Z",
"host": "antivirus1.domain.net",
"tags": [
"antivirus",
"test",
"boston"
],
"file": "/antivirus/log/SSE20140225.log",
"type": "symantecav-logs",
"typecode": "0",
"filename": "UMTL300X.rtf",
"client": "client.domain.net",
"scan duration": 0.167,
"connect duration": 0.232,
"extension": "rtf"
},
"sort": [
1393368016000,
1393368016000
]
}

My goal is to visualize the max and mean of the scan and connect duration
over time as a line graph within Kibana. Is this possible with the widgets
currently available? I've been trying out several but haven't had much luck
getting them to do what I'm looking for.

Here are are the ES queries I'm using on the Kibana dashboard:
type:"symantecav-logs" AND tags:"test" AND host:"antivirus1.domain.net"
type:"symantecav-logs" AND tags:"test" AND host:"antivirus2.domain.net"

thanks for any and all help you can lend to a neophyte such as myself!
-Dave

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/fc93669e-6d95-4f99-b00d-63ad997865d2%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

When you add a Histogram panel, look in the setting Chart Value. There are
options for max and mean in there and then in the Value Field, you can
specify "scan duration" (or "connect duration") - I'm not 100% sure if the
spaces in your field name might fail but if it does, you'll probably need
to fix your LS config to output field names with no spaces. The only
limitation right now is you can't plot multiple time series stats (Chart
Value + Value Field) in 1 histogram at the moment. So you'll need to create
separate histograms per Chart Value + Value Field.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/80167728-cacd-4be1-829b-9fc2abb1ab3a%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

That did the trick! I was able to keep the spaces in the field name, but
did need to cast the field to a float in logstash for the metric to work.

Really loving how quickly valuable data hidden in the logs can be drawn out
and visualized with logstash+elasticsearch+kibana. Props to y'all for
making it happen.

Thanks!
-Dave

On Wed, Feb 26, 2014 at 9:52 AM, Binh Ly binhly_es@yahoo.com wrote:

When you add a Histogram panel, look in the setting Chart Value. There are
options for max and mean in there and then in the Value Field, you can
specify "scan duration" (or "connect duration") - I'm not 100% sure if the
spaces in your field name might fail but if it does, you'll probably need
to fix your LS config to output field names with no spaces. The only
limitation right now is you can't plot multiple time series stats (Chart
Value + Value Field) in 1 histogram at the moment. So you'll need to create
separate histograms per Chart Value + Value Field.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/mc7bmixJGe8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/80167728-cacd-4be1-829b-9fc2abb1ab3a%40googlegroups.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAA%3DO8vL7m1CJ9GRrmnLyJOHmWKKNoFgQc2vp-0b6pBC8ZmdHXw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

4

Oh yeah forgot about the datatype - that's good that you caught that. Good
to hear!

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/00e68528-8744-4a43-bf3b-f42982929230%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

5

My Apologies for picking up an old thread.

But I have the same issue. I am not able to understand what to put inside
the Value field when I select the Chart Value as Mean or Max or total.

I have a log whose field names are

Timestamp, elapsed, label, bytes, Latency
2014-11-13T13:05:45.430+0530,1184,PageName,7512,980
2014-11-13T13:05:45.447+0530,1167,PageName,7512,963
2014-11-13T13:05:45.449+0530,1262,PageName,7512,1049
2014-11-13T13:05:45.430+0530,1332,PageName,7512,1103

The Value field says it needs an numerical value.

I want to see how the Latency is over time.. which is like timestamp vs
Latency or timestamp vs elapsed.

Any ideas.

Thanks,
Krishna

On Wednesday, February 26, 2014 7:31:47 PM UTC+5:30, Dave Snigier wrote:

Howdy everyone,
I have events with the following structure in ES:

{
"_index": "logstash-2014.02.25",
"_type": "symantecav-logs",
"_id": "_5Hig6lPTUi2p-palnuplA",
"_score": null,
"_source": {
"message": [
"1393368016|0|2|5|3|69.16.1.13/UMTL300X.rtf|4|UMTL300X.rtf|39|192.168.23.7|17|0.167|18|0.232|43|192.168.25.22|44|9003|45|12133924 http://69.16.1.13/UMTL300X.rtf|4|UMTL300X.rtf|39|192.168.23.7|17|0.167|18|0.232|43|192.168.25.22|44|9003|45|12133924"
],
"@version": 1,
"@timestamp": "2014-02-25T22:40:16.000Z",
"host": "antivirus1.domain.net",
"tags": [
"antivirus",
"test",
"boston"
],
"file": "/antivirus/log/SSE20140225.log",
"type": "symantecav-logs",
"typecode": "0",
"filename": "UMTL300X.rtf",
"client": "client.domain.net",
"scan duration": 0.167,
"connect duration": 0.232,
"extension": "rtf"
},
"sort": [
1393368016000,
1393368016000
]
}

My goal is to visualize the max and mean of the scan and connect duration
over time as a line graph within Kibana. Is this possible with the widgets
currently available? I've been trying out several but haven't had much luck
getting them to do what I'm looking for.

Here are are the ES queries I'm using on the Kibana dashboard:
type:"symantecav-logs" AND tags:"test" AND host:"antivirus1.domain.net"
type:"symantecav-logs" AND tags:"test" AND host:"antivirus2.domain.net"

thanks for any and all help you can lend to a neophyte such as myself!
-Dave

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/53be466c-0b98-46cc-9568-a921fb32e135%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.