Hi, I'm trying to set up a real time streaming dashboard for logs, which
would collect logs using fluentd or similar, and all I would want to do is
extract running statistics from the data eg firewall denies / second. I
don't want to keep the data for any time, I just want to get the stats, so
ideally I wouldn't even want to store the logs in ElasticSearch at all (but
I know I have to for a short period)
Any suggestions on how I would do that, with a low retention time for logs,
so that Kibana could be used as a live dashboard?
Thanks!
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/33476786-71a6-443c-893d-90a51c0e6ad7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Kibana only knows to query Elasticsearch, so you essentially have to have
ES running in the background. You can use a short TTL on messages so they
expire automatically after a few sec/mins.
But to me it seems using Kibana is going overboard if all you want to do is
some graphics on top of some custom API?
--
Itamar Syn-Hershko
http://code972.com | @synhershko https://twitter.com/synhershko
Freelance Developer & Consultant
Author of RavenDB in Action http://manning.com/synhershko/
On Mon, Apr 7, 2014 at 7:27 PM, Chris Holt zuzzyuk@gmail.com wrote:
Hi, I'm trying to set up a real time streaming dashboard for logs, which
would collect logs using fluentd or similar, and all I would want to do is
extract running statistics from the data eg firewall denies / second. I
don't want to keep the data for any time, I just want to get the stats, so
ideally I wouldn't even want to store the logs in Elasticsearch at all (but
I know I have to for a short period)
Any suggestions on how I would do that, with a low retention time for
logs, so that Kibana could be used as a live dashboard?
Thanks!
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/33476786-71a6-443c-893d-90a51c0e6ad7%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/33476786-71a6-443c-893d-90a51c0e6ad7%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZteAXyjgWVyKZXN8%3DAD2oAmxN4XO1xB6_rmAs775i5%3DxQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Would SQLstream be applicable here?
On Monday, April 7, 2014 9:27:58 AM UTC-7, Chris Holt wrote:
Hi, I'm trying to set up a real time streaming dashboard for logs, which
would collect logs using fluentd or similar, and all I would want to do is
extract running statistics from the data eg firewall denies / second. I
don't want to keep the data for any time, I just want to get the stats, so
ideally I wouldn't even want to store the logs in Elasticsearch at all (but
I know I have to for a short period)
Any suggestions on how I would do that, with a low retention time for
logs, so that Kibana could be used as a live dashboard?
Thanks!
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8bfb977a-021d-4804-8ca5-1769e558ac91%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.