# Real time streaming dashboard using Kibana **URL:** \ **Category:** Elasticsearch\ **Created:** [April 7, 2014, 4:27pm UTC](https://discuss.elastic.co/t/real-time-streaming-dashboard-using-kibana/16849 "2014-04-07T16:27:58Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![Chris\_Holt](https://avatars.discourse-cdn.com/v4/letter/c/e36b37/32.png) [@Chris\_Holt](https://discuss.elastic.co/u/Chris_Holt)\ **Post date:** [April 7, 2014, 4:27pm UTC](https://discuss.elastic.co/t/real-time-streaming-dashboard-using-kibana/16849/1 "2014-04-07T16:27:58Z")
Hi, I'm trying to set up a real time streaming dashboard for logs, which would collect logs using fluentd or similar, and all I would want to do is extract running statistics from the data eg firewall denies / second. I don't want to keep the data for any time, I just want to get the stats, so ideally I wouldn't even want to store the logs in ElasticSearch at all (but I know I have to for a short period) Any suggestions on how I would do that, with a low retention time for logs, so that Kibana could be used as a live dashboard? Thanks! -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/33476786-71a6-443c-893d-90a51c0e6ad7%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/33476786-71a6-443c-893d-90a51c0e6ad7%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Itamar\_Syn\_Hershko](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/itamar_syn_hershko/32/725_2.png) [@Itamar\_Syn\_Hershko](https://discuss.elastic.co/u/Itamar_Syn_Hershko)\ **Post date:** [April 7, 2014, 4:51pm UTC](https://discuss.elastic.co/t/real-time-streaming-dashboard-using-kibana/16849/2 "2014-04-07T16:51:25Z")
Kibana only knows to query Elasticsearch, so you essentially have to have ES running in the background. You can use a short TTL on messages so they expire automatically after a few sec/mins. But to me it seems using Kibana is going overboard if all you want to do is some graphics on top of some custom API? -- Itamar Syn-Hershko [http://code972.com](http://code972.com) | @synhershko [https://twitter.com/synhershko](https://twitter.com/synhershko) Freelance Developer & Consultant Author of RavenDB in Action [http://manning.com/synhershko/](http://manning.com/synhershko/) On Mon, Apr 7, 2014 at 7:27 PM, Chris Holt [zuzzyuk@gmail.com](mailto:zuzzyuk@gmail.com) wrote: > Hi, I'm trying to set up a real time streaming dashboard for logs, which > would collect logs using fluentd or similar, and all I would want to do is > extract running statistics from the data eg firewall denies / second. I > don't want to keep the data for any time, I just want to get the stats, so > ideally I wouldn't even want to store the logs in Elasticsearch at all (but > I know I have to for a short period) > > Any suggestions on how I would do that, with a low retention time for > logs, so that Kibana could be used as a live dashboard? > > Thanks! > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/33476786-71a6-443c-893d-90a51c0e6ad7%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/33476786-71a6-443c-893d-90a51c0e6ad7%40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/33476786-71a6-443c-893d-90a51c0e6ad7%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/33476786-71a6-443c-893d-90a51c0e6ad7%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZteAXyjgWVyKZXN8%3DAD2oAmxN4XO1xB6\_rmAs775i5%3DxQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZteAXyjgWVyKZXN8%3DAD2oAmxN4XO1xB6_rmAs775i5%3DxQ%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Andrew\_Bare](https://avatars.discourse-cdn.com/v4/letter/a/df788c/32.png) [@Andrew\_Bare](https://discuss.elastic.co/u/Andrew_Bare)\ **Post date:** [April 9, 2014, 5:08pm UTC](https://discuss.elastic.co/t/real-time-streaming-dashboard-using-kibana/16849/3 "2014-04-09T17:08:23Z")
Would SQLstream be applicable here? On Monday, April 7, 2014 9:27:58 AM UTC-7, Chris Holt wrote: > Hi, I'm trying to set up a real time streaming dashboard for logs, which > would collect logs using fluentd or similar, and all I would want to do is > extract running statistics from the data eg firewall denies / second. I > don't want to keep the data for any time, I just want to get the stats, so > ideally I wouldn't even want to store the logs in Elasticsearch at all (but > I know I have to for a short period) > > Any suggestions on how I would do that, with a low retention time for > logs, so that Kibana could be used as a live dashboard? > > Thanks! -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/8bfb977a-021d-4804-8ca5-1769e558ac91%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/8bfb977a-021d-4804-8ca5-1769e558ac91%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:37am UTC](https://discuss.elastic.co/t/real-time-streaming-dashboard-using-kibana/16849/4 "2017-07-06T01:37:10Z")