# Need some advice to build a log central **URL:** \ **Category:** Elasticsearch\ **Created:** [August 25, 2014, 2:26am UTC](https://discuss.elastic.co/t/need-some-advice-to-build-a-log-central/19427 "2014-08-25T02:26:19Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![zkidkid](https://avatars.discourse-cdn.com/v4/letter/z/fbc32d/32.png) [@zkidkid](https://discuss.elastic.co/u/zkidkid)\ **Post date:** [August 25, 2014, 2:26am UTC](https://discuss.elastic.co/t/need-some-advice-to-build-a-log-central/19427/1 "2014-08-25T02:26:19Z")
Hi All, I am going to build a log central using ElasticSearch. I need some advice from anyone who have built it already. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/5af88543-5806-4021-83a5-41abc5b2bed6%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/5af88543-5806-4021-83a5-41abc5b2bed6%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![vineeth\_mohan\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vineeth_mohan_2/32/747_2.png) [@vineeth\_mohan\_2](https://discuss.elastic.co/u/vineeth_mohan_2)\ **Post date:** [August 25, 2014, 3:24am UTC](https://discuss.elastic.co/t/need-some-advice-to-build-a-log-central/19427/2 "2014-08-25T03:24:43Z")
Hello Sang , As this is a question answer forum , we highly recommend you to take a shot yourself and post questions if you have hit a dead end. Thanks Vineeth On Mon, Aug 25, 2014 at 7:56 AM, Sang Dang [zkidkid@gmail.com](mailto:zkidkid@gmail.com) wrote: > Hi All, > I am going to build a log central using Elasticsearch. > I need some advice from anyone who have built it already. > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/5af88543-5806-4021-83a5-41abc5b2bed6%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/5af88543-5806-4021-83a5-41abc5b2bed6%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/5af88543-5806-4021-83a5-41abc5b2bed6%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/5af88543-5806-4021-83a5-41abc5b2bed6%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAGdPd5%3DFpaQo8togCSKFR5J6bsqMBw\_8-39uNPPj\_Q6H2ag%2Bow%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAGdPd5%3DFpaQo8togCSKFR5J6bsqMBw_8-39uNPPj_Q6H2ag%2Bow%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![zkidkid](https://avatars.discourse-cdn.com/v4/letter/z/fbc32d/32.png) [@zkidkid](https://discuss.elastic.co/u/zkidkid)\ **Post date:** [August 25, 2014, 8:05am UTC](https://discuss.elastic.co/t/need-some-advice-to-build-a-log-central/19427/3 "2014-08-25T08:05:21Z")
Hi Vineeth Mohan, My log central will contain 2 type of log, one is for log to debug/monitor, other is for stats. I have 2 ways to achieve it: #1 , I use only ES, it's ok to log for debug/monitor (using kibana). To do stats, I will build some extra api (base on filter/facet/agregration...) #2, I use ES as external data storage, and write data to ES use Apache Hive ([https://github.com/elasticsearch/elasticsearch-hadoop#apache-hive](https://github.com/elasticsearch/elasticsearch-hadoop#apache-hive)) this approach will help me alot in doing stats, but I don't know whether it's good for logging other info ( to debug/monitor purpose). I really appreciate your help 🙂 Best Regards. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/20091177-e41d-45be-bd0a-c535d7c65871%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/20091177-e41d-45be-bd0a-c535d7c65871%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![zkidkid](https://avatars.discourse-cdn.com/v4/letter/z/fbc32d/32.png) [@zkidkid](https://discuss.elastic.co/u/zkidkid)\ **Post date:** [August 26, 2014, 2:33am UTC](https://discuss.elastic.co/t/need-some-advice-to-build-a-log-central/19427/4 "2014-08-26T02:33:24Z")
Hello All, I have selected #2 as my solution. I write data to ES, and use kibana+ to realtime monitor. For stats, I use Hive. Each project, I will create a index, for each type of log I will put in a ES Type, ex: ProjectX\>\>log\_debug \>\>log\_error \>\>Stats\_API \>\>Stats\_PageView \>\>Stats\_XYZ I am wonder whether it's good ? Should I separate by time for each type of project ? Regards. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/35487688-4204-4f4d-aa2e-2a9b6a43aa82%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/35487688-4204-4f4d-aa2e-2a9b6a43aa82%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![vineeth\_mohan\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vineeth_mohan_2/32/747_2.png) [@vineeth\_mohan\_2](https://discuss.elastic.co/u/vineeth_mohan_2)\ **Post date:** [August 26, 2014, 12:55pm UTC](https://discuss.elastic.co/t/need-some-advice-to-build-a-log-central/19427/5 "2014-08-26T12:55:07Z")
Hello Sang , Can i know why you are using Hive. I feel you can do the analysis in Elasticsearch itself. Rest seems good to me. Thanks Vineeth On Tue, Aug 26, 2014 at 8:03 AM, Sang Dang [zkidkid@gmail.com](mailto:zkidkid@gmail.com) wrote: > Hello All, > I have selected #2 as my solution. > I write data to ES, and use kibana+ to realtime monitor. > For stats, I use Hive. > > Each project, I will create a index, for each type of log I will put in a > ES Type, > ex: ProjectX\>\>log\_debug > \>\>log\_error > \>\>Stats\_API > \>\>Stats\_PageView > \>\>Stats\_XYZ > > I am wonder whether it's good ? > Should I separate by time for each type of project ? > > Regards. > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/35487688-4204-4f4d-aa2e-2a9b6a43aa82%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/35487688-4204-4f4d-aa2e-2a9b6a43aa82%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/35487688-4204-4f4d-aa2e-2a9b6a43aa82%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/35487688-4204-4f4d-aa2e-2a9b6a43aa82%40googlegroups.com?utm_medium=email&utm_source=footer) > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAGdPd5mvMAK%3DkKqg%3DTyzb-J0Boo\_CVPUnC\_vY0j%2BhNn\_rH8\_%3Dw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAGdPd5mvMAK%3DkKqg%3DTyzb-J0Boo_CVPUnC_vY0j%2BhNn_rH8_%3Dw%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:06am UTC](https://discuss.elastic.co/t/need-some-advice-to-build-a-log-central/19427/6 "2017-07-06T01:06:26Z")