# Full data in HDFS and Elasticsearch keeps the index pointer **URL:** \ **Category:** Elasticsearch\ **Created:** [September 22, 2014, 9:14am UTC](https://discuss.elastic.co/t/full-data-in-hdfs-and-elasticsearch-keeps-the-index-pointer/19906 "2014-09-22T09:14:55Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Hsu\_Orozco](https://avatars.discourse-cdn.com/v4/letter/h/e495f1/32.png) [@Hsu\_Orozco](https://discuss.elastic.co/u/Hsu_Orozco)\ **Post date:** [September 22, 2014, 9:14am UTC](https://discuss.elastic.co/t/full-data-in-hdfs-and-elasticsearch-keeps-the-index-pointer/19906/1 "2014-09-22T09:14:55Z")
Hi all, In order to solve the query/ aggregation issue, I want to park index data in HDFS and ES keeps the index pointer. Does anyone know how to do that? Please give me a guide, thx. I know someone use MySQL to keep the index data, it's very like what I am talking about. However HDFS is more attractive to me, becasue HDFS is a centralize place that I can share it to different hadoop ecosystem. Best Regards, Orozco -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/7224ed04-4cc0-4aaf-a08f-038827023c4e%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/7224ed04-4cc0-4aaf-a08f-038827023c4e%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![costin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/costin/32/44950_2.png) [@costin](https://discuss.elastic.co/u/costin)\ **Post date:** [September 22, 2014, 9:20am UTC](https://discuss.elastic.co/t/full-data-in-hdfs-and-elasticsearch-keeps-the-index-pointer/19906/2 "2014-09-22T09:20:15Z")
What is the query/aggregation issue that you are referring to? On 9/22/14 12:14 PM, Hsu Orozco wrote: > Hi all, > > In order to solve the query/ aggregation issue, I want to park index data in HDFS and ES keeps the index pointer. > > Does anyone know how to do that? Please give me a guide, thx. I know someone use MySQL to keep the index data, it's very > like what I am talking about. > > However HDFS is more attractive to me, becasue HDFS is a centralize place that I can share it to different hadoop ecosystem. > > Best Regards, > Orozco > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) [mailto:elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/7224ed04-4cc0-4aaf-a08f-038827023c4e%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/7224ed04-4cc0-4aaf-a08f-038827023c4e%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/7224ed04-4cc0-4aaf-a08f-038827023c4e%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/7224ed04-4cc0-4aaf-a08f-038827023c4e%40googlegroups.com?utm_medium=email&utm_source=footer). > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- Costin -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/541FE9CF.7030607%40gmail.com](https://groups.google.com/d/msgid/elasticsearch/541FE9CF.7030607%40gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Hsu\_Orozco](https://avatars.discourse-cdn.com/v4/letter/h/e495f1/32.png) [@Hsu\_Orozco](https://discuss.elastic.co/u/Hsu_Orozco)\ **Post date:** [September 23, 2014, 6:34am UTC](https://discuss.elastic.co/t/full-data-in-hdfs-and-elasticsearch-keeps-the-index-pointer/19906/3 "2014-09-23T06:34:53Z")
Hi Costin, my query looks like this: POST \_search { "query": { "filtered": { "query": { "match\_all": {} }, "filter": { "bool": { "must\_not": [ { "terms": { "host": ["","Null"] } },{ "terms": { "curr\_subscr\_id": ["","Null"] } } ] } } } }, "aggs":{ "host\_aggs":{ "terms":{ "field":"host" }, "aggs":{ "subscr\_aggs":{ "cardinality":{ "field":"curr\_subscr\_id" } } } } } ,"size": 1 ,"fields": [ "age","start\_time","host","\_source" ] } -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/fb3b27b0-9ff2-435a-a216-f76079541dc3%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/fb3b27b0-9ff2-435a-a216-f76079541dc3%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Hsu\_Orozco](https://avatars.discourse-cdn.com/v4/letter/h/e495f1/32.png) [@Hsu\_Orozco](https://discuss.elastic.co/u/Hsu_Orozco)\ **Post date:** [September 23, 2014, 6:38am UTC](https://discuss.elastic.co/t/full-data-in-hdfs-and-elasticsearch-keeps-the-index-pointer/19906/4 "2014-09-23T06:38:22Z")
And another one thing, I was misunderstanding, ES can't parallel work with another repository such like MySQL. I mean ES can't keep index pointer in itself, and index data in MySQL. All searching process must be done in ES. The only way improving performane seems like adding more machines, so far I think. Hsu Orozco於 2014年9月22日星期一UTC+8下午5時14分56秒寫道： > Hi all, > > In order to solve the query/ aggregation issue, I want to park index data > in HDFS and ES keeps the index pointer. > > Does anyone know how to do that? Please give me a guide, thx. I know > someone use MySQL to keep the index data, it's very like what I am talking > about. > > However HDFS is more attractive to me, becasue HDFS is a centralize place > that I can share it to different hadoop ecosystem. > > Best Regards, > Orozco -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/aed46066-bf80-4f21-a329-d5779734f316%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/aed46066-bf80-4f21-a329-d5779734f316%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:00am UTC](https://discuss.elastic.co/t/full-data-in-hdfs-and-elasticsearch-keeps-the-index-pointer/19906/5 "2017-07-06T01:00:30Z")