# HDFS as elastic search data repository **URL:** \ **Category:** Elasticsearch\ **Created:** [April 1, 2016, 12:26pm UTC](https://discuss.elastic.co/t/hdfs-as-elastic-search-data-repository/46056 "2016-04-01T12:26:07Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![yogeshmsharma](https://avatars.discourse-cdn.com/v4/letter/y/a183cd/32.png) [@yogeshmsharma](https://discuss.elastic.co/u/yogeshmsharma)\ **Post date:** [April 1, 2016, 12:26pm UTC](https://discuss.elastic.co/t/hdfs-as-elastic-search-data-repository/46056/1 "2016-04-01T12:26:07Z")
For large volume of logs we are thinking of using HDFS as data repository for elastic search. **apache flume** is one of option suggested by few blogs on internet. Would like to get more information from community on this to come on conclusion. How do I instruct Elastic search to store data in HDFS and query/index same every time. LOGSTASH ---\> ELASTIC SEARCH ( want to use Kibana for visualization ) -- \> HDFS ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [April 2, 2016, 6:29pm UTC](https://discuss.elastic.co/t/hdfs-as-elastic-search-data-repository/46056/2 "2016-04-02T18:29:25Z")
Except for [ES-Hadoop](https://www.elastic.co/products/hadoop) I don't believe HDFS is a supported storage backend for Elasticsearch. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [April 3, 2016, 1:55am UTC](https://discuss.elastic.co/t/hdfs-as-elastic-search-data-repository/46056/3 "2016-04-03T01:55:06Z")
It is not. ---
**Author:** ![yogeshmsharma](https://avatars.discourse-cdn.com/v4/letter/y/a183cd/32.png) [@yogeshmsharma](https://discuss.elastic.co/u/yogeshmsharma)\ **Post date:** [April 6, 2016, 6:46am UTC](https://discuss.elastic.co/t/hdfs-as-elastic-search-data-repository/46056/4 "2016-04-06T06:46:18Z")
Thanks @magnusbaeck. ES-hadoop as far as I understood from documentation is to do search on Hadoop echo system. Where as my requirement is to just use high storage capacity of Hadoop. I will be collecting huge logs from 'n' number of Micro services and need to store this in faster storage for better retrieval. any thought on that ?. I am getting tilt towards mongoDb a bit, doing research on elastic and mongodb for the moment ---
**Author:** ![yogeshmsharma](https://avatars.discourse-cdn.com/v4/letter/y/a183cd/32.png) [@yogeshmsharma](https://discuss.elastic.co/u/yogeshmsharma)\ **Post date:** [April 13, 2016, 1:08pm UTC](https://discuss.elastic.co/t/hdfs-as-elastic-search-data-repository/46056/5 "2016-04-13T13:08:09Z")
any link/guide to setup elastic indices in Hadoop? ---
**Author:** ![thn](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thn/32/8061_2.png) [@thn](https://discuss.elastic.co/u/thn)\ **Post date:** [April 13, 2016, 1:37pm UTC](https://discuss.elastic.co/t/hdfs-as-elastic-search-data-repository/46056/6 "2016-04-13T13:37:20Z")
I don't think it is supported... if you want it, you need to build one yourself... do it for the community 😉 Here is an idea that you can think about, check out the link below > **[Indexing and Searching on a Hadoop Distributed File System](http://www.drdobbs.com/parallel/indexing-and-searching-on-a-hadoop-distr/226300241)** > > Finding the needle in an Internet haystack ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [April 13, 2016, 2:16pm UTC](https://discuss.elastic.co/t/hdfs-as-elastic-search-data-repository/46056/7 "2016-04-13T14:16:16Z")
You should really not do this. Really... That being said, may be you can mount an [NFS drive running on hadoop](https://hadoop.apache.org/docs/r2.4.1/hadoop-project-dist/hadoop-hdfs/HdfsNfsGateway.html) and put your indices in it but again, I would not do it... Why not sending your logs directly to elasticsearch? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 10:59pm UTC](https://discuss.elastic.co/t/hdfs-as-elastic-search-data-repository/46056/8 "2017-07-05T22:59:36Z")