# Query HDFS

**URL:** <https://discuss.elastic.co/t/query-hdfs/78996>\
**Category:** Elasticsearch\
**Created:** [March 17, 2017, 10:04am UTC](https://discuss.elastic.co/t/query-hdfs/78996 "2017-03-17T10:04:48Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![shin1](https://avatars.discourse-cdn.com/v4/letter/s/51bf81/32.png) [@shin1](https://discuss.elastic.co/u/shin1)\
**Post date:** [March 17, 2017, 10:04am UTC](https://discuss.elastic.co/t/query-hdfs/78996/1 "2017-03-17T10:04:48Z")

</div>

Hi Gurus,

Does anybody know if Elasticsearch can directory query HDFS where the target data stored?  
In other words, is it possible for Elasticsearch without having any data to make a query?  
I'm not familiar with Hadoop HDFS Repository Plugin, but this is a right way?

Many thanks in advance for your help.  
Thanks,

---

<div class="post-metadata">

**Author:** ![ywelsch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ywelsch/32/7751_2.png) [@ywelsch](https://discuss.elastic.co/u/ywelsch)\
**Post date:** [March 17, 2017, 10:11am UTC](https://discuss.elastic.co/t/query-hdfs/78996/2 "2017-03-17T10:11:41Z")

</div>

no, that's not possible. Data needs to be indexed into Elasticsearch to be available for queries.

---

<div class="post-metadata">

**Author:** ![shin1](https://avatars.discourse-cdn.com/v4/letter/s/51bf81/32.png) [@shin1](https://discuss.elastic.co/u/shin1)\
**Post date:** [March 17, 2017, 10:57am UTC](https://discuss.elastic.co/t/query-hdfs/78996/3 "2017-03-17T10:57:04Z")

</div>

Hi Yannick,

Thank you very much for an update!

I just learned there's \_source enabled option and thought it might be a possible way if it's set to disabled, am not 100% sure, though..

BTW, you are saying that elasticsearch can't completely query hdfs?  
I'm looking into a reliable and proven architecture between ES and hdfs as I've heard that master data shouldn't be in ES but HDFS instead because of the safety of data.

Thanks,

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [March 19, 2017, 10:54pm UTC](https://discuss.elastic.co/t/query-hdfs/78996/4 "2017-03-19T22:54:28Z")

</div>

> [@shin1](#):
>
> BTW, you are saying that elasticsearch can't completely query hdfs?

Yes.

> [@shin1](#):
>
> I'm looking into a reliable and proven architecture between ES and hdfs as I've heard that master data shouldn't be in ES but HDFS instead because of the safety of data.

Where did you hear that?

---

<div class="post-metadata">

**Author:** ![shin1](https://avatars.discourse-cdn.com/v4/letter/s/51bf81/32.png) [@shin1](https://discuss.elastic.co/u/shin1)\
**Post date:** [March 20, 2017, 12:51am UTC](https://discuss.elastic.co/t/query-hdfs/78996/5 "2017-03-20T00:51:01Z")

</div>

OK, let me rephrase my questions.

-Can ES be a storage for master data? and does it require any data backup strategy?(Can ES be a reliable data storage?)  
Not 100% sure for now, but my guess is that hdfs as ES's data backup storage and ES requiring to have the searchable data. (Data needs to be loaded into ES)

-Can ES search even in PDF files?

-Does ES require ES-Hadoop connector to exchange data between ES and hdfs? Any alternatives?

Sorry for posting silly questions.  
I'm newbie to ES and learning it from scratch now, but it takes time...Your help is great pointer so that I can (re)direct my learning.

Cheers,

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [March 20, 2017, 2:34am UTC](https://discuss.elastic.co/t/query-hdfs/78996/6 "2017-03-20T02:34:35Z")

</div>

> [@shin1](#):
>
> -Can ES be a storage for master data? and does it require any data backup strategy?

Yes and Yes.

> [@shin1](#):
>
> -Can ES search even in PDF files?

If you index them in ES, yes.

> [@shin1](#):
>
> -Does ES require ES-Hadoop connector to exchange data between ES and hdfs? Any alternatives?

Yes. DIY.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 17, 2017, 2:34am UTC](https://discuss.elastic.co/t/query-hdfs/78996/7 "2017-04-17T02:34:37Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.