# Elasticsearch hadoop plugin

**URL:** <https://discuss.elastic.co/t/elasticsearch-hadoop-plugin/10978>\
**Category:** Elasticsearch\
**Created:** [March 3, 2013, 12:13am UTC](https://discuss.elastic.co/t/elasticsearch-hadoop-plugin/10978 "2013-03-03T00:13:24Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Alexis\_Okuwa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alexis_okuwa/32/1983_2.png) [@Alexis\_Okuwa](https://discuss.elastic.co/u/Alexis_Okuwa)\
**Post date:** [March 3, 2013, 12:13am UTC](https://discuss.elastic.co/t/elasticsearch-hadoop-plugin/10978/1 "2013-03-03T00:13:24Z")

</div>

I was thinking about using the hadoop plugin for a project and saw that it  
is getting depercated but the project seems to be active on github, is  
there a reason its getting depercated

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\
**Post date:** [March 4, 2013, 9:43am UTC](https://discuss.elastic.co/t/elasticsearch-hadoop-plugin/10978/2 "2013-03-04T09:43:34Z")

</div>

The reason is, shared gateways do not scale well enough (local is much  
better) and stand in the way for implementing a new smart backup/restore  
mechanism. In fact, most people intend backups on HDFS/S3/NFS, but  
setting a shared gateway is just a snapshot of the index shared by many  
nodes, not a scalable backup/restore solution.

> <https://github.com/elastic/elasticsearch/issues/2458>
>
> Shared gateways (shared FS storage or S3 for example) are problematic performanc…e wise since they constantly need to snapshot the state of the index to a shared location, and then use that as the system of record. The local gateway on the other hand doesn't need it, and performs much better.
> 
> The main benefit of a shared gateway is the fact that the data is actually stored on another persistent location (i.e. using ephemeral disks on AWS, but still having the data on s3), but then its actually abusing the shared gateway design (to be used as a backup).
> 
> In the near future, we will have a proper snapshot(backup)/restore API, which will be the proper way to do backups, but relaying on the shared gateway for that is problematic. Note, backups can still be made by "rsync" the data location for each node "manually".

I think, after backup/restore is added to ES, you will be able to use  
HDFS again.

Jörg

Am 03.03.2013 01:13, schrieb Wojons Tech:

> I was thinking about using the hadoop plugin for a project and saw  
> that it is getting depercated but the project seems to be active on  
> github, is there a reason its getting depercated --  
> You received this message because you are subscribed to the Google  
> Groups "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send  
> an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 2:48am UTC](https://discuss.elastic.co/t/elasticsearch-hadoop-plugin/10978/3 "2017-07-06T02:48:39Z")

</div>


