# Indexing in Hadoop **URL:** \ **Category:** Elasticsearch\ **Created:** [November 18, 2011, 9:13pm UTC](https://discuss.elastic.co/t/indexing-in-hadoop/5914 "2011-11-18T21:13:12Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![camechis](https://avatars.discourse-cdn.com/v4/letter/c/34f0e0/32.png) [@camechis](https://discuss.elastic.co/u/camechis)\ **Post date:** [November 18, 2011, 9:13pm UTC](https://discuss.elastic.co/t/indexing-in-hadoop/5914/1 "2011-11-18T21:13:12Z")
I am attempting to create a M/R job that reads data from Hbase, creates indexes via lucene and save the index to hdfs/local. I am trying to figure out the integration point with elastic search. Do I have to do something different for elastic search? How does es read the indexes and auto shard it? I am still some one new with hadoop and I am just looking into elastic search. ---
**Author:** ![otisg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/otisg/32/492_2.png) [@otisg](https://discuss.elastic.co/u/otisg)\ **Post date:** [November 20, 2011, 3:38am UTC](https://discuss.elastic.co/t/indexing-in-hadoop/5914/2 "2011-11-20T03:38:00Z")
Hello, I think you should think about this a little differently. For example, think about sending documents formed from data in HBase directly to ES via its API instead of thinking how to index with Lucene. When you do that, you'll learn answers to your questions as you learn about using the ES API to index data. In terms or reading data from HBase, you could start by looking at HBase's Export MR job. For indexing to ES: [Elastic — The Search AI Company | Elastic](http://www.elasticsearch.org/guide/reference/api/index_.html) ## Otis Sematext :: [http://sematext.com/](http://sematext.com/) :: Solr - Lucene - Hadoop - HBase Lucene ecosystem search :: [http://search-lucene.com/](http://search-lucene.com/) On Nov 18, 4:13 pm, camechis [camec...@gmail.com](mailto:camec...@gmail.com) wrote: > I am attempting to create a M/R job that reads data from Hbase, > creates indexes via lucene and save the index to hdfs/local. I am > trying to figure out the integration point with Elasticsearch. Do I > have to do something different for Elasticsearch? How does es read > the indexes and auto shard it? I am still some one new with hadoop > and I am just looking into Elasticsearch. ---
**Author:** ![camechis](https://avatars.discourse-cdn.com/v4/letter/c/34f0e0/32.png) [@camechis](https://discuss.elastic.co/u/camechis)\ **Post date:** [November 21, 2011, 2:01pm UTC](https://discuss.elastic.co/t/indexing-in-hadoop/5914/3 "2011-11-21T14:01:18Z")
Ok, I think I understand based on some other things I found. Basically read the records out of HBase and form some type of document ( JSON, etc ). Once the document is created post it to the Elasticsearch cluster via the JAVA api. On Nov 19, 10:38 pm, Otis Gospodnetic [otis.gospodne...@gmail.com](mailto:otis.gospodne...@gmail.com) wrote: > Hello, > > I think you should think about this a little differently. For > example, think about sending documents formed from data in HBase > directly to ES via its API instead of thinking how to index with > Lucene. When you do that, you'll learn answers to your questions as > you learn about using the ES API to index data. > > In terms or reading data from HBase, you could start by looking at > HBase's Export MR job. > For indexing to ES:[Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/index_.html) > > ## Otis > > Sematext ::[http://sematext.com/::](http://sematext.com/::) Solr - Lucene - Hadoop - HBase > Lucene ecosystem search ::[http://search-lucene.com/](http://search-lucene.com/) > > On Nov 18, 4:13 pm, camechis [camec...@gmail.com](mailto:camec...@gmail.com) wrote: > > > I am attempting to create a M/R job that reads data from Hbase, > > creates indexes via lucene and save the index to hdfs/local. I am > > trying to figure out the integration point with Elasticsearch. Do I > > have to do something different for Elasticsearch? How does es read > > the indexes and auto shard it? I am still some one new with hadoop > > and I am just looking into Elasticsearch. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:48am UTC](https://discuss.elastic.co/t/indexing-in-hadoop/5914/4 "2017-07-06T03:48:04Z")