# When do you index your data?

**URL:** <https://discuss.elastic.co/t/when-do-you-index-your-data/6110>\
**Category:** Elasticsearch\
**Created:** [December 9, 2011, 7:16pm UTC](https://discuss.elastic.co/t/when-do-you-index-your-data/6110 "2011-12-09T19:16:17Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Royce](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/royce/32/3016_2.png) [@Royce](https://discuss.elastic.co/u/Royce)\
**Post date:** [December 9, 2011, 7:16pm UTC](https://discuss.elastic.co/t/when-do-you-index-your-data/6110/1 "2011-12-09T19:16:17Z")

</div>

Hi,

I'm new to search and ES. Can anyone discuss when they index their  
data? Which method is effective?

In my mind, there's two options:

1. Parellal with CRUD operations to my relational database

2. A batch process that runs a SQL statement and then indexes data in  
bulk.

Look forward to hearing from the ES community.

Take care,

Royce

---

<div class="post-metadata">

**Author:** ![Karussell1](https://avatars.discourse-cdn.com/v4/letter/k/50afbb/32.png) [@Karussell1](https://discuss.elastic.co/u/Karussell1)\
**Post date:** [December 9, 2011, 7:39pm UTC](https://discuss.elastic.co/t/when-do-you-index-your-data/6110/2 "2011-12-09T19:39:35Z")

</div>

It depends on your requirements. If you need real time then method 1,  
if not or you want it simple then method 2.

BTW: this project can help [https://github.com/Aconex/scrutineer](https://github.com/Aconex/scrutineer)

Peter.

On 9 Dez., 20:16, Royce [royce.hay...@gmail.com](mailto:royce.hay...@gmail.com) wrote:

> Hi,
> 
> I'm new to search and ES. Can anyone discuss when they index their  
> data? Which method is effective?
> 
> In my mind, there's two options:
> 
> 1. Parellal with CRUD operations to my relational database
> 
> 2. A batch process that runs a SQL statement and then indexes data in  
> bulk.
> 
> Look forward to hearing from the ES community.
> 
> Take care,
> 
> Royce

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [December 9, 2011, 8:02pm UTC](https://discuss.elastic.co/t/when-do-you-index-your-data/6110/3 "2011-12-09T20:02:26Z")

</div>

For my project, I choose to :  
Persist my top entity with Hibernate  
Send this _top_ object to an ActiveMQ queue in the same transaction

Then, my _jms_ queue handle some work (pdf rendering, JSON convert and push to ES)

So it's not real time but in most case, near to real time.

BTW, pushing documents to ES is not really the most time consuming (not sure it's the right english expression !). I only make it asynchronous because I have other works to do with my entities.

HTH and hope it makes sense.

David 😉  
@dadoonet

Le 9 déc. 2011 à 20:39, Karussell [tableyourtime@googlemail.com](mailto:tableyourtime@googlemail.com) a écrit :

> It depends on your requirements. If you need real time then method 1,  
> if not or you want it simple then method 2.
> 
> BTW: this project can help [https://github.com/Aconex/scrutineer](https://github.com/Aconex/scrutineer)
> 
> Peter.
> 
> On 9 Dez., 20:16, Royce [royce.hay...@gmail.com](mailto:royce.hay...@gmail.com) wrote:
> 
> > Hi,
> > 
> > I'm new to search and ES. Can anyone discuss when they index their  
> > data? Which method is effective?
> > 
> > In my mind, there's two options:
> > 
> > 1. Parellal with CRUD operations to my relational database
> > 
> > 2. A batch process that runs a SQL statement and then indexes data in  
> > bulk.
> > 
> > Look forward to hearing from the ES community.
> > 
> > Take care,
> > 
> > Royce

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 3:45am UTC](https://discuss.elastic.co/t/when-do-you-index-your-data/6110/4 "2017-07-06T03:45:55Z")

</div>