# Poor performance updating
**URL:** https://discuss.elastic.co/t/poor-performance-updating/6703
**Category:** Elasticsearch
**Created:** [February 14, 2012, 10:16am UTC](https://discuss.elastic.co/t/poor-performance-updating/6703 "2012-02-14T10:16:21Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@haarts](https://discuss.elastic.co/u/haarts)
#### Post date: [February 14, 2012, 10:16am UTC](https://discuss.elastic.co/t/poor-performance-updating/6703/1 "2012-02-14T10:16:21Z")
Dear list,
We were stoked when we found out about the updating feature in the recent
0.19.0rc2 release. We have been eagerly experimenting with it but are
disappointed by it's performance. Hopefully you can tell us we are doing
something wrong.
We roughly use this model: [https://gist.github.com/1751349](https://gist.github.com/1751349). Starting from a
clean index it takes 7 seconds to index 1000 documents (ok-ish). After
indexing 3 million documents performance degrades to 30 seconds per 1000
documents (prohibitively slow). We expect to insert 500 million documents
plus 4 million a day.
Our approach inserting documents is as follows:
We first try to update a document, if that returns an error we instead
create it.
The resulting documents can contain hundreds and possibly thousands of
'interactions' growing the document size to about 3Mb.
Are there ways of speeding this process up?
With kind regards,
Harm
---
### Author:  [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [February 14, 2012, 2:53pm UTC](https://discuss.elastic.co/t/poor-performance-updating/6703/2 "2012-02-14T14:53:56Z")
My guess is that it simply gets slower since you index bigger documents with more interactions. The update API still reindex the document. You might turn things around and index and interaction as its own document.
On Tuesday, February 14, 2012 at 12:16 PM, haarts wrote:
> Dear list,
>
> We were stoked when we found out about the updating feature in the recent 0.19.0rc2 release. We have been eagerly experimenting with it but are disappointed by it's performance. Hopefully you can tell us we are doing something wrong.
>
> We roughly use this model: [ES Data Model Skylines · GitHub](https://gist.github.com/1751349). Starting from a clean index it takes 7 seconds to index 1000 documents (ok-ish). After indexing 3 million documents performance degrades to 30 seconds per 1000 documents (prohibitively slow). We expect to insert 500 million documents plus 4 million a day.
>
> Our approach inserting documents is as follows:
> We first try to update a document, if that returns an error we instead create it.
> The resulting documents can contain hundreds and possibly thousands of 'interactions' growing the document size to about 3Mb.
>
> Are there ways of speeding this process up?
>
> With kind regards,
> Harm
---
### Author:  [@Karussell1](https://discuss.elastic.co/u/Karussell1)
#### Post date: [February 14, 2012, 9:56pm UTC](https://discuss.elastic.co/t/poor-performance-updating/6703/3 "2012-02-14T21:56:37Z")
You could also shard or split the index which will improve indexing
speed or tune lucene options for the indexing process only (e.g.
increase merge factor)
> **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)**
>
> Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today.
Did you also thought about another model? E.g. feeding interactions
instead of documents? This way you avoid updating but would require
more search logic
Peter.
On 14 Feb., 11:16, haarts [harmaa...@gmail.com](mailto:harmaa...@gmail.com) wrote:
> Dear list,
>
> We were stoked when we found out about the updating feature in the recent
> 0.19.0rc2 release. We have been eagerly experimenting with it but are
> disappointed by it's performance. Hopefully you can tell us we are doing
> something wrong.
>
> We roughly use this model:[ES Data Model Skylines · GitHub](https://gist.github.com/1751349). Starting from a
> clean index it takes 7 seconds to index 1000 documents (ok-ish). After
> indexing 3 million documents performance degrades to 30 seconds per 1000
> documents (prohibitively slow). We expect to insert 500 million documents
> plus 4 million a day.
>
> Our approach inserting documents is as follows:
> We first try to update a document, if that returns an error we instead
> create it.
> The resulting documents can contain hundreds and possibly thousands of
> 'interactions' growing the document size to about 3Mb.
>
> Are there ways of speeding this process up?
>
> With kind regards,
> Harm
---
### Author:  [@haarts](https://discuss.elastic.co/u/haarts)
#### Post date: [February 15, 2012, 2:11pm UTC](https://discuss.elastic.co/t/poor-performance-updating/6703/4 "2012-02-15T14:11:44Z")
That is what I thought as well.
You pointed out in an other reply that this parent/child functionality
might be what I was looking for. I've looked into it and have one remaining
question;
I want a query searching for 'tree AND house' and returning the parent
which has a child containing 'tree' and a child containing 'house'.
Based on your Gist [https://gist.github.com/758398](https://gist.github.com/758398): my Gist[https://gist.github.com/1835953](https://gist.github.com/1835953)
.
Is such a thing possible?
With kind regards,
---
### Author:  [@haarts](https://discuss.elastic.co/u/haarts)
#### Post date: [February 15, 2012, 2:17pm UTC](https://discuss.elastic.co/t/poor-performance-updating/6703/5 "2012-02-15T14:17:20Z")
Ah. I will dig into these options. Thanks!
I considered an other model as well. Especially a parent/child model as to
prevent reindexing the entire document.
But I haven't been able to get a particular kind of search working with
this. Imagine a particular parent having two children. One child has the
content 'tree' and the other 'house', I require the search 'tree AND house'
to return this parent. A concrete example can be found here[https://gist.github.com/1835953](https://gist.github.com/1835953).
Is that even possible?
With kind regards,
---
### Author:  [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [February 15, 2012, 7:40pm UTC](https://discuss.elastic.co/t/poor-performance-updating/6703/6 "2012-02-15T19:40:36Z")
Yes, it will work, you will use the has\_child filter / query to filter those and get back the parents.
On Wednesday, February 15, 2012 at 4:17 PM, haarts wrote:
> Ah. I will dig into these options. Thanks!
>
> I considered an other model as well. Especially a parent/child model as to prevent reindexing the entire document.
> But I haven't been able to get a particular kind of search working with this. Imagine a particular parent having two children. One child has the content 'tree' and the other 'house', I require the search 'tree AND house' to return this parent. A concrete example can be found here ([gist:1835953 · GitHub](https://gist.github.com/1835953)). Is that even possible?
>
> With kind regards,
---
### Author:  [@haarts](https://discuss.elastic.co/u/haarts)
#### Post date: [February 16, 2012, 9:55am UTC](https://discuss.elastic.co/t/poor-performance-updating/6703/7 "2012-02-16T09:55:23Z")
Excellent! We'll set to work. Thank you very much for your help.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 3:39am UTC](https://discuss.elastic.co/t/poor-performance-updating/6703/8 "2017-07-06T03:39:07Z")