Frequent updates to documents

Elastic Stack Elasticsearch
1

Hi,

We are using ES 1.0.3. In our application we do frequent updates to the
documents and this is causes delete count to increase quickly and frequent
merges.

Due to Lucene version issues with our application and ES API, we are not
able to use the API and we have written a module to interact with ES. For
document creation and updates, we use HTTP POST and always send all fields
for updates also.

Would sending just the updated fields, reduce the delete count? OR
Does update to single field also triggers whole document re-indexing?

Thanks in advance for your help!!

Regards,
Jinal Shah

--
The information contained in this e-mail message and any accompanying files
is or may be confidential. If you are not the intended recipient, any use,
dissemination, reliance, forwarding, printing or copying of this e-mail or
any attached files is unauthorised. This e-mail is subject to copyright. No
part of it should be reproduced, adapted or communicated without the
written consent of the copyright owner. If you have received this e-mail in
error please advise the sender immediately by return e-mail or telephone
and delete all copies. Fairfax Media does not guarantee the accuracy or
completeness of any information contained in this e-mail or attached files.
Internet communications are not secure, therefore Fairfax Media does not
accept legal responsibility for the contents of this message or attached
files.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/fcd97638-c934-4d12-8a8e-165b4ae1fdc9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

A small change costs as much as a large one. Your best bet is to batch
multiple updates for the same document together if possible. Also make sure
that your updates actually change something. Sending the exact same
document with the same ID still does an update.
On Dec 12, 2014 12:24 AM, "Jinal Shah" jinal.shah@fairfaxmedia.com.au
wrote:

Hi,

We are using ES 1.0.3. In our application we do frequent updates to the
documents and this is causes delete count to increase quickly and frequent
merges.

Due to Lucene version issues with our application and ES API, we are not
able to use the API and we have written a module to interact with ES. For
document creation and updates, we use HTTP POST and always send all fields
for updates also.

Would sending just the updated fields, reduce the delete count? OR
Does update to single field also triggers whole document re-indexing?

Thanks in advance for your help!!

Regards,
Jinal Shah

The information contained in this e-mail message and any accompanying
files is or may be confidential. If you are not the intended recipient, any
use, dissemination, reliance, forwarding, printing or copying of this
e-mail or any attached files is unauthorised. This e-mail is subject to
copyright. No part of it should be reproduced, adapted or communicated
without the written consent of the copyright owner. If you have received
this e-mail in error please advise the sender immediately by return e-mail
or telephone and delete all copies. Fairfax Media does not guarantee the
accuracy or completeness of any information contained in this e-mail or
attached files. Internet communications are not secure, therefore Fairfax
Media does not accept legal responsibility for the contents of this message
or attached files.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/fcd97638-c934-4d12-8a8e-165b4ae1fdc9%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/fcd97638-c934-4d12-8a8e-165b4ae1fdc9%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPmjWd1EoSpapW9s9R4oW-BVu7f7tmxCh%2B6%3DuL3h2S4WiMb_Wg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

Thanks for the Nikolas.

Users want to search data instantly after the save, so we are unable to use
batch updates. It is good to know that even an update in single field means
whole document reindex.

Thanks,
Jinal

On Friday, 12 December 2014 16:23:55 UTC+11, Jinal Shah wrote:

Hi,

We are using ES 1.0.3. In our application we do frequent updates to the
documents and this is causes delete count to increase quickly and frequent
merges.

Due to Lucene version issues with our application and ES API, we are not
able to use the API and we have written a module to interact with ES. For
document creation and updates, we use HTTP POST and always send all fields
for updates also.

Would sending just the updated fields, reduce the delete count? OR
Does update to single field also triggers whole document re-indexing?

Thanks in advance for your help!!

Regards,
Jinal Shah

The information contained in this e-mail message and any accompanying
files is or may be confidential. If you are not the intended recipient, any
use, dissemination, reliance, forwarding, printing or copying of this
e-mail or any attached files is unauthorised. This e-mail is subject to
copyright. No part of it should be reproduced, adapted or communicated
without the written consent of the copyright owner. If you have received
this e-mail in error please advise the sender immediately by return e-mail
or telephone and delete all copies. Fairfax Media does not guarantee the
accuracy or completeness of any information contained in this e-mail or
attached files. Internet communications are not secure, therefore Fairfax
Media does not accept legal responsibility for the contents of this message
or attached files.

--
The information contained in this e-mail message and any accompanying files
is or may be confidential. If you are not the intended recipient, any use,
dissemination, reliance, forwarding, printing or copying of this e-mail or
any attached files is unauthorised. This e-mail is subject to copyright. No
part of it should be reproduced, adapted or communicated without the
written consent of the copyright owner. If you have received this e-mail in
error please advise the sender immediately by return e-mail or telephone
and delete all copies. Fairfax Media does not guarantee the accuracy or
completeness of any information contained in this e-mail or attached files.
Internet communications are not secure, therefore Fairfax Media does not
accept legal responsibility for the contents of this message or attached
files.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/f20e4365-eb20-4b0a-be58-4bdddba46cdc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

4

Thanks for the reply Nikolas.

Users want to search data instantly after the save, so we are unable to use
batch updates. It is good to know that even an update in single field means
whole document reindex.

Thanks,
Jinal

On Friday, 12 December 2014 16:23:55 UTC+11, Jinal Shah wrote:

Hi,

We are using ES 1.0.3. In our application we do frequent updates to the
documents and this is causes delete count to increase quickly and frequent
merges.

Due to Lucene version issues with our application and ES API, we are not
able to use the API and we have written a module to interact with ES. For
document creation and updates, we use HTTP POST and always send all fields
for updates also.

Would sending just the updated fields, reduce the delete count? OR
Does update to single field also triggers whole document re-indexing?

Thanks in advance for your help!!

Regards,
Jinal Shah

The information contained in this e-mail message and any accompanying
files is or may be confidential. If you are not the intended recipient, any
use, dissemination, reliance, forwarding, printing or copying of this
e-mail or any attached files is unauthorised. This e-mail is subject to
copyright. No part of it should be reproduced, adapted or communicated
without the written consent of the copyright owner. If you have received
this e-mail in error please advise the sender immediately by return e-mail
or telephone and delete all copies. Fairfax Media does not guarantee the
accuracy or completeness of any information contained in this e-mail or
attached files. Internet communications are not secure, therefore Fairfax
Media does not accept legal responsibility for the contents of this message
or attached files.

--
The information contained in this e-mail message and any accompanying files
is or may be confidential. If you are not the intended recipient, any use,
dissemination, reliance, forwarding, printing or copying of this e-mail or
any attached files is unauthorised. This e-mail is subject to copyright. No
part of it should be reproduced, adapted or communicated without the
written consent of the copyright owner. If you have received this e-mail in
error please advise the sender immediately by return e-mail or telephone
and delete all copies. Fairfax Media does not guarantee the accuracy or
completeness of any information contained in this e-mail or attached files.
Internet communications are not secure, therefore Fairfax Media does not
accept legal responsibility for the contents of this message or attached
files.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ffd2084b-0fb2-4741-a2d5-52e20252fea6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.