What do you think about partial update plugin?

Elastic Stack Elasticsearch
1

Hi use ES as a caching system for my objects.

Some of these objects use a lot of lazy loading so only part of their
properties may be loaded. When they need to be saved, if I index them, some
of the properties will be juste removed from the index if they have not been
loaded and loaded them again is a pain in the ... for performances.

I've read some posts
about https://github.com/medcl/ElasticSearch.PartialUpdate.

What do you think ?

2

the partial update plugin still reindexes the whole document, there is no
way around it.

On Thu, Oct 13, 2011 at 8:33 PM, Alexandre Heimburger <
ahb@bluekiwi-software.com> wrote:

Hi use ES as a caching system for my objects.

Some of these objects use a lot of lazy loading so only part of their
properties may be loaded. When they need to be saved, if I index them, some
of the properties will be juste removed from the index if they have not been
loaded and loaded them again is a pain in the ... for performances.

I've read some posts about
https://github.com/medcl/ElasticSearch.PartialUpdate.

What do you think ?

3

Hi Shay,

I've been using this plugin for a while. It seems at least to reduce
network traffic which is a good thing.

Would it be possible to extend the bulk API to support this plugin (I
think it introduces the _update call)?

Thanks,

On Oct 14, 1:23 pm, Shay Banon kim...@gmail.com wrote:

the partial update plugin still reindexes the whole document, there is no
way around it.

On Thu, Oct 13, 2011 at 8:33 PM, Alexandre Heimburger <

a...@bluekiwi-software.com> wrote:

Hi use ES as a caching system for my objects.

Some of these objects use a lot of lazy loading so only part of their
properties may be loaded. When they need to be saved, if I index them, some
of the properties will be juste removed from the index if they have not been
loaded and loaded them again is a pain in the ... for performances.

I've read some posts about
https://github.com/medcl/ElasticSearch.PartialUpdate.

What do you think ?

4

I've pushed preliminary ground work to support update endpoint (that will
still mean reindexing the docs). And, one of elasticsearch users is helping
with actually implementing an Update endpoint (it will start with "update by
script"). The initial solution will be update a single doc, not bulk update.

On Mon, Oct 17, 2011 at 9:08 PM, mordant asmint3@hotmail.co.uk wrote:

Hi Shay,

I've been using this plugin for a while. It seems at least to reduce
network traffic which is a good thing.

Would it be possible to extend the bulk API to support this plugin (I
think it introduces the _update call)?

Thanks,

On Oct 14, 1:23 pm, Shay Banon kim...@gmail.com wrote:

the partial update plugin still reindexes the whole document, there is no
way around it.

On Thu, Oct 13, 2011 at 8:33 PM, Alexandre Heimburger <

a...@bluekiwi-software.com> wrote:

Hi use ES as a caching system for my objects.

Some of these objects use a lot of lazy loading so only part of their
properties may be loaded. When they need to be saved, if I index them,
some
of the properties will be juste removed from the index if they have not
been
loaded and loaded them again is a pain in the ... for performances.

I've read some posts about
https://github.com/medcl/ElasticSearch.PartialUpdate.

What do you think ?

5

I played around with this idea a few days ago.

This is a derivation of the partial update plugin. It was more or less a
thought experiment on building a _reindex endpoint that could (internally)
migrate indices. I'd like to see webhooks introduced into the bulk APIs so
the caller could just register callback methods (HTTP POST) similar to
jQuery (onSuccess, onFailure, onComplete).

I tested this lightly with 100K small-ish records and reindexing was, of
course, faster than the original index times since everything is handled on
the server side. Ideally, this plugin would accept a query to run. At the
moment it just uses a match match_all query.