I have a situation where I need to perform consolidation, analysis and
reporting tasks on data indexed in ES.
I would like to leverage ES to load balance or distribute these
processing tasks across a group of servers.
I am using the java API.
I have looked at the possibility of using native scripts and my
understanding is that native scripts are executed within the context
of queries whereby query requests are sent to all shards in an index
and each shard executes the native script locally for each search hit
and adds the output of the native script to the search hit. Not quite
what I am looking for.
The behaviour I am looking for is more like index and get requests
which are routed to a shard based on the index routing rules and
document key values (index/type/id).
Is there a mechanism for executing native scripts for index/put/get
requests whereby I could sending a request to ES which gets routed to
a cluster node based on a document key and then have that node execute
a native script and return a result document generated by that script?
The native script may also create new documents which are added to the
index for later retrieval. This key based routing would also allow
each cluster node to maintain working data in memory or local disc for
the tasks it performs.
If this is not already supported doe the ES architecture make it
possible for me to write a plug in that adds custom actions and action
handlers that are compatible with the ES transports and index/get
request routing? If this is possible can you direct me to java source
code examples of this and/or give me an overview of the ES classes I
would have to subclass and how to register these via a plugin.