Best practices for extending TransportBroadcastOperationAction

Elastic Stack Elasticsearch
1

Hi!
In general I wanted to ask, if there are any resources about best practices
when writing a plugin that does a TransportBroadcastOperationAction?
My only help so far was reading through other plugins, but I am still
missing documentation about the best practices in doing so...

More specific: I am trying to write a plugin that operates on a specific
shard (ideally on a specific node). In order to achieve that, I overwrite
TransportBroadcastOperationAction#shards(...) to return a
GroupShardsIterator that only contains my target.

Testing it from a java application I figured the following: It seems to
work, when I am using a TransportClient (I have not tested with more than
one active node in the cluster) – however, it does not, when I am using a
native Client via NodeBuilder.nodeBuilder().client(true)...

The failure seems to be caused by the shard-request not being routed to the
correct node, because I am recieving a NPE when calling super.writeTo(out)
in my ShardRequest.

Caused by: java.lang.NullPointerException
at
org.elasticsearch.common.io.stream.HandlesStreamOutput.writeString(HandlesStreamOutput.java:55)
at
org.elasticsearch.action.support.broadcast.BroadcastShardOperationRequest.writeTo(BroadcastShardOperationRequest.java:63)

I assume, I am doing things wrong, so where can I find documentation about
how to write a correct TransportBroadcastOperationAction-Plugin (and how to
tinker with request routing)?

Thanks!

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

Just some first aid: ES is very picky and throws NPEs if you do not
transmit an "index" value. These NPEs are not critical since they are
expected to get fixed by the programmer - if you want to address all
indices or no indices, you can't just use null and expect that ES can
figure out what you want to do.

In case you don't need to address an index, a different class may be a
better choice, for example, TransportSingleCustomOperationAction, which
is the "simplest" one of the transport action family, it does not rely
on indices or shards values.

Take care that you extend the action request/response classes properly
and always use the constructors of the inherited classes so private
variables are filled with non-null values.

You can look at//github.com/jprante/elasticsearch-index-termlist for an
example of a transport broadcast action, it is a very simple action
implementation, I wrote it for self-educational purpose.

Node client and Transport Client work without any difference in respect
to action implementations, they are exchangeable. I would like to give
more advise, but you have to show your code, for better insight of what
you intend to do.

Jörg

Am 06.03.13 17:47, schrieb konrad:

Hi!
In general I wanted to ask, if there are any resources about best
practices when writing a plugin that does a
TransportBroadcastOperationAction?
My only help so far was reading through other plugins, but I am still
missing documentation about the best practices in doing so...

More specific: I am trying to write a plugin that operates on a
specific shard (ideally on a specific node). In order to achieve that,
I overwrite TransportBroadcastOperationAction#shards(...) to return a
GroupShardsIterator that only contains my target.

Testing it from a java application I figured the following: It seems
to work, when I am using a TransportClient (I have not tested with
more than one active node in the cluster) – however, it does not, when
I am using a native Client via NodeBuilder.nodeBuilder().client(true)...

The failure seems to be caused by the shard-request not being routed
to the correct node, because I am recieving a NPE when calling
super.writeTo(out) in my ShardRequest.

Caused by: java.lang.NullPointerException
at
org.elasticsearch.common.io.stream.HandlesStreamOutput.writeString(HandlesStreamOutput.java:55)
at
org.elasticsearch.action.support.broadcast.BroadcastShardOperationRequest.writeTo(BroadcastShardOperationRequest.java:63)

I assume, I am doing things wrong, so where can I find documentation
about how to write a correct TransportBroadcastOperationAction-Plugin
(and how to tinker with request routing)?

Thanks!

You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

Hi Jörg thanks for your answer!

Am Mittwoch, 6. März 2013 20:22:30 UTC+1 schrieb Jörg Prante:

Just some first aid: ES is very picky and throws NPEs if you do not
transmit an "index" value. These NPEs are not critical since they are
expected to get fixed by the programmer - if you want to address all
indices or no indices, you can't just use null and expect that ES can
figure out what you want to do. Take care that you extend the action
request/response classes properly

and always use the constructors of the inherited classes so private

variables are filled with non-null values.

That last bit was the right hint: Although i was always calling super(...)
in my constructors, I missed to propagate parameters everywhere applicable.

E.g.:

public class ShardCustomRequest extends BroadcastShardOperationRequest {

protected ShardCustomRequest(CustomRequest request) {
//super(); <- wrong
super(request.index(), request.shard(), request); //<- correct
//...
}
}

What I found also confusing are the parameterless methods in
TransportBroadCastOperationAction for newRequest(), newShardRequest() and
newShardResponse(), since I don't see, what they are needed for...

You can look at//github.com/jprante/elasticsearch-index-termlist for an
example of a transport broadcast action,

Thanks, that was already one of the plugins I was drawing my insights from.

Node client and Transport Client work without any difference in respect

to action implementations, they are exchangeable.

Well, I figured they seem to slightly differ in the way a request is routed
and (re)constructed, since for me a

client.execute(MyCustomAction.INSTANCE, request).actionGet();

did yield the NPE in native mode, while working with transport.

I would like to give

more advise, but you have to show your code, for better insight of what
you intend to do.

For now this is also only self education and I don't have the guts yet to
show :wink: This might however change, once I cleaned up and tested properly.

The remaining question is, whether there exists documentation apart from
plugin code examples about the do's and don't's?

Regards,
Konrad

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.