newResponse returning failed shard responses

[m6800]

I am experimenting with custom Action plugins in Elasticsearch 2.4.4 using Java. I'm creating an index with 10000 documents spread out over 5 shards. I am trying to generate a list of documents at the shard level, and then aggregate the result from each shard response. About 90% of the time, I receive at least one failed shard response. 10% of the time all 5 shards return a List to the newResponse method, where I can then aggregate to a single List as expected. I am doing my testing using JUnits integration testing for elasticsearch.

I'm following examples from the Elasticsearch Cookbook to guide me in this task. Any ideas or pointers in debugging my code?

[venkata_sreekanth_bh (Venkata Sreekanth Bhagavatula)]

@m6800

lowing examples from the Elasticsearch Cookbook to guide me in this task. Any ideas or pointers in debugging my code?

can you post the message you receive when you have failed shard response

[m6800]

Yes, I'm receiving:
[ellipse][[ellipse][2]] BroadcastShardOperationFailedException[]; nested: RemoteTransportException[[node_s0][local[1]][consolidateAction[s]]]; nested: NotSerializableExceptionWrapper[i_o_exception: Can't write type [class com.project.elasticsearch.test.CustomGeoPoint]];
at org.elasticsearch.action.support.broadcast.TransportBroadcastAction$AsyncBroadcastAction.setFailure(TransportBroadcastAction.java:256)
at org.elasticsearch.action.support.broadcast.TransportBroadcastAction$AsyncBroadcastAction.onOperation(TransportBroadcastAction.java:216)
at org.elasticsearch.action.support.broadcast.TransportBroadcastAction$AsyncBroadcastAction$1.handleException(TransportBroadcastAction.java:193)
at org.elasticsearch.transport.local.LocalTransport.handleException(LocalTransport.java:379)
at org.elasticsearch.transport.local.LocalTransport.handleResponseError(LocalTransport.java:370)
at org.elasticsearch.transport.local.LocalTransport.messageReceived(LocalTransport.java:252)
at org.elasticsearch.transport.local.LocalTransportChannel$1.run(LocalTransportChannel.java:102)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: RemoteTransportException[[node_s0][local[1]][consolidateAction[s]]]; nested: NotSerializableExceptionWrapper[i_o_exception: Can't write type [class com.project.elasticsearch.test.CustomGeoPoint]];
Caused by: NotSerializableExceptionWrapper[i_o_exception: Can't write type [class com.project.elasticsearch.test.CustomGeoPoint]]
at org.elasticsearch.common.io.stream.StreamOutput.writeGenericValue(StreamOutput.java:436)
at org.elasticsearch.common.io.stream.StreamOutput.writeGenericValue(StreamOutput.java:376)
at com.project.elasticsearch.test.consolidate.ShardConsolidateResponse.writeTo(ShardConsolidateResponse.java:75)
at org.elasticsearch.transport.local.LocalTransportChannel.sendResponse(LocalTransportChannel.java:81)
at org.elasticsearch.transport.local.LocalTransportChannel.sendResponse(LocalTransportChannel.java:70)
at org.elasticsearch.transport.DelegatingTransportChannel.sendResponse(DelegatingTransportChannel.java:58)
at org.elasticsearch.transport.RequestHandlerRegistry$TransportChannelWrapper.sendResponse(RequestHandlerRegistry.java:140)
at org.elasticsearch.action.support.broadcast.TransportBroadcastAction$ShardTransportHandler.messageReceived(TransportBroadcastAction.java:282)
at org.elasticsearch.action.support.broadcast.TransportBroadcastAction$ShardTransportHandler.messageReceived(TransportBroadcastAction.java:278)
at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:77)
at org.elasticsearch.transport.local.LocalTransport$3.doRun(LocalTransport.java:304)
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

I have a List of objects of type CustomGeoPoint which is a class I've made. Thanks!

[m6800]

It looks like StreamOutput.writeGenericValues does not support custom objects.

Java Api for UpdateByQuery requests

OK, that was a bit of magic sauce I wasn't aware of. It got me a little further. Now getting write exception when attempting to write my custom objects. How do I pass the structures I want to update down to the script? Even if it's serializable, the script won't have access to the class itself. In the Sense playground, I can just create the JSON and magic happens. I tried turning it into JSON first, but then groovy script complains about strings not having the fields I'm looking for. What's the …