Java update API marshalling of DateTime

Per · May 2, 2012, 1:01pm

If seems like the UpdateRequestBuilder won't accept DateTime (or Date)
instances as script params (see stacktrace below). DateTime works fine
for the index api. A workaround is to manually (toString) convert the
DateTime instance to a String.

Is this a bug or is it the intended behaivor?

/Per.
...
HashMap<String, Object> map = new HashMap<String, Object>();
map.put("mydate", DateTime.now());
updateRequestBuilder.setScriptParams(map);
..

Caused by: java.io.IOException: Can't write type [class
org.joda.time.DateTime]
at
org.elasticsearch.common.io.stream.StreamOutput.writeGenericValue(StreamOutput.java:
266)
at
org.elasticsearch.common.io.stream.StreamOutput.writeGenericValue(StreamOutput.java:
260)
at
org.elasticsearch.common.io.stream.StreamOutput.writeMap(StreamOutput.java:
205)
at
org.elasticsearch.action.update.UpdateRequest.writeTo(UpdateRequest.java:
358)
at
org.elasticsearch.transport.support.TransportStreams.buildRequest(TransportStreams.java:
117)
at
org.elasticsearch.transport.netty.NettyTransport.sendRequest(NettyTransport.java:
452)
at
org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:
185)
at
org.elasticsearch.action.support.single.instance.TransportInstanceSingleOperationAction
$AsyncSingleAction.start(TransportInstanceSingleOperationAction.java:
191)
at
org.elasticsearch.action.support.single.instance.TransportInstanceSingleOperationAction
$AsyncSingleAction.start(TransportInstanceSingleOperationAction.java:
119)
at
org.elasticsearch.action.support.single.instance.TransportInstanceSingleOperationAction.doExecute(TransportInstanceSingleOperationAction.java:
71)
at
org.elasticsearch.action.support.single.instance.TransportInstanceSingleOperationAction.doExecute(TransportInstanceSingleOperationAction.java:
49)
at
org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:
61)
at org.elasticsearch.client.node.NodeClient.execute(NodeClient.java:
83)
at
org.elasticsearch.client.support.AbstractClient.update(AbstractClient.java:
101)
at
org.elasticsearch.action.update.UpdateRequestBuilder.doExecute(UpdateRequestBuilder.java:
177)

kimchy · May 2, 2012, 1:32pm

Yea, it does not do typical Java serialization, what you should do is
either convert it to the ISO format string, or the milliseconds value.

On Wed, May 2, 2012 at 4:01 PM, Per dev2null@gmail.com wrote:

If seems like the UpdateRequestBuilder won't accept DateTime (or Date)
instances as script params (see stacktrace below). DateTime works fine
for the index api. A workaround is to manually (toString) convert the
DateTime instance to a String.

Is this a bug or is it the intended behaivor?

/Per.
...
HashMap<String, Object> map = new HashMap<String, Object>();
map.put("mydate", DateTime.now());
updateRequestBuilder.setScriptParams(map);
..

Caused by: java.io.IOException: Can't write type [class
org.joda.time.DateTime]
at

org.elasticsearch.common.io.stream.StreamOutput.writeGenericValue(StreamOutput.java:
266)
at

org.elasticsearch.common.io.stream.StreamOutput.writeGenericValue(StreamOutput.java:
260)
at
org.elasticsearch.common.io.stream.StreamOutput.writeMap(StreamOutput.java:
205)
at
org.elasticsearch.action.update.UpdateRequest.writeTo(UpdateRequest.java:
358)
at

org.elasticsearch.transport.support.TransportStreams.buildRequest(TransportStreams.java:
117)
at

org.elasticsearch.transport.netty.NettyTransport.sendRequest(NettyTransport.java:
452)
at

org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:
185)
at

org.elasticsearch.action.support.single.instance.TransportInstanceSingleOperationAction
$AsyncSingleAction.start(TransportInstanceSingleOperationAction.java:
191)
at

org.elasticsearch.action.support.single.instance.TransportInstanceSingleOperationAction
$AsyncSingleAction.start(TransportInstanceSingleOperationAction.java:
119)
at

org.elasticsearch.action.support.single.instance.TransportInstanceSingleOperationAction.doExecute(TransportInstanceSingleOperationAction.java:
71)
at

org.elasticsearch.action.support.single.instance.TransportInstanceSingleOperationAction.doExecute(TransportInstanceSingleOperationAction.java:
49)
at

org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:
61)
at org.elasticsearch.client.node.NodeClient.execute(NodeClient.java:
83)
at
org.elasticsearch.client.support.AbstractClient.update(AbstractClient.java:
101)
at

org.elasticsearch.action.update.UpdateRequestBuilder.doExecute(UpdateRequestBuilder.java:
177)

Per · May 2, 2012, 7:14pm

I understand that it's not "typical" Java serialization.

The thing that surprised me was that the IndexRequestBuilder and the
UpdateRequestBuilder behaved differently when feeding them with a map
with a Date instance and for me the reason is not obvious.

The IndexRequestBuilder#setSource(Map<String, Object> source) method
ends up using the XContentMapConverter which knows how to serialize
java.util.Date and it uses Object#toString() as a fallback for unknown
types, e.g. joda times DateTime class.

UpdateRequestBuilder#setScriptParams(Map<String, Object> scriptParams)
uses StreamOutput which doesn't know java.util.Date and defaults to
throwing an IOException for unkown types.

I guess that the best approach is to always manually serialize date
vales to a ISO format strings before talking to the ES java api.

On May 2, 3:32 pm, Shay Banon kim...@gmail.com wrote:

Yea, it does not do typical Java serialization, what you should do is
either convert it to the ISO format string, or the milliseconds value.

On Wed, May 2, 2012 at 4:01 PM, Per dev2n...@gmail.com wrote:

If seems like the UpdateRequestBuilder won't accept DateTime (or Date)
instances as script params (see stacktrace below). DateTime works fine
for the index api. A workaround is to manually (toString) convert the
DateTime instance to a String.

Is this a bug or is it the intended behaivor?

/Per.
...
HashMap<String, Object> map = new HashMap<String, Object>();
map.put("mydate", DateTime.now());
updateRequestBuilder.setScriptParams(map);
..

Caused by: java.io.IOException: Can't write type [class
org.joda.time.DateTime]
at

org.elasticsearch.common.io.stream.StreamOutput.writeGenericValue(StreamOutput.java:
266)
at

org.elasticsearch.common.io.stream.StreamOutput.writeGenericValue(StreamOutput.java:
260)
at
org.elasticsearch.common.io.stream.StreamOutput.writeMap(StreamOutput.java:
205)
at
org.elasticsearch.action.update.UpdateRequest.writeTo(UpdateRequest.java:
358)
at

org.elasticsearch.transport.support.TransportStreams.buildRequest(TransportStreams.java:
117)
at

org.elasticsearch.transport.netty.NettyTransport.sendRequest(NettyTransport.java:
452)
at

org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:
185)
at

org.elasticsearch.action.support.single.instance.TransportInstanceSingleOperationAction
$AsyncSingleAction.start(TransportInstanceSingleOperationAction.java:
191)
at

org.elasticsearch.action.support.single.instance.TransportInstanceSingleOperationAction
$AsyncSingleAction.start(TransportInstanceSingleOperationAction.java:
119)
at

org.elasticsearch.action.support.single.instance.TransportInstanceSingleOperationAction.doExecute(TransportInstanceSingleOperationAction.java:
71)
at

org.elasticsearch.action.support.single.instance.TransportInstanceSingleOperationAction.doExecute(TransportInstanceSingleOperationAction.java:
49)
at

org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:
61)
at org.elasticsearch.client.node.NodeClient.execute(NodeClient.java:
83)
at
org.elasticsearch.client.support.AbstractClient.update(AbstractClient.java:
101)
at

org.elasticsearch.action.update.UpdateRequestBuilder.doExecute(UpdateRequestBuilder.java:
177)

kimchy · May 2, 2012, 8:50pm

We can easily support Date, here is the issue:
Support Java Date when serializing update parameters · Issue #1902 · elastic/elasticsearch · GitHub.

On Wed, May 2, 2012 at 10:14 PM, Per dev2null@gmail.com wrote:

I understand that it's not "typical" Java serialization.

The thing that surprised me was that the IndexRequestBuilder and the
UpdateRequestBuilder behaved differently when feeding them with a map
with a Date instance and for me the reason is not obvious.

The IndexRequestBuilder#setSource(Map<String, Object> source) method
ends up using the XContentMapConverter which knows how to serialize
java.util.Date and it uses Object#toString() as a fallback for unknown
types, e.g. joda times DateTime class.

UpdateRequestBuilder#setScriptParams(Map<String, Object> scriptParams)
uses StreamOutput which doesn't know java.util.Date and defaults to
throwing an IOException for unkown types.

I guess that the best approach is to always manually serialize date
vales to a ISO format strings before talking to the ES java api.

On May 2, 3:32 pm, Shay Banon kim...@gmail.com wrote:

Yea, it does not do typical Java serialization, what you should do is
either convert it to the ISO format string, or the milliseconds value.

On Wed, May 2, 2012 at 4:01 PM, Per dev2n...@gmail.com wrote:

If seems like the UpdateRequestBuilder won't accept DateTime (or Date)
instances as script params (see stacktrace below). DateTime works fine
for the index api. A workaround is to manually (toString) convert the
DateTime instance to a String.

Is this a bug or is it the intended behaivor?

/Per.
...
HashMap<String, Object> map = new HashMap<String, Object>();
map.put("mydate", DateTime.now());
updateRequestBuilder.setScriptParams(map);
..

Caused by: java.io.IOException: Can't write type [class
org.joda.time.DateTime]
at

org.elasticsearch.common.io.stream.StreamOutput.writeGenericValue(StreamOutput.java:

at

org.elasticsearch.common.io.stream.StreamOutput.writeGenericValue(StreamOutput.java:

at

org.elasticsearch.common.io.stream.StreamOutput.writeMap(StreamOutput.java:

at

org.elasticsearch.action.update.UpdateRequest.writeTo(UpdateRequest.java:

at

org.elasticsearch.transport.support.TransportStreams.buildRequest(TransportStreams.java:

at

org.elasticsearch.transport.netty.NettyTransport.sendRequest(NettyTransport.java:

at

org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:

at

org.elasticsearch.action.support.single.instance.TransportInstanceSingleOperationAction

$AsyncSingleAction.start(TransportInstanceSingleOperationAction.java:
191)
at

org.elasticsearch.action.support.single.instance.TransportInstanceSingleOperationAction

$AsyncSingleAction.start(TransportInstanceSingleOperationAction.java:
119)
at

org.elasticsearch.action.support.single.instance.TransportInstanceSingleOperationAction.doExecute(TransportInstanceSingleOperationAction.java:

at

org.elasticsearch.action.support.single.instance.TransportInstanceSingleOperationAction.doExecute(TransportInstanceSingleOperationAction.java:

at

org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:

at
org.elasticsearch.client.node.NodeClient.execute(NodeClient.java:

at

org.elasticsearch.client.support.AbstractClient.update(AbstractClient.java:

at

org.elasticsearch.action.update.UpdateRequestBuilder.doExecute(UpdateRequestBuilder.java: