Elasticsearch inserting date type documents as UTC timezone datetime while indexing

Elastic Stack Elasticsearch
1

Hi,

I'm using below code to insert some documents in elasticsearch index. But
when inserting in the es the time is coming as UTC format rather than
original GMT+5:30Z format. Where as the Sysout before indexing is giving me
correct format as 2014-08-11T18:23:13.447+05:30 . Please let me know how
to keep the orizinal time format.

public static IndexResponse addDocumentsQuota(Client client, String index,
String type, String categoryName) {

IndexResponse indexResponse = null;
DateTime date = new DateTime();
System.out.println(date);
try {
indexResponse = client.prepareIndex(index, type)
.setSource(jsonBuilder()
.startObject()
.field("@timestamp", date)
.field("category_name", categoryName)
.field("alert_message", "alert message")
.field("creation_time", date)
.endObject())
.execute()
.actionGet();
} catch (ElasticsearchException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return indexResponse;
}

After indexing:

"_source": {
"@timestamp": "2014-08-11T11:57:59.839Z",
"category_name": "newCat1",
"alert_message": "alert message",
"creation_time": "2014-08-11T11:57:59.852Z"
}

Thanks,
Subhadip

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5c82c9ab-a627-46a9-ad18-dda334524ab1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

Hi,

Any ideas how to prevent the time changing while indexing in es, or to
convert in correct format while query ?

Thanks,
Subhadip

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/33363b16-b2df-4bef-aded-db198c13d5ce%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

3

Hi,

Can someone please give me a hint, I'm having trouble getting a solution
for this.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d42cb94b-681d-4bd3-bf21-d955cd0af729%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

4

Hi,

Please help with suggestion.

On Monday, August 11, 2014 7:51:00 PM UTC+5:30, Subhadip Bagui wrote:

Hi,

I'm using below code to insert some documents in elasticsearch index. But
when inserting in the es the time is coming as UTC format rather than
original GMT+5:30Z format. Where as the Sysout before indexing is giving me
correct format as 2014-08-11T18:23:13.447+05:30 . Please let me know
how to keep the orizinal time format.

public static IndexResponse addDocumentsQuota(Client client, String index,
String type, String categoryName) {

IndexResponse indexResponse = null;
DateTime date = new DateTime();
System.out.println(date);
try {
indexResponse = client.prepareIndex(index, type)
.setSource(jsonBuilder()
.startObject()
.field("@timestamp", date)
.field("category_name", categoryName)
.field("alert_message", "alert message")
.field("creation_time", date)
.endObject())
.execute()
.actionGet();
} catch (ElasticsearchException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return indexResponse;
}

After indexing:

"_source": {
"@timestamp": "2014-08-11T11:57:59.839Z",
"category_name": "newCat1",
"alert_message": "alert message",
"creation_time": "2014-08-11T11:57:59.852Z"
}

Thanks,
Subhadip

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a0a623d6-524c-4928-a35d-d042855c1834%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

5

Hi,

Please help with suggestions.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/02d96b0a-99a7-43f3-b245-13418dece674%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

6

Try to insert using Milliseconds.

Thanks,
Sumit

7

HI,

XContentBuilder (I assume jsonBuilder() returns it) serialises date using UTC timezone by default.
If you’d like to use different format, you’d need to build your own DateTimeFormatter and pass it when you add date type field.

For example, to use ISO date time format with specified timezone:

DateTimeFormatter formatter = ISODateTimeFormat.dateTime().withZone(DateTimeZone.forTimeZone(TimeZone.getTimeZone(ZONE_NAME)));
:
.field("@timestamp", date, formatter)
:
.field("creation_time", date, formatter)
:

Masaru

On January 5, 2015 at 13:45:19, Subhadip Bagui (i.bagui@gmail.com) wrote:

Hi,

Please help with suggestions.

--
You received this message because you are subscribed to the Google Groups "elasticsearch"
group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/02d96b0a-99a7-43f3-b245-13418dece674%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.54ab8619.6b8b4567.13b%40citra.local.
For more options, visit https://groups.google.com/d/optout.

8

Hi,

I tried with what you suggested using below code. But getting exception.

DateTime dateTime = new org.joda.time.DateTime();
System.out.println(dateTime);
DateTimeFormatter dateTimeFormatter =
ISODateTimeFormat.dateTime().withZone(DateTimeZone.forTimeZone(TimeZone.getTimeZone("Asia/Calcutta")));

try {
response = client
.prepareIndex("node-test", "node-entry")
.setSource(XContentFactory.jsonBuilder()
.startObject()
.field("NodeName", nodeName)
.field("NodeStatus", nodeStatus)
.field("@timestamp", dateTime, dateTimeFormatter)
.endObject())
.execute().actionGet();
} catch (ElasticsearchException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}

org.elasticsearch.transport.TransportSerializationException: Failed to
deserialize exception response from stream
at
org.elasticsearch.transport.netty.MessageChannelHandler.handlerResponseError(MessageChannelHandler.java:169)
at
org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:123)
at
org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at
org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:296)

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/eab0ae36-cd4e-42cc-bf8b-8e59ed24e678%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

9

Hi,

Not sure what’s failing because of deserialize issue. It usually happens when you use different version of java.
You can see what the problem is by checking ES log instead too.

To check you are sending document in correct format, you may want to print JSON before sending index request:

XContentBuilder builder = XContentFactory.jsonBuilder().startObject()
:
.field("@timestamp", dateTime, dateTimeFormatter)
.endObject();

System.out.println(builder.string());

try{
client.prepareIndex(“node-test”, “node-entry”).setSource(builder).execute().actionGet();
} catch (Exception e) {
e.printStackTrace();
}

Masaru

On January 7, 2015 at 20:12:59, Subhadip Bagui (i.bagui@gmail.com) wrote:

Hi,

I tried with what you suggested using below code. But getting exception.

DateTime dateTime = new org.joda.time.DateTime();
System.out.println(dateTime);
DateTimeFormatter dateTimeFormatter =
ISODateTimeFormat.dateTime().withZone(DateTimeZone.forTimeZone(TimeZone.getTimeZone("Asia/Calcutta")));

try {
response = client
.prepareIndex("node-test", "node-entry")
.setSource(XContentFactory.jsonBuilder()
.startObject()
.field("NodeName", nodeName)
.field("NodeStatus", nodeStatus)
.field("@timestamp", dateTime, dateTimeFormatter)
.endObject())
.execute().actionGet();
} catch (ElasticsearchException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}

org.elasticsearch.transport.TransportSerializationException: Failed to
deserialize exception response from stream
at
org.elasticsearch.transport.netty.MessageChannelHandler.handlerResponseError(MessageChannelHandler.java:169)
at
org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:123)
at
org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at
org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:296)

--
You received this message because you are subscribed to the Google Groups "elasticsearch"
group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/eab0ae36-cd4e-42cc-bf8b-8e59ed24e678%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.54ae2239.74b0dc51.677%40citra.local.
For more options, visit https://groups.google.com/d/optout.