Convert unix timestamp (seconds) to java (milliseconds)

Elastic Stack Elasticsearch
1

Greetings,

My primary data store has a date_created field stored as a unix timestamp.
I've verified that if I multiply that value by 1000 and then index
it, Elasticsearch correctly picks the field up as a date. But updating the
data in my primary data store is not very feasible, therefore, I'm looking
for some way to have Elasticsearch do the conversion for me. Lazy, eh?

I've tried adding this to my mapping

    "transform" : {
        "script" : "ctx._source['date_created'] =

ctx._source['date_created'] * 1000;",
"lang": "groovy"
},
"properties" : { ...

I was expecting that to multiply the date_created field by 1000 prior to
parsing the field as a date, but it doesn't seem to be working.

Any ideas what I might be doing wrong, or suggestions for an alternate
approach?

Nick

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5544e84b-06e5-4199-adff-17bd4039ac19%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

Replying to my own question for other searchers:

My problem had to do with types. The mapping that ended up working is:

{
"test" : {
"transform" : {
"script" : "ctx._source['date_created'] =
ctx._source['date_created'].toLong() * 1000",
"lang": "groovy"
},
"properties" : {
"date_created" : {"type" : "date"},
}
}
}

Note the "toLong()".

Also, once a script is modified, it appears as though the type has to be
deleted and the mapping re-applied for the changes to take effect.

Nick

On Wednesday, December 3, 2014 2:52:10 PM UTC-7, Nick Wood wrote:

Greetings,

My primary data store has a date_created field stored as a unix timestamp.
I've verified that if I multiply that value by 1000 and then index
it, Elasticsearch correctly picks the field up as a date. But updating the
data in my primary data store is not very feasible, therefore, I'm looking
for some way to have Elasticsearch do the conversion for me. Lazy, eh?

I've tried adding this to my mapping

    "transform" : {
        "script" : "ctx._source['date_created'] =

ctx._source['date_created'] * 1000;",
"lang": "groovy"
},
"properties" : { ...

I was expecting that to multiply the date_created field by 1000 prior to
parsing the field as a date, but it doesn't seem to be working.

Any ideas what I might be doing wrong, or suggestions for an alternate
approach?

Nick

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/991be6f5-31dc-444c-8748-383b8262d074%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

3

Transform doesn't change the source, just how it is indexed. I made it that
way because I figured I'd you want to change the source you can do it on
the application feeding elasticsearch. Transform is a way to index stuff
but leave it out of the source. Its copy_to on steroids.

Another reason transform doesn't change the source is that it'd make
updates and reindexing (like scan/scroll from one index to another)
complex.

There is an option that will retransform results on the fly but I don't
know it offhand. I believe its on the transform page in the docs.

Nik
On Dec 27, 2014 7:10 PM, "Nick Wood" nwood888@gmail.com wrote:

Replying to my own question for other searchers:

My problem had to do with types. The mapping that ended up working is:

{
"test" : {
"transform" : {
"script" : "ctx._source['date_created'] =
ctx._source['date_created'].toLong() * 1000",
"lang": "groovy"
},
"properties" : {
"date_created" : {"type" : "date"},
}
}
}

Note the "toLong()".

Also, once a script is modified, it appears as though the type has to be
deleted and the mapping re-applied for the changes to take effect.

Nick

On Wednesday, December 3, 2014 2:52:10 PM UTC-7, Nick Wood wrote:

Greetings,

My primary data store has a date_created field stored as a unix
timestamp. I've verified that if I multiply that value by 1000 and then
index it, Elasticsearch correctly picks the field up as a date. But
updating the data in my primary data store is not very feasible, therefore,
I'm looking for some way to have Elasticsearch do the conversion for me.
Lazy, eh?

I've tried adding this to my mapping

    "transform" : {
        "script" : "ctx._source['date_created'] =

ctx._source['date_created'] * 1000;",
"lang": "groovy"
},
"properties" : { ...

I was expecting that to multiply the date_created field by 1000 prior to
parsing the field as a date, but it doesn't seem to be working.

Any ideas what I might be doing wrong, or suggestions for an alternate
approach?

Nick

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/991be6f5-31dc-444c-8748-383b8262d074%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/991be6f5-31dc-444c-8748-383b8262d074%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPmjWd2E8qCT7GRfLYJo_GzB_5x%2BC%3D_FuJ49M%2BNfD%3D5Lhi4zCQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.