Is there a way to convert @timestamp of ES to Unix epoch time in milliseconds?

Hi,

As I mentioned at the title of this question, I wonder is there a way
to convert @timestamp of ES to Unix epoch time in milliseconds by
using ES query?
For instance, "2014-11-10T15:00:00.000Z" to "1415599200000".
Any help will be great.

Thanks.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CADQPeWwW_bitzy5GA_R8ukpXDCEj%3DXCM66N994iY35KpYPtdRw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

This is exactly what elasticsearch does behind the scene.
It index ms since epoch.

David

Le 5 déc. 2014 à 05:29, Mungeol Heo mungeol.heo@gmail.com a écrit :

Hi,

As I mentioned at the title of this question, I wonder is there a way
to convert @timestamp of ES to Unix epoch time in milliseconds by
using ES query?
For instance, "2014-11-10T15:00:00.000Z" to "1415599200000".
Any help will be great.

Thanks.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CADQPeWwW_bitzy5GA_R8ukpXDCEj%3DXCM66N994iY35KpYPtdRw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/349D0B40-1CB3-4567-A35B-9535783AF8B5%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.

Hi, David

I still want to know how to get epoch time from @timestamp by using query.
I mean I want a query result like below.
Suppose the "unixtime'' is not a indexed field and converted from @timestamp.

{
"_index": "test",
"_type": "test",
"_id": "JGaR1BCFSW2lArRtbZMcGg",
"_score": 1,
"fields": {
@timestamp": [
"2014-11-10T15:03:34.000Z"
],
"unixtime": [
"1415599200000"
]
}
}

Thanks.

On Fri, Dec 5, 2014 at 3:05 PM, David Pilato david@pilato.fr wrote:

This is exactly what elasticsearch does behind the scene.
It index ms since epoch.

David

Le 5 déc. 2014 à 05:29, Mungeol Heo mungeol.heo@gmail.com a écrit :

Hi,

As I mentioned at the title of this question, I wonder is there a way
to convert @timestamp of ES to Unix epoch time in milliseconds by
using ES query?
For instance, "2014-11-10T15:00:00.000Z" to "1415599200000".
Any help will be great.

Thanks.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CADQPeWwW_bitzy5GA_R8ukpXDCEj%3DXCM66N994iY35KpYPtdRw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/349D0B40-1CB3-4567-A35B-9535783AF8B5%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CADQPeWwX2%2B%3Dxj%2BykTvRqAuH8-TsjRv_%2B1vhO%3DF%2BHtyBN0Df2gA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

You should try to add store: true in your mapping for this field and ask for it as a field in the search query. I think this should work.

Still I don't understand what you are trying to do? Do you want to compute anything based on that?

David

Le 5 déc. 2014 à 08:29, Mungeol Heo mungeol.heo@gmail.com a écrit :

Hi, David

I still want to know how to get epoch time from @timestamp by using query.
I mean I want a query result like below.
Suppose the "unixtime'' is not a indexed field and converted from @timestamp.

{
"_index": "test",
"_type": "test",
"_id": "JGaR1BCFSW2lArRtbZMcGg",
"_score": 1,
"fields": {
@timestamp": [
"2014-11-10T15:03:34.000Z"
],
"unixtime": [
"1415599200000"
]
}
}

Thanks.

On Fri, Dec 5, 2014 at 3:05 PM, David Pilato david@pilato.fr wrote:
This is exactly what elasticsearch does behind the scene.
It index ms since epoch.

David

Le 5 déc. 2014 à 05:29, Mungeol Heo mungeol.heo@gmail.com a écrit :

Hi,

As I mentioned at the title of this question, I wonder is there a way
to convert @timestamp of ES to Unix epoch time in milliseconds by
using ES query?
For instance, "2014-11-10T15:00:00.000Z" to "1415599200000".
Any help will be great.

Thanks.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CADQPeWwW_bitzy5GA_R8ukpXDCEj%3DXCM66N994iY35KpYPtdRw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/349D0B40-1CB3-4567-A35B-9535783AF8B5%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CADQPeWwX2%2B%3Dxj%2BykTvRqAuH8-TsjRv_%2B1vhO%3DF%2BHtyBN0Df2gA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/055D8277-8B56-4461-A024-3E1B31989CAC%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.

First of all, thanks your advice.
And what I am trying to do is use the @timestamp of ES, includes
milliseconds, in hive while using elasticsearch-hadoop.
The problem is the timestamp type of hive, which is in seconds, can not help me.
So, I just want to figure out how many options do I have, and choose
one of them to solve my problem.
Thank you again.

mungeol

On Fri, Dec 5, 2014 at 6:14 PM, David Pilato david@pilato.fr wrote:

You should try to add store: true in your mapping for this field and ask for it as a field in the search query. I think this should work.

Still I don't understand what you are trying to do? Do you want to compute anything based on that?

David

Le 5 déc. 2014 à 08:29, Mungeol Heo mungeol.heo@gmail.com a écrit :

Hi, David

I still want to know how to get epoch time from @timestamp by using query.
I mean I want a query result like below.
Suppose the "unixtime'' is not a indexed field and converted from @timestamp.

{
"_index": "test",
"_type": "test",
"_id": "JGaR1BCFSW2lArRtbZMcGg",
"_score": 1,
"fields": {
@timestamp": [
"2014-11-10T15:03:34.000Z"
],
"unixtime": [
"1415599200000"
]
}
}

Thanks.

On Fri, Dec 5, 2014 at 3:05 PM, David Pilato david@pilato.fr wrote:
This is exactly what elasticsearch does behind the scene.
It index ms since epoch.

David

Le 5 déc. 2014 à 05:29, Mungeol Heo mungeol.heo@gmail.com a écrit :

Hi,

As I mentioned at the title of this question, I wonder is there a way
to convert @timestamp of ES to Unix epoch time in milliseconds by
using ES query?
For instance, "2014-11-10T15:00:00.000Z" to "1415599200000".
Any help will be great.

Thanks.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CADQPeWwW_bitzy5GA_R8ukpXDCEj%3DXCM66N994iY35KpYPtdRw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/349D0B40-1CB3-4567-A35B-9535783AF8B5%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CADQPeWwX2%2B%3Dxj%2BykTvRqAuH8-TsjRv_%2B1vhO%3DF%2BHtyBN0Df2gA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/055D8277-8B56-4461-A024-3E1B31989CAC%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CADQPeWw%3DQitRQaz3mjX%3D4yHuWt-ZXUg7SVoX0Tu0vV2D4T_puw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

IMHO you'd better ask "how I can transform an elasticsearch date field into typestamp field in Hive using elasticsearch-hadoop?"

I'm pretty sure there are some es-hadoop experts who can give you some advices.

David

Le 8 déc. 2014 à 01:02, Mungeol Heo mungeol.heo@gmail.com a écrit :

First of all, thanks your advice.
And what I am trying to do is use the @timestamp of ES, includes
milliseconds, in hive while using elasticsearch-hadoop.
The problem is the timestamp type of hive, which is in seconds, can not help me.
So, I just want to figure out how many options do I have, and choose
one of them to solve my problem.
Thank you again.

mungeol

On Fri, Dec 5, 2014 at 6:14 PM, David Pilato david@pilato.fr wrote:
You should try to add store: true in your mapping for this field and ask for it as a field in the search query. I think this should work.

Still I don't understand what you are trying to do? Do you want to compute anything based on that?

David

Le 5 déc. 2014 à 08:29, Mungeol Heo mungeol.heo@gmail.com a écrit :

Hi, David

I still want to know how to get epoch time from @timestamp by using query.
I mean I want a query result like below.
Suppose the "unixtime'' is not a indexed field and converted from @timestamp.

{
"_index": "test",
"_type": "test",
"_id": "JGaR1BCFSW2lArRtbZMcGg",
"_score": 1,
"fields": {
@timestamp": [
"2014-11-10T15:03:34.000Z"
],
"unixtime": [
"1415599200000"
]
}
}

Thanks.

On Fri, Dec 5, 2014 at 3:05 PM, David Pilato david@pilato.fr wrote:
This is exactly what elasticsearch does behind the scene.
It index ms since epoch.

David

Le 5 déc. 2014 à 05:29, Mungeol Heo mungeol.heo@gmail.com a écrit :

Hi,

As I mentioned at the title of this question, I wonder is there a way
to convert @timestamp of ES to Unix epoch time in milliseconds by
using ES query?
For instance, "2014-11-10T15:00:00.000Z" to "1415599200000".
Any help will be great.

Thanks.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CADQPeWwW_bitzy5GA_R8ukpXDCEj%3DXCM66N994iY35KpYPtdRw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/349D0B40-1CB3-4567-A35B-9535783AF8B5%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CADQPeWwX2%2B%3Dxj%2BykTvRqAuH8-TsjRv_%2B1vhO%3DF%2BHtyBN0Df2gA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/055D8277-8B56-4461-A024-3E1B31989CAC%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CADQPeWw%3DQitRQaz3mjX%3D4yHuWt-ZXUg7SVoX0Tu0vV2D4T_puw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CCCC2662-2B3B-4608-9C2F-958766F32845%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.

To be honest, I already have several ways to solve my problem.
For now, one of the best solutions is using a customized hive UDF
which converts time string with given pattern to unix time stamp in
milliseconds.
And, I am seeking another way at ES side which makes me have one more
option to choose, so I can finally decide the best way to use.
As far as I know, there is no much help I can seek from
elasticsearch-hadoop itself.
Again, Thank you so much.

Mungeol

On Mon, Dec 8, 2014 at 12:20 PM, David Pilato david@pilato.fr wrote:

IMHO you'd better ask "how I can transform an elasticsearch date field into typestamp field in Hive using elasticsearch-hadoop?"

I'm pretty sure there are some es-hadoop experts who can give you some advices.

David

Le 8 déc. 2014 à 01:02, Mungeol Heo mungeol.heo@gmail.com a écrit :

First of all, thanks your advice.
And what I am trying to do is use the @timestamp of ES, includes
milliseconds, in hive while using elasticsearch-hadoop.
The problem is the timestamp type of hive, which is in seconds, can not help me.
So, I just want to figure out how many options do I have, and choose
one of them to solve my problem.
Thank you again.

mungeol

On Fri, Dec 5, 2014 at 6:14 PM, David Pilato david@pilato.fr wrote:
You should try to add store: true in your mapping for this field and ask for it as a field in the search query. I think this should work.

Still I don't understand what you are trying to do? Do you want to compute anything based on that?

David

Le 5 déc. 2014 à 08:29, Mungeol Heo mungeol.heo@gmail.com a écrit :

Hi, David

I still want to know how to get epoch time from @timestamp by using query.
I mean I want a query result like below.
Suppose the "unixtime'' is not a indexed field and converted from @timestamp.

{
"_index": "test",
"_type": "test",
"_id": "JGaR1BCFSW2lArRtbZMcGg",
"_score": 1,
"fields": {
@timestamp": [
"2014-11-10T15:03:34.000Z"
],
"unixtime": [
"1415599200000"
]
}
}

Thanks.

On Fri, Dec 5, 2014 at 3:05 PM, David Pilato david@pilato.fr wrote:
This is exactly what elasticsearch does behind the scene.
It index ms since epoch.

David

Le 5 déc. 2014 à 05:29, Mungeol Heo mungeol.heo@gmail.com a écrit :

Hi,

As I mentioned at the title of this question, I wonder is there a way
to convert @timestamp of ES to Unix epoch time in milliseconds by
using ES query?
For instance, "2014-11-10T15:00:00.000Z" to "1415599200000".
Any help will be great.

Thanks.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CADQPeWwW_bitzy5GA_R8ukpXDCEj%3DXCM66N994iY35KpYPtdRw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/349D0B40-1CB3-4567-A35B-9535783AF8B5%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CADQPeWwX2%2B%3Dxj%2BykTvRqAuH8-TsjRv_%2B1vhO%3DF%2BHtyBN0Df2gA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/055D8277-8B56-4461-A024-3E1B31989CAC%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CADQPeWw%3DQitRQaz3mjX%3D4yHuWt-ZXUg7SVoX0Tu0vV2D4T_puw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CCCC2662-2B3B-4608-9C2F-958766F32845%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CADQPeWyuG%3Dte_dYdCmRK0GajY%2BN0THYWhC_apTgtw18Jy%2B5JOQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.