Unix timestamp (epoch) in date type

Elastic Stack Elasticsearch
1

From:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-core-types.html#date

"The date type will also accept a long number representing UTC
milliseconds since the epoch, regardless of the format it can handle."

Does that mean that I can save the epoch time + "000" and that
will be a valid date field?

Example:
Epoch: 1381041397
ES-Epoch: 1381041397000

Will that work also for the Kibana @timestamp field out of the box etc?

Regards,
Edward

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

Hi Edward,

Assuming your epoch time is in second, then yes, you need to multiply by
1000.

About kibana - if it is mapped as a date field it will work - regardless of
the input form - be it milliseconds epochs or an ISO formatted string.

Cheers,
Boaz

On Sunday, October 6, 2013 8:01:18 PM UTC+2, Edward Fjellskål wrote:

From:

Elasticsearch Platform — Find real-time answers at scale | Elastic

"The date type will also accept a long number representing UTC
milliseconds since the epoch, regardless of the format it can handle."

Does that mean that I can save the epoch time + "000" and that
will be a valid date field?

Example:
Epoch: 1381041397
ES-Epoch: 1381041397000

Will that work also for the Kibana @timestamp field out of the box etc?

Regards,
Edward

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

Hi Boaz,

So my epoch is not a date field:

"epoch" : {
"type" : "long"
},

I can probably test this fairly easy, but would that above work for
kibana if I store epoch as "epoch X 1000" ?

or would I need change the mapping to:

"epoch" : {
"type" : "date"
},

and insert "epoch x 1000" ?

Thanks for your resonse :slight_smile:

Edward

On 10/09/2013 12:30 PM, Boaz Leskes wrote:

Hi Edward,

Assuming your epoch time is in second, then yes, you need to multiply by
1000.

About kibana - if it is mapped as a date field it will work - regardless
of the input form - be it milliseconds epochs or an ISO formatted string.

Cheers,
Boaz

On Sunday, October 6, 2013 8:01:18 PM UTC+2, Edward Fjellskål wrote:

From:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-core-types.html#date
<http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-core-types.html#date>


"The date type will also accept a long number representing UTC
milliseconds since the epoch, regardless of the format it can handle."

Does that mean that I can save the epoch time + "000" and that
will be a valid date field?

Example:
Epoch: 1381041397
ES-Epoch: 1381041397000

Will that work also for the Kibana @timestamp field out of the box etc?

Regards,
Edward

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

4

Hi Edward,

"type" should be "date" for Kibana to work.

On Thu, Oct 10, 2013 at 9:20 PM, Edward Fjellskål
edwardfjellskaal@gmail.com wrote:

Hi Boaz,

So my epoch is not a date field:

"epoch" : {
"type" : "long"
},

I can probably test this fairly easy, but would that above work for
kibana if I store epoch as "epoch X 1000" ?

or would I need change the mapping to:

"epoch" : {
"type" : "date"
},

and insert "epoch x 1000" ?

Thanks for your resonse :slight_smile:

Edward

On 10/09/2013 12:30 PM, Boaz Leskes wrote:

Hi Edward,

Assuming your epoch time is in second, then yes, you need to multiply by
1000.

About kibana - if it is mapped as a date field it will work - regardless
of the input form - be it milliseconds epochs or an ISO formatted string.

Cheers,
Boaz

On Sunday, October 6, 2013 8:01:18 PM UTC+2, Edward Fjellskål wrote:

From:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-core-types.html#date
<http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-core-types.html#date>


"The date type will also accept a long number representing UTC
milliseconds since the epoch, regardless of the format it can handle."

Does that mean that I can save the epoch time + "000" and that
will be a valid date field?

Example:
Epoch: 1381041397
ES-Epoch: 1381041397000

Will that work also for the Kibana @timestamp field out of the box etc?

Regards,
Edward

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Regards,
Abhijeet Rastogi (shadyabhi)

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

5

HI Edward,

As Abhijeet has already said - for kibana to work you must have your epoch
field must be of the type date and if you send longs, it must be in
milliseconds since the epoch.

Cheers,
Boaz

On Fri, Oct 11, 2013 at 6:30 AM, Abhijeet Rastogi
abhijeet.1989@gmail.comwrote:

Hi Edward,

"type" should be "date" for Kibana to work.

On Thu, Oct 10, 2013 at 9:20 PM, Edward Fjellskål
edwardfjellskaal@gmail.com wrote:

Hi Boaz,

So my epoch is not a date field:

"epoch" : {
"type" : "long"
},

I can probably test this fairly easy, but would that above work for
kibana if I store epoch as "epoch X 1000" ?

or would I need change the mapping to:

"epoch" : {
"type" : "date"
},

and insert "epoch x 1000" ?

Thanks for your resonse :slight_smile:

Edward

On 10/09/2013 12:30 PM, Boaz Leskes wrote:

Hi Edward,

Assuming your epoch time is in second, then yes, you need to multiply by
1000.

About kibana - if it is mapped as a date field it will work - regardless
of the input form - be it milliseconds epochs or an ISO formatted
string.

Cheers,
Boaz

On Sunday, October 6, 2013 8:01:18 PM UTC+2, Edward Fjellskål wrote:

From:

Elasticsearch Platform — Find real-time answers at scale | Elastic

<

Elasticsearch Platform — Find real-time answers at scale | Elastic

"The date type will also accept a long number representing UTC
milliseconds since the epoch, regardless of the format it can

handle."

Does that mean that I can save the epoch time + "000" and that
will be a valid date field?

Example:
Epoch: 1381041397
ES-Epoch: 1381041397000

Will that work also for the Kibana @timestamp field out of the box

etc?

Regards,
Edward

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Regards,
Abhijeet Rastogi (shadyabhi)

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

6

Thanks for the input...
Im coding up something now :slight_smile:

Regards,
Edward

On 10/11/13, Boaz Leskes b.leskes@gmail.com wrote:

HI Edward,

As Abhijeet has already said - for kibana to work you must have your epoch
field must be of the type date and if you send longs, it must be in
milliseconds since the epoch.

Cheers,
Boaz

On Fri, Oct 11, 2013 at 6:30 AM, Abhijeet Rastogi
abhijeet.1989@gmail.comwrote:

Hi Edward,

"type" should be "date" for Kibana to work.

On Thu, Oct 10, 2013 at 9:20 PM, Edward Fjellskål
edwardfjellskaal@gmail.com wrote:

Hi Boaz,

So my epoch is not a date field:

"epoch" : {
"type" : "long"
},

I can probably test this fairly easy, but would that above work for
kibana if I store epoch as "epoch X 1000" ?

or would I need change the mapping to:

"epoch" : {
"type" : "date"
},

and insert "epoch x 1000" ?

Thanks for your resonse :slight_smile:

Edward

On 10/09/2013 12:30 PM, Boaz Leskes wrote:

Hi Edward,

Assuming your epoch time is in second, then yes, you need to multiply
by
1000.

About kibana - if it is mapped as a date field it will work -
regardless
of the input form - be it milliseconds epochs or an ISO formatted
string.

Cheers,
Boaz

On Sunday, October 6, 2013 8:01:18 PM UTC+2, Edward Fjellskål wrote:

From:

Elasticsearch Platform — Find real-time answers at scale | Elastic

<

Elasticsearch Platform — Find real-time answers at scale | Elastic

"The date type will also accept a long number representing UTC
milliseconds since the epoch, regardless of the format it can

handle."

Does that mean that I can save the epoch time + "000" and that
will be a valid date field?

Example:
Epoch: 1381041397
ES-Epoch: 1381041397000

Will that work also for the Kibana @timestamp field out of the box

etc?

Regards,
Edward

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Regards,
Abhijeet Rastogi (shadyabhi)

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Edward Bjarte Fjellskål
Senior Security Analyst
http://www.gamelinux.org/

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

7

Does it work? after changing the {"type" : "date"}
And setting a date format -

{
"messages" : {
"_timestamp" : {
"enabled" : true
},
"properties" : {
"app_event_time" : {
"type" : "date",
"format" : "yyyy/MM/dd HH:mm:ss"
},
"event_time" : {
"type" : "date",
"format" : "yyyy/MM/dd HH:mm:ss"
},

What could be the problem?

On Saturday, October 12, 2013 1:10:39 AM UTC+5:30, Edward Fjellskål wrote:

Thanks for the input...
Im coding up something now :slight_smile:

Regards,
Edward

On 10/11/13, Boaz Leskes <b.le...@gmail.com <javascript:>> wrote:

HI Edward,

As Abhijeet has already said - for kibana to work you must have your
epoch
field must be of the type date and if you send longs, it must be in
milliseconds since the epoch.

Cheers,
Boaz

On Fri, Oct 11, 2013 at 6:30 AM, Abhijeet Rastogi
<abhije...@gmail.com <javascript:>>wrote:

Hi Edward,

"type" should be "date" for Kibana to work.

On Thu, Oct 10, 2013 at 9:20 PM, Edward Fjellskål
<edwardfj...@gmail.com <javascript:>> wrote:

Hi Boaz,

So my epoch is not a date field:

"epoch" : {
"type" : "long"
},

I can probably test this fairly easy, but would that above work for
kibana if I store epoch as "epoch X 1000" ?

or would I need change the mapping to:

"epoch" : {
"type" : "date"
},

and insert "epoch x 1000" ?

Thanks for your resonse :slight_smile:

Edward

On 10/09/2013 12:30 PM, Boaz Leskes wrote:

Hi Edward,

Assuming your epoch time is in second, then yes, you need to
multiply
by
1000.

About kibana - if it is mapped as a date field it will work -
regardless
of the input form - be it milliseconds epochs or an ISO formatted
string.

Cheers,
Boaz

On Sunday, October 6, 2013 8:01:18 PM UTC+2, Edward Fjellskål wrote:

From:

Elasticsearch Platform — Find real-time answers at scale | Elastic

<

Elasticsearch Platform — Find real-time answers at scale | Elastic

"The date type will also accept a long number representing UTC 
milliseconds since the epoch, regardless of the format it can

handle."

Does that mean that I can save the epoch time + "000" and that 
will be a valid date field? 

Example: 
Epoch: 1381041397 
ES-Epoch: 1381041397000 

Will that work also for the Kibana @timestamp field out of the

box

etc?

Regards, 
Edward

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send
an email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send
an email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
Regards,
Abhijeet Rastogi (shadyabhi)

--
You received this message because you are subscribed to the Google
Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
Edward Bjarte Fjellskål
Senior Security Analyst
http://www.gamelinux.org/

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/299b6673-b6e2-4662-8e10-b50cb78c9fad%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

8

Does it work? after changing the {"type" : "date"}

And setting a date format -

{
"messages" : {
"_timestamp" : {
"enabled" : true
},
"properties" : {
"app_event_time" : {
"type" : "date",
"format" : "yyyy/MM/dd HH:mm:ss"
},
"event_time" : {
"type" : "date",
"format" : "yyyy/MM/dd HH:mm:ss"
},

What could be the problem?

Cause mine does not work

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/0f3bfa6a-c8b1-4e68-8230-1a0eb8f61d33%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.