Terms facet changing "date" to "long"

Hello all,

The issue is I have a terms panel in Kibana that I want to group events by
a "date" field from each record (Not the @timestamp field). The terms
panel is taking my nicely formatted dates (2014-07-31) and turning them
into longs since UTC (1403568000000). I did a quick test by creating a new
index, giving it a mapping, then running both a search and a facet query,
and sure enough, the facet query returns the long format instead of the
date format! I tried two types of "dates", just to see if that made a
difference. It did not.

=====
#Create mapping for index
PUT /test_index_jerry/test/_mapping
{
"test": {
"properties": {
"date1": {
"type": "date",
"format": "dateOptionalTime"
},
"date2": {
"type": "date",
"format": "date"
}
}
}
}

#Put some data
POST /test_index_jerry/test
{
"date1":"2014-06-30",
"date2":"2014-06-30"
}

#Execute a basic query
GET /test_index_jerry/test/_search
{
"query": {
"match_all": {}
}
}

It returns dates in "date" format

{
"took": 0,
"timed_out": false,
"_shards": {
"total": 2,
"successful": 2,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"_index": "test_index_jerry",
"_type": "test",
"_id": "VUOeBuiUTGeqBS2Zl8--lg",
"_score": 1,
"_source": {
"date1": "2014-06-30",
"date2": "2014-06-30"
}
}
]
}
}

#Execute a terms facet
GET /test_index_jerry/test/_search
{
"facets": {
"terms": {
"terms": {
"field": "date1",
"size": 10,
"order": "count",
"exclude": []
}
}
}
}

#Now we have longs
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 2,
"successful": 2,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"_index": "test_index_jerry",
"_type": "test",
"_id": "VUOeBuiUTGeqBS2Zl8--lg",
"_score": 1,
"_source": {
"date1": "2014-06-30",
"date2": "2014-06-30"
}
}
]
},
"facets": {
"terms": {
"_type": "terms",
"missing": 0,
"total": 1,
"other": 0,
"terms": [
{
"term": 1404086400000,
"count": 1
}
]
}
}
}

Is there some way I can get the term to stay in date formatted buckets? I
also tried the date histogram facet, but it returned longs as well.

Very much appreciate the help :slight_smile:
Thanks,
Chris

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAND3Dpjkq0BFL3E_-CgDB4%3DMayZPHCDm%2BcafQef2u3xfNt4qQg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Please excuse the bump of my own question. :slight_smile: After almost 8 months, I
still have this question! Just wanted to get it in front of people's eyes
again.

Is there a way to have date fields stored in ES displayed in a terms panel
as nicely formatted dates instead of epoch time?

Very much appreciated!
Chris

On Mon, Jun 30, 2014 at 3:21 PM, Chris Neal chris.neal@derbysoft.net
wrote:

Hello all,

The issue is I have a terms panel in Kibana that I want to group events by
a "date" field from each record (Not the @timestamp field). The terms
panel is taking my nicely formatted dates (2014-07-31) and turning them
into longs since UTC (1403568000000). I did a quick test by creating a new
index, giving it a mapping, then running both a search and a facet query,
and sure enough, the facet query returns the long format instead of the
date format! I tried two types of "dates", just to see if that made a
difference. It did not.

=====
#Create mapping for index
PUT /test_index_jerry/test/_mapping
{
"test": {
"properties": {
"date1": {
"type": "date",
"format": "dateOptionalTime"
},
"date2": {
"type": "date",
"format": "date"
}
}
}
}

#Put some data
POST /test_index_jerry/test
{
"date1":"2014-06-30",
"date2":"2014-06-30"
}

#Execute a basic query
GET /test_index_jerry/test/_search
{
"query": {
"match_all": {}
}
}

It returns dates in "date" format

{
"took": 0,
"timed_out": false,
"_shards": {
"total": 2,
"successful": 2,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"_index": "test_index_jerry",
"_type": "test",
"_id": "VUOeBuiUTGeqBS2Zl8--lg",
"_score": 1,
"_source": {
"date1": "2014-06-30",
"date2": "2014-06-30"
}
}
]
}
}

#Execute a terms facet
GET /test_index_jerry/test/_search
{
"facets": {
"terms": {
"terms": {
"field": "date1",
"size": 10,
"order": "count",
"exclude": []
}
}
}
}

#Now we have longs
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 2,
"successful": 2,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"_index": "test_index_jerry",
"_type": "test",
"_id": "VUOeBuiUTGeqBS2Zl8--lg",
"_score": 1,
"_source": {
"date1": "2014-06-30",
"date2": "2014-06-30"
}
}
]
},
"facets": {
"terms": {
"_type": "terms",
"missing": 0,
"total": 1,
"other": 0,
"terms": [
{
"term": 1404086400000,
"count": 1
}
]
}
}
}

Is there some way I can get the term to stay in date formatted buckets? I
also tried the date histogram facet, but it returned longs as well.

Very much appreciate the help :slight_smile:
Thanks,
Chris

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAND3DphaBX2%2BZ_mRuS4vtx39EQKs9k9EnH08nrJBN58hj6yCYA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

"terms panel" you mean Kibana? take a look at Kibana 4, they are doing this
automatically in most places

--

Itamar Syn-Hershko
http://code972.com | @synhershko https://twitter.com/synhershko
Freelance Developer & Consultant
Lucene.NET committer and PMC member

On Thu, Feb 5, 2015 at 7:29 PM, Chris Neal chris.neal@derbysoft.net wrote:

Please excuse the bump of my own question. :slight_smile: After almost 8 months, I
still have this question! Just wanted to get it in front of people's eyes
again.

Is there a way to have date fields stored in ES displayed in a terms panel
as nicely formatted dates instead of epoch time?

Very much appreciated!
Chris

On Mon, Jun 30, 2014 at 3:21 PM, Chris Neal chris.neal@derbysoft.net
wrote:

Hello all,

The issue is I have a terms panel in Kibana that I want to group events
by a "date" field from each record (Not the @timestamp field). The terms
panel is taking my nicely formatted dates (2014-07-31) and turning them
into longs since UTC (1403568000000). I did a quick test by creating a new
index, giving it a mapping, then running both a search and a facet query,
and sure enough, the facet query returns the long format instead of the
date format! I tried two types of "dates", just to see if that made a
difference. It did not.

=====
#Create mapping for index
PUT /test_index_jerry/test/_mapping
{
"test": {
"properties": {
"date1": {
"type": "date",
"format": "dateOptionalTime"
},
"date2": {
"type": "date",
"format": "date"
}
}
}
}

#Put some data
POST /test_index_jerry/test
{
"date1":"2014-06-30",
"date2":"2014-06-30"
}

#Execute a basic query
GET /test_index_jerry/test/_search
{
"query": {
"match_all": {}
}
}

It returns dates in "date" format

{
"took": 0,
"timed_out": false,
"_shards": {
"total": 2,
"successful": 2,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"_index": "test_index_jerry",
"_type": "test",
"_id": "VUOeBuiUTGeqBS2Zl8--lg",
"_score": 1,
"_source": {
"date1": "2014-06-30",
"date2": "2014-06-30"
}
}
]
}
}

#Execute a terms facet
GET /test_index_jerry/test/_search
{
"facets": {
"terms": {
"terms": {
"field": "date1",
"size": 10,
"order": "count",
"exclude": []
}
}
}
}

#Now we have longs
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 2,
"successful": 2,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"_index": "test_index_jerry",
"_type": "test",
"_id": "VUOeBuiUTGeqBS2Zl8--lg",
"_score": 1,
"_source": {
"date1": "2014-06-30",
"date2": "2014-06-30"
}
}
]
},
"facets": {
"terms": {
"_type": "terms",
"missing": 0,
"total": 1,
"other": 0,
"terms": [
{
"term": 1404086400000,
"count": 1
}
]
}
}
}

Is there some way I can get the term to stay in date formatted buckets?
I also tried the date histogram facet, but it returned longs as well.

Very much appreciate the help :slight_smile:
Thanks,
Chris

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAND3DphaBX2%2BZ_mRuS4vtx39EQKs9k9EnH08nrJBN58hj6yCYA%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAND3DphaBX2%2BZ_mRuS4vtx39EQKs9k9EnH08nrJBN58hj6yCYA%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZsvwQKtabn8-%3D1JE_nDrwEwEy-eFB6KsXnj%3Dg4mzzOCKw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Thanks Itamar!

I'm looking into Kibana 4 right now actually. :slight_smile:
It's going to be a bit though before I can get us migrated, so I was hoping
there was an interim solution that could be applied to Kibana 3.

Thanks for your help!
Chris

On Thu, Feb 5, 2015 at 3:53 PM, Itamar Syn-Hershko itamar@code972.com
wrote:

"terms panel" you mean Kibana? take a look at Kibana 4, they are doing
this automatically in most places

--

Itamar Syn-Hershko
http://code972.com | @synhershko https://twitter.com/synhershko
Freelance Developer & Consultant
Lucene.NET committer and PMC member

On Thu, Feb 5, 2015 at 7:29 PM, Chris Neal chris.neal@derbysoft.net
wrote:

Please excuse the bump of my own question. :slight_smile: After almost 8 months, I
still have this question! Just wanted to get it in front of people's eyes
again.

Is there a way to have date fields stored in ES displayed in a terms
panel as nicely formatted dates instead of epoch time?

Very much appreciated!
Chris

On Mon, Jun 30, 2014 at 3:21 PM, Chris Neal chris.neal@derbysoft.net
wrote:

Hello all,

The issue is I have a terms panel in Kibana that I want to group events
by a "date" field from each record (Not the @timestamp field). The terms
panel is taking my nicely formatted dates (2014-07-31) and turning them
into longs since UTC (1403568000000). I did a quick test by creating a new
index, giving it a mapping, then running both a search and a facet query,
and sure enough, the facet query returns the long format instead of the
date format! I tried two types of "dates", just to see if that made a
difference. It did not.

=====
#Create mapping for index
PUT /test_index_jerry/test/_mapping
{
"test": {
"properties": {
"date1": {
"type": "date",
"format": "dateOptionalTime"
},
"date2": {
"type": "date",
"format": "date"
}
}
}
}

#Put some data
POST /test_index_jerry/test
{
"date1":"2014-06-30",
"date2":"2014-06-30"
}

#Execute a basic query
GET /test_index_jerry/test/_search
{
"query": {
"match_all": {}
}
}

It returns dates in "date" format

{
"took": 0,
"timed_out": false,
"_shards": {
"total": 2,
"successful": 2,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"_index": "test_index_jerry",
"_type": "test",
"_id": "VUOeBuiUTGeqBS2Zl8--lg",
"_score": 1,
"_source": {
"date1": "2014-06-30",
"date2": "2014-06-30"
}
}
]
}
}

#Execute a terms facet
GET /test_index_jerry/test/_search
{
"facets": {
"terms": {
"terms": {
"field": "date1",
"size": 10,
"order": "count",
"exclude": []
}
}
}
}

#Now we have longs
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 2,
"successful": 2,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"_index": "test_index_jerry",
"_type": "test",
"_id": "VUOeBuiUTGeqBS2Zl8--lg",
"_score": 1,
"_source": {
"date1": "2014-06-30",
"date2": "2014-06-30"
}
}
]
},
"facets": {
"terms": {
"_type": "terms",
"missing": 0,
"total": 1,
"other": 0,
"terms": [
{
"term": 1404086400000,
"count": 1
}
]
}
}
}

Is there some way I can get the term to stay in date formatted buckets?
I also tried the date histogram facet, but it returned longs as well.

Very much appreciate the help :slight_smile:
Thanks,
Chris

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAND3DphaBX2%2BZ_mRuS4vtx39EQKs9k9EnH08nrJBN58hj6yCYA%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAND3DphaBX2%2BZ_mRuS4vtx39EQKs9k9EnH08nrJBN58hj6yCYA%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZsvwQKtabn8-%3D1JE_nDrwEwEy-eFB6KsXnj%3Dg4mzzOCKw%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZsvwQKtabn8-%3D1JE_nDrwEwEy-eFB6KsXnj%3Dg4mzzOCKw%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAND3Dpic3N3VM5cZjw9iK7cL9RkCN2ypbvkS7NRq4QojPdbMBw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.