Searching by date using long as parameter

Michal_Domagala_2 · May 17, 2012, 3:47pm

Documantation states that
"The date type will also accept a long number representing UTC
milliseconds since the epoch, regardless of the format it can handle."
http://www.elasticsearch.org/guide/reference/mapping/core-types.html

Please answer my understanding is correct or not

I put mapping that attribut mydate is a date

curl -XPOST 'http://127.0.0.1:9200/ind1?pretty=true'
{
"mappings": {
"docdate": {
"properties": {
"mydate": {"type": "date"}
} } }}

I put document with date, Date is passed as long

curl -XPOST ' 'http://127.0.0.1:9200/ind1/docdate/1'
{"mydate":1000000}

I do search by date. Date i passed as long

curl -XPOST ' 'http://127.0.0.1:9200/ind1/_search
{"query":{"term":{"mydate":1000000}}}

The search returns nothing but expect document inserted in step2

Is it a bug or misunderstanding?

Michal Domagala

kimchy · May 20, 2012, 7:58pm

Its actually only supported when indexing, not when searching. We could
also try and support it when searching, a bit more complicated in this case
(the joda parser is a bit too relaxed for us in this case). Open an issue?

On Thu, May 17, 2012 at 5:47 PM, Michal Domagala outsider404@gmail.comwrote:

Documantation states that
"The date type will also accept a long number representing UTC
milliseconds since the epoch, regardless of the format it can handle."
Elasticsearch Platform — Find real-time answers at scale | Elastic

Please answer my understanding is correct or not

I put mapping that attribut mydate is a date

curl -XPOST 'http://127.0.0.1:9200/ind1?pretty=true'
{
"mappings": {
"docdate": {
"properties": {
"mydate": {"type": "date"}
} } }}

I put document with date, Date is passed as long

curl -XPOST ' 'http://127.0.0.1:9200/ind1/docdate/1'
{"mydate":1000000}

I do search by date. Date i passed as long

curl -XPOST ' 'http://127.0.0.1:9200/ind1/_search
{"query":{"term":{"mydate":1000000}}}

The search returns nothing but expect document inserted in step2

Is it a bug or misunderstanding?

Michal Domagala

Michal_Domagala_2 · May 22, 2012, 9:54pm

Actually I don't need search date by long. Thanks for explanation

On 20 Maj, 21:58, Shay Banon kim...@gmail.com wrote:

Its actually only supported when indexing, not when searching. We could
also try and support it when searching, a bit more complicated in this case
(the joda parser is a bit too relaxed for us in this case). Open an issue?

On Thu, May 17, 2012 at 5:47 PM, Michal Domagala outsider...@gmail.comwrote:

Documantation states that
"The date type will also accept a long number representing UTC
milliseconds since the epoch, regardless of the format it can handle."
Elasticsearch Platform — Find real-time answers at scale | Elastic

Please answer my understanding is correct or not

I put mapping that attribut mydate is a date

curl -XPOST 'http://127.0.0.1:9200/ind1?pretty=true'
{
"mappings": {
"docdate": {
"properties": {
"mydate": {"type": "date"}
} } }}

I put document with date, Date is passed as long

curl -XPOST ' 'http://127.0.0.1:9200/ind1/docdate/1'
{"mydate":1000000}

I do search by date. Date i passed as long

curl -XPOST ' 'http://127.0.0.1:9200/ind1/_search
{"query":{"term":{"mydate":1000000}}}

The search returns nothing but expect document inserted in step2

Is it a bug or misunderstanding?

Michal Domagala

Michal_Domagala_2 · May 23, 2012, 9:30pm

Regardless I don't need search date by long, I observed strange
behavior (0.19.4)

curl -XPOST 'http://127.0.0.1:9200/ind1' -d '{"mappings": {"docdate":
{"properties": {"mydate":{"type":"date"}}}}}'

curl -XPOST 'http://127.0.0.1:9200/ind1/docdate' -d '{"mydate":
1000000}'

curl -XPOST 'http://127.0.0.1:9200/ind1/docdate' -d '{"mydate":
2764800000}'

curl -XPOST 'http://127.0.0.1:9200/ind1/_search' -d '{"query":{"term":
{"mydate":1000000}}}'

curl -XPOST 'http://127.0.0.1:9200/ind1/_search' -d '{"query":{"term":
{"mydate":2764800000}}}'

I expect that search hist should be equal

On 22 Maj, 23:54, Michal Domagala outsider...@gmail.com wrote:

Actually I don't need search date by long. Thanks for explanation

On 20 Maj, 21:58, Shay Banon kim...@gmail.com wrote:

Its actually only supported when indexing, not when searching. We could
also try and support it when searching, a bit more complicated in this case
(the joda parser is a bit too relaxed for us in this case). Open an issue?

On Thu, May 17, 2012 at 5:47 PM, Michal Domagala outsider...@gmail.comwrote:

Documantation states that
"The date type will also accept a long number representing UTC
milliseconds since the epoch, regardless of the format it can handle."
Elasticsearch Platform — Find real-time answers at scale | Elastic

Please answer my understanding is correct or not

I put mapping that attribut mydate is a date

curl -XPOST 'http://127.0.0.1:9200/ind1?pretty=true'
{
"mappings": {
"docdate": {
"properties": {
"mydate": {"type": "date"}
} } }}

I put document with date, Date is passed as long

curl -XPOST ' 'http://127.0.0.1:9200/ind1/docdate/1'
{"mydate":1000000}

I do search by date. Date i passed as long

curl -XPOST ' 'http://127.0.0.1:9200/ind1/_search
{"query":{"term":{"mydate":1000000}}}

The search returns nothing but expect document inserted in step2

Is it a bug or misunderstanding?

Michal Domagala

Michal_Domagala_2 · May 24, 2012, 12:23am

Similar problem occurs when I use Java API:

import static
org.elasticsearch.common.xcontent.XContentFactory.jsonBuilder;
import static org.elasticsearch.index.query.FilterBuilders.;
import java.io.IOException;
import java.util.Date;
import static org.elasticsearch.node.NodeBuilder.;
import org.elasticsearch.ElasticSearchException;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.action.search.SearchType;
import org.elasticsearch.client.Client;
import org.elasticsearch.node.Node;

public class Test {

public static void main(String[] args) throws ElasticSearchException,

IOException {
Node node = nodeBuilder().local(true).node();
Client client = node.client();

	String mapping = jsonBuilder().startObject().startObject("docdate")
			.startObject("properties").startObject("mydate").field("type",

"date").string();

	client.admin().indices().prepareCreate("ind1").addMapping("docdate",

mapping).execute().actionGet();

	client.prepareIndex("ind1", "docdate")
	    .setSource(jsonBuilder().startObject().field("mydate",

"1000000"))
.setRefresh(true).execute().actionGet();

	client.prepareIndex("ind1", "docdate")
    	.setSource(jsonBuilder().startObject().field("mydate",

"2764800000"))
.setRefresh(true).execute().actionGet();

	SearchResponse searchResponse = client.prepareSearch("ind1")
	        .setSearchType(SearchType.COUNT)
	        .setFilter(termFilter("mydate", new Date(1000000)))
	        .execute()
	        .actionGet();
	System.out.println("Hits:"+

searchResponse.getHits().getTotalHits()); //0

	searchResponse = client.prepareSearch("ind1")
	        .setSearchType(SearchType.COUNT)
	        .setFilter(termFilter("mydate", new Date(2764800000L)))
	        .execute()
	        .actionGet();
	System.out.println("Hits:"+

searchResponse.getHits().getTotalHits()); //1

	node.close();
}

}

The problem occurs when I have to store third-party JSON document on
my database. I expect that by mapping definition I will force Elastic
search to correct value interpretations.
In current case "mydate" is a date and "1000000" should be mapped as
date, but filter [date = new Date(1000000)] fails

kimchy · May 25, 2012, 10:42pm

The indexing part that decides to treat it as millis since the epoch
detects it based on the json type, if its numeric, then its millis since
the epoch. In your Java sample, you index it as a String, which means the
(very lenient) joda parser will be used to parse the string value. Once you
change the indexing code to index it as numeric, and not string, it will
work.

On Thu, May 24, 2012 at 2:23 AM, Michal Domagala outsider404@gmail.comwrote:

Similar problem occurs when I use Java API:

import static
org.elasticsearch.common.xcontent.XContentFactory.jsonBuilder;
import static org.elasticsearch.index.query.FilterBuilders.;
import java.io.IOException;
import java.util.Date;
import static org.elasticsearch.node.NodeBuilder.;
import org.elasticsearch.ElasticSearchException;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.action.search.SearchType;
import org.elasticsearch.client.Client;
import org.elasticsearch.node.Node;

public class Test {
   public static void main(String[] args) throws
ElasticSearchException,
IOException {
Node node = nodeBuilder().local(true).node();
Client client = node.client();
           String mapping =
jsonBuilder().startObject().startObject("docdate")

.startObject("properties").startObject("mydate").field("type",
"date").string();

client.admin().indices().prepareCreate("ind1").addMapping("docdate",
mapping).execute().actionGet();
           client.prepareIndex("ind1", "docdate")
               .setSource(jsonBuilder().startObject().field("mydate",
"1000000"))
.setRefresh(true).execute().actionGet();
           client.prepareIndex("ind1", "docdate")
           .setSource(jsonBuilder().startObject().field("mydate",
"2764800000"))
.setRefresh(true).execute().actionGet();
           SearchResponse searchResponse = client.prepareSearch("ind1")
                   .setSearchType(SearchType.COUNT)
                   .setFilter(termFilter("mydate", new Date(1000000)))
                   .execute()
                   .actionGet();
           System.out.println("Hits:"+
searchResponse.getHits().getTotalHits()); //0
           searchResponse = client.prepareSearch("ind1")
                   .setSearchType(SearchType.COUNT)
                   .setFilter(termFilter("mydate", new
Date(2764800000L)))
.execute()
.actionGet();
System.out.println("Hits:"+
searchResponse.getHits().getTotalHits()); //1
           node.close();
   }
}

The problem occurs when I have to store third-party JSON document on
my database. I expect that by mapping definition I will force Elastic
search to correct value interpretations.
In current case "mydate" is a date and "1000000" should be mapped as
date, but filter [date = new Date(1000000)] fails