Problems with numericRangeFilter / date range query

[Andreas_Fink]

Hi List.

I am trying to index a type (event) that contains a date.
Then i want to find all events that lie between two dates.
When querying via the java api, i get the following Exception

-- 8< --

org.elasticsearch.action.search.SearchPhaseExecutionException: Failed to execute phase [dfs], total failure; shardFailures {[1][gdp][0]: SearchParseException[[gdp][0]: query[ConstantScore(:)],from[0],size[60]: Parse Failure [Failed to parse source [
򃦲omsize$¸„query򈭡tch_all򻻅filter򌮵meric_range򏥶ent_start_date򀉱1/02/2012toI12/02/2012Œinclude_lower#Œinclude_upper#󻻻]]]; nested: QueryParsingException[[gdp] Field [event_start_date] is not a numeric type]; }{[1][gdp][3]: SearchParseException[[gdp][3]: query[ConstantScore(:)],from[0],size[60]: Parse Failure [Failed to parse source [
򃦲omsize$¸„query򈭡tch_all򻻅filter򌮵meric_range򏥶ent_start_date򀉱1/02/2012toI12/02/2012Œinclude_lower#Œinclude_upper#󻻻]]]; nested: QueryParsingException[[gdp] Field [event_start_date] is not a numeric type]; }{[1][gdp][2]: SearchParseException[[gdp][2]: query[ConstantScore(:)],from[0],size[60]: Parse Failure [Failed to parse source [
򃦲omsize$¸„query򈭡tch_all򻻅filter򌮵meric_range򏥶ent_start_date򀉱1/02/2012toI12/02/2012Œinclude_lower#Œinclude_upper#󻻻]]]; nested: QueryParsingException[[gdp] Field [event_start_date] is not a numeric type]; }{[1][gdp][4]: SearchParseException[[gdp][4]: query[ConstantScore(:)],from[0],size[60]: Parse Failure [Failed to parse source [
򃦲omsize$¸„query򈭡tch_all򻻅filter򌮵meric_range򏥶ent_start_date򀉱1/02/2012toI12/02/2012Œinclude_lower#Œinclude_upper#󻻻]]]; nested: QueryParsingException[[gdp] Field [event_start_date] is not a numeric type]; }{[1][gdp][1]: SearchParseException[[gdp][1]: query[ConstantScore(:)],from[0],size[60]: Parse Failure [Failed to parse source [
򃦲omsize$¸„query򈭡tch_all򻻅filter򌮵meric_range򏥶ent_start_date򀉱1/02/2012toI12/02/2012Œinclude_lower#Œinclude_upper#󻻻]]]; nested: QueryParsingException[[gdp] Field [event_start_date] is not a numeric type]; }
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.onFirstPhaseResult(TransportSearchTypeAction.java:258)
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$3.onFailure(TransportSearchTypeAction.java:211)
at org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteDfs(SearchServiceTransportAction.java:107)
at org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchDfsQueryThenFetchAction.java:86)
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:205)
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:192)
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:178)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)

-- 8< --

Yes, the trace contains strange characters (Eclipse/Junit).
I see that ES complains about the "event_start_date" not being numeric but from the docs it seems my mapping is alright.
It is as following:

-- 8< --

{
event: {
properties: {
title_id: {
type: "long",
index: "not_analyzed"
},
event_id: {
type: "long",
index: "not_analyzed"
},
event_start_date: {
type: "date",
format: "dd/MM/yyyy"
},
event_end_date: {
type: "date",
format: "dd/MM/yyyy"
},
venue_id: {
type: "long",
index: "not_analyzed"
}
}
}
}

-- 8< --

My query looks like this:

-- 8< --

final SearchResponse res = _client.prepareSearch(index)
.setTypes(type)
.setSearchType(SearchType.DFS_QUERY_THEN_FETCH)
.setQuery(QueryBuilders.matchAllQuery())
.setFilter(FilterBuilders.numericRangeFilter(field)
.gte("11/02/2012")
.lte("13/02/2012")
)
.execute()
.actionGet();

-- 8< --

And a sample json of data:

-- 8< --

{"title_id":9000961601,"event_id":10019937040,"event_start_date":"12/02/2012","event_end_date":"12/02/2012","venue_id":9000961600}

-- 8< --

Does anybody have an idea?
I am new to Elasticsearch, just used Compass before. Please bear with me if this has been answered before but i couldn't find anything related online.

Thanks a lot,
Andi.

[kimchy (Shay Banon)]

It seems like your mapping is correct, can you make sure that it has been applied properly by getting the mapping and validating? If you use curl, there is a specific get mapping endpoint: Elasticsearch Platform — Find real-time answers at scale | Elastic. If you use Java, you use the cluster state API to get the mapping.

On Monday, February 6, 2012 at 9:56 AM, Andreas Fink wrote:

Hi List.

I am trying to index a type (event) that contains a date.
Then i want to find all events that lie between two dates.
When querying via the java api, i get the following Exception

-- 8< --

org.elasticsearch.action.search.SearchPhaseExecutionException: Failed to execute phase [dfs], total failure; shardFailures {[1][gdp][0]: SearchParseException[[gdp][0]: query[ConstantScore(:)],from[0],size[60]: Parse Failure [Failed to parse source [
򃦲omsize$¸„query򈭡tch_all򻻅filter򌮵meric_range򏥶ent_start_date򀉱1/02/2012toI12/02/2012Œinclude_lower#Œinclude_upper#󻻻]]]; nested: QueryParsingException[[gdp] Field [event_start_date] is not a numeric type]; }{[1][gdp][3]: SearchParseException[[gdp][3]: query[ConstantScore(:)],from[0],size[60]: Parse Failure [Failed to parse source [
򃦲omsize$¸„query򈭡tch_all򻻅filter򌮵meric_range򏥶ent_start_date򀉱1/02/2012toI12/02/2012Œinclude_lower#Œinclude_upper#󻻻]]]; nested: QueryParsingException[[gdp] Field [event_start_date] is not a numeric type]; }{[1][gdp][2]: SearchParseException[[gdp][2]: query[ConstantScore(:)],from[0],size[60]: Parse Failure [Failed to parse source [
򃦲omsize$¸„query򈭡tch_all򻻅filter򌮵meric_range򏥶ent_start_date򀉱1/02/2012toI12/02/2012Œinclude_lower#Œinclude_upper#󻻻]]]; nested: QueryParsingException[[gdp] Field [event_start_date] is not a numeric type]; }{[1][gdp][4]: SearchParseException[[gdp][4]: query[ConstantScore(:)],from[0],size[60]: Parse Failure [Failed to parse source [
򃦲omsize$¸„query򈭡tch_all򻻅filter򌮵meric_range򏥶ent_start_date򀉱1/02/2012toI12/02/2012Œinclude_lower#Œinclude_upper#󻻻]]]; nested: QueryParsingException[[gdp] Field [event_start_date] is not a numeric type]; }{[1][gdp][1]: SearchParseException[[gdp][1]: query[ConstantScore(:)],from[0],size[60]: Parse Failure [Failed to parse source [
򃦲omsize$¸„query򈭡tch_all򻻅filter򌮵meric_range򏥶ent_start_date򀉱1/02/2012toI12/02/2012Œinclude_lower#Œinclude_upper#󻻻]]]; nested: QueryParsingException[[gdp] Field [event_start_date] is not a numeric type]; }
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.onFirstPhaseResult(TransportSearchTypeAction.java:258)
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$3.onFailure(TransportSearchTypeAction.java:211)
at org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteDfs(SearchServiceTransportAction.java:107)
at org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchDfsQueryThenFetchAction.java:86)
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:205)
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:192)
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:178)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)

-- 8< --

Yes, the trace contains strange characters (Eclipse/Junit).
I see that ES complains about the "event_start_date" not being numeric but from the docs it seems my mapping is alright.
It is as following:

-- 8< --

{
event: {
properties: {
title_id: {
type: "long",
index: "not_analyzed"
},
event_id: {
type: "long",
index: "not_analyzed"
},
event_start_date: {
type: "date",
format: "dd/MM/yyyy"
},
event_end_date: {
type: "date",
format: "dd/MM/yyyy"
},
venue_id: {
type: "long",
index: "not_analyzed"
}
}
}
}

-- 8< --

My query looks like this:

-- 8< --

final SearchResponse res = _client.prepareSearch(index)
.setTypes(type)
.setSearchType(SearchType.DFS_QUERY_THEN_FETCH)
.setQuery(QueryBuilders.matchAllQuery())
.setFilter(FilterBuilders.numericRangeFilter(field)
.gte("11/02/2012")
.lte("13/02/2012")
)
.execute()
.actionGet();

-- 8< --

And a sample json of data:

-- 8< --

{"title_id":9000961601,"event_id":10019937040,"event_start_date":"12/02/2012","event_end_date":"12/02/2012","venue_id":9000961600}

-- 8< --

Does anybody have an idea?
I am new to Elasticsearch, just used Compass before. Please bear with me if this has been answered before but i couldn't find anything related online.

Thanks a lot,
Andi.