Date type in Java API

Elastic Stack Elasticsearch
1

Hey community...

I've been struggling with the Date type in ElasticSearch. Let me first
explain what I need, then I'll explain to you how I'm trying to get there...

I have a field in ES, configured as (copied from the output of the
head-plugin):

myDate: {

  • index: analyzed
  • store: yes
  • format: dateOptionalTime
  • type: date

}

What I'd like to be able to do is filter (RangeFilter on SearchRequest)
documents based on a specified Date-range. The dates I have is in the
standard Java Date format (I'm using the Java API).

The line I use to populate this field is as follows:

...jsonBuilder().startObject().field("myDate", dateObject).endObject()...

Here, dateObject is of the standard Java Date type. To take from this is
that I'm hoping that the API knows what to do with the Date-type object.
Alternatively, I can place a String-formatted date (but which format?) or
a Long value of the instant since epoch. I know not what is required
here...

When I query the database, I then try to extract the date value with:

Date dateObject = new Date((Long) fieldMap.get("myDate").getValue()); //
effectively GetField.getValue()

I noticed that the type in the map (fieldMap is a Map<String, GetField>) is
Long, hence the type-casting.

With this implementation the filter on the search-query does not work. I
can get the date from the returned result, but I can't get the query to
work. I suspect the format of the query is not correct. Here is how I set
that up:

RangeFilterBuilder dateRangeFilter = FilterBuilders.rangeFilter("myDate");
dateRangeFilter.from(dateObject.getTime());
dateRangeFilter.to(dateObject.getTime());

As you may or may not know, the getTime() method on a Java Date object
returnes the time since epoch in millies. I have also tried this with the
numeric range filter.

My question then, in three parts:

  1. What is the type to use when saving the value through the API
    (setting the field)?
  2. What is the variable type I can expect the get from the GetField?
  3. What is the type I need to use when setting the ranges in the
    RangeFilter? (and should I use RangeFilter, or NumericRangeFilter)

Thank you, once again.

Kind regards,
Thinus

2

This may help:

On Thursday, April 26, 2012 10:27:13 AM UTC-4, Thinus Prinsloo wrote:

Hey community...

I've been struggling with the Date type in Elasticsearch. Let me first
explain what I need, then I'll explain to you how I'm trying to get there...

I have a field in ES, configured as (copied from the output of the
head-plugin):

myDate: {

  • index: analyzed
  • store: yes
  • format: dateOptionalTime
  • type: date

}

What I'd like to be able to do is filter (RangeFilter on SearchRequest)
documents based on a specified Date-range. The dates I have is in the
standard Java Date format (I'm using the Java API).

The line I use to populate this field is as follows:

...jsonBuilder().startObject().field("myDate", dateObject).endObject()...

Here, dateObject is of the standard Java Date type. To take from this is
that I'm hoping that the API knows what to do with the Date-type object.
Alternatively, I can place a String-formatted date (but which format?) or
a Long value of the instant since epoch. I know not what is required
here...

When I query the database, I then try to extract the date value with:

Date dateObject = new Date((Long) fieldMap.get("myDate").getValue()); //
effectively GetField.getValue()

I noticed that the type in the map (fieldMap is a Map<String, GetField>)
is Long, hence the type-casting.

With this implementation the filter on the search-query does not work. I
can get the date from the returned result, but I can't get the query to
work. I suspect the format of the query is not correct. Here is how I set
that up:

RangeFilterBuilder dateRangeFilter = FilterBuilders.rangeFilter("myDate");
dateRangeFilter.from(dateObject.getTime());
dateRangeFilter.to(dateObject.getTime());

As you may or may not know, the getTime() method on a Java Date object
returnes the time since epoch in millies. I have also tried this with the
numeric range filter.

My question then, in three parts:

  1. What is the type to use when saving the value through the API
    (setting the field)?
  2. What is the variable type I can expect the get from the GetField?
  3. What is the type I need to use when setting the ranges in the
    RangeFilter? (and should I use RangeFilter, or NumericRangeFilter)

Thank you, once again.

Kind regards,
Thinus

3

Internally, a date type is stored as a long value milliseconds since the
epoch. When you provide a Date object to teh range filter, it will be
formatted as the default ISO format (dateOptionalTime). You will get the
value as the string formatted value though (String). I do wonder though why
do you store the field, cause if you don't store it, it will be
automatically extracted from _source.

In general, what you do is fine, can you maybe share a sample code that
repro this? (create index, put mapping, index a sample doc, and show the
search failure)?

On Thu, Apr 26, 2012 at 5:27 PM, Thinus Prinsloo
thinus.prinsloo@gmail.comwrote:

Hey community...

I've been struggling with the Date type in Elasticsearch. Let me first
explain what I need, then I'll explain to you how I'm trying to get there...

I have a field in ES, configured as (copied from the output of the
head-plugin):

myDate: {

  • index: analyzed
  • store: yes
  • format: dateOptionalTime
  • type: date

}

What I'd like to be able to do is filter (RangeFilter on SearchRequest)
documents based on a specified Date-range. The dates I have is in the
standard Java Date format (I'm using the Java API).

The line I use to populate this field is as follows:

...jsonBuilder().startObject().field("myDate", dateObject).endObject()...

Here, dateObject is of the standard Java Date type. To take from this is
that I'm hoping that the API knows what to do with the Date-type object.
Alternatively, I can place a String-formatted date (but which format?) or
a Long value of the instant since epoch. I know not what is required
here...

When I query the database, I then try to extract the date value with:

Date dateObject = new Date((Long) fieldMap.get("myDate").getValue()); //
effectively GetField.getValue()

I noticed that the type in the map (fieldMap is a Map<String, GetField>)
is Long, hence the type-casting.

With this implementation the filter on the search-query does not work. I
can get the date from the returned result, but I can't get the query to
work. I suspect the format of the query is not correct. Here is how I set
that up:

RangeFilterBuilder dateRangeFilter = FilterBuilders.rangeFilter("myDate");
dateRangeFilter.from(dateObject.getTime());
dateRangeFilter.to(dateObject.getTime());

As you may or may not know, the getTime() method on a Java Date object
returnes the time since epoch in millies. I have also tried this with the
numeric range filter.

My question then, in three parts:

  1. What is the type to use when saving the value through the API
    (setting the field)?
  2. What is the variable type I can expect the get from the GetField?
  3. What is the type I need to use when setting the ranges in the
    RangeFilter? (and should I use RangeFilter, or NumericRangeFilter)

Thank you, once again.

Kind regards,
Thinus