# Microsecond dates?

**URL:** <https://discuss.elastic.co/t/microsecond-dates/8952>\
**Category:** Elasticsearch\
**Created:** [September 7, 2012, 11:34am UTC](https://discuss.elastic.co/t/microsecond-dates/8952 "2012-09-07T11:34:27Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Mark\_Hinkle](https://avatars.discourse-cdn.com/v4/letter/m/9fc29f/32.png) [@Mark\_Hinkle](https://discuss.elastic.co/u/Mark_Hinkle)\
**Post date:** [September 7, 2012, 11:34am UTC](https://discuss.elastic.co/t/microsecond-dates/8952/1 "2012-09-07T11:34:27Z")

</div>

Question regarding "date" type precision - is milliseconds the highest date  
resolution? This post  
([http://elasticsearch-users.115913.n3.nabble.com/Sorting-on-a-date-field-td1873287.html#none](http://elasticsearch-users.115913.n3.nabble.com/Sorting-on-a-date-field-td1873287.html#none))  
says "The `date` type in elasticsearch actually indexes a long value,  
parsing the relevant date to a long. The resolution of the long value is  
based on the date string passed." However, this gist  
([https://gist.github.com/3665247](https://gist.github.com/3665247)) seems to indicate a millisecond limit -  
even when specifically setting the date format with microseconds as shown,  
the results are out of order and the long shown in the "sort" result is  
short a few digits. I know that I can stuff the whole thing into a long as  
microseconds since epoch or stuff seconds into an additional float property  
and simply "sort":{"time":{"order":"desc"},"seconds":{"order":"desc"}} to  
get microsecond date ordering in the results but I am hoping someone will  
say "You're doing it wrong." Thanks in advance.

--

---

<div class="post-metadata">

**Author:** ![phill](https://avatars.discourse-cdn.com/v4/letter/p/779978/32.png) [@phill](https://discuss.elastic.co/u/phill)\
**Post date:** [September 7, 2012, 3:46pm UTC](https://discuss.elastic.co/t/microsecond-dates/8952/2 "2012-09-07T15:46:10Z")

</div>

I'm pretty sure the long in question is the long stored in a Java.util.Date

[http://docs.oracle.com/javase/6/docs/api/java/util/Date.html](http://docs.oracle.com/javase/6/docs/api/java/util/Date.html)  
"The class |Date| represents a specific instant in time, with  
millisecond precision."  
"the number of milliseconds since January 1, 1970, 00:00:00 GMT  
represented by this date." Just like in Unix systems, but in a 64 bit  
long instead of a traditional int (32 bit).

One problem is that dates created when asking the OS for a millisecond  
value at the moment often doesn't have millisecond precision since some  
system clocks and the system call to get that value will only respond at  
an interval larger than 1 millisecond.

The value formatted for output in the results is probably NOT the long  
value used for sorting;  
Hopefully the code uses the actual long value, but returns a friendly  
string -- a string that happens to be missing a few digits.

If you really have a source of higher precision values you should store  
them in something else, but I would use some combination of integers not  
a float.

-Paul

On 9/7/2012 4:34 AM, Mark Hinkle wrote:

> Question regarding "date" type precision - is milliseconds the highest  
> date resolution? This post  
> ([http://elasticsearch-users.115913.n3.nabble.com/Sorting-on-a-date-field-td1873287.html#none](http://elasticsearch-users.115913.n3.nabble.com/Sorting-on-a-date-field-td1873287.html#none))  
> says "The `date` type in elasticsearch actually indexes a long value,  
> parsing the relevant date to a long. The resolution of the long value  
> is based on the date string passed." However, this gist  
> ([microsecond date format test · GitHub](https://gist.github.com/3665247)) seems to indicate a millisecond  
> limit - even when specifically setting the date format with  
> microseconds as shown, the results are out of order and the long shown  
> in the "sort" result is short a few digits. I know that I can stuff  
> the whole thing into a long as microseconds since epoch or stuff  
> seconds into an additional float property and simply  
> "sort":{"time":{"order":"desc"},"seconds":{"order":"desc"}} to get  
> microsecond date ordering in the results but I am hoping someone will  
> say "You're doing it wrong." Thanks in advance.

--

---

<div class="post-metadata">

**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\
**Post date:** [September 7, 2012, 8:57pm UTC](https://discuss.elastic.co/t/microsecond-dates/8952/3 "2012-09-07T20:57:04Z")

</div>

Hi Mark,

no matter what you do, when you index a date, in the end you store a long  
in the Lucene index, representing the UTC date in millisecond resolution.  
More info at

> **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)**
>
> Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today.

If you want to use float, be aware of the explanations of FloatField at  
[http://lucene.apache.org/core/4\_0\_0-BETA/core/org/apache/lucene/document/FloatField.html](http://lucene.apache.org/core/4_0_0-BETA/core/org/apache/lucene/document/FloatField.html)

Hint: set precision\_step to a very large value (Integer.MAX) if you only  
sort the date values, but never do range queries. It will take less space  
on heap while sorting.

Best regards,

Jörg

On Friday, September 7, 2012 1:34:27 PM UTC+2, Mark Hinkle wrote:

> Question regarding "date" type precision - is milliseconds the highest  
> date resolution? This post (  
> [http://elasticsearch-users.115913.n3.nabble.com/Sorting-on-a-date-field-td1873287.html#none](http://elasticsearch-users.115913.n3.nabble.com/Sorting-on-a-date-field-td1873287.html#none))  
> says "The `date` type in elasticsearch actually indexes a long value,  
> parsing the relevant date to a long. The resolution of the long value is  
> based on the date string passed." However, this gist (  
> [microsecond date format test · GitHub](https://gist.github.com/3665247)) seems to indicate a millisecond limit -  
> even when specifically setting the date format with microseconds as shown,  
> the results are out of order and the long shown in the "sort" result is  
> short a few digits. I know that I can stuff the whole thing into a long as  
> microseconds since epoch or stuff seconds into an additional float property  
> and simply "sort":{"time":{"order":"desc"},"seconds":{"order":"desc"}} to  
> get microsecond date ordering in the results but I am hoping someone will  
> say "You're doing it wrong." Thanks in advance.

--

---

<div class="post-metadata">

**Author:** ![Mark\_Hinkle](https://avatars.discourse-cdn.com/v4/letter/m/9fc29f/32.png) [@Mark\_Hinkle](https://discuss.elastic.co/u/Mark_Hinkle)\
**Post date:** [September 7, 2012, 11:31pm UTC](https://discuss.elastic.co/t/microsecond-dates/8952/4 "2012-09-07T23:31:08Z")

</div>

Thank you both for your replies. Disappointing but not unexpected  
given the observable behavior. Since neither Shay's reply in the older  
forum thread I mentioned nor the docs page mentioned by Jörg qualify  
date precision to milliseconds, I had held out hope. 🙂

One might question the usefulness or clarity of allowing a date format  
pattern with more than 3 digits in precision when in reality those  
extra digits will never be useable as part of the date/time. On the  
other hand, in the (typical?) case where the time zone is at the end  
of the date, I can see the necessity of getting past those "extra"  
digits in an effort to reach the timezone.

I will take your warnings about the use of a float to heart and follow  
Paul's suggestion of just using integer for the fractional seconds.

Thanks again.

--  
Mark Hinkle

--

---

<div class="post-metadata">

**Author:** ![phill](https://avatars.discourse-cdn.com/v4/letter/p/779978/32.png) [@phill](https://discuss.elastic.co/u/phill)\
**Post date:** [September 8, 2012, 12:58am UTC](https://discuss.elastic.co/t/microsecond-dates/8952/5 "2012-09-08T00:58:01Z")

</div>

On 9/7/2012 4:31 PM, Mark Hinkle wrote:

> One might question the usefulness or clarity of allowing a date format  
> pattern with more than 3 digits in precision when in reality those  
> extra digits will never be useable as part of the date/time. On the  
> other hand, in the (typical?) case where the time zone is at the end  
> of the date, I can see the necessity of getting past those "extra"  
> digits in an effort to reach the timezone.

Again the behavior is inherited from Java library classes. The  
SimpleDateFormat strings allow you to specify 1 or more capital "S"s.  
1,2 or 3 "S"s can parse and generate milliseconds producing a value  
without trailing zeros.

Curiously, simple date format given TOO MANY "S"s after the decimal  
actually will parse the larger value after the dot as total millis (4  
digits would mean whole seconds!) and add ALL OF THEM TO THE TIME  
INCREASING THE LARGER VALUES like seconds and minute etc.

When formatting using a format with TOO MANY "S"s it will format the  
value with leading zeros which does not look like millis, but will  
reparse correctly.

I leave the following code for you on Friday night when I should already  
be gone from the office. I have been using Java since version 0.8  
(1995) and never knew what would happen if you put more than 3 "S"s  
after the dot. Yes, I might question the usefulness too, but I wouldn't  
bother to try to get Oracle to fix it now.  
It ain't Shays's fault! 🙂

By the way, do you really have better than millisecond precision?

-Paul

public class Foo {

```
 static DateFormat sdf1 = new SimpleDateFormat("yyyy-MM-dd G 'at' HH:mm:ss.S z"); // just one S

 static DateFormat sdf2 = new SimpleDateFormat("yyyy-MM-dd G 'at' HH:mm:ss.SSSS z"); // an extra "S"

 public static void main(String[] args) throws Exception {

     System.out.println( sdf1.format(sdf1.parse("2012-09-07 AD at 05:14:23.9 PDT") )); //2012-09-07 AD at 05:14:23.9 PDT

     System.out.println( sdf1.format(sdf1.parse("2012-09-07 AD at 05:14:23.98 PDT") )); //2012-09-07 AD at 05:14:23.98 PDT

     System.out.println( sdf1.format(sdf1.parse("2012-09-07 AD at 05:14:23.987 PDT") )); //2012-09-07 AD at 05:14:23.987 PDT

     System.out.println( sdf1.format(sdf1.parse("2012-09-07 AD at 05:14:23.9876 PDT") )); //2012-09-07 AD at 05:14:32.876 PDT <-- added 9 seconds!

     

     System.out.println( sdf1.format(sdf2.parse("2012-09-07 AD at 05:14:23.9 PDT") )); // 2012-09-07 AD at 05:14:23.9 PDT

     System.out.println( sdf1.format(sdf2.parse("2012-09-07 AD at 05:14:23.98 PDT") )); // 2012-09-07 AD at 05:14:23.98 PDT

     System.out.println( sdf1.format(sdf2.parse("2012-09-07 AD at 05:14:23.987 PDT") )); // 2012-09-07 AD at 05:14:23.987 PDT

     System.out.println( sdf1.format(sdf2.parse("2012-09-07 AD at 05:14:23.9876 PDT") )); // 2012-09-07 AD at 05:14:32.876 PDT <-- added 9 seconds

     System.out.println( sdf1.format(sdf2.parse("2012-09-07 AD at 05:14:23.98765 PDT") )); // 2012-09-07 AD at 05:16:01.765 PDT <-- added 98 seconds

     System.out.println( sdf2.format(sdf1.parse("2012-09-07 AD at 05:14:23.9 PDT") )); // 2012-09-07 AD at 05:14:23.0009 PDT <-- formatted with misleading zeros

     System.out.println( sdf2.format(sdf1.parse("2012-09-07 AD at 05:14:23.98 PDT") )); // 2012-09-07 AD at 05:14:23.0098 PDT <-- formatted with misleading zeros

     System.out.println( sdf2.format(sdf1.parse("2012-09-07 AD at 05:14:23.987 PDT") )); // 2012-09-07 AD at 05:14:23.0987 PDT <-- formatted with misleading zeros

 }

```

--

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 3:13am UTC](https://discuss.elastic.co/t/microsecond-dates/8952/6 "2017-07-06T03:13:36Z")

</div>