ISO8601 Durations

Elastic Stack Elasticsearch
1

I would like to use ISO8601 style duration values in my indexed documents, but ES does not support it natively. Is there a way to use custom analyzers or some other approach to convert these time periods to milliseconds on indexing and querying?

1 Like
2

I'm reasonably sure you can write a plugin that adds a new datatype for
that. Or you can convert on your side which is probably less work unless
you are particularly used to Elasticsearch.

I don't think durations like that map to milliseconds though. You'd need to
know the start dates and leap year rules for that. Maybe leap second rules
if you are being super careful.

3

Durations map ok to numerical values. What you are referring to is a time interval, which has the implication of a date component.

I don't see any documentation on how to achieve the type of mapping I want. I did review the docs on custom transforms, which is promising from the indexing point of view, but would not help me when it comes time to query.

Perhaps it doesn't make sense to query for a duration using ISO8601 formats, even though it might make sense to index in that way.

4

I'm pretty sure they don't but that isn't really the issue here.

Transform isn't what you want at all - its move like copy_to on steroids. If you could do this you'd do it by writing a plugin. But now that I dug around a little while I couldn't find out how plugins could register new types. So you might be out of luck.

5

I also have to store ISO8601 durations.
Did you build your own plugin? Is this feature available in a current version of elasticsearch without any plugin?

EDIT: As a eorkaround I just convert the ISO8601 durations to seconds.
When I read values from elasticsearch, I convert them back from seconds to ISO8601. But It would be great if elasticsearch would support ISO8601 anyway.

6

We didn't write a plugin, we convert before write. I understand that I t's difficult for Elasticsearch to introduce a way of supporting higher level types because of Lucene dependencies, but it would be nice to have a clean way of converting a higher level construct into a number, Boolean, date or string value.