Get the difference between different format of datetime

Elastic Stack Elasticsearch
1

I have data with two date field like this

 "@buildTimestamp": "2020-12-01T18:55:34.895-0800",
 "@timestamp": "2020-12-02T03:02:16.723Z"

I'd like to use ingest pipeline script processor to get the difference.
How can I converse this two to the same format, then do the comparison?
I cannot get it right

I tried something like this

POST _ingest/pipeline/_simulate
{
  "pipeline": {
    "processors": [

      {
        "script": {
          "lang": "painless",
          "source": """
            def test;
            test = ChronoUnit.MILLIS.between(ZonedDateTime.parse(ctx['@buildTimestamp']), ZonedDateTime.parse(ctx['@timestamp']))/1000;
          """
        }
      }
    ]
  },
  "docs": [
    {
      "_index": "test-2020.12",
      "_type": "_doc",
      "_id": "ojlmIXYBFr9SJ001ii0C",
      "_source": {
        "@buildTimestamp": "2020-12-01T18:55:34.895-0800",
        "@timestamp": "2020-12-02T03:02:16.723Z"
      }
    }
  ]
}

but the error shows

{
  "docs" : [
    {
      "error" : {
        "root_cause" : [
          {
            "type" : "exception",
            "reason" : "java.lang.IllegalArgumentException: ScriptException[runtime error]; nested: DateTimeParseException[Text '2020-12-01T18:55:34.895-0800' could not be parsed, unparsed text found at index 26];",
            "header" : {
              "processor_type" : "script"
            }
          }
        ],
        "type" : "exception",
        "reason" : "java.lang.IllegalArgumentException: ScriptException[runtime error]; nested: DateTimeParseException[Text '2020-12-01T18:55:34.895-0800' could not be parsed, unparsed text found at index 26];",
        "caused_by" : {
          "type" : "illegal_argument_exception",
          "reason" : "ScriptException[runtime error]; nested: DateTimeParseException[Text '2020-12-01T18:55:34.895-0800' could not be parsed, unparsed text found at index 26];",
          "caused_by" : {
            "type" : "script_exception",
            "reason" : "runtime error",
            "script_stack" : [
              "java.base/java.time.format.DateTimeFormatter.parseResolved0(DateTimeFormatter.java:2052)",
              "java.base/java.time.format.DateTimeFormatter.parse(DateTimeFormatter.java:1951)",
              "java.base/java.time.ZonedDateTime.parse(ZonedDateTime.java:598)",
              "java.base/java.time.ZonedDateTime.parse(ZonedDateTime.java:583)",
              "test = ChronoUnit.MILLIS.between(ZonedDateTime.parse(ctx['@buildTimestamp']), ZonedDateTime.parse(ctx['@timestamp']))/1000;",
              "                                                        ^---- HERE"
            ],
            "script" : "            def test;\n            test = ChronoUnit.MILLIS.between(ZonedDateTime.parse(ctx['@buildTimestamp']), ZonedDateTime.parse(ctx['@timestamp']))/1000;",
            "lang" : "painless",
            "caused_by" : {
              "type" : "date_time_parse_exception",
              "reason" : "Text '2020-12-01T18:55:34.895-0800' could not be parsed, unparsed text found at index 26"
            }
          }
        },
        "header" : {
          "processor_type" : "script"
        }
      }
    }
  ]
}
2

Nevermind, I get it working
something like this

def buildts = ctx['@buildTimestamp'];
DateTimeFormatter dtf = DateTimeFormatter.ofPattern("[yyyyMMdd][yyyy-MM-dd][yyyy-DDD]['T'[HHmmss][HHmm][HH:mm:ss][HH:mm][.SSS][XXXX]");
ZonedDateTime zdt1 = ZonedDateTime.parse(buildts, dtf);
ZonedDateTime zdt2 = ZonedDateTime.parse(ctx['@timestamp']);
long differencInMillis = ChronoUnit.MILLIS.between(zdt1, zdt2);
ctx['build_duration'] = differencInMillis
1 Like
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.