Watcher, Convert timestap field in loop

[ddoroshenko (Dmitriy)]

Hi everyone,

I' d like to iterate through ctx.payload.hits.hits and convert @timestamp field from ISO8601 to milliseconds.

I have the following configuration:

"actions": {
  "log_hits": {
    "foreach": "ctx.payload.hits.hits",
    "max_iterations": 10,
    "transform": {
      "script": {
        "source": """
          String timestamp = ctx.payload['_source']['@timestamp'];
          ZonedDateTime datetime = ZonedDateTime.parse(timestamp);
          long milliSinceEpoch = datetime.toInstant().toEpochMilli();
          return ['epochtime':milliSinceEpoch];
        """,
        "lang": "painless"
      }
    },
    "logging": {
      "text": "{{ctx.payload.epochtime}}"
    }
  }

But in result I get an error

              "type" : "script_exception",
              "reason" : "runtime error",
              "script_stack" : [
                """timestamp = ctx.payload['_source']['@timestamp'];
          ZonedDateTime """,
                "                       ^---- HERE"
              ],
              "script" : " ...",
              "lang" : "painless",
              "position" : {
                "offset" : 41,
                "start" : 18,
                "end" : 92
              },
              "caused_by" : {
                "type" : "null_pointer_exception",
                "reason" : "cannot access method/field [normalizeIndex] from a null def reference"
              }
            }
          },
          "reason" : "Failed to transform payload"

Suppose I have a mistake somewhere in my loop.

[stu (Stuart Tettemer)]

From the error, it appears ctx.payload is null, which shouldn't be the case in the watcher transform context.

The normalizeIndex part of the message is code that tries to handle indices on def types that hold on array.

The transform can check for ctx.payload == null or, if you can narrow down the troublesome result, use Debug.explain(ctx.payload) to dump the contents of ctx.payload.

[ddoroshenko (Dmitriy)]

well, as I supposed, the problem was in incorrectly placed transform unit

this works

"transform": {
    "script": """
      def n = [];
      def hits = ctx.payload.hits.hits;
      for (hit in hits) {
        hit.epochtime = ZonedDateTime.parse(hit['_source']['@timestamp']).toInstant().toEpochMilli();
        n.add(hit);
      }
      return ['new':n]
    """
  },
  "actions": {
    "log_hits": {
      "foreach": "ctx.payload.new",
      "max_iterations": 10,
      "logging": {
        "text": "{{ctx.payload.epochtime}}"
      }
    }
  }