Unexpected type's in Painless update script

Elastic Stack Elasticsearch
1

I'm using a Painless update-script in conjunction with some entity-centric aggregation scripts as recommended here.

The code isn't working the way I expect it to. In this example the @timestamp is a Date, with the following mapping:

  "@timestamp": {
    "type": "date"
  }

When I launch my script for the first time to update documents, it run's perfectly and creates my aggregated documents.

If I launch it a second time to grab new events and update the documents with new events, I get an error because one of the Date fields is now recognized as a String.

Here is a simplified version of the code with some comments inline about where I see issues:

//Copy doc source to local variable with shorter name
def docSrc = ctx._source;

// Initialize if new document
if("create".equals(ctx.op)){
  docSrc.eventCount = 1;
  docSrc.duplicateEventsSkipped = 0;
}

// Consolidate latest batch of events
for (event in params.events) {
   // @timestamp is a Date, but event['@timestamp'].date gives this error:
   // "Unable to find dynamic field [date] for class [java.lang.String]."
  def timestamp = Date.from( Instant.parse( event['@timestamp'] ));

  // Skip duplicate events.
  // This code run's fine for the first iteration, the second gives this error:
  // "Unable to find dynamic method [getTime] with [0] arguments for class [java.lang.String]."
  if (docSrc.activity != null && timestamp.getTime() < docSrc.activity.getTime()) {
    docSrc.duplicateEventsSkipped ++;
    continue;
  }
}
2

Found a similar question here: Painless dates in update script

I've rewritten my script to cache any Date fields when they are initialized/updated and assume that they are String otherwise.

I'm really hoping there is an alternative to this, but in case this is the expected behavior here is one solution for anyone else who comes across this problem:

//Copy doc source to local variable with shorter name
def docSrc = ctx._source;

// Initialize if new document
if("create".equals(ctx.op)){
  docSrc.eventCount = 1;
  docSrc.duplicateEventsSkipped = 0;
}

// Cache date fields.
def activityInstant = null;
if (docSrc.activity != null) {
  activityInstant = Instant.parse(docSrc.activity);
}

def otherDateField = null;
if (docSrc.otherDateField != null) {
  otherDateField = Instant.parse(docSrc.activity);
}

// Consolidate latest batch of events
for (event in params.events) {
  def timestamp = Instant.parse( event['@timestamp'] );

  // Skip duplicate
  if (activityInstant != null && timestamp.isBefore(activityInstant)) {
    docSrc.duplicateEventsSkipped ++;
    continue;
  }

  docSrc.eventCount ++; 
  docSrc.activity = Date.from(timestamp);
  docSrc.otherDateField = Date.from(Instant.parse(event['otherDateField']))
}                                                                                                                                   

// Compute any additional data using the cached dates rather than the values updated in ctx._source.
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.