Duplicate : Scripted metric not working inside transform

Elastic Stack Kibana
,
1

First of all, I'd like to clarify that the reason I'm posting this question because I am unable to add any replies or edits to the original one.
I'm finding myself unable to post questions as well using that account(error - You are not permitted to view the requested resource.). So I resorted to using another account I created previously.

Hi, I have an index which contains events as documents and I'm trying to aggregate them by sessionId. I created the following transform and so far I have been able to generate the startTime, endTime, and sessionDuration, along with term aggregation for the event field using the following JSON -

{
  "group_by": {
    "sessionId": {
      "terms": {
        "field": "sessionId"
      }
    }
  },
  "aggregations": {
    "events": {
      "terms": {
        "field": "event",
        "size": 10
      }
    },
    "endTime": {
      "max": {
        "field": "timestamp"
      }
    },
    "startTime": {
      "min": {
        "field": "timestamp"
      }
    },
    "sessionDuration": {
      "bucket_script": {
        "buckets_path": {
          "start": "startTime",
          "end": "endTime"
        },
        "script": "((params.end - params.start)/1000)"
      }
    }
  }
}

But here I'm only able to get term aggregation of events. But I also want to know exactly what events were fired in a session, and their order.
So, I created the following search query to get a list of all the events in an index - which returns all the events for the entire index as a list, which is exactly what I need.

POST test30/_search?size=0
{
  "query": {
    "match_all": {}
  },
  "aggs": {
    "eventFlow": {
      "scripted_metric": {
        "init_script": "state.allEvents = [];", 
        "map_script": "state.allEvents.add(doc.event.value)",
        "combine_script": "return state.allEvents",
        "reduce_script": "List newAllEvents = new ArrayList(); for (a in states){ newAllEvents.add(a)} return newAllEvents"
      }
    }
  }
}

So I added the above scripted metric query to the transform like this -

{
  "group_by": {
    "sessionId": {
      "terms": {
        "field": "sessionId"
      }
    }
  },
  "aggregations": {
    "events": {
      "terms": {
        "field": "event",
        "size": 10
      }
    },
    "endTime": {
      "max": {
        "field": "timestamp"
      }
    },
    "startTime": {
      "min": {
        "field": "timestamp"
      }
    },
    "sessionDuration": {
      "bucket_script": {
        "buckets_path": {
          "start": "startTime",
          "end": "endTime"
        },
        "script": "((params.end - params.start)/1000)"
      }
    },
    "eventFlow": {
      "scripted_metric": {
        "init_script": "state.allEvents = [];",
        "map_script": "state.allEvents.add(doc.event.value)",
        "combine_script": "return state.allEvents",
        "reduce_script": "List newAllEvents = new ArrayList(); for (a in states){ newAllEvents.add(a)} return newAllEvents"
      }
    }
  }
}

But I don't see any new field created in the preview section of the transform. Am I missing something here?

here is the events data for an example session to try and replicate this issue

{'event': 'landed', 'website': 'rate.medicarefaq.com', 'elementType': '', 'elementId': '', 'path': '/ms', 'secondsElapsed': 17, 'userId': 'Y1VTSL6', 'timestamp': '2022-10-09T16:23:28.699236', 'eventId': '520M255', 'sessionId': '6DTD37W'}
{'elementType': 'button', 'elementId': 'get-free-quotes-now-bottom', 'event': 'click', 'website': 'rate.medicarefaq.com', 'path': '/ms', 'secondsElapsed': 10, 'userId': 'Y1VTSL6', 'timestamp': '2022-10-09T16:23:28.699281', 'eventId': 'R1DWJ61', 'sessionId': '6DTD37W'}
{'elementType': 'text-input', 'elementId': 'zip-code-input-top', 'event': 'text-input-entered', 'website': 'rate.medicarefaq.com', 'path': '/ms', 'secondsElapsed': 18, 'userId': 'Y1VTSL6', 'timestamp': '2022-10-09T16:23:28.699295', 'eventId': '1X7DN5L', 'sessionId': '6DTD37W'}
{'elementType': 'drop-down-input', 'elementId': 'country-input-bottom', 'event': 'drop-down-input-entered', 'website': 'rate.medicarefaq.com', 'path': '/ms', 'secondsElapsed': 14, 'userId': 'Y1VTSL6', 'timestamp': '2022-10-09T16:23:28.699306', 'eventId': 'YLG8FPX', 'sessionId': '6DTD37W'}
{'elementType': 'button', 'elementId': 'get-free-quotes-now-top', 'event': 'click', 'website': 'rate.medicarefaq.com', 'path': '/ms', 'secondsElapsed': 19, 'userId': 'Y1VTSL6', 'timestamp': '2022-10-09T16:23:28.699316', 'eventId': '520SIF7', 'sessionId': '6DTD37W'}
{'elementType': 'text-input', 'elementId': 'first-name-bottom', 'event': 'text-input-entered', 'website': 'rate.medicarefaq.com', 'path': '/ms', 'secondsElapsed': 13, 'userId': 'Y1VTSL6', 'timestamp': '2022-10-09T16:23:28.699551', 'eventId': 'OB586GD', 'sessionId': '6DTD37W'}
{'elementType': 'button', 'elementId': 'footer-compare-rates-button', 'event': 'click', 'website': 'rate.medicarefaq.com', 'path': '/ms', 'secondsElapsed': 13, 'userId': 'Y1VTSL6', 'timestamp': '2022-10-09T16:23:28.699586', 'eventId': 'YGP5KGT', 'sessionId': '6DTD37W'}
{'elementType': 'text-input', 'elementId': 'first-name-bottom', 'event': 'text-input-entered', 'website': 'rate.medicarefaq.com', 'path': '/ms', 'secondsElapsed': 11, 'userId': 'Y1VTSL6', 'timestamp': '2022-10-09T16:23:28.699600', 'eventId': '7CAJDYV', 'sessionId': '6DTD37W'}
{'elementType': 'text-input', 'elementId': 'first-name-bottom', 'event': 'text-input-entered', 'website': 'rate.medicarefaq.com', 'path': '/ms', 'secondsElapsed': 11, 'userId': 'Y1VTSL6', 'timestamp': '2022-10-09T16:23:28.699611', 'eventId': 'LJKY6EK', 'sessionId': '6DTD37W'}
{'event': 'idle', 'idleTimeInSeconds': '60', 'website': 'rate.medicarefaq.com', 'path': '/ms', 'secondsElapsed': 17, 'elementType': '', 'elementId': '', 'userId': 'Y1VTSL6', 'timestamp': '2022-10-09T16:23:28.701535', 'eventId': 'A7X1FIQ', 'sessionId': '6DTD37W'}
{'elementType': 'drop-down-input', 'elementId': 'country-input-bottom', 'event': 'drop-down-input-entered', 'website': 'rate.medicarefaq.com', 'path': '/ms', 'secondsElapsed': 16, 'userId': 'Y1VTSL6', 'timestamp': '2022-10-09T16:23:28.701606', 'eventId': 'YBCFAPT', 'sessionId': '6DTD37W'}
{'elementType': 'text-input', 'elementId': 'first-name-bottom', 'event': 'text-input-entered', 'website': 'rate.medicarefaq.com', 'path': '/ms', 'secondsElapsed': 19, 'userId': 'Y1VTSL6', 'timestamp': '2022-10-09T16:23:28.701623', 'eventId': '5LCJ6JT', 'sessionId': '6DTD37W'}
{'event': 'exited', 'website': 'rate.medicarefaq.com', 'path': '/ms', 'secondsElapsed': 11, 'elementType': '', 'elementId': '', 'userId': 'Y1VTSL6', 'timestamp': '2022-10-09T16:23:28.701635', 'eventId': 'WG1YGHK', 'sessionId': '6DTD37W'}
2

There's no reason that should be, are you getting an error or something?

3

No, i'm just not able to see the edit button for it. I think it may an issue with the question in particular since I can't reply to it even from my second account - the reply button just isn't there.
The only error I got was when I tried posting another question from that account - You are not permitted to view the requested resource.

4

I am chasing down why this is happening.

5

Ok it's been fixed, so will close this :slight_smile:

1 Like