Extracting values from text messages and summing them in real time queries

Hi,

I have event description fields that look this way:

"User1 have executed 3 actions"
"User2 have executed 5 actions"
....

And then many unnstructured message descriptions around.

I'm trying to select only messages that have that pattern, and count the sum of actions listed in these messages.

The ingest pipeline is configured as :

"event_descr": {
      "type": "text"
    },

There is already billions of messages so I don't think I can change the index template.

I have tried using a match query on have AND executed AND actions, which returns the right messages, then using a grok pattern in a runtime mapping to return the actions nr:

"runtime_mappings": {
    "actions": {
      "type": "keyword",
      "script": """
        String actions=grok('%{USERNAME} %{WORD} %{WORD} %{NUMBER:actions:keyword} %{WORD}').extract(doc["event_descr"].value)?.actions;
        if (actions != null) emit(actions);
      """
    }
  },

However I get a nasty error:

"caused_by" : {
"type" : "illegal_argument_exception",
"reason" : "Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [message] in order to load field data by uninverting the inverted index. Note that this can use significant memory."
}
}

And I'm not even attempting to convert the value extracted to a number and sum it. :confused:

Is there any other way to do this given my constraints?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.