Tried making a runtime script to modify field, but now visualize with the index shows all empty fields

Elastic Stack Kibana
1

I have an index with 130 fields. One field I would like to change ranges over longs: 0, 1, 2.

In order to do this, I added a runtime field with the following description:

def names = ['0': 'Other', '1':'Friendly', '2': 'Opposing'];

emit(doc['attributes.forceId'].size() > 0 ? names[doc['attributes.forceId'].value.toString()] : null);

But I now the index is full of empty fields, with the correct time range, whereas before these fields were full. attributes.forceId has some null data but is filled with 0,1, and 2's. Not sure why the data isn't showing up after adding the runtime field described above.

Error is:

"Cannot invoke \"String.length()\" because \"v\" is null"
2

A runtime field mapped as a keyword cannot emit a null value.

You need to return a string value instead of null in your ternary operator.

My test:

# Cleanup
DELETE discuss-338708

# Create an index
PUT discus-338708
{
  "mappings": {
    "properties": {
      "ts": {"type": "date"},
      "forceId": { "type": "keyword"}
    }
  }
}

# Add some data
POST discuss-338708/_bulk
{ "index": {}}
{ "ts": "2023-07-19T12:42:55+0200", "forceId": 0}
{ "index": {}}
{ "ts": "2023-07-19T12:22:55+0200", "forceId": 0}
{ "index": {}}
{ "ts": "2023-07-19T12:12:55+0200", "forceId": 1}
{ "index": {}}
{ "ts": "2023-07-19T12:02:55+0200", "forceId": 1}
{ "index": {}}
{ "ts": "2023-07-19T11:52:55+0200", "forceId": 2}
{ "index": {}}
{ "ts": "2023-07-19T11:42:55+0200", "forceId": 2}
{ "index": {}}
{ "ts": "2023-07-19T11:32:55+0200"}
{ "index": {}}
{ "ts": "2023-07-19T11:22:55+0200"}


# Search with a runtime field
GET discuss-338708/_search
{
  "_source": false, 
  "fields": [
    "ts","forceId","mapped"
  ], 
  "runtime_mappings": {
    "mapped": {
      "type": "keyword",
      "script": {
        "source": """
def names = ['0': 'Other', '1':'Friendly', '2': 'Opposing'];
emit(doc['forceId'].size() > 0 ? names[doc['forceId'].value.toString()] : "N/A");
        """
      }
    }
  }
}
4

I am still getting the error 'cannot invoke string.length() because "v" is null'. I do not understand why, as I am outputting a string in my ternary operator.

5

Can you share a reproducible test as I did to check your script and mappings?

6

Hi Jsanz, I'm sorry but I don't really understand where you run the code you showed above. I'm using Kibana. The data I'm being sent is logs generated by other containers that eventually makes its way to Kibana, so I don't understand how to reproduce your test.

Is there a different engine, like at the command line with a logstash build, where you're executing the above?

7

Sorry, I was not explicit about that. The code shared above is expected to be run from the Kibana Console in the Dev Tools section of the Management block in the Kibana sidebar. That application is independent of Logstash.

image
image385×569 23.9 KB

The code I shared creates an index, then insert some documents using the Bulk API, to finally execute the runtime field directly on a search request.

That script, of course, can later be added in the index mapping, or added to the Kibana Data View, depending on your needs.

8

Hey, I'm actually not sure of the problem, as from what I can see in discover, all of the forceID's are 0's, 1's, or 2's; but I'm solving this problem by translating the forceID in a logger that preprocesses the data being sent to kibana.

Thanks for your help though! I'm sure I'll have more questions soon enough.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.