Kibana data view containing runtime composite fields

Elastic Stack Kibana
1

Kibana data views. Cool stuff and thank you for them!

For some stupid reason I cannot figure out how to use "composite" fields in the runtime mapping.
Consider the following simple example:

{
  "data_view": {
    "id": "log.enhanced",
    "name": "Data view with a runtime field",
    "timeFieldName": "@timestamp",
    "title": "logs-*",
    "runtimeFieldMap": {
      "foo_bar_baz": {
        "type": "keyword",
        "script": {
          "source": "emit('computed')"
        }
      }
    }
  }
}

So far, so good. The runtime field is accessible as foo_bar_baz in the data view and contains the expected value of computed.

However I'd need the runtime field be named as foo.bar.baz, with the proper hierarchy (consider I'd like to have other runtime fields such as foo.bar.boh, foo.bom.bee and so on).

So I tried the following (simply replacing _ with . in the field name):

{
  "data_view": {
    "id": "log.enhanced",
    "name": "Data view with a runtime field",
    "timeFieldName": "@timestamp",
    "title": "logs-*",
    "runtimeFieldMap": {
      "foo.bar.baz": {
        "type": "keyword",
        "script": {
          "source": "emit('computed')"
        }
      }
    }
  }
}

But now the runtime field foo.bar.baz in the data view is empty - it does not contain the runtime value.

I also tried to use composite type instead of keyword, following this example. But I could not get it working.

Any suggestions?

2

Hi maybe this helps, I could create a data view for the e-commerce sample data with a composite runtime field that maps a couple of keyword fields. No problem on having a dot in the field name as far as I can see


POST kbn:/api/data_views/data_view
{
  "data_view": {
    "title": "kibana_sample_data_ecommerce*",
    "name": "Kibana Sample Data Commerce with custom runtime field",
    "timeFieldName": "order_date",
    "runtimeFieldMap": {
      "composite.test": {
        "type": "composite",
        "script": {
          "source": """
          Map fields = new HashMap();
          fields.put('gender',doc['customer_gender'].value);
          fields.put('currency',doc['currency'].value);
          emit(fields);
        """
        },
        "fields": {
          "gender": {"type": "keyword"},
          "currency": {"type": "keyword"}
        }
      }
    }
  }
}
3

Thank you very much, @jsanz!
Your example is nice and clear.
The problem on my side was with the field mappings clash.

Anyway, would you mind adding your example to the official Elastic documentation [1]?
There is just one example for a composite field using grok but it does not help really when you'd like to use a script, like you did.

[1] Explore your data with runtime fields | Elasticsearch Guide [8.9] | Elastic

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.