How to use scripted fields in vega/vega-lite visualizations?

[tonyskulk]

Simple question, but I could not find any documentation about that. On Discover view I can see scripted field values but on my vega visualizations I can not.

This is my current spec:

{
  $schema: "https://vega.github.io/schema/vega-lite/v2.json"
  mark: line
  data: {
    url: {
      %context%: true
      %timefield%: timestamp
      index: gaptrader.heartbeat
      body: {
        size: 10000,
        // _source: ['timestamp', 'lastSignal', 'lastSignalSeconds']
      }
    }
    format: { property: "hits.hits" }
  }
  transform: [
    {
      calculate: "toDate(datum._source['timestamp'])"
      as: "time"
    }
  ]
  encoding: {
    x: {
      field: time
      type: temporal
      axis: { title: false }
    }
    y: {
      field: lastSignalSeconds
      type: quantitative
      axis: { title: false }
    }
  }
}

and I want my (already created) scripted field lastSignalSeconds to be shown on my chart. I tried many things like

lastSignalSeconds
datum.lastSignalSeconds
_source.lastSignalSeconds
_source['lastSignalSeconds']

but nothing seems to work. What is the correct syntax accessing scripted fields?

[thomasneirynck]

hi @tonyskulk,

Scripted fields are a Kibana-concept and part of the Kibana index-pattern. When creating Vega-visualizations you are not actually querying through the index pattern-framework of Kibana. Rather, you are submitting a raw Elasticsearch query that bypasses the Kibana index-pattern configurations.

You will have to write the raw ES-query. So you will have to define the script-field again in that Elasticsearch-query (body property). For more info, see https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-script-fields.html.

[tonyskulk]

Thanks @thomasneirynck for clarification.
This helped alot.
My working spec is now looking like that:

data: {
  url: {
    %context%: true
    %timefield%: timestamp
    index: gaptrader.heartbeat
    body: {
      size: 10000,
      _source: ['timestamp', 'lastSignal'],
      script_fields : {
        lastSignalSeconds : {
          script : {
            lang: 'painless',
            source: (doc['timestamp'].value.getMillis() - doc['lastSignal'].value.getMillis()) / 1000
          }
        }
      }
    }
  }
  format: { property: "hits.hits" }
}

And in my encoding part I can access scripted fields by

fields.lastSignalSeconds