Accessing Elasticsearch from Vega to create custom visualizations


I bet this problem will be easy for someone out there - but I can't see what I'm doing wrong. I have data in an Elasticsearch index called "plotting_test" that I'd like to plot with Vega. I'm looking at the blog entry for reference.

All of the documents in the index look like this:

  "_index": "plotting_test",
  "_type": "text",
  "_id": "CgNrA20BpOHJNpjkfGzX",
  "_version": 1,
  "_score": 0,
  "_source": {
    "time": "2019-08-14",
    "ds": "2019-08-14T00:00:00",
    "data": 0.4997,
    "yhat": 0.5541273960954143,
    "yhat_lower": 0.2884054750552231,
    "yhat_upper": 0.8083939855874178
  "fields": {
    "ds": [
  "time": [

I'm trying to graph x = ds and y = yhat. This is the Vega code I have so far:

  data: {
    url: {
      %context%: true
      %timefield%: "ds"
      index: plotting_test
      body: {
        size: 10000
        _source: ["ds", "yhat"]
    # We only need the content of hits.hits array
    format: {property: "hits.hits"}
  # Parse timestamp into a javascript date value
  transform: [
    {calculate: "toDate(datum._source['ds'])", as: "time"}
  # Draw a circle, with x being the time field, and y - number of bytes
  mark: circle
  encoding: {
    x: {field: "ds", type: "temporal"}
    y: {field: "yhat", type: "quantitative"}

Using the Kibana dev tool I can get the data no problem by simply writing

GET /plotting_test/_search
  "query": {
      "match_all": {}
  "_source":["yhat", "ds"]

I can't figure out what is wrong with the Vega code. If I can just get one example to work locally I can start doing all of my analytics plots in Vega.

Thank you!

I think your XY encoding is not matching any fields. Because of the document shape you showed earlier, you still need _source as a prefix. Also, I see that you are using the transform clause but not actually encoding it. Putting it together, here is what I think you could try:

  encoding: {
    x: {field: "time", type: "temporal"}
    y: {field: "_source.yhat", type: "quantitative"}

I only changed the X and Y encodings from your previous example.

1 Like

Thank you so much! It worked!


This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.