# How to plot (raw) data, Using Vega

**URL:** <https://discuss.elastic.co/t/how-to-plot-raw-data-using-vega/238259>\
**Category:** Kibana\
**Created:** [June 23, 2020, 12:43pm UTC](https://discuss.elastic.co/t/how-to-plot-raw-data-using-vega/238259 "2020-06-23T12:43:26Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![defalt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/defalt/32/71379_2.png) [@defalt](https://discuss.elastic.co/u/defalt)\
**Post date:** [June 23, 2020, 12:43pm UTC](https://discuss.elastic.co/t/how-to-plot-raw-data-using-vega/238259/1 "2020-06-23T12:43:26Z")

</div>

Hi,  
I want to plot my **raw time series data**. As far as I know this is not possible with normal visualizations. This is why I am currently looking into Vega-Lite. The data structure looks like this:

```auto
{
  "_index": "index_new",
  "_type": "index",
  "_id": "ABCD.....",
  "_version": 1,
  "_score": null,
  "_source": {
    "Value1": 56.60521,
    "Value2": 0,
    "Value3": 9.46315,
.
.
.
  },
  "fields": {
    "@timestamp": [
      "20XX-08-31T1X:XX:XX.759Z"
    ],
    "timestamp": [
      "20XX-08-31TXX:XX:XX.759Z"
    ]
  },
  "sort": [
    1267276399749
  ]
}

```

I can plot this with the following Vega-Lite script:

```auto
{
  $schema: "https://vega.github.io/schema/vega-lite/v2.json"
  mark: line
  data: {
    url: {
      %context%: true
      %timefield%: timestamp
      index: index_new
      body: {
        size: 10000, //This is used to plot the raw data, thats why I need Vega
        _source: ["timestamp", "Value1"]
      },
    }
    format: { property: "hits.hits" }
  }
    transform: [
    {
      calculate: "toDate(datum._source['timestamp'])"
      as: "time"
    }
  ]

  encoding: {
    x: {
      field: time
      type: temporal
      axis: { title: "Zeit" }
    }
    y: {
      field: _source.Value1
      type: quantitative
      // Specify the label for this axis
      axis: { title: "Value" }
    }
    color: {
      field: _source.extension
      type: nominal
      legend: { title: 'File type' }
    }
  }
}

```

Result:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/0/8/0803216278394a415bf88167f016b93ce96791ad.png)

It should look something like this:

 ![Graph](https://us1.discourse-cdn.com/elastic/original/3X/b/6/b64aec64959def17d285df40d82f5cce9dda88b4.png)

**My Questions:**

1. How can I draw different lines on the same diagram?
2. How can I change the timescale? I am in europe and don't use AM, PM.
3. How can I use different scales on the Y Axis for Value1, Value2 etc.?

I tried it with something like this:

```auto
  "mark": "line",
  "encoding": {
    "x": {"field": "Value1", "type": "temporal"},
    "y": {"field": "Value2", "type": "quantitative"},
    "color": {"field": "_source", "type": "nominal"}
  }
}

```

but it doesn't show anything. Found it here: [https://vega.github.io/vega-lite/examples/line\_color.html](https://vega.github.io/vega-lite/examples/line_color.html)

Can someone help with me with my Vega-Problems.  
Thanks,  
Defalt

---

<div class="post-metadata">

**Author:** ![defalt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/defalt/32/71379_2.png) [@defalt](https://discuss.elastic.co/u/defalt)\
**Post date:** [June 25, 2020, 8:10am UTC](https://discuss.elastic.co/t/how-to-plot-raw-data-using-vega/238259/3 "2020-06-25T08:10:29Z")

</div>

Bump

---

<div class="post-metadata">

**Author:** ![timroes](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/timroes/32/19712_2.png) [@timroes](https://discuss.elastic.co/u/timroes)\
**Post date:** [June 25, 2020, 11:57am UTC](https://discuss.elastic.co/t/how-to-plot-raw-data-using-vega/238259/4 "2020-06-25T11:57:05Z")

</div>

Hi @defalt,

I think for the complex chart you want, you won't be able to achieve that with _vega-lite_ anymore and will need to use _vega_ instead, e.g. I don't think vega-lite has a way to have more then two y-axis (one on the left, one on the right).

In Vega then:

> How can I draw different lines on the same diagram?

In Vega, you'll have `marks` which can be an array and you can add as many marks (e.g. all of type: line) that you want within the chart. That way you can add multiple lines, that are based of different points in the data, by referencing different fields for each mark.

I've put together a small example for you in the Vega editor, with an [example spec](https://vega.github.io/editor/#/url/vega/N4IgJAzgxgFgpgWwIYgFwhgF0wBwqgegIDc4BzJAOjIEtMYBXAI0poHsDp5kTykSArJQBWENgDsQAGhAATONABONHJnaT0AQQAETJBBpRtAGxri422EkWZtcAB5IEOY3ErSQAdxqz6aAQAMATLwNGRYaABMQTI4SLKyZmT+MrJImCioANqg4k5waCAZTK4exEjGDApoOSD2aACMAgCcAMwBABwCAOyRACyRHa0yAJ5RHTJQaK0AvlKg9ahNbZ09rQNDo2h9wyBTqAJzC40t7V3dfRu7Y6gdDZONkUd1JyvnAldbS80PqM3Pi2WZx6ADZPiAbgJIr8Gk95i8lqdVt0LoNrlE+jC4cdEW8eh1wTcOt1fpEGgDXsDus1CY0SXtGgEKbjgR0ArTUCCJgyluT4YCkV07hy+j8eQ0mQBdGaSmTQCrVbK5fKFeoyTAjHAFdBqBAFGSKJDiMjarw+PypNjIMxoUBpDKFYqlGQAMxocGMslVIBmzzyesKY3VmtNpnM1g8huNptC4UwHnEhm1mEUVRkAC84Io2GgU2m5FakDbUHb0pkikgSvqQG6PV70GNffD-aaRpEPBqtYUw3AIwajSbCrGIjJE1Bk6m4BmsznUHmpwXrRpSw6dZXnTX3Z7ClNfbKQEh7IramxlHBxPH0Ew2NgrR55aV0Gqa6fkJeQABSAASqA-CB98Knu6F7dnALrxnKUAKoGAGgEB57vq44H3lBj4Qu2MhsC6LoQHAl7tJaS4AMJsMYp6FAAxAAYlRQQ0R4xiVh6JFkYolE0XRVEdoYADWLHkeg1G0QE9EyjIyCKDxEA1KAnahmY1Yutm-4lnIZaOuuBRzCA55QGw8i2jpF5ZoZiygA+prPrW25PgBEKGRZMGulu9b2b6TaySG3YKR4Sl3qp9rlk6WkyLp+naqACEmapZkgI5tnOXW3raTc5moa2GGbkl6C7nKKZsDxEUgOUlSmkJnE+u50pAA).

You can see how I added different marks for each line there.

> How can I change the timescale? I am in europe and don't use AM, PM.

On each axis you can specify the `format` property (also included that in the example linked above), to specify a [time format string](https://github.com/d3/d3-time-format#locale_format), which can use `%H` to reference 24h format instead of `%I` which references 12 hour format.

> How can I use different scales on the Y Axis for Value1, Value2 etc.?

If you use different `scales` (see example), each of them can reference an own `domain` and therefore different scale that it will represent.

Cheers,  
Tim

---

<div class="post-metadata">

**Author:** ![defalt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/defalt/32/71379_2.png) [@defalt](https://discuss.elastic.co/u/defalt)\
**Post date:** [June 26, 2020, 8:44am UTC](https://discuss.elastic.co/t/how-to-plot-raw-data-using-vega/238259/5 "2020-06-26T08:44:53Z")

</div>

Thanks @timroes for your help yesterday. I tried to change your script so that I can import data from elasticsearch but it doesn't work. Do you know what I did wrong?

```auto
{
  "$schema": "https://vega.github.io/schema/vega/v4.3.json",
  "description": "A basic line chart example.",
  "padding": 5,
  
  data: {
  name:"table"
    url: {
      %context%: true
      %timefield%: timestamp
      index: modbusdata_new
      body: {
        size: 10000,
        _source: ["timestamp", "T107","C106"]
      },
    }
  }
    transform: [
    {
      calculate: "toDate(datum._source['timestamp'])"
      as: "time"
    }
  ],
  
  "scales": [
    {
      "name": "timex", //Changed the scale names so that I wouldnt confuse the y scale with the y value and so on
      "type": "time",
      "range": "width",
      "domain": {"data": "table", "field": "time"}
    },
    {
      "name": "NameT107",
      "type": "linear",
      "range": "height",
      "nice": true,
      "zero": true,
      "domain": {"data": "table", "field": "_source.T107"} // "_source.['T107']" or "T107" doesn't work either
    },
    {
      "name": "NameC107",
      "type": "linear",
      "range": "height",
      "nice": true,
      "zero": true,
      "domain": {"data": "table", "field": "_source.C106"}
    }
  ],
  "axes": [
    {"orient": "bottom", "scale": "timex", "format": "%H:%m"},
    {"orient": "left", "scale": "NameT107"},
    {
      "orient": "left",
      "scale": "NameC107",
      "offset": 30,
      "domainColor": "#FF00FF",
      "labelColor": "#FF00FF",
      "tickColor": "#FF00FF"
    }
  ],
  "marks": [
    {
      "type": "line",
      "from": {"data": "table"},
      "encode": {
        "enter": {
          "x": {"scale": "timex", "field": "time"},
          "y": {"scale": "NameT107", "field": "_source.T107"}
        }
      }
    },
    {
      "type": "line",
      "from": {"data": "table"},
      "encode": {
        "enter": {
          "x": {"scale": "timex", "field": "time"},
          "y": {"scale": "NameC107", "field": "_source.C106"},
          "stroke": {"value": "#FF00FF"}
        }
      }
    }
  ]
}

```

I get the error: `Cannot read property 'T107' of undefined`.  
If I change the line `"domain": {"data": "table", "field": "_source.T107"} `  
to `"domain": {"data": "table", "field": "Whatever"} ` the error changes to `Cannot read property 'Whatever' of undefined`.

The output of F12-\>Console as [requested](https://discuss.elastic.co/t/visualize-cannot-read-property-some-of-undefined/134374/4) is:

```auto
Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-P5polb1UreUSOe5V/Pv7tc+yeZuJXiOi/3fqhGsU7BE='), or a nonce ('nonce-...') is required to enable inline execution.

```

but below it it says: ` A single error about an inline script not firing due to content security policy is expected!`  
Haven't found a solution for this. Any ideas?  
Thanks,  
defalt.

---

<div class="post-metadata">

**Author:** ![Stratoula\_Kalafateli](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stratoula_kalafateli/32/70923_2.png) [@Stratoula\_Kalafateli](https://discuss.elastic.co/u/Stratoula_Kalafateli)\
**Post date:** [June 26, 2020, 10:56am UTC](https://discuss.elastic.co/t/how-to-plot-raw-data-using-vega/238259/6 "2020-06-26T10:56:28Z")

</div>

Hey @defalt. I think that the `format: { property: "hits.hits" }` is missing on `data` field. If you hit on the Dev Console of kibana the:

```auto
GET modbusdata_new/_search 
    {
      "size": 10000,
      "_source": ["timestamp", "T107","C106"]
}
```

You will get a response like that:

```auto
"hits" : {
    "total" : {
      "value" : 5,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "example",
        "_id" : "1",
        "_score" : 1.0,
        "_source" : {
          "timestamp" : "A",
          "T107" : 8,
          "C106": 9
        }
      },
```

The path to `_source` is `hits.hits` so you should also include it. Your data should be like:

```auto
  data: {
    url: {
      %context%: true
      %timefield%: timestamp
      index: kibana_sample_data_logs
      body: {
        size: 10000
        _source: ["timestamp", "T107","C106"]
      }
    }
    format: {
      property: hits.hits
    }
  }
```

So now you will be able to get the info as `_source.T107`

Hope it helps 🙂

Stratoula

---

<div class="post-metadata">

**Author:** ![defalt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/defalt/32/71379_2.png) [@defalt](https://discuss.elastic.co/u/defalt)\
**Post date:** [June 29, 2020, 6:42am UTC](https://discuss.elastic.co/t/how-to-plot-raw-data-using-vega/238259/7 "2020-06-29T06:42:38Z")

</div>

This helped a lot @Stratoula_Kalafateli . Now I can see data. But the problem is the data isn't drawn accross the X-Axis. Its drawn in a single line (All timestamps are in a single point I guess). I couldn't find a solution to this.

```auto
{
  "$schema": "https://vega.github.io/schema/vega/v4.3.json",
  "description": "A basic line chart example.",
  "padding": 5,
  
  data: {
  name:"table"
    url: {
      %context%: true
      %timefield%: timestamp
      index: modbusdata_new
      body: {
        "size": 10000,
        "_source": ["timestamp", "T107","C106"]
      },
    }
    format: { property: "hits.hits" }
  }
    transform: [
    {
      calculate: "toDate(datum._source['timestamp'])"
      as: "time"
    }
  ],
  
  "scales": [
    {
      "name": "timex",
      "type": "time",
      "range": "width",
      "domain": {"data": "table", "field": "time"}
    },
    {
      "name": "Namet",
      "type": "linear",
      "range": "height",
      "nice": true,
      "zero": true,
      "domain": {"data": "table", "field": "_source.T107"}
    },
    {
      "name": "Namec",
      "type": "linear",
      "range": "height",
      "nice": true,
      "zero": true,
      "domain": {"data": "table", "field": "_source.C106"}
    }
  ],
  "axes": [
    {"orient": "bottom", "scale": "timex", "format": "%d:%m:%H:%m"},
    {"orient": "left", "scale": "Namet"},
    {
      "orient": "left",
      "scale": "Namec",
      "offset": 30,
      "domainColor": "#FF00FF",
      "labelColor": "#FF00FF",
      "tickColor": "#FF00FF"
    }
  ],
  "marks": [
  {
      "type": "line",
      "from": {"data": "table"},
      "encode": {
        "enter": {
          "x": {"scale": "timex", "field": "time"},
          "y": {"scale": "Namec", "field": "_source.C106"},
          "stroke": {"value": "#FF00FF"}
        }
      }
    },
    {
      "type": "line",
      "from": {"data": "table"},
      "encode": {
        "enter": {
          "x": {"scale": "timex", "field": "time"},
          "y": {"scale": "Namet", "field": "_source.T107"}
        }
      }
    }
  ]
}

```

Output:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/0/c/0cde6ea0317a214afcd81558ac59e81330862331.png)

When I change the timescale the data changes too. So its some type of a time error.  
**Thanks** for your help.

---

<div class="post-metadata">

**Author:** ![Stratoula\_Kalafateli](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stratoula_kalafateli/32/70923_2.png) [@Stratoula\_Kalafateli](https://discuss.elastic.co/u/Stratoula_Kalafateli)\
**Post date:** [June 29, 2020, 8:14am UTC](https://discuss.elastic.co/t/how-to-plot-raw-data-using-vega/238259/8 "2020-06-29T08:14:49Z")

</div>

Hey @defalt! I assume that the problem here is that as we are using Vega now (instead of vega-lite) the "transform" pipeline now [should appear on `data`](https://vega.github.io/vega/docs/transforms/). You also need to replace "calculate" with the Vega ["formula"](https://vega.github.io/vega/docs/transforms/formula/).  
So it should be something like:

```auto
data: {
  name:"table"
    url: {
      %context%: true
      %timefield%: timestamp
      index: modbusdata_new
      body: {
        "size": 10000,
        "_source": ["timestamp", "T107","C106"]
      },
    }
    format: { property: "hits.hits" },
    transform: [
        { type: "formula",as: "time", expr: "datetime(datum._source.timestamp) "}
      ]
  }
```

Also check the supported date functions [here](https://vega.github.io/vega/docs/expressions/#datetime-functions).

---

<div class="post-metadata">

**Author:** ![defalt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/defalt/32/71379_2.png) [@defalt](https://discuss.elastic.co/u/defalt)\
**Post date:** [June 29, 2020, 12:22pm UTC](https://discuss.elastic.co/t/how-to-plot-raw-data-using-vega/238259/9 "2020-06-29T12:22:54Z")

</div>

Awesome, it works now. Is there a way to display data from different indices too? If I write `index: index1,index2` its really buggy and doesn't display the right things.  
Lets say I have Index 1 with T107 and C106 and Index 2 with O145. Is there a way to display them in the same graph?

---

<div class="post-metadata">

**Author:** ![Stratoula\_Kalafateli](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stratoula_kalafateli/32/70923_2.png) [@Stratoula\_Kalafateli](https://discuss.elastic.co/u/Stratoula_Kalafateli)\
**Post date:** [June 29, 2020, 1:08pm UTC](https://discuss.elastic.co/t/how-to-plot-raw-data-using-vega/238259/10 "2020-06-29T13:08:52Z")

</div>

Yes you can! `data` can be an array on vega so you could have:

```auto
data: [
{
  name:"table"
    url: {
      %context%: true
      %timefield%: timestamp
      index: modbusdata_new
      body: {
        "size": 10000,
        "_source": ["timestamp", "T107","C106"]
      },
    }
    format: { property: "hits.hits" },
    transform: [
        { type: "formula",as: "time", expr: "datetime(datum._source.timestamp) "}
      ]
  },
{
  name:"table2"
    url: {
      %context%: true
      %timefield%: timestamp
      index: my_second_index
      body: {
        "size": 10000,
        "_source": ["timestamp", "O145"]
      },
    }
    format: { property: "hits.hits" },
    transform: [
        { type: "formula",as: "time", expr: "datetime(datum._source.timestamp) "}
      ]
  }
]
```

Then you could add a new scale with domain the new dataset ` "domain": {"data": "table2", "field": "_source.O145"}`, a new axis and a new mark:

```auto
{
      "type": "line",
      "from": {"data": "table2"},
      "encode": {
        "enter": {
          "x": {"scale": "x", "field": "time"},
          "y": {"scale": "y2", "field": "_source.O145"},
          "stroke": {"value": "#FF00FF"}
        }
      }
    }
```

---

<div class="post-metadata">

**Author:** ![defalt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/defalt/32/71379_2.png) [@defalt](https://discuss.elastic.co/u/defalt)\
**Post date:** [June 30, 2020, 7:17am UTC](https://discuss.elastic.co/t/how-to-plot-raw-data-using-vega/238259/11 "2020-06-30T07:17:18Z")

</div>

Thanks for all your help @Stratoula_Kalafateli . Do you have an idea why my timing scale is weird? I don't think it displays correct data and it doesn't change with the timeframe set in kibana. It jumps from 11:05 to 12:05

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/a/1/a1ecd8a0f99b1c863be4304e8c4a2f02c13bf78c.png)

```auto
"axes": [
    {"orient": "bottom", "scale": "timex", "format": "%H:%m","grid":"true"},

```

---

<div class="post-metadata">

**Author:** ![Stratoula\_Kalafateli](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stratoula_kalafateli/32/70923_2.png) [@Stratoula\_Kalafateli](https://discuss.elastic.co/u/Stratoula_Kalafateli)\
**Post date:** [June 30, 2020, 7:24am UTC](https://discuss.elastic.co/t/how-to-plot-raw-data-using-vega/238259/12 "2020-06-30T07:24:46Z")

</div>

Hey @defalt 🙂 I can't imagine why this happens. Do you want to copy paste your configuration? Maybe it rings a bell.

---

<div class="post-metadata">

**Author:** ![defalt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/defalt/32/71379_2.png) [@defalt](https://discuss.elastic.co/u/defalt)\
**Post date:** [June 30, 2020, 7:38am UTC](https://discuss.elastic.co/t/how-to-plot-raw-data-using-vega/238259/13 "2020-06-30T07:38:53Z")

</div>

Config

```auto
{
  "$schema": "https://vega.github.io/schema/vega/v4.3.json",
  "description": "A basic line chart example.",
  "padding": 5,
  
data: [
{
  name:"table"
    url: {
      %context%: true
      %timefield%: @timestamp
      index: index1
      body: {
        "size": 17000,
        "_source": ["@timestamp", "value1","value2","value3"],
        "sort" : { "@timestamp" : "desc" }
      },
    }
    format: { property: "hits.hits" },
    transform: [
        { type: "formula",as: "time", expr: "datetime(datum._source['@timestamp']) "}
      ]
  },
{
  name:"table2"
    url: {
      %context%: true
      %timefield%: @timestamp
      index: index2
      body: {
        "size": 17000,
        "_source": ["@timestamp", "value4"],
        "sort" : { "@timestamp" : "desc" }
      },
    }
    format: { property: "hits.hits" },
    transform: [
        { type: "formula",as: "time2", expr: "datetime(datum._source['@timestamp']) "}
      ]
  }
]
  "scales": [
    {
      "name": "timex",
      "type": "time",
      "range": "width",
      "domain": {"data": "table", "field": "time"}
    },
    {
      "name": "Namet",
      "type": "linear",
      "range": "height",
      "nice": true,
      "zero": true,
      "domain": {"data": "table", "field": "_source.value1"}
    },
    {
      "name": "Namec",
      "type": "linear",
      "range": "height",
      "nice": true,
      "zero": true,
      "domain": {"data": "table", "field": "_source.value2"}
    },
    {
      "name": "Namevalue3",
      "type": "linear",
      "range": "height",
      "nice": true,
      "zero": true,
      "domain": {"data": "table", "field": "_source.value3"}
    },
    {
      "name": "NameValue4",
      "type": "linear",
      "range": "height",
      "nice": true,
      "zero": true,
      "domain": {"data": "table2", "field": "_source.value4"}
    }
  ],
  "axes": [
    {
    "orient": "bottom", 
    "scale": "timex", 
    "format": "%s",
    "grid":"true"},
    {
    "orient": "left", 
    "scale": "Namet",
    "grid":"true"},
    {
      "orient": "left",
      "scale": "Namec",
      "offset": 30,
      "domainColor": "#FF00FF",
      "labelColor": "#FF00FF",
      "tickColor": "#FF00FF",
      "grid":"true"
    },
    {
      "orient": "left",
      "scale": "NameValue4",
      "offset": 60,
      "domainColor": "#CCCCCC",
      "labelColor": "#CCCCCC",
      "tickColor": "#CCCCCC",
      "grid":"true"
    },
    {
      "orient": "left",
      "scale": "Namevalue3",
      "offset": 90,
      "domainColor": "#444444",
      "labelColor": "#444444",
      "tickColor": "#444444",
      "grid":"true"
    }
  ],
  "marks": [
  {
      "type": "line",
      "from": {"data": "table"},
      "encode": {
        "enter": {
          "x": {"scale": "timex", "field": "time"},
          "y": {"scale": "Namec", "field": "_source.value2"},
          "stroke": {"value": "#FF00FF"}
        }
      }
    },
    {
      "type": "line",
      "from": {"data": "table"},
      "encode": {
        "enter": {
          "x": {"scale": "timex", "field": "time"},
          "y": {"scale": "Namet", "field": "_source.value1"}
        }
      }
    },
    {
      "type": "line",
      "from": {"data": "table2"},
      "encode": {
        "enter": {
          "x": {"scale": "timex", "field": "time2"},
          "y": {
          "scale": "NameValue4",
          "field": "_source.value4"
          },
          "stroke": {"value": "#CCCCCC"}
        }
      }
    },
    {
      "type": "line",
      "from": {"data": "table"},
      "encode": {
        "enter": {
          "x": {"scale": "timex", "field": "time"},
          "y": {
          "scale": "Namevalue3", "field": "_source.value3"},
          "stroke": {"value": "#444444"}
        }
      }
    }
  ]
}

```

This is the scale in %s, seconds.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/9/a/9aa5f1cea1723f9a72ed0ab85c64b9adc3f3a4e4.png)  
According to the scale i woud insert data in nanoseconds. But thats not the case:  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/d/a/daadf251bea8fb667158f82564a56cb7daec64ce.png)

Here you can see what the timescale should look like.

 ![Screenshot_1](https://us1.discourse-cdn.com/elastic/original/3X/d/6/d68278bbb69896261e597d4a4f2bb03373eaa4d5.png)

---

<div class="post-metadata">

**Author:** ![Stratoula\_Kalafateli](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stratoula_kalafateli/32/70923_2.png) [@Stratoula\_Kalafateli](https://discuss.elastic.co/u/Stratoula_Kalafateli)\
**Post date:** [June 30, 2020, 7:58am UTC](https://discuss.elastic.co/t/how-to-plot-raw-data-using-vega/238259/14 "2020-06-30T07:58:08Z")

</div>

ok, I see that you set `time` and `time2`. I would use the same variable `time`.

So this is an example configuration with two lines (I have used the kibana sample datasets)

```auto
{
  "$schema": "https://vega.github.io/schema/vega/v4.3.json",
  "description": "A basic line chart example.",
  "padding": 5,
  "data": [
    {
  	// query ES based on the currently selected time range and filter string
  	name: table
  	url: {
    	%context%: true
    	%timefield%: timestamp
    	index: kibana_sample_data_flights
    	body: {
        "size": 10000,
        "_source": ["timestamp", "DistanceKilometers"],
        "sort" : { "timestamp" : "desc" }
      }
  	}
  	// From the result, take just the data we are interested in
  	format: {property: "hits.hits"}
  	transform: [
    	{type: "formula", expr: "datetime(datum._source.timestamp)", as: "time"}
  	]
	},
	{
  	// query ES based on the currently selected time range and filter string
  	name: table2
  	url: {
    	%context%: true
    	%timefield%: timestamp
    	index: kibana_sample_data_logs
    	body: {
        "size": 10000,
        "_source": ["timestamp", "bytes"],
         "sort" : { "timestamp" : "desc" }
      }
  	}
  	// From the result, take just the data we are interested in
  	format: {property: "hits.hits"}
  	transform: [
    	{type: "formula", expr: "datetime(datum._source.timestamp)", as: "time"}
  	]
	}
  ],
  "scales": [
    {
      "name": "x",
      "type": "time",
      "range": "width",
      "domain": {"data": "table", "field": "time"}
    },
    {
      "name": "y",
      "type": "linear",
      "range": "height",
      "nice": true,
      "zero": true,
      "domain": {"data": "table", "field": "_source.DistanceKilometers"}
    },
    {
      "name": "y2",
      "type": "linear",
      "range": "height",
      "nice": true,
      "zero": true,
      "domain": {"data": "table2", "field": "_source.bytes"}
    }
  ],
  "axes": [
    {"orient": "bottom", "scale": "x", "grid": true, "format": "%s",},
    {"orient": "left", "scale": "y"},
    {
      "orient": "left",
      "scale": "y2",
      "offset": 50,
      "domainColor": "#FF00FF",
      "labelColor": "#FF00FF",
      "tickColor": "#FF00FF"
    }
  ],
  "marks": [
    {
      "type": "line",
      "from": {"data": "table"},
      "encode": {
        "enter": {
          "x": {"scale": "x", "field": "time"},
          "y": {"scale": "y", "field": "_source.DistanceKilometers"}
        }
      }
    },
    {
      "type": "line",
      "from": {"data": "table2"},
      "encode": {
        "enter": {
          "x": {"scale": "x", "field": "time"},
          "y": {"scale": "y2", "field": "_source.bytes"},
          "stroke": {"value": "#FF00FF"}
        }
      }
    }
  ]
}
```

The above example changes with the kibana timepicker and the time scale seems to work properly.

---

<div class="post-metadata">

**Author:** ![defalt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/defalt/32/71379_2.png) [@defalt](https://discuss.elastic.co/u/defalt)\
**Post date:** [June 30, 2020, 8:08am UTC](https://discuss.elastic.co/t/how-to-plot-raw-data-using-vega/238259/15 "2020-06-30T08:08:35Z")

</div>

Replaced time2 with time. Still the same error tho. But thanks for the try. Any ideas on how to debug this?

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/f/2/f2f4a6a394827c790797fde183e2494dc1f954f4.png)

---

<div class="post-metadata">

**Author:** ![Stratoula\_Kalafateli](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stratoula_kalafateli/32/70923_2.png) [@Stratoula\_Kalafateli](https://discuss.elastic.co/u/Stratoula_Kalafateli)\
**Post date:** [June 30, 2020, 8:12am UTC](https://discuss.elastic.co/t/how-to-plot-raw-data-using-vega/238259/16 "2020-06-30T08:12:08Z")

</div>

If you remove completely the line from the second index, does it work correctly ? I would try these combinations to understand when the problem is introduced. Also check my example, I think it will help. Finally, also check your datasets, the @timestamp field.

---

<div class="post-metadata">

**Author:** ![defalt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/defalt/32/71379_2.png) [@defalt](https://discuss.elastic.co/u/defalt)\
**Post date:** [June 30, 2020, 8:34am UTC](https://discuss.elastic.co/t/how-to-plot-raw-data-using-vega/238259/17 "2020-06-30T08:34:41Z")

</div>

1. If I remove transform:` [{ type: "formula",as: "time", expr: "datetime(datum._source['@timestamp']) "} ]` the grey line is gone. It needs the timeframe to be drawn.
2. I think there is no real difference between your example and my code. I always tried to stay close to the code you and tim gave me.
3. The Timestamps look like this.

```auto
{
  "_index": "index",
  "_type": "index",
  "_id": "AXAJ3VEaaIwsdfT2yAntx",
  "_version": 1,
  "_score": null,
  "_source": {
   "valueXY": XYZ
    "@timestamp": "2020-02-02T08:17:47.959Z",
  },
  "fields": {
    "@timestamp": [
      "2020-02-02T08:17:47.959Z"
    ],
    "timestamp": [
      "2020-02-02T09:17:47.959Z"
    ]
  },
}

```

If I change my code from @timestamp to timestamp I get "Cannot read property 'hits' of undefined".

---

<div class="post-metadata">

**Author:** ![Stratoula\_Kalafateli](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stratoula_kalafateli/32/70923_2.png) [@Stratoula\_Kalafateli](https://discuss.elastic.co/u/Stratoula_Kalafateli)\
**Post date:** [June 30, 2020, 8:44am UTC](https://discuss.elastic.co/t/how-to-plot-raw-data-using-vega/238259/18 "2020-06-30T08:44:01Z")

</div>

My suggestion was to remove temporarily the one line from the second index in order to check if the other three lines from the first index are displayed correctly (for debugging purposes).

Something like that:

```auto
{
  "$schema": "https://vega.github.io/schema/vega/v4.3.json",
  "description": "A basic line chart example.",
  "padding": 5,
  
data: [
{
  name:"table"
    url: {
      %context%: true
      %timefield%: @timestamp
      index: index1
      body: {
        "size": 17000,
        "_source": ["@timestamp", "value1","value2","value3"],
        "sort" : { "@timestamp" : "desc" }
      },
    }
    format: { property: "hits.hits" },
    transform: [
        { type: "formula",as: "time", expr: "datetime(datum._source['@timestamp']) "}
      ]
  }
]
  "scales": [
    {
      "name": "timex",
      "type": "time",
      "range": "width",
      "domain": {"data": "table", "field": "time"}
    },
    {
      "name": "Namet",
      "type": "linear",
      "range": "height",
      "nice": true,
      "zero": true,
      "domain": {"data": "table", "field": "_source.value1"}
    },
    {
      "name": "Namec",
      "type": "linear",
      "range": "height",
      "nice": true,
      "zero": true,
      "domain": {"data": "table", "field": "_source.value2"}
    },
    {
      "name": "Namevalue3",
      "type": "linear",
      "range": "height",
      "nice": true,
      "zero": true,
      "domain": {"data": "table", "field": "_source.value3"}
    }
  ],
  "axes": [
    {
    "orient": "bottom", 
    "scale": "timex", 
    "format": "%s",
    "grid":"true"},
    {
    "orient": "left", 
    "scale": "Namet",
    "grid":"true"},
    {
      "orient": "left",
      "scale": "Namec",
      "offset": 30,
      "domainColor": "#FF00FF",
      "labelColor": "#FF00FF",
      "tickColor": "#FF00FF",
      "grid":"true"
    },
    {
      "orient": "left",
      "scale": "Namevalue3",
      "offset": 90,
      "domainColor": "#444444",
      "labelColor": "#444444",
      "tickColor": "#444444",
      "grid":"true"
    }
  ],
  "marks": [
  {
      "type": "line",
      "from": {"data": "table"},
      "encode": {
        "enter": {
          "x": {"scale": "timex", "field": "time"},
          "y": {"scale": "Namec", "field": "_source.value2"},
          "stroke": {"value": "#FF00FF"}
        }
      }
    },
    {
      "type": "line",
      "from": {"data": "table"},
      "encode": {
        "enter": {
          "x": {"scale": "timex", "field": "time"},
          "y": {"scale": "Namet", "field": "_source.value1"}
        }
      }
    },
    {
      "type": "line",
      "from": {"data": "table"},
      "encode": {
        "enter": {
          "x": {"scale": "timex", "field": "time"},
          "y": {
          "scale": "Namevalue3", "field": "_source.value3"},
          "stroke": {"value": "#444444"}
        }
      }
    }
  ]
}
```

So if the three lines are displayed correctly, the error is on the 4th line.  
@timestamp seems fine to me.

---

<div class="post-metadata">

**Author:** ![defalt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/defalt/32/71379_2.png) [@defalt](https://discuss.elastic.co/u/defalt)\
**Post date:** [June 30, 2020, 9:07am UTC](https://discuss.elastic.co/t/how-to-plot-raw-data-using-vega/238259/19 "2020-06-30T09:07:58Z")

</div>

Got rid of table 2, still the same problem.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/1/e/1ec92403ad6f340be057cf39fe1f24ffd4e1b643.png)

**Edit:**  
I will try to solve this "the hard way". I will create a new index with sample data. If the error stays the same the code is faulty, if the error is gone, my data is in some way faulty.

@timroes are there any plans to make raw data visualization a standard feature in kibana? I think everyone who uses this for measuring voltages etc. will thank you for it 😉

---

<div class="post-metadata">

**Author:** ![Stratoula\_Kalafateli](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stratoula_kalafateli/32/70923_2.png) [@Stratoula\_Kalafateli](https://discuss.elastic.co/u/Stratoula_Kalafateli)\
**Post date:** [June 30, 2020, 9:21am UTC](https://discuss.elastic.co/t/how-to-plot-raw-data-using-vega/238259/20 "2020-06-30T09:21:59Z")

</div>

Sounds good, looking forward to the results 🙂

---

<div class="post-metadata">

**Author:** ![defalt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/defalt/32/71379_2.png) [@defalt](https://discuss.elastic.co/u/defalt)\
**Post date:** [June 30, 2020, 10:53am UTC](https://discuss.elastic.co/t/how-to-plot-raw-data-using-vega/238259/21 "2020-06-30T10:53:01Z")

</div>

The same problem occurs with different data. When I zoomed out it looks like everything is correct and the time fits but when the timescale is smaller the same error appears.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/2/3/23a23ee5b491b9a35f896d84f27755ea5e316ac8.png)  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/9/9/99b55225d70203518c97167549dc6e97145ef570.png)

```auto
{
  "$schema": "https://vega.github.io/schema/vega/v4.3.json",
  "description": "A basic line chart example.",
  "padding": 5,
  
data: [
{
  name:"table"
    url: {
      %context%: true
      %timefield%: @timestamp
      index: elasticoffee
      body: {
        "size": 10000,
        "_source": ["@timestamp", "quadId"],
        "sort" : { "@timestamp" : "desc" }
      },
    }
    format: { property: "hits.hits" },
    transform: [
        { type: "formula",as: "time", expr: "datetime(datum._source['@timestamp'])"}
      ]
  }
],
  "scales": [
    {
      "name": "timex",
      "type": "time",
      "range": "width",
      "domain": {"data": "table", "field": "time"}
    },
    {
      "name": "NameBeverage",
      "type": "linear",
      "range": "height",
      "nice": true,
      "zero": true,
      "domain": {"data": "table", "field": "_source.quadId"}
    }
  ],
  "axes": [
    {
    "orient": "bottom", 
    "scale": "timex", 
    "format": "%d %H:%m",
    "grid":"true"
    },
    {
    "orient": "left", 
    "scale": "NameBeverage",
    "grid":"true"},
  ],
  "marks": [
  {
      "type": "line",
      "from": {"data": "table"},
      "encode": {
        "enter": {
          "x": {"scale": "timex", "field": "time"},
          "y": {"scale": "NameBeverage", "field": "_source.quadId"},
          "stroke": {"value": "#FF00FF"},
          "strokeWidth": "1"
        }
      }
    }
  ]
}

```

The used data was from [Elastic Coffee](https://www.elastic.co/de/blog/elasticon-kibana-canvas-story-elasticoffee) and looks something like this:

```auto
POST _bulk
{ "index" : { "_index" : "elasticoffee", "_type" : "doc", "_id" : "5005" } }
{"sceneID": "2", "sceneData": "0", "entityID": "zwave.quad2", "quadId": 15, "quadMod": "1", "@timestamp": "2018-08-30T08:26:39Z", "beverageClass": "Hot Beverages", "beverage": "Latte", "beverageSide": "left", "beverageIndex": 5, "quantity": 1}

```

I used quadId for my testing purpose. I am clueless why this time shift happens. May best guess is that is has something to do with the transform function, its the only thing that can manipulate the time.

[Next page](https://discuss.elastic.co/t/how-to-plot-raw-data-using-vega/238259.md?page=2)