# Can't Get Timestamp Transform To Work For A Basic Vega Scatter Plot

**URL:** <https://discuss.elastic.co/t/cant-get-timestamp-transform-to-work-for-a-basic-vega-scatter-plot/264254>\
**Category:** Kibana\
**Tags:** vega\
**Created:** [February 14, 2021, 7:08pm UTC](https://discuss.elastic.co/t/cant-get-timestamp-transform-to-work-for-a-basic-vega-scatter-plot/264254 "2021-02-14T19:08:53Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Webtrend\_Inc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/webtrend_inc/32/70034_2.png) [@Webtrend\_Inc](https://discuss.elastic.co/u/Webtrend_Inc)\
**Post date:** [February 14, 2021, 7:08pm UTC](https://discuss.elastic.co/t/cant-get-timestamp-transform-to-work-for-a-basic-vega-scatter-plot/264254/1 "2021-02-14T19:08:53Z")

</div>

Here is my Vega code

```auto
{
  "$schema": "https://vega.github.io/schema/vega/v4.json",
  "description": "A basic scatter plot",
  "padding": 5,

  "data": {
    "name": "source",
    "url": {
      "%context%": "true",
      "%timefield%": "@timestamp",
      "index": "alb-logs-*",
      "body": {
        "size": 1000,
        "_source": ["@timestamp", "elb_received_bytes"],
      }
    },
    "format": {"property": "hits.hits"},
  },
  "transform": [
      {
        calculate: "toDate(datum._source['@timestamp'])" as: "time"
      }
    ]
  "scales": [
    {
      "name": "x",
      "type": "time",
      "round": true,
      "nice": true,
      "zero": true,
      "domain": {"data": "source", "field": "time"},
      "range": "width"
    },
    {
      "name": "y",
      "type": "linear",
      "round": true,
      "nice": true,
      "zero": true,
      "domain": {"data": "source", "field": "_source.elb_received_bytes"},
      "range": "height"
    }
  ],

  "axes": [
    {
      "scale": "x",
      "grid": true,
      "domain": false,
      "orient": "bottom",
      "tickCount": 5,
      "title": "Time"
    },
    {
      "scale": "y",
      "grid": true,
      "domain": false,
      "orient": "left",
      "titlePadding": 5,
      "title": "Bytes"
    }
  ],

  

  "marks": [
    {
      "name": "marks",
      "type": "symbol",
      "from": {"data": "source"},
      "encode": {
        "update": {
          "x": {"scale": "x", "field": "time"},
          "y": {"scale": "y", "field": "_source.elb_received_bytes"},
          "shape": {"value": "circle"},
          "strokeWidth": {"value": 2},
          "opacity": {"value": 0.5},
          "stroke": {"value": "#4682b4"},
          "fill": {"value": "transparent"}
        }
      }
    }
  ]
}

```

When I run the code, my time data is missing and I am seeing following output:

 ![8279db38-577f-449e-9181-6e92b1d6b42d](https://us1.discourse-cdn.com/elastic/original/3X/a/3/a3b9892a11b4aa3a93dbf70535afbd01a6f6cbbf.png)

IN my web console, when I run `VEGA_DEBUG.view.data('source')`, I don't see the "time" field in my data array. So, I believe that my transform function is not working properly.  
Here is what I see in my web console

```auto
0:
Symbol(vega_id): 3935923
_id: "Man2mHcBbBSZp3usZgN-"
_index: "alb-logs-2020.12"
_score: 1
_source:
@timestamp: "2020-12-16T20:00:03.303Z"
elb_received_bytes: "241"

```

---

<div class="post-metadata">

**Author:** ![Webtrend\_Inc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/webtrend_inc/32/70034_2.png) [@Webtrend\_Inc](https://discuss.elastic.co/u/Webtrend_Inc)\
**Post date:** [February 14, 2021, 11:09pm UTC](https://discuss.elastic.co/t/cant-get-timestamp-transform-to-work-for-a-basic-vega-scatter-plot/264254/2 "2021-02-14T23:09:24Z")

</div>

OK. I made some progress. Apparently, I was using Vega-lite expression for transform. I changed the expression and made it vega specific :

```auto
"transform": [
      {
        "type": "formula", "as": "time", "expr": "toDate(datum._source['@timestamp'])" 
      }
    ]

```

and now it appears to be working but it led to another problem as shown below

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/b/7/b7e7a87b3e6d6fa1ad5ee861e08c0d131d4738fa.png)

As you can see, the x-axis is not scaling automatically according to the timestamp range of the data points.

Also, why can't I still see the transformed field "time" values in chrome console? I only see the 2 original elastic index fields as mentioned in my previous comment. How does one see the transformed fields in browser console?

---

<div class="post-metadata">

**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\
**Post date:** [February 16, 2021, 10:45pm UTC](https://discuss.elastic.co/t/cant-get-timestamp-transform-to-work-for-a-basic-vega-scatter-plot/264254/3 "2021-02-16T22:45:04Z")

</div>

Could you [provide the spec](https://www.elastic.co/guide/en/kibana/master/vega.html#asking-for-help-with-a-vega-spec)?

---

<div class="post-metadata">

**Author:** ![Webtrend\_Inc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/webtrend_inc/32/70034_2.png) [@Webtrend\_Inc](https://discuss.elastic.co/u/Webtrend_Inc)\
**Post date:** [February 18, 2021, 1:06am UTC](https://discuss.elastic.co/t/cant-get-timestamp-transform-to-work-for-a-basic-vega-scatter-plot/264254/4 "2021-02-18T01:06:23Z")

</div>

@aaron-nimocks thanks for your response. Here are the specs. I have reduced the number of data points to 10 so that it can be easily accommodated here. The problem still persists even with 10 data points.

```auto
{
  "$schema": "https://vega.github.io/schema/vega/v4.json",
  "description": "A basic scatter plot",
  "padding": 5,
  "data": {
    "name": "source",
    "format": {
      "property": "hits.hits"
    },
    "transform": [
      {
        "type": "formula",
        "as": "time",
        "expr": "toDate(datum._source['@timestamp'])"
      }
    ],
    "url": {
      "index": "alb-logs-*",
      "body": {
        "size": 10,
        "_source": [
          "@timestamp",
          "elb_received_bytes"
        ],
        "query": {
          "bool": {
            "must": [
              {
                "range": {
                  "@timestamp": {
                    "gte": "2021-02-18T00:47:15.704Z",
                    "lte": "2021-02-18T01:02:15.704Z",
                    "format": "strict_date_optional_time"
                  }
                }
              }
            ],
            "filter": [
              {
                "match_all": {}
              }
            ],
            "should": [],
            "must_not": []
          }
        }
      }
    },
    "values": {
      "took": 6,
      "timed_out": false,
      "_shards": {
        "total": 3,
        "successful": 3,
        "skipped": 2,
        "failed": 0
      },
      "hits": {
        "total": 1487,
        "max_score": 1,
        "hits": [
          {
            "_index": "alb-logs-2021.02",
            "_type": "_doc",
            "_id": "6ViesncBHSONY29jDXEY",
            "_score": 1,
            "_source": {
              "@timestamp": "2021-02-18T00:47:36.638Z",
              "elb_received_bytes": "236"
            },
            "time": 1613609256638
          },
          {
            "_index": "alb-logs-2021.02",
            "_type": "_doc",
            "_id": "6liesncBHSONY29jDXEY",
            "_score": 1,
            "_source": {
              "@timestamp": "2021-02-18T00:48:04.753Z",
              "elb_received_bytes": "236"
            },
            "time": 1613609284753
          },
          {
            "_index": "alb-logs-2021.02",
            "_type": "_doc",
            "_id": "61iesncBHSONY29jDXEY",
            "_score": 1,
            "_source": {
              "@timestamp": "2021-02-18T00:47:23.721Z",
              "elb_received_bytes": "202"
            },
            "time": 1613609243721
          },
          {
            "_index": "alb-logs-2021.02",
            "_type": "_doc",
            "_id": "7FiesncBHSONY29jDXEY",
            "_score": 1,
            "_source": {
              "@timestamp": "2021-02-18T00:47:48.237Z",
              "elb_received_bytes": "449"
            },
            "time": 1613609268237
          },
          {
            "_index": "alb-logs-2021.02",
            "_type": "_doc",
            "_id": "7ViesncBHSONY29jDXEY",
            "_score": 1,
            "_source": {
              "@timestamp": "2021-02-18T00:47:53.094Z",
              "elb_received_bytes": "241"
            },
            "time": 1613609273094
          },
          {
            "_index": "alb-logs-2021.02",
            "_type": "_doc",
            "_id": "7liesncBHSONY29jDXEY",
            "_score": 1,
            "_source": {
              "@timestamp": "2021-02-18T00:47:59.482Z",
              "elb_received_bytes": "69"
            },
            "time": 1613609279482
          },
          {
            "_index": "alb-logs-2021.02",
            "_type": "_doc",
            "_id": "71iesncBHSONY29jDXEY",
            "_score": 1,
            "_source": {
              "@timestamp": "2021-02-18T00:48:11.755Z",
              "elb_received_bytes": "165"
            },
            "time": 1613609291755
          },
          {
            "_index": "alb-logs-2021.02",
            "_type": "_doc",
            "_id": "8FiesncBHSONY29jDXEY",
            "_score": 1,
            "_source": {
              "@timestamp": "2021-02-18T00:48:20.981Z",
              "elb_received_bytes": "1373"
            },
            "time": 1613609300981
          },
          {
            "_index": "alb-logs-2021.02",
            "_type": "_doc",
            "_id": "8ViesncBHSONY29jDXEY",
            "_score": 1,
            "_source": {
              "@timestamp": "2021-02-18T00:48:20.258Z",
              "elb_received_bytes": "161"
            },
            "time": 1613609300258
          },
          {
            "_index": "alb-logs-2021.02",
            "_type": "_doc",
            "_id": "8liesncBHSONY29jDXEY",
            "_score": 1,
            "_source": {
              "@timestamp": "2021-02-18T00:48:21.460Z",
              "elb_received_bytes": "90"
            },
            "time": 1613609301460
          }
        ]
      }
    }
  },
  "scales": [
    {
      "name": "x",
      "type": "time",
      "round": true,
      "nice": true,
      "zero": true,
      "domain": {
        "data": "source",
        "field": "time"
      },
      "range": "width"
    },
    {
      "name": "y",
      "type": "linear",
      "round": true,
      "nice": true,
      "zero": true,
      "domain": {
        "data": "source",
        "field": "_source.elb_received_bytes"
      },
      "range": "height"
    }
  ],
  "axes": [
    {
      "scale": "x",
      "grid": true,
      "domain": false,
      "orient": "bottom",
      "tickCount": 5,
      "title": "Time"
    },
    {
      "scale": "y",
      "grid": true,
      "domain": false,
      "orient": "left",
      "titlePadding": 5,
      "title": "Bytes"
    }
  ],
  "marks": [
    {
      "name": "marks",
      "type": "symbol",
      "from": {
        "data": "source"
      },
      "encode": {
        "update": {
          "x": {
            "scale": "x",
            "field": "time"
          },
          "y": {
            "scale": "y",
            "field": "_source.elb_received_bytes"
          },
          "shape": {
            "value": "circle"
          },
          "strokeWidth": {
            "value": 2
          },
          "opacity": {
            "value": 0.5
          },
          "stroke": {
            "value": "#4682b4"
          },
          "fill": {
            "value": "transparent"
          }
        }
      }
    }
  ],
  "config": {
    "range": {
      "category": {
        "scheme": "elastic"
      }
    },
    "arc": {
      "fill": "#54B399"
    },
    "area": {
      "fill": "#54B399"
    },
    "line": {
      "stroke": "#54B399"
    },
    "path": {
      "stroke": "#54B399"
    },
    "rect": {
      "fill": "#54B399"
    },
    "rule": {
      "stroke": "#54B399"
    },
    "shape": {
      "stroke": "#54B399"
    },
    "symbol": {
      "fill": "#54B399"
    },
    "trail": {
      "fill": "#54B399"
    },
    "title": {
      "color": "#343741"
    },
    "style": {
      "guide-label": {
        "fill": "#69707d"
      },
      "guide-title": {
        "fill": "#343741"
      },
      "group-title": {
        "fill": "#343741"
      },
      "group-subtitle": {
        "fill": "#343741"
      }
    },
    "axis": {
      "tickColor": "#eef0f3",
      "domainColor": "#eef0f3",
      "gridColor": "#eef0f3"
    },
    "background": "transparent"
  },
  "width": "container",
  "height": "container",
  "autosize": {
    "type": "fit",
    "contains": "padding"
  }
}

```

---

<div class="post-metadata">

**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\
**Post date:** [February 18, 2021, 1:45am UTC](https://discuss.elastic.co/t/cant-get-timestamp-transform-to-work-for-a-basic-vega-scatter-plot/264254/5 "2021-02-18T01:45:23Z")

</div>

There is a new aggregate called [timeunit](https://vega.github.io/vega/docs/transforms/timeunit/) but it's in V5 and [here](https://vega.github.io/editor/#/edited) is the example.

I made all the changes [here](https://vega.github.io/editor/#/url/vega/N4IgJAzgxgFgpgWwIYgFwhgF0wBwqgegIDc4BzJAOjIEtMYBXAI0poHsDp5kTykSArJQBWENgDsQAGhAATONABONHJnaT0AQQAETJBBpRt0JNjiLtOADZtM0kDiSzZNcWTQCZs0ylQBtUHEkBDg0EDEGRShQmQAzNkVkO1RQHEU2HHNMAE8wmDoISnzMCBAAXxlMRSRxCHjEtACQHMywtRCGcTp7WJo4K1k2mhD7ToLG8LgoCVlSgF0ZAzIgqzamV3KpUBbQ9CQyMkU+TBiQQ7YGHCZc-xAxzAAGEAWQDNLbpFJqslCX3v7ZhMAPoRKJwSj9JhAo7RGikWRA64neYyfQTSHQqZwOFwBFIhTPMovYhIKwMAkpZpsNgAazQADZKsNcUCLslYqSIHAZCCYEhFIDKZhbKS0ABmRYMKDRCB1BirVAS8I0lSZQaoABMcSQNCsuLQDwqGHGQpFCoAjAAWAAcAHYZMgAB4g6ZHNDmmTFd5NIGueSOsKkpgAWhsZAgwY1Dw15so0fsQJ2YSBsjYUATNHVIHpADU+hBxFAAEIACQAygB5AByAE0NQBOYQAEQAGgBRGsJ6AJXYekAgi5gtCgAAC7QUmGCODCUZjwejwfNDwAKg8HqhLbbFfTKPSxdaAFr2DEw7HwxHZZEzsX0zbNZnu+nmm8PesagT0vfWiqgX3if2BlYIZhhGs6xvGPJJugKZphmWb0lY+aFqWla1g2zbtp2PLdm6qB9gOkTRMOIBjsyECTgg07oGB84aou5qruuNqoA8lqULaAhikeMgnliOJ4peFIgBqN53uOj7PvSr4atam6cT+-Z+nAAZ7EBoZsOGkbRuBGoJlB-apumPKZmET5IcW5bVnWjath2Xaur22GDkRlKkSE5FTjO2m0Yu1qMRuW4iexMbcSAvGwue+LvMJ8ZGuJeFPi+b6WmKtoxgpf4AapwEaaB2lxrpkHZK00GGXBYS2gAYuZKFWehtlYf2OGOU1zm7KO44eZRXlzgu5p+WuAUbtalAibaoXhWeLJRWElqWvWYkPglknSfS1pjRlSkqSAQbqZpYEFXpxW7AZsHGVmtp5goyGWWhNmYfZPbuk5hHtSRnUUVRMW9XR-X+ZuqCcXG9aWhNQGYhF02CdFGqWuai0jMtSUaraYqvpam3-spgE5ft+UQf2+kwUZikXYh10Wah1kYXZ2EOc9rWvcRbkTp51HeX1A1MVuAj1pQNoamDUKnvxF5Xug9ILXFS3molUlvraIPrZjWU7WpIFaTGh1FSVp0k76F3mjVt3Uw1j24fhoIuR1ZGfT15o+X9g3MeasYcQIQsQ1NAniyAssCAjvZy9J9bmu7KvY9le15VrBOJsdyZledYTWtVFO1XdNONS6T14S9Q6uR9bPfQ7nP-damoPJQ9bWuansi5F0NhM+qOBxJSVo6+tcR9tu0awdcdE0npMp1dBaU3V920019N54zBc2+5dvsz9vnl5Xo0CIex7gw3UO+7L8PS4jh8d2u77flsilY736u5ZrOlHbrxPleg1rk+PGemw9dO55bbXMyLt1Fepdfpc2GpqWMlopL1z4o3X29YnjHyDitesaMrRSSJGUbBLwTB6m9IEYIJ0AyVATugPQ-57DVDcCdAA7pmeg9hHDOFcO4FicZPAgHSJ0dUVRyReDYMgDYlIlgrCGCEMscAACO5JCxwAABSYHWOISg9wpDaCUa4SgHlFCYHUZolR5EMgAEpygKSCIjEAuRSG60QuIOA-IqEXEoagPh3IQBdBcm4mQAAvcwbA0DeLkIInUGhQDeEnGEK2px-gDDCJNUWM0jTUJ+HkbEZAsDlBeEgR0FImgJD6OIZIIAmC2GFAgeweDiE9ASEkZcZD7wjDiLU0wxFRGinQOOAAql0TAZZMhQBoP8RQijlGqN6RAdRwAECuAYMiVA2gADkiyyimOwVfKpYRrFnGULwxQ-DglCI0ByKwXIZAFLgEUsIepYh2CZJgPUAAFJwLg3AeHuXqMIRYm5EgdPyGkBCPFELCMgRQAL7D6QgNkBApTVhxHSBUykETfDhAAUaS50x5DEUuBEt6KlQCbPQCQkAsSsz3CQTIG4BKoCkhOts0l8Td5wP3gSI0EA+QlVACSMkJ1BlRE+WyqotI4AAHUGEwGIty8kaANRGgyEgQZORJWkmlewgQgr0g0jelKk6ABiaB60mCWjvL0KwCouUqpOlUGoEBHBHCudgnBMh6GyEYagRBDxPTpMyRuNcZQgA) to make that work. Let me know if this all makes sense.

---

<div class="post-metadata">

**Author:** ![Webtrend\_Inc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/webtrend_inc/32/70034_2.png) [@Webtrend\_Inc](https://discuss.elastic.co/u/Webtrend_Inc)\
**Post date:** [February 18, 2021, 8:12am UTC](https://discuss.elastic.co/t/cant-get-timestamp-transform-to-work-for-a-basic-vega-scatter-plot/264254/6 "2021-02-18T08:12:38Z")

</div>

Got it to work. Thanks for the solution 🙂

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/9/7/97b73f3636d69e762d101c71867dfea153fd99e6.png)

---

<div class="post-metadata">

**Author:** ![Webtrend\_Inc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/webtrend_inc/32/70034_2.png) [@Webtrend\_Inc](https://discuss.elastic.co/u/Webtrend_Inc)\
**Post date:** [February 18, 2021, 5:35pm UTC](https://discuss.elastic.co/t/cant-get-timestamp-transform-to-work-for-a-basic-vega-scatter-plot/264254/7 "2021-02-18T17:35:49Z")

</div>

I am just seeking one more clarification. What is the purpose of following transform?

```auto
{
          "type": "aggregate",
          "groupby": ["unit0"],
          "ops": ["average"],
          "fields": ["_source.elb_received_bytes"],
          "as": ["elb_received_bytes"]
        }

```

When I insert this into my code, I get an error `Cannot read property 'elb_received_bytes' of undefined`

---

<div class="post-metadata">

**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\
**Post date:** [February 18, 2021, 5:43pm UTC](https://discuss.elastic.co/t/cant-get-timestamp-transform-to-work-for-a-basic-vega-scatter-plot/264254/8 "2021-02-18T17:43:33Z")

</div>

That one groups the `unit0` field that was created with the `timeunit` aggregation and adds in your aggregation for your `y-axis`.

So if in `timeunit` you specify `minutes` then that aggregation will take the `average bytes` and group them by `unit0` so every minute is accounted for and you can create the chart.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 18, 2021, 5:43pm UTC](https://discuss.elastic.co/t/cant-get-timestamp-transform-to-work-for-a-basic-vega-scatter-plot/264254/9 "2021-03-18T17:43:52Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.