Obtaining `now` on a kibana visualization

[jmartori (Jordi Martori)]

We have our messages with a field timestamp, and I'd like to have a plot to view the difference from now to the max(timestamp). This way we can know if we are ingesting properly and how long has it been seen we did.

We are currently using elk 5.6.9, but we are planning on migrating soon to newer versions.

Thanks for the help,

[flash1293 (Joe Reuter)]

I couldn't think of a way to do this with existing Kibana visualizations because you can't add client side calculations there, but fortunately there is the Vega visualization which allows a lot of flexibility.

The following spec queries elasticsearch with a max aggregation on the @timestamp field and then does the calculation against the current time client side using a vega expression (of course you have to substitute your index pattern name and possibly time field name):

{
  "$schema": "https://vega.github.io/schema/vega/v4.json",
  "data": {
    "name": "es",
    "url": {
     "%context%": true,
      "index": "YOUR_INDEX_NAME",
      "body": {"aggs": {"max_ts": {"max": {"field": "@timestamp"}}}, "size": 0}
    },
    "format": {"property": "aggregations.max_ts.value"}
  },
  "marks": [
    {
      "type": "text",
      "encode": {
        "enter": {
          "fill": {"value": "#000"},
          "text": {"signal": "(now() - data('es')[0].data) / 1000"},
          "fontSize": {"value": 25}
        },
        "update": {"x": {"signal": "10"}, "y": {"signal": "10"}}
      }
    },
    {
      "type": "text",
      "encode": {
        "enter": {
          "fill": {"value": "#000"},
          "text": {"signal": "'Seconds since the last document'"},
          "fontSize": {"value": 25}
        },
        "update": {"x": {"signal": "10"}, "y": {"signal": "40"}}
      }
    }
  ]
}

The results looks like this:

[jmartori (Jordi Martori)]

I'll look into that, thanks for the answer.