Kibana Vega - Drawing the line with dynamic StrokeWidth is not working for me


#1

Hi,

Quick summary of what I want the visualization to be like ...
I have multiple IHS proxys which route requests to a bunch of servers. I want to draw a parallel coordinates graph that would show the number of requests getting routed from each IHS proxy to each server. That way, if there is an imbalance in the routing logic, we would come to know. I want the thickness of the lines to indicate the number of requests (I also plan to add the tooltip later which would show the number of requests).

My data:
All the IHS logs are sent to elastic. For easy sharing and simulation, I have uploaded the output of the elastic query to gist.github.com and running the visualization on that.

High Level Logic:
Extract IHS and AppServer fields from elastic index (now gist.github json) and form a dataset. Let's call it ihsdata.
On ihsdata, apply group by on IHS and AppServer fields, and for a new dataset ihssummary.
Now ihssummary has data IHS-AppServer-count.
...
...
Create a scale 'StrokeWidth' over a range [1-10] (which is what I want my strokewidth to be) on the domain ihssummary.count.
...
...
Draw the parallel coordinates visualization by specifying strokewidth on scale "StrokeWidth" and field "{parent: {datum: count}}"

Problem:
All my lines are coming with same width.

My Vega Code

{
  "$schema": "https://vega.github.io/schema/vega/v3.json",
  "width": 700,
  "height": 400,
  "padding": 5,
  "autosize": "pad",

  "config": {
    "axisY": {
      "titleX": -2,
      "titleY": 410,
      "titleAngle": 0,
      "titleAlign": "right",
      "titleBaseline": "top"
    }
  },

  "data": [
    {
      "name": "ihsdata",
      "url": "https://gist.githubusercontent.com/tanmaydeuskar/630fbc30e94322f7442ea1f2042c53ec/raw/889bf99786e7463afbafa1c2eb8278278c08e166/ihs_vega_es_output.json",
      "format": {"property": "hits.hits"},
      "transform": [
        {"type": "formula", "as": "IHS", "expr": "datum._source.IHS"},
        {"type": "formula", "as": "AppServer", "expr": "datum._source.AppServer"}
      ]
    },
    {
      "name": "ihssummary", "source": "ihsdata",
      "transform": [
        { "type": "aggregate", "groupby": ["IHS", "AppServer"] }
      ]
    },
    {
      "name": "distinct-ihs", "source": "ihssummary",
      "transform": [
        { "type": "aggregate", "groupby": ["IHS"] },
        { "type": "collect", "sort": {"field": "IHS"} }
      ]
    },
    {
      "name": "distinct-appserver", "source": "ihssummary",
      "transform": [
        { "type": "aggregate", "groupby": ["AppServer"] },
        { "type": "collect", "sort": {"field": "AppServer"} }
      ]
    },
    {
      "name": "fields",
      "values": [
        "IHS",
        "AppServer"
      ]
    }
  ],

  "scales": [
    {
      "name": "ord", "type": "point",
      "range": "width", "round": true,
      "domain": {"data": "fields", "field": "data"}
    },
    {
      "name": "IHS", "type": "point",
      "range": "height", "round": true,
      "domain": {"data": "distinct-ihs", "field": "IHS"}
    },
    {
      "name": "AppServer", "type": "point",
      "range": "height", "round": true,
      "domain": {"data": "distinct-appserver", "field": "AppServer"}
    },
    {
      "name": "StrokeWidth", "type": "linear",
      "range": [1,10],
      "domain": {"data": "ihssummary", "field": "count"}
    }
  ],

  "axes": [
    {
      "orient": "left", "zindex": 1,
      "scale": "IHS", "title": "IHS",
      "offset": {"scale": "ord", "value": "IHS", "mult": -1}
    },
    {
      "orient": "left", "zindex": 1,
      "scale": "AppServer", "title": "AppServer",
      "offset": {"scale": "ord", "value": "AppServer", "mult": -1}
    }
  ],

  "marks": [
    {
      "type": "group",
      "from": {"data": "ihssummary"},
      "marks": [
        {
          "type": "line",
          "from": {"data": "fields"},
          "encode": {
            "enter": {
              "x": {"scale": "ord", "field": "data"},
              "y": {
                "scale": {"datum": "data"},
                "field": {"parent": {"datum": "data"}}
              },
              "stroke": {"value": "steelblue"},
              "strokeWidth": {
                "scale": "StrokeWidth",
                "field": {"parent": {"datum": "count"}}
              },
              "strokeOpacity": {
                "value": 1
              }
            }
          }
        }
      ]
    }
  ]
}

This is how the visualization looks

Some of the vegadebug information (in case it helps):

Vega debug of dataset ihssummary

VEGA_DEBUG.view.data("ihssummary")
  (15) [{…}, {…}, {…}, {…}, {…}, {…}, {…}, {…}, {…}, {…}, {…}, {…}, {…}, {…}, {…}]
    0: {IHS: "IHS1", AppServer: "Server2", count: 67, Symbol(vega_id): 8469}
    1: {IHS: "IHS3", AppServer: "Server3", count: 21, Symbol(vega_id): 8470}
    2: {IHS: "IHS1", AppServer: "Server4", count: 70, Symbol(vega_id): 8471}
    3: {IHS: "IHS1", AppServer: "Server1", count: 104, Symbol(vega_id): 8472}
    4: {IHS: "IHS1", AppServer: "Server3", count: 63, Symbol(vega_id): 8473}
    5: {IHS: "IHS2", AppServer: "Server3", count: 29, Symbol(vega_id): 8474}
    6: {IHS: "IHS3", AppServer: "Server5", count: 11, Symbol(vega_id): 8475}
    7: {IHS: "IHS2", AppServer: "Server1", count: 41, Symbol(vega_id): 8476}
    8: {IHS: "IHS1", AppServer: "Server5", count: 38, Symbol(vega_id): 8477}
    9: {IHS: "IHS2", AppServer: "Server4", count: 18, Symbol(vega_id): 8478}
    10: {IHS: "IHS3", AppServer: "Server1", count: 29, Symbol(vega_id): 8479}
    11: {IHS: "IHS3", AppServer: "Server4", count: 29, Symbol(vega_id): 8480}
    12: {IHS: "IHS3", AppServer: "Server2", count: 20, Symbol(vega_id): 8481}
    13: {IHS: "IHS2", AppServer: "Server2", count: 19, Symbol(vega_id): 8482}
    14: {IHS: "IHS2", AppServer: "Server5", count: 11, Symbol(vega_id): 8483}
    length: 15
    __proto__: Array(0)

Any idea where I am going wrong? Thanks in advance for help.

  • Parag

#2

I couldn't attach one more piece of vega debug in the main body due to character limit, so adding it here ...

Here is Vega Debug of StrokeWidth scale (There is still too much data in the StrokeWidth scale, I am not finding an easy way to upload it - but will upload one way or other if it is necessary)

VEGA_DEBUG.view._runtime.scales.StrokeWidth
  Scale {id: 7053, value: ƒ, stamp: 1, rank: 146, qrank: 146, …}
    flags: 2
    id: 7053
    pulse: Pulse {dataflow: View, stamp: 1, add: Array(0), rem: Array(0), mod: Array(0), …}
    qrank: 146
    rank: 146
    stamp: 1
    value: ƒ scale(x)
    _argops: [{…}]
    _argval: Parameters {type: "linear", range: Array(2), domain: Array(2), _:mod:_: {…}}
    _targets: Array(15)
      0: Encode {id: 7191, value: null, stamp: 1, rank: 226, qrank: 226, …}
      1: Encode {id: 7209, value: null, stamp: 1, rank: 260, qrank: 260, …}
      2: Encode {id: 7227, value: null, stamp: 1, rank: 294, qrank: 294, …}
      3: Encode {id: 7245, value: null, stamp: 1, rank: 328, qrank: 328, …}
      4: Encode {id: 7263, value: null, stamp: 1, rank: 362, qrank: 362, …}
      5: Encode {id: 7281, value: null, stamp: 1, rank: 396, qrank: 396, …}
      6: Encode {id: 7299, value: null, stamp: 1, rank: 430, qrank: 430, …}
      7: Encode {id: 7317, value: null, stamp: 1, rank: 464, qrank: 464, …}
      8: Encode {id: 7335, value: null, stamp: 1, rank: 498, qrank: 498, …}
      9: Encode {id: 7353, value: null, stamp: 1, rank: 532, qrank: 532, …}
      10: Encode {id: 7371, value: null, stamp: 1, rank: 566, qrank: 566, …}
      11: Encode {id: 7389, value: null, stamp: 1, rank: 600, qrank: 600, …}
      12: Encode {id: 7407, value: null, stamp: 1, rank: 634, qrank: 634, …}
      13: Encode {id: 7425, value: null, stamp: 1, rank: 668, qrank: 668, …}
      14: Encode {id: 7443, value: null, stamp: 1, rank: 702, qrank: 702, …}
      add: ƒ (_)
      remove: ƒ (_)
      length: 15
      __proto__: Array(0)
    __proto__: Transform

(Yuri Astrakhan) #3

@deuskars thanks for such a thorough job on asking the question! I wish every post was like that :slight_smile:

Your strokeWidth expression is incorrect. You need to access parent's count field, e.g. "field": {"parent": "count"}. TBH, I personally prefer the signal notation - "signal": "scale('StrokeWidth', parent.count)",, as it is less confusing to me, but either works for me in the Vega editor. BTW, note that it now has a data debugger at the bottom, making it easier to inspect your data.


#4

@nyuriks, thanks for your kind words :blush: Probably my QA background helped :slight_smile:

And even more thanks for the solution. It worked like a charm. You mentioned two ways. Where do I find these multiple possible ways. I realize that finally I need provide a number value to strokeWidth. Is there a place I can look up where I can find different (possibly all the different) ways an expression can be written? I did look up https://vega.github.io/vega/docs/expressions/ but from there I couldn't figure this out. I couldn't understand how the "signal" expression comes into picture - I haven't even defined a signal. Maybe I just need to get more familiar with vega and then things will fall in place.

Thanks for the tip on the data viewer at the bottom of the vega editor. That's really awesome.

-- Parag


(Yuri Astrakhan) #5

@deuskars signal is really a misnomer here. Initially it was thought that signal would simply be a reference to a named signal value, but soon thereafter it became a generic dynamic expression. So just treat it if it was called "expression". Maybe Vega team will rename it to reduce confusion (feel free to file a github issue about it on the vega/vega github repo).

That said, this is not really spelled out anywhere I can see in the docs. You are welcome to try to change that -- each doc page has an edit button at the bottom :slight_smile: Perhaps there should be an explanation at the top of the signals page, to solve the confusion.


#6

@nyuriks thanks for the explanation. I will probably start contributing to docs but I need to gain some more expertise on vega before I do that. I don't feel qualified to make a correct and complete contribution yet.


(Yuri Astrakhan) #7

@deuskars be bold :slight_smile: Your changes won't go into docs right away - Vega people will review them, and possibly change your text. But adding to the docs will highlight the unclear points to them. Without that, they think the docs are perfect :slight_smile:


#8

ok. cool. :slight_smile:


(system) #9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.