Problem with data transformation for vega visualization

Hello,
I'm trying to create a network graph that can display domains, IPs, subdomains, etc. and associated links. For this, I found a code sample that allows you to draw this kind of graph: LINK

So I transformed my data to be the same form as used in this example. Then, I indexed the nodes in a nodes_index index, then in a links_index index, the links.

To load the data, I use:

  "data": [
    {
      "name": "link-data-raw",
      "url": {
        "index": "links_index",
        "body": {
          "size": 1000,
          "_source": ["source", "target", "value"]
        }
      },
      "format": {
        "property": "hits.hits",
        "type": "json",
        "parse": {
          "source": "string",
          "target": "string",
          "value": "number"
        }
      },
....
    {
      "name": "node-data",
      "url": {
        "index": "nodes_index",
        "body": {
          "size": 10000,
          "_source": ["name", "group"]
        }
      },
      "format": {
        "property": "hits.hits",
        "type": "json",
        "parse": {
          "name": "string",
          "group": "string"
        }
      },
....

The graph is displayed well, but there are problems with the arrows and the LinkDistance.

Arrows are not synchronized with links. In fact, the arrow is supposed to be the end of the link, but it remains fixed, even if the link moves.

Then, I have a "linkDistance" cursor, which, when I move it, gives the error:

"links is undefined".

The code parts for these features:

(line 433) for arrows

      "transform": [
        {
          "type": "linkpath",
          "require": {"signal": "force"},
          "shape": "line",
          "sourceX": {"expr": "scale('xscale', datum.datum.source.x)"},
          "sourceY": {"expr": "scale('yscale', datum.datum.source.y)"},
          "targetX": {"expr": "scale('xscale', datum.datum.target.x)"},
          "targetY": {"expr": "scale('yscale', datum.datum.target.y)"}
        },
        {
          "type": "formula",
          "expr": "atan2(datum.datum.target.y - datum.datum.source.y,datum.datum.source.x - datum.datum.target.x)",
          "as": "angle1"
        },
        {
          "type": "formula",
          "expr": "(datum.angle1>=0?datum.angle1:(2*PI + datum.angle1)) * (360 / (2*PI))",
          "as": "angle2"
        },
        {
          "type": "formula",
          "expr": "(360-datum.angle2)*(PI/180)",
          "as": "angle3"
        },
        {
          "type": "formula",
          "expr": "(cos(datum.angle3)*(nodeRadius+5))+(scale('xscale',datum.datum.target.x))",
          "as": "arrowX"
        },
        {
          "type": "formula",
          "expr": "(sin(datum.angle3)*(nodeRadius+5))+(scale('yscale',datum.datum.target.y))",
          "as": "arrowY"
        }
      ]
    },
    {
      "type": "symbol",
      "name": "arrows",
      "zindex": 1,
      "from": {"data": "links"},
      "encode": {
        "update": {
          "shape": {"value": "triangle"},
          "angle": {"signal": "-datum.angle2-90"},
          "x": {"signal": "datum.arrowX"},
          "y": {"signal": "datum.arrowY"},
          "text": {"signal": "'▲'"},
          "fill": {
            "signal": "datum.datum.source.index!=nodeHover.id && datum.datum.target.index!=nodeHover.id ? '#929399':merge(hsl(scale('color', datum.datum.source.group)), {l:0.64})"
          },
          "size": {"signal": "nodeRadius==1?0:60"}
        }
      }
    },
...

(line 369) for linkDistance

            {
              "force": "link",
              "links": "link-data-raw",
              "distance": {"signal": "linkDistance"},
              "id": "name"
            }

To try to identify the problem, I tried to use static data:

      "name": "link-data-raw",
      "values": [
          {"source": "NodeA", "target": "NodeB", "value": 1},
          {"source": "NodeB", "target": "NodeC", "value": 2},
          {"source": "NodeA", "target": "NodeC", "value": 3}],
      "format": {
        "type": "json"
      }
      "name": "node-data",
      "values": [
                {"name": "NodeA", "group": "Group1"},
                {"name": "NodeB", "group": "Group2"},
                {"name": "NodeC", "group": "Group3"}],
      "format": {
        "type": "json"
      }

And if I add a transformation, in the "data" block, like this (for link only):

      "transform": [
        {
          "type": "formula",
          "as": "source",
          "expr": "datum.source"
        },
        {
          "type": "formula",
          "as": "target",
          "expr": "datum.target"
        },
        {
          "type": "formula",
          "as": "value",
          "expr": "datum.value"
        }
      ]

I get my problems with arrows and linkDistance, however if I remove this transformation block, it works perfectly.

So, in fact in this case this transformation is useless anyway, but since in Kibana the data is encapsulated in _source, then I have to carry out a preliminary transformation.
I don't understand why this transformation poses a problem, yet we send back the same information? This allows you to access the data via "datum.source"....

Can you enlighten me on this problem?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.