# Dynamic fetch data from index to generate TreeFlow

**URL:** <https://discuss.elastic.co/t/dynamic-fetch-data-from-index-to-generate-treeflow/268993>\
**Category:** Kibana\
**Tags:** vega\
**Created:** [April 1, 2021, 7:31am UTC](https://discuss.elastic.co/t/dynamic-fetch-data-from-index-to-generate-treeflow/268993 "2021-04-01T07:31:08Z")\
**Posts on this page:** 12\
**Page:** 2

<div class="post-metadata">

**Author:** ![anjana1](https://avatars.discourse-cdn.com/v4/letter/a/22d042/32.png) [@anjana1](https://discuss.elastic.co/u/anjana1)\
**Post date:** [April 6, 2021, 10:13am UTC](https://discuss.elastic.co/t/dynamic-fetch-data-from-index-to-generate-treeflow/268993/21 "2021-04-06T10:13:49Z")

</div>

Yup document need to be skipped .. OK will explore that option .. Thanks

---

<div class="post-metadata">

**Author:** ![anjana1](https://avatars.discourse-cdn.com/v4/letter/a/22d042/32.png) [@anjana1](https://discuss.elastic.co/u/anjana1)\
**Post date:** [April 7, 2021, 9:51am UTC](https://discuss.elastic.co/t/dynamic-fetch-data-from-index-to-generate-treeflow/268993/22 "2021-04-07T09:51:57Z")

</div>

@markov00 Could you pls confirm if we can collapse and expand this tree flow. We have too much data and its cluttered . So wanted to check if we can collapse and expand the tree

---

<div class="post-metadata">

**Author:** ![markov00](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/markov00/32/33316_2.png) [@markov00](https://discuss.elastic.co/u/markov00)\
**Post date:** [April 7, 2021, 10:42am UTC](https://discuss.elastic.co/t/dynamic-fetch-data-from-index-to-generate-treeflow/268993/23 "2021-04-07T10:42:15Z")

</div>

mmm, I'm pretty sure something can be done but it can be complex.  
For sure you can filter out by `depth`, like showing only the first two layers for example, but collapse/expand on click can be probably a bit more complicated.

---

<div class="post-metadata">

**Author:** ![Ashvin\_Bhutekar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ashvin_bhutekar/32/86899_2.png) [@Ashvin\_Bhutekar](https://discuss.elastic.co/u/Ashvin_Bhutekar)\
**Post date:** [April 14, 2021, 7:01am UTC](https://discuss.elastic.co/t/dynamic-fetch-data-from-index-to-generate-treeflow/268993/24 "2021-04-14T07:01:37Z")

</div>

Hey Anjana,  
could you please share the final code, with which you were able to create tree flow dynamically? I am working on a similar thing.

---

<div class="post-metadata">

**Author:** ![anjana1](https://avatars.discourse-cdn.com/v4/letter/a/22d042/32.png) [@anjana1](https://discuss.elastic.co/u/anjana1)\
**Post date:** [April 14, 2021, 9:26am UTC](https://discuss.elastic.co/t/dynamic-fetch-data-from-index-to-generate-treeflow/268993/25 "2021-04-14T09:26:47Z")

</div>

```auto
{
  "$schema": "https://vega.github.io/schema/vega/v5.json",
  "description": "An example of Cartesian layouts for a node-link diagram of hierarchical data.",
  "width": 600,
  "height": 600,
  "padding": 5,
  "signals": [
    {
      "name": "labels",
      "value": true,
      "bind": {
        "input": "checkbox"
      }
    },
    {
      "name": "layout",
      "value": "tidy",
      "bind": {
        "input": "radio",
        "options": [
          "tidy",
          "cluster"
        ]
      }
    },
    {
      "name": "links",
      "value": "diagonal",
      "bind": {
        "input": "select",
        "options": [
          "line",
          "curve",
          "diagonal",
          "orthogonal"
        ]
      }
    },
    {
      "name": "separation",
      "value": false,
      "bind": {
        "input": "checkbox"
      }
    }
  ],
  "data": [
    {
      "name": "table",
      "url": {
      "%context%": true,
      "%timefield%": "@timestamp",
      "index": "apm-*",
      "body": {
        "aggs": {
          "time_buckets": {
            "date_histogram": {
              "field": "@timestamp",
              "interval": {"%autointerval%": true},
              "extended_bounds": {
                "min": {"%timefilter%": "min"},
                "max": {"%timefilter%": "max"}
              },
              "min_doc_count": 0
            }
          }
        },
        "size": 10
     } }
      "format": {"property": "hits.hits"
      } ,
   
      "transform": [
      
      {"type": "formula", "as": "OrgId", "expr": "datum._source['labels.OrgId']"},
       {"type": "formula", "as": "Region", "expr": "datum._source.OrgHier.Region"},
       {"type": "formula", "as": "StoreNum", "expr": "datum._source['labels.StoreNum']"},
        {"type": "formula", "as": "RegisterNum", "expr": "datum._source['labels.RegisterNum']"},
        {"type": "formula", "as": "ServiceName", "expr": "datum._source.ServiceName"},
        {"type": "formula", "as": "ServiceSummary", "expr": "datum._source.ServiceSummary"},
        {
          "type": "fold",
          "fields": [
            "OrgId",
            "Region",
            "StoreNum",
            "RegisterNum",
            "ServiceName",
            "ServiceSummary"
          ]
        },
        {
          "type": "formula",
        "as": "id",
          "expr": "datum.key == 'OrgId' ? datum.OrgId : datum.key == 'Region' ? join([datum.OrgId,datum.Region],'-') : datum.key == 'StoreNum' ? join([datum.OrgId, datum.Region, datum.StoreNum],'-') : datum.key == 'RegisterNum' ? join([datum.OrgId, datum.Region, datum.StoreNum, datum.RegisterNum],'-') : datum.key == 'ServiceName' ? join([datum.OrgId, datum.Region, datum.StoreNum, datum.RegisterNum,datum.ServiceName],'-'):datum.key == 'ServiceSummary' ? join([datum.OrgId, datum.Region, datum.StoreNum, datum.RegisterNum,datum.ServiceName,datum.ServiceSummary],'-'):null"
        },
        {
          "type": "formula",
          "as": "parent",
          "expr": "datum.key == 'OrgId' ? null : datum.key == 'Region' ? datum.OrgId : datum.key == 'StoreNum' ? join([datum.OrgId, datum.Region],'-') : datum.key == 'RegisterNum' ? join([datum.OrgId, datum.Region, datum.StoreNum],'-') : datum.key == 'ServiceName' ? join([datum.OrgId, datum.Region, datum.StoreNum,datum.RegisterNum],'-'): datum.key == 'ServiceSummary' ? join([datum.OrgId, datum.Region, datum.StoreNum,datum.RegisterNum,datum.ServiceName],'-'):null"
        },
        {
          "type": "filter",
          "expr": "isDefined(datum.value)"
        },
        {
          "type": "aggregate",
          "groupby": [
            "id",
            "parent"
          ]
        },
        {
          "type": "formula",
          "as": "label",
          "expr": "peek(split(datum.id,'-'))"
        }
      ]
    },
    {
      "name": "tree",
      "source": "table",
      "transform": [
        {
          "type": "stratify",
          "key": "id",
          "parentKey": "parent"
        },
        {
          "type": "tree",
          "method": {
            "signal": "layout"
          },
          "size": [
            {
              "signal": "height"
            },
            {
              "signal": "width - 100"
            }
          ],
          "separation": {
            "signal": "separation"
          },
          "as": [
            "y",
            "x",
            "depth",
            "children"
          ]
        }
      ]
    },
    {
      "name": "links",
      "source": "tree",
      "transform": [
        {
          "type": "treelinks"
        },
        {
          "type": "linkpath",
          "orient": "horizontal",
          "shape": {
            "signal": "links"
          }
        }
      ]
    }
  ],
  "scales": [
    {
      "name": "color",
      "type": "linear",
      "range": {
        "scheme": "magma"
      },
      "domain": {
        "data": "tree",
        "field": "depth"
      },
      "zero": true
    }
  ],
  "marks": [
    {
      "type": "path",
      "from": {
        "data": "links"
      },
      "encode": {
        "update": {
          "path": {
            "field": "path"
          },
          "stroke": {
            "value": "#ccc"
          }
        }
      }
    },
    {
      "type": "symbol",
      "from": {
        "data": "tree"
      },
      "encode": {
        "enter": {
          "size": {
            "value": 100
          },
          "stroke": {
            "value": "#fff"
          }
        },
        "update": {
          "x": {
            "field": "x"
          },
          "y": {
            "field": "y"
          },
          "fill": {
            "scale": "color",
            "field": "depth"
          }
        }
      }
    },
    {
      "type": "text",
      "from": {
        "data": "tree"
      },
      "encode": {
        "enter": {
          "text": {
            "field": "label"
          },
          "fontSize": {
            "value": 9
          },
          "baseline": {
            "value": "middle"
          }
        },
        "update": {
          "x": {
            "field": "x"
          },
          "y": {
            "field": "y"
          },
          "dx": {
            "signal": "datum.children ? -7 : 7"
          },
          "align": {
            "signal": "datum.children ? 'right' : 'left'"
          },
          "opacity": {
            "signal": "labels ? 1 : 0"
          }
        }
      }
    }
  ]
}

```

---

<div class="post-metadata">

**Author:** ![Ashvin\_Bhutekar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ashvin_bhutekar/32/86899_2.png) [@Ashvin\_Bhutekar](https://discuss.elastic.co/u/Ashvin_Bhutekar)\
**Post date:** [April 14, 2021, 11:35am UTC](https://discuss.elastic.co/t/dynamic-fetch-data-from-index-to-generate-treeflow/268993/26 "2021-04-14T11:35:59Z")

</div>

Thank you very much, Anjana! In my case, it shows multiple roots or ambiguous errors. As I am dealing with Urls child may refer to multiple parents (causing multiple root or ambiguous errors). Is it possible to resolve this ambiguity or multiple root errors? please suggest.

---

<div class="post-metadata">

**Author:** ![anjana1](https://avatars.discourse-cdn.com/v4/letter/a/22d042/32.png) [@anjana1](https://discuss.elastic.co/u/anjana1)\
**Post date:** [April 15, 2021, 7:45am UTC](https://discuss.elastic.co/t/dynamic-fetch-data-from-index-to-generate-treeflow/268993/27 "2021-04-15T07:45:51Z")

</div>

Hi @markov00 ,

I had another question here. The values that are shows here, like 1001, EAST etc how do we change its color. I have a black background and the values are not readable now Thanks

---

<div class="post-metadata">

**Author:** ![markov00](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/markov00/32/33316_2.png) [@markov00](https://discuss.elastic.co/u/markov00)\
**Post date:** [April 15, 2021, 8:59am UTC](https://discuss.elastic.co/t/dynamic-fetch-data-from-index-to-generate-treeflow/268993/28 "2021-04-15T08:59:20Z")

</div>

hi @anjana1  
Looking at your vega spec, one thing you can try is to change the color scale to ordinal (that should at least work better then the linear in your case)  
something like that should work:

```auto
 {
      "name": "color",
      "type": "ordinal",
      "range": {"scheme": "category20"},
      "domain": {
        "data": "tree",
        "field": "depth"
      },
      "zero": true
    }

```

and try one of the color schemes allowed: [Color Schemes | Vega](https://vega.github.io/vega/docs/schemes/)

---

<div class="post-metadata">

**Author:** ![anjana1](https://avatars.discourse-cdn.com/v4/letter/a/22d042/32.png) [@anjana1](https://discuss.elastic.co/u/anjana1)\
**Post date:** [April 15, 2021, 9:04am UTC](https://discuss.elastic.co/t/dynamic-fetch-data-from-index-to-generate-treeflow/268993/29 "2021-04-15T09:04:00Z")

</div>

Thanks @markov00

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/2/6/26a264df605fe88f948af50282270f6b52e02771.png)  
tried and it didn't change the text color . It only changes those node colors.

---

<div class="post-metadata">

**Author:** ![markov00](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/markov00/32/33316_2.png) [@markov00](https://discuss.elastic.co/u/markov00)\
**Post date:** [April 15, 2021, 9:53am UTC](https://discuss.elastic.co/t/dynamic-fetch-data-from-index-to-generate-treeflow/268993/30 "2021-04-15T09:53:21Z")

</div>

oh sorry, I was thinking at the nodes, sorry. For labels just add the `fill` to the text like:

```auto
{
      "type": "text",
      "from": {
        "data": "tree"
      },
      "encode": {
        "enter": {
          "text": {
            "field": "label"
          },
          "fill": {
            "value": "white"
          },
          "fontSize": {
            "value": 9
          },
          "baseline": {
            "value": "middle"
          }
        },

```

You can use css colors or hex values

---

<div class="post-metadata">

**Author:** ![markov00](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/markov00/32/33316_2.png) [@markov00](https://discuss.elastic.co/u/markov00)\
**Post date:** [April 15, 2021, 1:28pm UTC](https://discuss.elastic.co/t/dynamic-fetch-data-from-index-to-generate-treeflow/268993/31 "2021-04-15T13:28:00Z")

</div>

> [@Ashvin\_Bhutekar](#):
>
> As I am dealing with Urls child may refer to multiple parents (causing multiple root or ambiguous errors). Is it possible to resolve this ambiguity or multiple root errors? please suggest.

hei @Ashvin_Bhutekar can you please provide an example for that dataset and the kind of error you are receiving?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 13, 2021, 1:28pm UTC](https://discuss.elastic.co/t/dynamic-fetch-data-from-index-to-generate-treeflow/268993/32 "2021-05-13T13:28:56Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

[Previous page](https://discuss.elastic.co/t/dynamic-fetch-data-from-index-to-generate-treeflow/268993.md?page=1)