# Vegalite - Filter Flatten Data

**URL:** <https://discuss.elastic.co/t/vegalite-filter-flatten-data/323123>\
**Category:** Kibana\
**Tags:** vega\
**Created:** [January 13, 2023, 11:46am UTC](https://discuss.elastic.co/t/vegalite-filter-flatten-data/323123 "2023-01-13T11:46:27Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Siva\_Kumar\_Menta](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/siva_kumar_menta/32/104159_2.png) [@Siva\_Kumar\_Menta](https://discuss.elastic.co/u/Siva_Kumar_Menta)\
**Post date:** [January 13, 2023, 11:46am UTC](https://discuss.elastic.co/t/vegalite-filter-flatten-data/323123/1 "2023-01-13T11:46:27Z")

</div>

Hi All,

I have data in Elasticsearch index "testdata" with few fields in array format. While trying to apply filter on flattened data @ line #18 as {"filter":"ms \> 1"} getting error. Syntax wise it looks fine but not sure about cause of error.

Please suggest modifications required in script.

Error Screenshot:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/d/7/d72089e1f7dd516f98fcc062a7a3ab293fa6b440.png)

Sample Index Data:

```auto
{"index": {}}
{"SNI#": "HCP1", "filePrefix": "TR37609971", "vehicle": "TE1484625", "ECU": "HCP1_CANFD04", "filedate": "2022-10-18", "threshold": 200, "ECU_SNI": "HCP1_CANFD04-HCP1", "maxms": 69, "minms": 0, "TotalFreq": 0, "avgFreq": 21, "maxFreq": 0, "maxFreqDur": 0, "maxDur": 0, "maxDurFreq": 0, "Frequency": [2, 1, 1, 1], "ms": [0, 14, 20, 69]},{"index": {}},
{"SNI#": "HCP1", "filePrefix": "TR37609901", "vehicle": "TE1484610", "ECU": "HCP1_CANFD04", "filedate": "2022-10-18", "threshold": 200, "ECU_SNI": "HCP1_CANFD04-HCP1", "maxms": 70, "minms": 0, "TotalFreq": 0, "avgFreq": 21, "maxFreq": 0, "maxFreqDur": 0, "maxDur": 0, "maxDurFreq": 0, "Frequency": [2, 1, 1], "ms": [0, 14, 70]}

```

Syntax:

```auto
{
  "$schema": "https://vega.github.io/schema/vega-lite/v5.json",
  "title": "",
  "data": {
    "url": {
      "index": "testdata",
      "%context%": true, "body": {"size": 10000}
    },
    "format": {"property": "hits.hits"}
  }, 
   "transform": [
    {"calculate": "datum._source.ECU_SNI", "as": "ECU_SNI"},
    {"calculate": "datum._source.ECU", "as": "ECU"},    
    {"calculate": "datum._source.ms", "as": "ms"},
    {"calculate": "datum._source.Frequency", "as": "Frequency"},
    {"flatten": ["ms", "Frequency"] },

    {"filter":"ms > 1"},
    {
      "aggregate": [{
       "op": "sum",
       "field": "Frequency",
       "as": "TotFrequency"
      }],
      "groupby": ["ECU_SNI","ECU"] }
      ],
  
  "layer": [{
   "mark": {"type": "bar", "filled": true},
  "encoding": {
    
    "x": {"field": "TotFrequency", "type": "quantitative","title":"Frequency","aggregate": "sum"},
    "y": {"field": "ECU_SNI", "type": "nominal","title": "ECU_SNI"},
    
    "color": {"field": "ECU","title":"ECU","orient":"top"},
    
    "tooltip": [
      {"field": "ECU_SNI", "type": "ordinal", "title": "SNI", "tooltip": true},
      {"field": "TotFrequency", "type": "quantitative", "title": "frequency", "tooltip": true}
    ]
  }
 }],
  "config": {
    "legend": {"orient": "top", "layout": {"top": {"anchor": "left"}}}
  }
}

```

Thanks for reading through my problem / your help!  
Siva Kumar

---

<div class="post-metadata">

**Author:** ![Siva\_Kumar\_Menta](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/siva_kumar_menta/32/104159_2.png) [@Siva\_Kumar\_Menta](https://discuss.elastic.co/u/Siva_Kumar_Menta)\
**Post date:** [January 17, 2023, 9:21am UTC](https://discuss.elastic.co/t/vegalite-filter-flatten-data/323123/2 "2023-01-17T09:21:09Z")

</div>

> [@Siva\_Kumar\_Menta](#):
>
> `{"filter":"ms > 1"},`

Hi All,

Any inputs on the problem posted. Please note that I have a deliverable in 2 days so please suggest whether it's possible to apply filter on flattened data. If not any alternatives.

Looking forward for responses.

Thanks for your help!  
Siva Kumar

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 14, 2023, 9:21am UTC](https://discuss.elastic.co/t/vegalite-filter-flatten-data/323123/3 "2023-02-14T09:21:16Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.