# Filter columns that are in array,

**URL:** <https://discuss.elastic.co/t/filter-columns-that-are-in-array/147487>\
**Category:** Elasticsearch\
**Created:** [September 6, 2018, 3:47am UTC](https://discuss.elastic.co/t/filter-columns-that-are-in-array/147487 "2018-09-06T03:47:16Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![aashishgahlawat](https://avatars.discourse-cdn.com/v4/letter/a/b9bd4f/32.png) [@aashishgahlawat](https://discuss.elastic.co/u/aashishgahlawat)\
**Post date:** [September 6, 2018, 3:47am UTC](https://discuss.elastic.co/t/filter-columns-that-are-in-array/147487/1 "2018-09-06T03:47:16Z")

</div>

I have:

```
1 2 3 4 5
1 55 4 8 7
8 6 80 40 1

```

I have my `pandas` `dataframe` saved on `elasticsearch` `index`, each row is a document.  
  
Now I want to retrieve all columns that are in list `[1,5]`

i.e. output shall be:

```
1 5
1 7
8 1

```

When I do :

```
from elasticsearch import Elasticsearch
from pandas.io.json import json_normalize
res = es.search(index="index_name", body={ "query": {"match_all": {}}})
df = json_normalize(res['hits']['hits'])

```

I get my `dataframe`, but I don't want to load it completely, just want to retrieve specific columns, how do I do so?

**Probably this won't help me completely, but will be done via filter only** [ES filter-term-array](https://www.elastic.co/guide/en/elasticsearch/guide/current/_finding_multiple_exact_values.html)

# UPDATED

Documents on ES are like:

```
{
    "_index": "index_name",
    "_type": "dataframe",
    "_id": "0",
    "_score": 1,
    "_source": {
      "1": 1,
      "2": 55,
      "3": 4,
      "4": 8,
      "5": 7,
      "index": 0
    }

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 4, 2018, 3:47am UTC](https://discuss.elastic.co/t/filter-columns-that-are-in-array/147487/2 "2018-10-04T03:47:20Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.