# Implement the prune filter using painless script in ingest pipeline

**URL:** <https://discuss.elastic.co/t/implement-the-prune-filter-using-painless-script-in-ingest-pipeline/218619>\
**Category:** Elasticsearch\
**Tags:** painless\
**Created:** [February 10, 2020, 3:18pm UTC](https://discuss.elastic.co/t/implement-the-prune-filter-using-painless-script-in-ingest-pipeline/218619 "2020-02-10T15:18:30Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sarfaraz\_Ahmad](https://avatars.discourse-cdn.com/v4/letter/s/67e7ee/32.png) [@Sarfaraz\_Ahmad](https://discuss.elastic.co/u/Sarfaraz_Ahmad)\
**Post date:** [February 10, 2020, 3:18pm UTC](https://discuss.elastic.co/t/implement-the-prune-filter-using-painless-script-in-ingest-pipeline/218619/1 "2020-02-10T15:18:30Z")

</div>

Hi,

I see a remove processor for ingest pipelines in Elasticsearch. But I don't see any counterparts for logstash's prune filter. How can I implement one using painless ?  
Also I am unable to create deep copies of ctx.keySet(),  
so I can iterate over this new set and then ctx.remove() unnecessary fields from my document.

Any pointers would be welcome.

Regards,  
Ahmad

---

<div class="post-metadata">

**Author:** ![Sarfaraz\_Ahmad](https://avatars.discourse-cdn.com/v4/letter/s/67e7ee/32.png) [@Sarfaraz\_Ahmad](https://discuss.elastic.co/u/Sarfaraz_Ahmad)\
**Post date:** [February 10, 2020, 5:07pm UTC](https://discuss.elastic.co/t/implement-the-prune-filter-using-painless-script-in-ingest-pipeline/218619/2 "2020-02-10T17:07:16Z")

</div>

I got it. This works. But it seems dirty and performance intensive 😑

```
   {
 "script": {
   "lang": "painless",
   "source": """
    ctx.to_keep = new ArrayList();
    ctx.to_drop = new ArrayList();
    ctx.to_keep = ['field1', 'field2'];
    for (key in ctx.keySet()) {
      if (! ctx.to_keep.contains(key)) {
        ctx.to_drop.add(key);
      }
    }
    for (key in ctx.to_drop) {
        ctx.remove(key);
    }
   """
 }

```

I am not sure how efficient this is. Can someone from elastic please take a look here ?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 9, 2020, 5:07pm UTC](https://discuss.elastic.co/t/implement-the-prune-filter-using-painless-script-in-ingest-pipeline/218619/3 "2020-03-09T17:07:16Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.