# Wondering that can we lookup on a list to remove the field not on the list

**URL:** <https://discuss.elastic.co/t/wondering-that-can-we-lookup-on-a-list-to-remove-the-field-not-on-the-list/204018>\
**Category:** Logstash\
**Created:** [October 17, 2019, 10:10am UTC](https://discuss.elastic.co/t/wondering-that-can-we-lookup-on-a-list-to-remove-the-field-not-on-the-list/204018 "2019-10-17T10:10:29Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Izek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/izek/32/37113_2.png) [@Izek](https://discuss.elastic.co/u/Izek)\
**Post date:** [October 17, 2019, 10:10am UTC](https://discuss.elastic.co/t/wondering-that-can-we-lookup-on-a-list-to-remove-the-field-not-on-the-list/204018/1 "2019-10-17T10:10:29Z")

</div>

Hi folks,

We have an index with so many data form different application.  
we are currently looking for a way like schema restriction and not allow people to generate new field.  
Is there any way to use logstash to lookup on a list or sql data lookup, then remove all the other field not on the list.  
We had look at prune plugin, but we have thousands field out there.  
We cannot use logstash config to deal with thousands of field name

Let me know if you need more explain  
Appreciate it.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [October 17, 2019, 3:21pm UTC](https://discuss.elastic.co/t/wondering-that-can-we-lookup-on-a-list-to-remove-the-field-not-on-the-list/204018/2 "2019-10-17T15:21:38Z")

</div>

If you are writing data to elasticsearch you could turn off [dynamic mapping](https://www.elastic.co/guide/en/elasticsearch/reference/current/dynamic-mapping.html). If you need to do it in logstash you would have to write it in a ruby filter.

---

<div class="post-metadata">

**Author:** ![Izek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/izek/32/37113_2.png) [@Izek](https://discuss.elastic.co/u/Izek)\
**Post date:** [October 18, 2019, 2:19am UTC](https://discuss.elastic.co/t/wondering-that-can-we-lookup-on-a-list-to-remove-the-field-not-on-the-list/204018/3 "2019-10-18T02:19:37Z")

</div>

Dynamic mapping is good. but not in this case.  
If we enable dynamic mapping, then it will create tons of field.  
But it you disable it, it will drop the record even there only have one field you don't want to  
Currently we already have more than 7000 field.  
We are try to find a good way to control it.  
But we dont want to manage it manually.  
To manage the template with thousands of fields is terrible

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 15, 2019, 2:19am UTC](https://discuss.elastic.co/t/wondering-that-can-we-lookup-on-a-list-to-remove-the-field-not-on-the-list/204018/4 "2019-11-15T02:19:39Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.