# ES scripting help

**URL:** <https://discuss.elastic.co/t/es-scripting-help/97170>\
**Category:** Elasticsearch\
**Created:** [August 16, 2017, 2:48am UTC](https://discuss.elastic.co/t/es-scripting-help/97170 "2017-08-16T02:48:55Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ramon\_Zaro](https://avatars.discourse-cdn.com/v4/letter/r/2bfe46/32.png) [@Ramon\_Zaro](https://discuss.elastic.co/u/Ramon_Zaro)\
**Post date:** [August 16, 2017, 2:48am UTC](https://discuss.elastic.co/t/es-scripting-help/97170/1 "2017-08-16T02:48:55Z")

</div>

Hi all,  
I am developing a filesearch solution using fscrawler to push data into ES 5.5.0

the issue I am facing is a fields explosion (it's crossing 1000 fields in no time) as fscrawler creates new fields dynamically, 95% of which are "meta.\*" fields.  
details of the issue are here if anyone is interested : [Filesearch solution using ES 5.5.0](https://discuss.elastic.co/t/filesearch-solution-using-es-5-5-0/94377)

the solution I can see is using the remove processor in ingest node to get rid of the meta.\* fields.

I have tried directly using remove on meta.\* fields but that throws a javalang exception.

The only way around seems to be using script processor to extract the meta.\* fields and then using the remove to get rid of them.  
thing is, I have no experience of this kind of thing. how do I access the fields in ingest node in the first place ?  
any pointers would be much appreciated.

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [August 16, 2017, 1:16pm UTC](https://discuss.elastic.co/t/es-scripting-help/97170/2 "2017-08-16T13:16:40Z")

</div>

if you shared what you did, along with a sample document, pipeline and your setup (mapping) it would be tremendously helpful.

You can also configure the properties as part of the pipeline configuraiton, see [https://www.elastic.co/guide/en/elasticsearch/plugins/5.5/using-ingest-attachment.html](https://www.elastic.co/guide/en/elasticsearch/plugins/5.5/using-ingest-attachment.html)

---

<div class="post-metadata">

**Author:** ![Ramon\_Zaro](https://avatars.discourse-cdn.com/v4/letter/r/2bfe46/32.png) [@Ramon\_Zaro](https://discuss.elastic.co/u/Ramon_Zaro)\
**Post date:** [August 28, 2017, 3:23am UTC](https://discuss.elastic.co/t/es-scripting-help/97170/3 "2017-08-28T03:23:22Z")

</div>

sample document : could be anything, mostly .doc, docx, .pdf, .xls, .txt and some image/audio/video files.

because of this large variation in filetype I want just file.properties and content.

I am using ingest node from fscrawler to ES, as explained here : [https://github.com/dadoonet/fscrawler#using-ingest-node-pipeline](https://github.com/dadoonet/fscrawler#using-ingest-node-pipeline)

fscrawler creates mappings automatically, as it encounters new fields.

I want to write a script to pick all "meta.\*" fields it creates and get rid of them using 'remove' processor before ingesting the data in ES.

is that enough to go on or am I missing something ?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 25, 2017, 3:23am UTC](https://discuss.elastic.co/t/es-scripting-help/97170/4 "2017-09-25T03:23:22Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.