# Getting all terms/tokens for a field in a custom scoring script (in java)

**URL:** <https://discuss.elastic.co/t/getting-all-terms-tokens-for-a-field-in-a-custom-scoring-script-in-java/26351>\
**Category:** Elasticsearch\
**Created:** [July 27, 2015, 4:49pm UTC](https://discuss.elastic.co/t/getting-all-terms-tokens-for-a-field-in-a-custom-scoring-script-in-java/26351 "2015-07-27T16:49:00Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![lucaw](https://avatars.discourse-cdn.com/v4/letter/l/bc79bd/32.png) [@lucaw](https://discuss.elastic.co/u/lucaw)\
**Post date:** [July 27, 2015, 4:49pm UTC](https://discuss.elastic.co/t/getting-all-terms-tokens-for-a-field-in-a-custom-scoring-script-in-java/26351/1 "2015-07-27T16:49:00Z")

</div>

I have a similar question as to the one asked [here](https://discuss.elastic.co/t/native-scoring-script-iterate-over-indexed-terms-of-a-field/20750) but as that one has had no responses and is over 6 months old I hope it is ok to ask again.

I would like to implement a custom scoring function (as a native script in Java) that uses, as inputs, a collection of [common learning to rank features](http://research.microsoft.com/en-us/projects/mslr/feature.aspx). Given my understanding of custom scoring all of the logic for this computation is placed inside a function in a class extending [AbstractSearchScript](http://grepcode.com/file/repo1.maven.org/maven2/org.elasticsearch/elasticsearch/1.4.4/org/elasticsearch/script/AbstractSearchScript.java#AbstractSearchScript). In this context an [IndexLookup](http://grepcode.com/file/repo1.maven.org/maven2/org.elasticsearch/elasticsearch/1.4.4/org/elasticsearch/search/lookup/IndexLookup.java) object is accessible (allowing access to TF/IDF information for particular fields via its 'get' function which returns a [IndexField](http://grepcode.com/file/repo1.maven.org/maven2/org.elasticsearch/elasticsearch/1.4.4/org/elasticsearch/search/lookup/IndexField.java#IndexField) object).

Computing the features I would like requires that I know all of the terms/tokens for particular fields so that I can use the 'get' function of the IndexField class to retrieve TF/IDF information about the terms (in particular the summary functions sumdf and sumttf of the IndexField class are too coarse for me). Unfortunately I don't know what these terms are in general, as they are different for every query, and would have to retrieve them dynamically. Is this possible while in the custom scoring context? If not is there a better way to accomplish the above?

Thanks!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 11:58pm UTC](https://discuss.elastic.co/t/getting-all-terms-tokens-for-a-field-in-a-custom-scoring-script-in-java/26351/2 "2017-07-05T23:58:49Z")

</div>


