# Generate multiple scripted fields for the whole index to filter and aggregate

**URL:** <https://discuss.elastic.co/t/generate-multiple-scripted-fields-for-the-whole-index-to-filter-and-aggregate/58795>\
**Category:** Elasticsearch\
**Created:** [August 24, 2016, 9:41am UTC](https://discuss.elastic.co/t/generate-multiple-scripted-fields-for-the-whole-index-to-filter-and-aggregate/58795 "2016-08-24T09:41:34Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![dmitry\_polushkin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dmitry_polushkin/32/1326_2.png) [@dmitry\_polushkin](https://discuss.elastic.co/u/dmitry_polushkin)\
**Post date:** [August 24, 2016, 9:41am UTC](https://discuss.elastic.co/t/generate-multiple-scripted-fields-for-the-whole-index-to-filter-and-aggregate/58795/1 "2016-08-24T09:41:34Z")

</div>

I would like to generate multiple scripted fields for the whole index (per each document in the index), for `post_filter` and aggregate (with filter) by those dynamic values.

As I understand, at the moment the only solution to get the scripted value through all the documents in the index to use score, but it has limitations of:

- it's not possible to use in the post\_filter
- it's not allowed to have multiple scores

What should I do in such case?

---

<div class="post-metadata">

**Author:** ![gm42](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gm42/32/11804_2.png) [@gm42](https://discuss.elastic.co/u/gm42)\
**Post date:** [September 26, 2016, 7:14am UTC](https://discuss.elastic.co/t/generate-multiple-scripted-fields-for-the-whole-index-to-filter-and-aggregate/58795/2 "2016-09-26T07:14:54Z")

</div>

You could go a long way with Lucene expressions (because they're fast) and OR filters on the score to store multiple fields on a single score, but you would still bump a wall with the post\_filter.

The only possible solution I can think of is some custom metric aggregation, since there you have more control on the map/reduce, but again - I have not experimented with that and it would still be a terrible hack.

---

<div class="post-metadata">

**Author:** ![dmitry\_polushkin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dmitry_polushkin/32/1326_2.png) [@dmitry\_polushkin](https://discuss.elastic.co/u/dmitry_polushkin)\
**Post date:** [September 26, 2016, 9:05am UTC](https://discuss.elastic.co/t/generate-multiple-scripted-fields-for-the-whole-index-to-filter-and-aggregate/58795/3 "2016-09-26T09:05:49Z")

</div>

@gm42 can you please point a link how to use Lucene expressions to store multiple fields on a single score?  
I can think of pushing different values within one number as such (100 and 200): 10000000000200

---

<div class="post-metadata">

**Author:** ![dmitry\_polushkin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dmitry_polushkin/32/1326_2.png) [@dmitry\_polushkin](https://discuss.elastic.co/u/dmitry_polushkin)\
**Post date:** [September 26, 2016, 9:07am UTC](https://discuss.elastic.co/t/generate-multiple-scripted-fields-for-the-whole-index-to-filter-and-aggregate/58795/4 "2016-09-26T09:07:56Z")

</div>

I think to replace everything with mongodb, as mongo has aggregation pipeline.

---

<div class="post-metadata">

**Author:** ![gm42](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gm42/32/11804_2.png) [@gm42](https://discuss.elastic.co/u/gm42)\
**Post date:** [September 26, 2016, 12:16pm UTC](https://discuss.elastic.co/t/generate-multiple-scripted-fields-for-the-whole-index-to-filter-and-aggregate/58795/5 "2016-09-26T12:16:21Z")

</div>

Sorry, I was in a meeting and couldn't reply before.

In order to store multiple fields in the score one needs to treat it as a bitfield e.g. you assign bits 0 to 4 for a field, then bits 5 to 9 to another field etc.

This can only work if you need less than the total amount of bits available (which is 53 for an IEEE floating point).

As per aggregation pipeline: ElasticSearch has it too, see [https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-scripted-metric-aggregation.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-scripted-metric-aggregation.html)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 10:17pm UTC](https://discuss.elastic.co/t/generate-multiple-scripted-fields-for-the-whole-index-to-filter-and-aggregate/58795/6 "2017-07-05T22:17:20Z")

</div>


