# Pushed: more script goodness, including script fields, scripted facets, and script filter
**URL:** https://discuss.elastic.co/t/pushed-more-script-goodness-including-script-fields-scripted-facets-and-script-filter/3021
**Category:** Elasticsearch
**Created:** [June 16, 2010, 1:56am UTC](https://discuss.elastic.co/t/pushed-more-script-goodness-including-script-fields-scripted-facets-and-script-filter/3021 "2010-06-16T01:56:42Z")
**Posts on this page:** 4
**Page:** 1
### Author:  [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [June 16, 2010, 1:56am UTC](https://discuss.elastic.co/t/pushed-more-script-goodness-including-script-fields-scripted-facets-and-script-filter/3021/1 "2010-06-16T01:56:42Z")
Hi,
As promised, once I built the infrastructure, adding exciting features
becomes really simple. Here is a breakdown of some features added recently
(on top of the facets and custom\_score query):
- Script Fields:
The ability to return script evaluation per hit of fields that _are not
stored_. This make sense (as any other scripted field on numeric or
not\_analyzed fields). Issue is here:
[http://github.com/elasticsearch/elasticsearch/issues/221](http://github.com/elasticsearch/elasticsearch/issues/221). For example:
{
"query" : {
...
},
"script\_fields" : {
"test1" : {
"script" : "doc['my\_field\_name'].value \* 2"
},
"test2" : {
"script" : "doc['my\_field\_name'].value \* factor",
"params" : {
"factor" : 2.0
}
}
}
}
- Script filter
Allows to define a filter that uses a script to filter out results. Issue
here: [http://github.com/elasticsearch/elasticsearch/issues/226](http://github.com/elasticsearch/elasticsearch/issues/226). Sample:
"filtered" : {
"query" : {
...
},
"filter" : {
"script" : {
"script" : "doc['num1'].value \> 1"
}
}
}
- Scripted facets
Both statistical and histogram facets now allow to define scripts instead of
field names. Issues are:
[http://github.com/elasticsearch/elasticsearch/issues/227](http://github.com/elasticsearch/elasticsearch/issues/227), and
[http://github.com/elasticsearch/elasticsearch/issues/228](http://github.com/elasticsearch/elasticsearch/issues/228). Here is a sample
of of statistical facet:
{
"query" : {
"match\_all" : { }
},
"facets" : {
"test" : {
"statistical" : {
"script" : "(doc['num1'] + doc['num2']) \* factor",
"params" : {
"factor" : 2.2
}
},
"global" : false
}
}
}
-shay.banon
---
### Author:  [@otisg](https://discuss.elastic.co/u/otisg)
#### Post date: [June 16, 2010, 4:20am UTC](https://discuss.elastic.co/t/pushed-more-script-goodness-including-script-fields-scripted-facets-and-script-filter/3021/2 "2010-06-16T04:20:52Z")
Questions for HCG (Human Code Generator):
- Are script filters applied after the fact? (after the matches have
been scored and retrieved? If so, how does one deal with a situation
where N matches are needed at the end, but the filter(s) may end up
removing some unknown number of them? Does one simply have to get m x
N hits and hope that N or more hits are left after the filters?
- Can you explain what statistical and histogram facets are? The
example for the histogram shows, I think, that you can derive new
facet values from partial field values (e.g. from a date field you can
extract "minuteOfHour" and that becomes the new facet value)... but I
think I'm missing something here, otherwise you wouldn't call them
"histogram" facets?
Similarly, what makes statistical facets statistical? I can't figure
it out from the example in the issue, which I think shows that you can
create a new facet value from other fields.
- I think all examples I saw for the new "script" stuff involved
numeric or date fields. Can they be used with string fields, and if
so, can you give some examples? For example, if I have 2 fields,
"username" and "id", and they have 1:1 mapping, could I use the
"script" functionality to end up with facets of " "
format?
## Thanks, Otis
Sematext :: [http://sematext.com/](http://sematext.com/) :: Solr - Lucene - Nutch
Lucene ecosystem search :: [http://search-lucene.com/](http://search-lucene.com/)
On Jun 15, 9:56 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote:
> Hi,
>
> As promised, once I built the infrastructure, adding exciting features
> becomes really simple. Here is a breakdown of some features added recently
> (on top of the facets and custom\_score query):
>
> - Script Fields:
>
> The ability to return script evaluation per hit of fields that _are not
> stored_. This make sense (as any other scripted field on numeric or
> not\_analyzed fields). Issue is here:[Search: Allow to specify script fields to be loaded · Issue #221 · elastic/elasticsearch · GitHub](http://github.com/elasticsearch/elasticsearch/issues/221). For example:
>
> {
> "query" : {
> ...
> },
> "script\_fields" : {
> "test1" : {
> "script" : "doc['my\_field\_name'].value \* 2"
> },
> "test2" : {
> "script" : "doc['my\_field\_name'].value \* factor",
> "params" : {
> "factor" : 2.0
> }
> }
> }
>
> }
>
> - Script filter
>
> Allows to define a filter that uses a script to filter out results. Issue
> here:[Script Filter: Support providing a custom script as a filter · Issue #226 · elastic/elasticsearch · GitHub](http://github.com/elasticsearch/elasticsearch/issues/226). Sample:
>
> "filtered" : {
> "query" : {
> ...
> },
> "filter" : {
> "script" : {
> "script" : "doc['num1'].value \> 1"
> }
> }
>
> }
>
> - Scripted facets
>
> Both statistical and histogram facets now allow to define scripts instead of
> field names. Issues are:[Facets: Script statistical facets · Issue #227 · elastic/elasticsearch · GitHub](http://github.com/elasticsearch/elasticsearch/issues/227), andhttp://github.com/elasticsearch/elasticsearch/issues/228. Here is a sample
> of of statistical facet:
>
> {
> "query" : {
> "match\_all" : { }
> },
> "facets" : {
> "test" : {
> "statistical" : {
> "script" : "(doc['num1'] + doc['num2']) \* factor",
> "params" : {
> "factor" : 2.2
> }
> },
> "global" : false
> }
> }
>
> }
>
> -shay.banon
---
### Author:  [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [June 16, 2010, 8:35am UTC](https://discuss.elastic.co/t/pushed-more-script-goodness-including-script-fields-scripted-facets-and-script-filter/3021/3 "2010-06-16T08:35:51Z")
On Wed, Jun 16, 2010 at 7:20 AM, Otis [otis.gospodnetic@gmail.com](mailto:otis.gospodnetic@gmail.com) wrote:
> Questions for HCG (Human Code Generator):
>
> - Are script filters applied after the fact? (after the matches have
> been scored and retrieved? If so, how does one deal with a situation
> where N matches are needed at the end, but the filter(s) may end up
> removing some unknown number of them? Does one simply have to get m x
> N hits and hope that N or more hits are left after the filters?
Script filters work like any other filter, term filter / range filter and so
on. Its basically a Lucene Filer implementation.
> - Can you explain what statistical and histogram facets are? The
> example for the histogram shows, I think, that you can derive new
> facet values from partial field values (e.g. from a date field you can
> extract "minuteOfHour" and that becomes the new facet value)... but I
> think I'm missing something here, otherwise you wouldn't call them
> "histogram" facets?
> Similarly, what makes statistical facets statistical? I can't figure
> it out from the example in the issue, which I think shows that you can
> create a new facet value from other fields.
I explained it on another post (
[http://groups.google.com/a/elasticsearch.com/group/users/browse\_thread/thread/73a35a5168896b45#](http://groups.google.com/a/elasticsearch.com/group/users/browse_thread/thread/73a35a5168896b45#)),
the script is just an extension. The statistical one is explained here:
[Issues · elastic/elasticsearch · GitHub](http://github.com/elasticsearch/elasticsearch/issues/closed#issue/212). The
original one does min/max/count/mean/std for a numeric field, the script one
basically gives the same stats on a number value returned by the script. The
histogram ones is explained in the original post linked about, except now,
on top of fixed key and value fields, they can be scripts.
> - I think all examples I saw for the new "script" stuff involved
> numeric or date fields. Can they be used with string fields, and if
> so, can you give some examples? For example, if I have 2 fields,
> "username" and "id", and they have 1:1 mapping, could I use the
> "script" functionality to end up with facets of " "
> format?
Yes, they can be used with string fields, but only where it makes sense. For
example, script fields can return a concatenation of the "username" and "id"
if you want. For facets, you have the terms facet for strings, which has
nothing to do with scripts, other than that, in the statistical and
historical facets, the return values _must_ be numbers (they work on
numbers) but it does not mean you can't use the string fields in your
script.
> ## Thanks, Otis
>
> Sematext :: [http://sematext.com/](http://sematext.com/) :: Solr - Lucene - Nutch
> Lucene ecosystem search :: [http://search-lucene.com/](http://search-lucene.com/)
>
> On Jun 15, 9:56 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote:
>
> > Hi,
> >
> > As promised, once I built the infrastructure, adding exciting features
> > becomes really simple. Here is a breakdown of some features added
> > recently
> > (on top of the facets and custom\_score query):
> >
> > - Script Fields:
> >
> > The ability to return script evaluation per hit of fields that _are not
> > stored_. This make sense (as any other scripted field on numeric or
> > not\_analyzed fields). Issue is here:
> > [Search: Allow to specify script fields to be loaded · Issue #221 · elastic/elasticsearch · GitHub](http://github.com/elasticsearch/elasticsearch/issues/221). For example:
> >
> > {
> > "query" : {
> > ...
> > },
> > "script\_fields" : {
> > "test1" : {
> > "script" : "doc['my\_field\_name'].value \* 2"
> > },
> > "test2" : {
> > "script" : "doc['my\_field\_name'].value \* factor",
> > "params" : {
> > "factor" : 2.0
> > }
> > }
> > }
> >
> > }
> >
> > - Script filter
> >
> > Allows to define a filter that uses a script to filter out results. Issue
> > here:[Script Filter: Support providing a custom script as a filter · Issue #226 · elastic/elasticsearch · GitHub](http://github.com/elasticsearch/elasticsearch/issues/226). Sample:
> >
> > "filtered" : {
> > "query" : {
> > ...
> > },
> > "filter" : {
> > "script" : {
> > "script" : "doc['num1'].value \> 1"
> > }
> > }
> >
> > }
> >
> > - Scripted facets
> >
> > Both statistical and histogram facets now allow to define scripts instead
> > of
> > field names. Issues are:
> > [Facets: Script statistical facets · Issue #227 · elastic/elasticsearch · GitHub](http://github.com/elasticsearch/elasticsearch/issues/227), andhttp://
> > [Facets: Script Histogram facet · Issue #228 · elastic/elasticsearch · GitHub](http://github.com/elasticsearch/elasticsearch/issues/228). Here is a sample
> > of of statistical facet:
> >
> > {
> > "query" : {
> > "match\_all" : { }
> > },
> > "facets" : {
> > "test" : {
> > "statistical" : {
> > "script" : "(doc['num1'] + doc['num2']) \* factor",
> > "params" : {
> > "factor" : 2.2
> > }
> > },
> > "global" : false
> > }
> > }
> >
> > }
> >
> > -shay.banon
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 4:23am UTC](https://discuss.elastic.co/t/pushed-more-script-goodness-including-script-fields-scripted-facets-and-script-filter/3021/4 "2017-07-06T04:23:16Z")