### Author:  [@Valentin\_Pletzer](https://discuss.elastic.co/u/Valentin_Pletzer)
#### Post date: [November 24, 2012, 5:27pm UTC](https://discuss.elastic.co/t/facet-plugin-coding-tutorial/9778/13 "2012-11-24T17:27:09Z")
I guess I better explain what I try to achieve: I want to create a list of
hashes (using the map-script) and then create a bloomfilter with all that
hashes (using the combine-script) and then merge all the bloomfilter (using
the reduce-script).
What I dont know, is how to share the data between the different scripts.
On Fri, Nov 23, 2012 at 2:26 AM, Igor Motov [imotov@gmail.com](mailto:imotov@gmail.com) wrote:
> The plugin is script engine agnostic. So, whatever is available in MVEL
> should be available in javascript. This includes \_client which points to
> NodeClient for the node that the mapping script is running on. I am not
> sure what you mean by global var in this context.
>
> On Thursday, November 22, 2012 11:28:50 AM UTC-5, Valentin wrote:
>
> > 2 additional questions:
> >
> > Is it possible to access the \_client variable from JavaScript? And can I
> > use it as a (per node) global var?
> >
> > Am 22.11.2012 um 13:45 schrieb Igor Motov [imo...@gmail.com](mailto:imo...@gmail.com):
> >
> > Did you install javascript plugin[https://github.com/elasticsearch/elasticsearch-lang-javascript](https://github.com/elasticsearch/elasticsearch-lang-javascript)on all your nodes and restarted the cluster?
> >
> > On Thursday, November 22, 2012 7:42:40 AM UTC-5, Valentin wrote:
> >
> > > Sorry thats the correct error message from your example:
> > > {
> > > "error" : "\*\*SearchPhaseExecutionException[\*\*Failed to execute phase
> > > [query], total failure; shardFailures {[ht\_GQoo2R5S6CtHCTMjJKg][\*\*test-idx][0]:
> > > SearchParseException[[test-**idx][0]: query[ConstantScore(**
> > > NotDeleted(_:_))],from[-1],\*\*size[-1]: Parse Failure [Failed to parse
> > > source [{\n "query": {\n "query\_string": {\n
> > > "query": "_:_"\n }\n },\n "facets": {\n
> > > "facet1": {\n "script": {\n "map\_script":
> > > "facet.messages += encodeURI(\_fields.message.**value)",\n
> > > "lang": "javascript",\n "params" : {\n
> > > "facet" : {"messages": ""},\n "field" :
> > > "message"\n }\n }\n }\n }\n}\n]]];
> > > nested: ElasticSearchIllegalArgumentEx**ception[script\_lang not
> > > supported [javascript]]; }]",
> > > "status" : 500
> > > }
> > >
> > > On Thu, Nov 22, 2012 at 1:41 PM, Valentin Pletzer [ple...@gmail.com](mailto:ple...@gmail.com)wrote:
> > >
> > > > Basically it says Javascript is not supported:
> > > >
> > > > {
> > > > "error" : "\*\*SearchPhaseExecutionException[\*\*Failed to execute phase
> > > > [query], total failure; shardFailures {[ht\_GQoo2R5S6CtHCTMjJKg][\*\*test-idx][0]:
> > > > SearchParseException[[test-**idx][0]: query[ConstantScore(**
> > > > NotDeleted(_:_))],from[-1],\*\*size[-1]: Parse Failure [Failed to parse
> > > > source [{\n "query": {\n "query\_string": {\n
> > > > "query": "_:_"\n }\n },\n "facets": {\n
> > > > "facet1": {\n "script": {\n "map\_script":
> > > > "facet.messages += encodeURI(\_fields.message.**value)",\n
> > > > "lang": "js",\n "params" : {\n
> > > > "facet" : {"messages": ""},\n "field" :
> > > > "message"\n }\n }\n }\n }\n}\n]]];
> > > > nested: ElasticSearchIllegalArgumentEx**ception[script\_lang not
> > > > supported [js]]; }]",
> > > > "status" : 500
> > > > }
> > > >
> > > > I am using elasticsearch 0.19.9
> > > >
> > > > {
> > > > "ok" : true,
> > > > "status" : 200,
> > > > "name" : "webstats02",
> > > > "version" : {
> > > > "number" : "0.19.9",
> > > > "snapshot\_build" : false
> > > > },
> > > > "tagline" : "You Know, for Search"
> > > > }
> > > >
> > > > On Thu, Nov 22, 2012 at 1:12 PM, Igor Motov [imo...@gmail.com](mailto:imo...@gmail.com) wrote:
> > > >
> > > > > Like this: [https://gist.github.com/\*\*18966ac80256a3330605](https://gist.github.com/**18966ac80256a3330605)[https://gist.github.com/18966ac80256a3330605](https://gist.github.com/18966ac80256a3330605)
> > > > >
> > > > > How does it fail?
> > > > >
> > > > > On Thursday, November 22, 2012 6:45:54 AM UTC-5, Valentin wrote:
> > > > >
> > > > > > Hmm. But how? I tried using it with JavaScript and failed ☹
> > > > > >
> > > > > > On Thu, Nov 22, 2012 at 12:35 PM, Igor Motov [imo...@gmail.com](mailto:imo...@gmail.com)wrote:
> > > > > >
> > > > > > > Yes, it does.
> > > > > > >
> > > > > > > On Thursday, November 22, 2012 5:27:55 AM UTC-5, Valentin wrote:
> > > > > > >
> > > > > > > > Thanks for the help. Does the facet script plugin support the
> > > > > > > > lang-parameter?
> > > > > > > >
> > > > > > > > On Wed, Nov 21, 2012 at 5:39 PM, Igor Motov [imo...@gmail.com](mailto:imo...@gmail.com)wrote:
> > > > > > > >
> > > > > > > > > Unfortunately, I haven't seen any tutorial on the topic. You can
> > > > > > > > > start with this thread - [https://groups.google.com/d/](https://groups.google.com/d/)\*\*\*\*\*\*
> > > > > > > > > topic/elasticsearch/\*\*NlLoXMfx4f\*\*\*\*w/discussion[https://groups.google.com/d/topic/elasticsearch/NlLoXMfx4fw/discussion](https://groups.google.com/d/topic/elasticsearch/NlLoXMfx4fw/discussion)and then take a look at how other facets are implemented.
> > > > > > > > >
> > > > > > > > > For prototyping, I am typically using the facet script plugin[https://github.com/imotov/elasticsearch-facet-script](https://github.com/imotov/elasticsearch-facet-script)that allows me to quickly create custom facets using any scripting language
> > > > > > > > > that elasticsearch supports.
> > > > > > > > >
> > > > > > > > > On Wednesday, November 21, 2012 10:35:58 AM UTC-5, Valentin wrote:
> > > > > > > > >
> > > > > > > > > > Are there any tutorials where facet plugins are explained? I
> > > > > > > > > > would like to write one myself and wondered where to start looking.
> > > > > > > > >
> > > > > > > > > --
> > > > > > > >
> > > > > > > > --
> > > > > >
> > > > > > --
> > >
> > > --
> >
> > --
--
---
### Author:  [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)
#### Post date: [November 24, 2012, 11:50pm UTC](https://discuss.elastic.co/t/facet-plugin-coding-tutorial/9778/14 "2012-11-24T23:50:05Z")
The fields defined in the params parameter are passed to all map-script
calls as global variables. So these variables can be used by map-script to
store intermediate results. The same global variables are later available
to combine-script that can combine them into a facet for the shard. So,
you can create a parameter called "hash\_list" in the params and use it to
accumulate the list of hashes. You can retrieve this list in combine-script
and generate the bloomfilter. However, to pass this bloomfiter to
reduce-script, it has to be serializable, which means the result of
combine-script can contain only primitive data types, java.util.Date,
byte, Object, java.util.List, and java.util.Map.
On Saturday, November 24, 2012 12:27:13 PM UTC-5, Valentin wrote:
> I guess I better explain what I try to achieve: I want to create a list of
> hashes (using the map-script) and then create a bloomfilter with all that
> hashes (using the combine-script) and then merge all the bloomfilter (using
> the reduce-script).
>
> What I dont know, is how to share the data between the different scripts.
>
> On Fri, Nov 23, 2012 at 2:26 AM, Igor Motov \<[imo...@gmail.com](mailto:imo...@gmail.com)\
>
> > wrote:
>
> > The plugin is script engine agnostic. So, whatever is available in MVEL
> > should be available in javascript. This includes \_client which points to
> > NodeClient for the node that the mapping script is running on. I am not
> > sure what you mean by global var in this context.
> >
> > On Thursday, November 22, 2012 11:28:50 AM UTC-5, Valentin wrote:
> >
> > > 2 additional questions:
> > >
> > > Is it possible to access the \_client variable from JavaScript? And can I
> > > use it as a (per node) global var?
> > >
> > > Am 22.11.2012 um 13:45 schrieb Igor Motov [imo...@gmail.com](mailto:imo...@gmail.com):
> > >
> > > Did you install javascript plugin[https://github.com/elasticsearch/elasticsearch-lang-javascript](https://github.com/elasticsearch/elasticsearch-lang-javascript)on all your nodes and restarted the cluster?
> > >
> > > On Thursday, November 22, 2012 7:42:40 AM UTC-5, Valentin wrote:
> > >
> > > > Sorry thats the correct error message from your example:
> > > > {
> > > > "error" : "\*\*SearchPhaseExecutionException[\*\*Failed to execute phase
> > > > [query], total failure; shardFailures {[ht\_GQoo2R5S6CtHCTMjJKg][\*\*test-idx][0]:
> > > > SearchParseException[[test-**idx][0]: query[ConstantScore(**
> > > > NotDeleted(_:_))],from[-1],\*\*size[-1]: Parse Failure [Failed to parse
> > > > source [{\n "query": {\n "query\_string": {\n
> > > > "query": "_:_"\n }\n },\n "facets": {\n
> > > > "facet1": {\n "script": {\n "map\_script":
> > > > "facet.messages += encodeURI(\_fields.message.**value)",\n
> > > > "lang": "javascript",\n "params" : {\n
> > > > "facet" : {"messages": ""},\n "field" :
> > > > "message"\n }\n }\n }\n }\n}\n]]];
> > > > nested: ElasticSearchIllegalArgumentEx**ception[script\_lang not
> > > > supported [javascript]]; }]",
> > > > "status" : 500
> > > > }
> > > >
> > > > On Thu, Nov 22, 2012 at 1:41 PM, Valentin Pletzer [ple...@gmail.com](mailto:ple...@gmail.com)wrote:
> > > >
> > > > > Basically it says Javascript is not supported:
> > > > >
> > > > > {
> > > > > "error" : "\*\*SearchPhaseExecutionException[\*\*Failed to execute
> > > > > phase [query], total failure; shardFailures {[ht\_GQoo2R5S6CtHCTMjJKg][
> > > > > \*\*test-idx][0]: SearchParseException[[test-\*\*idx][0]:
> > > > > query[ConstantScore(\*_NotDeleted(:))],from[-1],size[-1]: Parse
> > > > > Failure [Failed to parse source [{\n "query": {\n
> > > > > "query\_string": {\n "query": ":"\n }\n },\n
> > > > > "facets": {\n "facet1": {\n "script": {\n
> > > > > "map\_script": "facet.messages += encodeURI(\_fields.message.
> > > > > \*value)",\n "lang": "js",\n
> > > > > "params" : {\n "facet" : {"messages": ""},\n
> > > > > "field" : "message"\n }\n
> > > > > }\n }\n }\n}\n]]]; nested: ElasticSearchIllegalArgumentEx_ception[script\_lang
> > > > > not supported [js]]; }]",
> > > > > "status" : 500
> > > > > }
> > > > >
> > > > > I am using elasticsearch 0.19.9
> > > > >
> > > > > {
> > > > > "ok" : true,
> > > > > "status" : 200,
> > > > > "name" : "webstats02",
> > > > > "version" : {
> > > > > "number" : "0.19.9",
> > > > > "snapshot\_build" : false
> > > > > },
> > > > > "tagline" : "You Know, for Search"
> > > > > }
> > > > >
> > > > > On Thu, Nov 22, 2012 at 1:12 PM, Igor Motov [imo...@gmail.com](mailto:imo...@gmail.com) wrote:
> > > > >
> > > > > > Like this: [https://gist.github.com/\*\*18966ac80256a3330605](https://gist.github.com/**18966ac80256a3330605)[https://gist.github.com/18966ac80256a3330605](https://gist.github.com/18966ac80256a3330605)
> > > > > >
> > > > > > How does it fail?
> > > > > >
> > > > > > On Thursday, November 22, 2012 6:45:54 AM UTC-5, Valentin wrote:
> > > > > >
> > > > > > > Hmm. But how? I tried using it with JavaScript and failed ☹
> > > > > > >
> > > > > > > On Thu, Nov 22, 2012 at 12:35 PM, Igor Motov [imo...@gmail.com](mailto:imo...@gmail.com)wrote:
> > > > > > >
> > > > > > > > Yes, it does.
> > > > > > > >
> > > > > > > > On Thursday, November 22, 2012 5:27:55 AM UTC-5, Valentin wrote:
> > > > > > > >
> > > > > > > > > Thanks for the help. Does the facet script plugin support the
> > > > > > > > > lang-parameter?
> > > > > > > > >
> > > > > > > > > On Wed, Nov 21, 2012 at 5:39 PM, Igor Motov [imo...@gmail.com](mailto:imo...@gmail.com)wrote:
> > > > > > > > >
> > > > > > > > > > Unfortunately, I haven't seen any tutorial on the topic. You can
> > > > > > > > > > start with this thread - [https://groups.google.com/d/](https://groups.google.com/d/)\*\*\*\*\*\*
> > > > > > > > > > topic/elasticsearch/\*\*NlLoXMfx4f\*\*\*\*w/discussion[https://groups.google.com/d/topic/elasticsearch/NlLoXMfx4fw/discussion](https://groups.google.com/d/topic/elasticsearch/NlLoXMfx4fw/discussion)and then take a look at how other facets are implemented.
> > > > > > > > > >
> > > > > > > > > > For prototyping, I am typically using the facet script plugin[https://github.com/imotov/elasticsearch-facet-script](https://github.com/imotov/elasticsearch-facet-script)that allows me to quickly create custom facets using any scripting language
> > > > > > > > > > that elasticsearch supports.
> > > > > > > > > >
> > > > > > > > > > On Wednesday, November 21, 2012 10:35:58 AM UTC-5, Valentin wrote:
> > > > > > > > > >
> > > > > > > > > > > Are there any tutorials where facet plugins are explained? I
> > > > > > > > > > > would like to write one myself and wondered where to start looking.
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > >
> > > > > > > > > --
> > > > > > >
> > > > > > > --
> > > >
> > > > --
> > >
> > > --
--
---
### Author:  [@Valentin\_Pletzer](https://discuss.elastic.co/u/Valentin_Pletzer)
#### Post date: [December 2, 2012, 2:03pm UTC](https://discuss.elastic.co/t/facet-plugin-coding-tutorial/9778/15 "2012-12-02T14:03:02Z")
Hi Igor. Thanks for your help. I got it up and running. I
really appreciate your script-facet-plugin.
But there may be a still some room for optimization in my code. I am using
Javascript to read and write a list (of hashes) in the facet-parameter. At
the beginning of every map-call I have to use JSON.parse to get a list and
at the end JSON.stringify to store the result. Is there a better way?
On Sunday, November 25, 2012 12:50:05 AM UTC+1, Igor Motov wrote:
> The fields defined in the params parameter are passed to all map-script
> calls as global variables. So these variables can be used by map-script to
> store intermediate results. The same global variables are later available
> to combine-script that can combine them into a facet for the shard. So,
> you can create a parameter called "hash\_list" in the params and use it to
> accumulate the list of hashes. You can retrieve this list in combine-script
> and generate the bloomfilter. However, to pass this bloomfiter to
> reduce-script, it has to be serializable, which means the result of
> combine-script can contain only primitive data types, java.util.Date,
> byte, Object, java.util.List, and java.util.Map.
--
---
### Author:  [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)
#### Post date: [December 3, 2012, 5:09pm UTC](https://discuss.elastic.co/t/facet-plugin-coding-tutorial/9778/16 "2012-12-03T17:09:40Z")
If you initialize a facet parameter in params with an empty map {}, it will
be available in the script as an object. By assigning values to this
object's properties, you will be actually writing these values to the
underlying map. Here is an example: [javascript\_facets.sh · GitHub](https://gist.github.com/4196323) (I
omitted reduce script for clarity)
On Sunday, December 2, 2012 9:03:02 AM UTC-5, Valentin wrote:
> Hi Igor. Thanks for your help. I got it up and running. I
> really appreciate your script-facet-plugin.
>
> But there may be a still some room for optimization in my code. I am using
> Javascript to read and write a list (of hashes) in the facet-parameter. At
> the beginning of every map-call I have to use JSON.parse to get a list and
> at the end JSON.stringify to store the result. Is there a better way?
>
> On Sunday, November 25, 2012 12:50:05 AM UTC+1, Igor Motov wrote:
>
> > The fields defined in the params parameter are passed to all map-script
> > calls as global variables. So these variables can be used by map-script to
> > store intermediate results. The same global variables are later available
> > to combine-script that can combine them into a facet for the shard. So,
> > you can create a parameter called "hash\_list" in the params and use it to
> > accumulate the list of hashes. You can retrieve this list in combine-script
> > and generate the bloomfilter. However, to pass this bloomfiter to
> > reduce-script, it has to be serializable, which means the result of
> > combine-script can contain only primitive data types, java.util.Date,
> > byte, Object, java.util.List, and java.util.Map.
--
---
### Author:  [@sujoysett](https://discuss.elastic.co/u/sujoysett)
#### Post date: [December 4, 2012, 11:29am UTC](https://discuss.elastic.co/t/facet-plugin-coding-tutorial/9778/17 "2012-12-04T11:29:40Z")
Hi,
I am in the process of developing a lemma clustering facet. It will work
like terms facet only, but will group together syntactically different
tokens having same lemma. (eg. If there is token 'interested' and
'interesting', both will be counted together and shown under term
'interest').
I do not want to store the lemma itself during indexing, as I will be also
be performing normal terms faceting on these fields.
I am trying to use a custom analyzer that will store lemma of a token as
associated meta information along with a token (Lucene payload perhaps). I
want to use this extra information in a custom facet while combining tokens.
What would be the best way to achieve this?
[Currently I am doing this by doing extracting lemma while faceting in my
custom facet. Thus I am uselessly repeating the lemma operation on every
facet call. If I can have the information stored from indexing time, I
could cut a lot on faceting time.]
Thanks in advance,
-- Sujoy.
On Monday, December 3, 2012 10:39:40 PM UTC+5:30, Igor Motov wrote:
> If you initialize a facet parameter in params with an empty map {}, it
> will be available in the script as an object. By assigning values to this
> object's properties, you will be actually writing these values to the
> underlying map. Here is an example: [javascript\_facets.sh · GitHub](https://gist.github.com/4196323) (I
> omitted reduce script for clarity)
>
> On Sunday, December 2, 2012 9:03:02 AM UTC-5, Valentin wrote:
>
> > Hi Igor. Thanks for your help. I got it up and running. I
> > really appreciate your script-facet-plugin.
> >
> > But there may be a still some room for optimization in my code. I am
> > using Javascript to read and write a list (of hashes) in the
> > facet-parameter. At the beginning of every map-call I have to use
> > JSON.parse to get a list and at the end JSON.stringify to store the result.
> > Is there a better way?
> >
> > On Sunday, November 25, 2012 12:50:05 AM UTC+1, Igor Motov wrote:
> >
> > > The fields defined in the params parameter are passed to all map-script
> > > calls as global variables. So these variables can be used by map-script to
> > > store intermediate results. The same global variables are later available
> > > to combine-script that can combine them into a facet for the shard. So,
> > > you can create a parameter called "hash\_list" in the params and use it to
> > > accumulate the list of hashes. You can retrieve this list in combine-script
> > > and generate the bloomfilter. However, to pass this bloomfiter to
> > > reduce-script, it has to be serializable, which means the result of
> > > combine-script can contain only primitive data types, java.util.Date,
> > > byte, Object, java.util.List, and java.util.Map.
--
---
### Author:  [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)
#### Post date: [December 4, 2012, 11:39am UTC](https://discuss.elastic.co/t/facet-plugin-coding-tutorial/9778/18 "2012-12-04T11:39:19Z")
Is there any reason you cannot index your field twice using MultiField[http://www.elasticsearch.org/guide/reference/mapping/multi-field-type.html](http://www.elasticsearch.org/guide/reference/mapping/multi-field-type.html) type
and then use normal terms facets?
On Tuesday, December 4, 2012 6:29:40 AM UTC-5, Sujoy Sett wrote:
> Hi,
>
> I am in the process of developing a lemma clustering facet. It will work
> like terms facet only, but will group together syntactically different
> tokens having same lemma. (eg. If there is token 'interested' and
> 'interesting', both will be counted together and shown under term
> 'interest').
>
> I do not want to store the lemma itself during indexing, as I will be also
> be performing normal terms faceting on these fields.
>
> I am trying to use a custom analyzer that will store lemma of a token as
> associated meta information along with a token (Lucene payload perhaps). I
> want to use this extra information in a custom facet while combining tokens.
>
> What would be the best way to achieve this?
>
> [Currently I am doing this by doing extracting lemma while faceting in my
> custom facet. Thus I am uselessly repeating the lemma operation on every
> facet call. If I can have the information stored from indexing time, I
> could cut a lot on faceting time.]
>
> Thanks in advance,
> -- Sujoy.
>
> On Monday, December 3, 2012 10:39:40 PM UTC+5:30, Igor Motov wrote:
>
> > If you initialize a facet parameter in params with an empty map {}, it
> > will be available in the script as an object. By assigning values to this
> > object's properties, you will be actually writing these values to the
> > underlying map. Here is an example: [javascript\_facets.sh · GitHub](https://gist.github.com/4196323) (I
> > omitted reduce script for clarity)
> >
> > On Sunday, December 2, 2012 9:03:02 AM UTC-5, Valentin wrote:
> >
> > > Hi Igor. Thanks for your help. I got it up and running. I
> > > really appreciate your script-facet-plugin.
> > >
> > > But there may be a still some room for optimization in my code. I am
> > > using Javascript to read and write a list (of hashes) in the
> > > facet-parameter. At the beginning of every map-call I have to use
> > > JSON.parse to get a list and at the end JSON.stringify to store the result.
> > > Is there a better way?
> > >
> > > On Sunday, November 25, 2012 12:50:05 AM UTC+1, Igor Motov wrote:
> > >
> > > > The fields defined in the params parameter are passed to all map-script
> > > > calls as global variables. So these variables can be used by map-script to
> > > > store intermediate results. The same global variables are later available
> > > > to combine-script that can combine them into a facet for the shard. So,
> > > > you can create a parameter called "hash\_list" in the params and use it to
> > > > accumulate the list of hashes. You can retrieve this list in combine-script
> > > > and generate the bloomfilter. However, to pass this bloomfiter to
> > > > reduce-script, it has to be serializable, which means the result of
> > > > combine-script can contain only primitive data types, java.util.Date,
> > > > byte, Object, java.util.List, and java.util.Map.
--
---
### Author:  [@sujoysett](https://discuss.elastic.co/u/sujoysett)
#### Post date: [December 4, 2012, 11:54am UTC](https://discuss.elastic.co/t/facet-plugin-coding-tutorial/9778/19 "2012-12-04T11:54:29Z")
Yes, MultiField will solve this particular case. But I have further text
analysis planned in pipeline, which will not be possible by
using MultiField. Sorry that I had presented only a portion of my use-case
here. I should have explained more.
I want access to several other token metadata like parts-of-speech tag,
co-reference tag, token span, etc., and often multiple of them together,
for text processing purpose. MultiField can't help me there, unless I can
access the associated MultiField values in the Aggregator in my custom
facet.
Is it possible? Or any other hint?
Thanks,
-- Sujoy.
On Tuesday, December 4, 2012 5:09:19 PM UTC+5:30, Igor Motov wrote:
> Is there any reason you cannot index your field twice using MultiField[http://www.elasticsearch.org/guide/reference/mapping/multi-field-type.html](http://www.elasticsearch.org/guide/reference/mapping/multi-field-type.html) type
> and then use normal terms facets?
>
> On Tuesday, December 4, 2012 6:29:40 AM UTC-5, Sujoy Sett wrote:
>
> > Hi,
> >
> > I am in the process of developing a lemma clustering facet. It will work
> > like terms facet only, but will group together syntactically different
> > tokens having same lemma. (eg. If there is token 'interested' and
> > 'interesting', both will be counted together and shown under term
> > 'interest').
> >
> > I do not want to store the lemma itself during indexing, as I will be
> > also be performing normal terms faceting on these fields.
> >
> > I am trying to use a custom analyzer that will store lemma of a token as
> > associated meta information along with a token (Lucene payload perhaps). I
> > want to use this extra information in a custom facet while combining tokens.
> >
> > What would be the best way to achieve this?
> >
> > [Currently I am doing this by doing extracting lemma while faceting in my
> > custom facet. Thus I am uselessly repeating the lemma operation on every
> > facet call. If I can have the information stored from indexing time, I
> > could cut a lot on faceting time.]
> >
> > Thanks in advance,
> > -- Sujoy.
> >
> > On Monday, December 3, 2012 10:39:40 PM UTC+5:30, Igor Motov wrote:
> >
> > > If you initialize a facet parameter in params with an empty map {}, it
> > > will be available in the script as an object. By assigning values to this
> > > object's properties, you will be actually writing these values to the
> > > underlying map. Here is an example: [javascript\_facets.sh · GitHub](https://gist.github.com/4196323) (I
> > > omitted reduce script for clarity)
> > >
> > > On Sunday, December 2, 2012 9:03:02 AM UTC-5, Valentin wrote:
> > >
> > > > Hi Igor. Thanks for your help. I got it up and running. I
> > > > really appreciate your script-facet-plugin.
> > > >
> > > > But there may be a still some room for optimization in my code. I am
> > > > using Javascript to read and write a list (of hashes) in the
> > > > facet-parameter. At the beginning of every map-call I have to use
> > > > JSON.parse to get a list and at the end JSON.stringify to store the result.
> > > > Is there a better way?
> > > >
> > > > On Sunday, November 25, 2012 12:50:05 AM UTC+1, Igor Motov wrote:
> > > >
> > > > > The fields defined in the params parameter are passed to all
> > > > > map-script calls as global variables. So these variables can be used by
> > > > > map-script to store intermediate results. The same global variables are
> > > > > later available to combine-script that can combine them into a facet for
> > > > > the shard. So, you can create a parameter called "hash\_list" in the params
> > > > > and use it to accumulate the list of hashes. You can retrieve this list in
> > > > > combine-script and generate the bloomfilter. However, to pass this
> > > > > bloomfiter to reduce-script, it has to be serializable, which means the
> > > > > result of combine-script can contain only primitive data types,
> > > > > java.util.Date, byte, Object, java.util.List, and java.util.Map.
--
---
### Author:  [@sujoysett](https://discuss.elastic.co/u/sujoysett)
#### Post date: [December 6, 2012, 7:35am UTC](https://discuss.elastic.co/t/facet-plugin-coding-tutorial/9778/20 "2012-12-06T07:35:24Z")
Hi All,
Any hint on how to approach for the above requirement?
Thanks,
-- Sujoy.
On Tuesday, December 4, 2012 5:24:29 PM UTC+5:30, Sujoy Sett wrote:
> Yes, MultiField will solve this particular case. But I have further text
> analysis planned in pipeline, which will not be possible by
> using MultiField. Sorry that I had presented only a portion of my use-case
> here. I should have explained more.
>
> I want access to several other token metadata like parts-of-speech tag,
> co-reference tag, token span, etc., and often multiple of them together,
> for text processing purpose. MultiField can't help me there, unless I can
> access the associated MultiField values in the Aggregator in my custom
> facet.
>
> Is it possible? Or any other hint?
>
> Thanks,
> -- Sujoy.
>
> On Tuesday, December 4, 2012 5:09:19 PM UTC+5:30, Igor Motov wrote:
>
> > Is there any reason you cannot index your field twice using MultiField[http://www.elasticsearch.org/guide/reference/mapping/multi-field-type.html](http://www.elasticsearch.org/guide/reference/mapping/multi-field-type.html) type
> > and then use normal terms facets?
> >
> > On Tuesday, December 4, 2012 6:29:40 AM UTC-5, Sujoy Sett wrote:
> >
> > > Hi,
> > >
> > > I am in the process of developing a lemma clustering facet. It will work
> > > like terms facet only, but will group together syntactically different
> > > tokens having same lemma. (eg. If there is token 'interested' and
> > > 'interesting', both will be counted together and shown under term
> > > 'interest').
> > >
> > > I do not want to store the lemma itself during indexing, as I will be
> > > also be performing normal terms faceting on these fields.
> > >
> > > I am trying to use a custom analyzer that will store lemma of a token as
> > > associated meta information along with a token (Lucene payload perhaps). I
> > > want to use this extra information in a custom facet while combining tokens.
> > >
> > > What would be the best way to achieve this?
> > >
> > > [Currently I am doing this by doing extracting lemma while faceting in
> > > my custom facet. Thus I am uselessly repeating the lemma operation on every
> > > facet call. If I can have the information stored from indexing time, I
> > > could cut a lot on faceting time.]
> > >
> > > Thanks in advance,
> > > -- Sujoy.
> > >
> > > On Monday, December 3, 2012 10:39:40 PM UTC+5:30, Igor Motov wrote:
> > >
> > > > If you initialize a facet parameter in params with an empty map {}, it
> > > > will be available in the script as an object. By assigning values to this
> > > > object's properties, you will be actually writing these values to the
> > > > underlying map. Here is an example: [javascript\_facets.sh · GitHub](https://gist.github.com/4196323) (I omitted reduce script for clarity)
> > > >
> > > > On Sunday, December 2, 2012 9:03:02 AM UTC-5, Valentin wrote:
> > > >
> > > > > Hi Igor. Thanks for your help. I got it up and running. I
> > > > > really appreciate your script-facet-plugin.
> > > > >
> > > > > But there may be a still some room for optimization in my code. I am
> > > > > using Javascript to read and write a list (of hashes) in the
> > > > > facet-parameter. At the beginning of every map-call I have to use
> > > > > JSON.parse to get a list and at the end JSON.stringify to store the result.
> > > > > Is there a better way?
> > > > >
> > > > > On Sunday, November 25, 2012 12:50:05 AM UTC+1, Igor Motov wrote:
> > > > >
> > > > > > The fields defined in the params parameter are passed to all
> > > > > > map-script calls as global variables. So these variables can be used by
> > > > > > map-script to store intermediate results. The same global variables are
> > > > > > later available to combine-script that can combine them into a facet for
> > > > > > the shard. So, you can create a parameter called "hash\_list" in the params
> > > > > > and use it to accumulate the list of hashes. You can retrieve this list in
> > > > > > combine-script and generate the bloomfilter. However, to pass this
> > > > > > bloomfiter to reduce-script, it has to be serializable, which means the
> > > > > > result of combine-script can contain only primitive data types,
> > > > > > java.util.Date, byte, Object, java.util.List, and java.util.Map.
--
[Next page](https://discuss.elastic.co/t/facet-plugin-coding-tutorial/9778.md?page=2)