How do you access the various fields for a multi-field type from an MVEL script?


(hjc1710) #1

So, in a nutshell, I have a mapping with a simple multi-field type in an
index type called track. It has two fields, one is Genre (which is the
actual field) and is a string analyzed with the standard analyzer. Another
is Genre.raw, which just inserts the string unindexed. I need to write a
script that takes one argument, an array of strings (we'll call it
genreTags), and returns true if the track document contains all of those,
and only all of those, strings that are given in genreTags, however we must
ensure that they match the Genre.raw field and not the normal Genre field.
I can get the script to run and can scaffold the logic in the MVEL shell,
but I cannot for the life of me figure out how to access the Genre.raw
field from the MVEL script. I've tried doc['Genre.raw'], doc['Genre'].raw,
doc['Genre'].values.raw and nothing seems to work.

Can someone point me in the right direction?

Here's a gist that contains a simple CURL recreation of the above, but, in
the end, I just need to figure out how to access each individual field in a
multi-field type in MVEL (and I can't find that in the docs either).

https://gist.github.com/hjc1710/e08d8e329f87ba942513

With the above script, ideally, after the query is run the document with
just item1 in it should get the boost and should be returned first, but the
one with both item1 and item2 is returned first.

Thanks in advance.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Igor Motov) #2

There are much faster ways to implement this behavior, such as indexing
sorted set of all elements of the field Genre as a single not analyzed
string and matching on this entire string in order to boost. But, to answer
your question about the script, you should simply replace .length with
.size() in it to make it work.

            "script": "res = true; foreach(t : genreTags) { res = res 

&& doc["Genre.raw"].values contains t }; res = res && genreTags.size() ==
doc["Genre.raw"].values.size(); return res;",

On Friday, October 4, 2013 12:26:18 PM UTC-4, Hayden Chudy wrote:

So, in a nutshell, I have a mapping with a simple multi-field type in an
index type called track. It has two fields, one is Genre (which is the
actual field) and is a string analyzed with the standard analyzer. Another
is Genre.raw, which just inserts the string unindexed. I need to write a
script that takes one argument, an array of strings (we'll call it
genreTags), and returns true if the track document contains all of those,
and only all of those, strings that are given in genreTags, however we must
ensure that they match the Genre.raw field and not the normal Genre field.
I can get the script to run and can scaffold the logic in the MVEL shell,
but I cannot for the life of me figure out how to access the Genre.raw
field from the MVEL script. I've tried doc['Genre.raw'], doc['Genre'].raw,
doc['Genre'].values.raw and nothing seems to work.

Can someone point me in the right direction?

Here's a gist that contains a simple CURL recreation of the above, but, in
the end, I just need to figure out how to access each individual field in a
multi-field type in MVEL (and I can't find that in the docs either).

https://gist.github.com/hjc1710/e08d8e329f87ba942513

With the above script, ideally, after the query is run the document with
just item1 in it should get the boost and should be returned first, but the
one with both item1 and item2 is returned first.

Thanks in advance.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(hjc1710) #3

Thank you SOOOO much!!! That was what I was missing. The various types in
MVEL, especially having lists and arrays, really throws me for a loop. I
just didn't even know where to really start with this (I now know arrays in
elasticsearch are represented as lists, and not arrays, in MVEL). Given
that the MVEL documentation is just about terrible, posting some guidelines
as to what every type in elasticsearch becomes when its represented in MVEL
would be very useful. I assume that, for the most part, its a 1:1 mapping,
but knowing that arrays are turned into MVEL lists and not MVEL arrays is a
nice piece of knowledge to have.

Now, about your improvement to this script. This is how I read that: add
another field to the index type, we'll call it genreSortString for now.
genreSortString will consist of a string that is made by first sorting all
strings in alphabetical order in the Genre field, then by joining all of
those sorted strings into one larger string (that is possibly comma
delimited). This string is not analyzed and, when I go to write this query
and search for the genreSortString field, I rebuild the desired string with
the query information I have now and match on that. Is that a correct
interpretation? If so, it sounds like a damn good idea. Is there anyway I
can make it so elasticsearch will build this string for me every time the
Genre field is updated or inserted? Or does this need to be done
procedurally and inserted manually into elasticsearch?

Obviously, I would like to avoid running the script, so if this is easy to
implement, I will happily use it.

Thank you so much.

On Friday, October 4, 2013 1:59:14 PM UTC-4, Igor Motov wrote:

There are much faster ways to implement this behavior, such as indexing
sorted set of all elements of the field Genre as a single not analyzed
string and matching on this entire string in order to boost. But, to answer
your question about the script, you should simply replace .length with
.size() in it to make it work.

            "script": "res = true; foreach(t : genreTags) { res = res 

&& doc["Genre.raw"].values contains t }; res = res && genreTags.size() ==
doc["Genre.raw"].values.size(); return res;",

On Friday, October 4, 2013 12:26:18 PM UTC-4, Hayden Chudy wrote:

So, in a nutshell, I have a mapping with a simple multi-field type in an
index type called track. It has two fields, one is Genre (which is the
actual field) and is a string analyzed with the standard analyzer. Another
is Genre.raw, which just inserts the string unindexed. I need to write a
script that takes one argument, an array of strings (we'll call it
genreTags), and returns true if the track document contains all of those,
and only all of those, strings that are given in genreTags, however we must
ensure that they match the Genre.raw field and not the normal Genre field.
I can get the script to run and can scaffold the logic in the MVEL shell,
but I cannot for the life of me figure out how to access the Genre.raw
field from the MVEL script. I've tried doc['Genre.raw'], doc['Genre'].raw,
doc['Genre'].values.raw and nothing seems to work.

Can someone point me in the right direction?

Here's a gist that contains a simple CURL recreation of the above, but,
in the end, I just need to figure out how to access each individual field
in a multi-field type in MVEL (and I can't find that in the docs either).

https://gist.github.com/hjc1710/e08d8e329f87ba942513

With the above script, ideally, after the query is run the document with
just item1 in it should get the boost and should be returned first, but the
one with both item1 and item2 is returned first.

Thanks in advance.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Igor Motov) #4

You got it. Unfortunately, you will have to generate this string yourself,
elasticsearch doesn't have any facilities that would build such string for
you.

On Friday, October 4, 2013 2:10:04 PM UTC-4, Hayden Chudy wrote:

Thank you SOOOO much!!! That was what I was missing. The various types in
MVEL, especially having lists and arrays, really throws me for a loop.
I just didn't even know where to really start with this (I now know arrays
in elasticsearch are represented as lists, and not arrays, in MVEL). Given
that the MVEL documentation is just about terrible, posting some guidelines
as to what every type in elasticsearch becomes when its represented in MVEL
would be very useful. I assume that, for the most part, its a 1:1 mapping,
but knowing that arrays are turned into MVEL lists and not MVEL arrays is a
nice piece of knowledge to have.

Now, about your improvement to this script. This is how I read that: add
another field to the index type, we'll call it genreSortString for now.
genreSortString will consist of a string that is made by first sorting all
strings in alphabetical order in the Genre field, then by joining all of
those sorted strings into one larger string (that is possibly comma
delimited). This string is not analyzed and, when I go to write this query
and search for the genreSortString field, I rebuild the desired string with
the query information I have now and match on that. Is that a correct
interpretation? If so, it sounds like a damn good idea. Is there anyway I
can make it so elasticsearch will build this string for me every time the
Genre field is updated or inserted? Or does this need to be done
procedurally and inserted manually into elasticsearch?

Obviously, I would like to avoid running the script, so if this is easy to
implement, I will happily use it.

Thank you so much.

On Friday, October 4, 2013 1:59:14 PM UTC-4, Igor Motov wrote:

There are much faster ways to implement this behavior, such as indexing
sorted set of all elements of the field Genre as a single not analyzed
string and matching on this entire string in order to boost. But, to answer
your question about the script, you should simply replace .length with
.size() in it to make it work.

            "script": "res = true; foreach(t : genreTags) { res = res 

&& doc["Genre.raw"].values contains t }; res = res && genreTags.size() ==
doc["Genre.raw"].values.size(); return res;",

On Friday, October 4, 2013 12:26:18 PM UTC-4, Hayden Chudy wrote:

So, in a nutshell, I have a mapping with a simple multi-field type in an
index type called track. It has two fields, one is Genre (which is the
actual field) and is a string analyzed with the standard analyzer. Another
is Genre.raw, which just inserts the string unindexed. I need to write a
script that takes one argument, an array of strings (we'll call it
genreTags), and returns true if the track document contains all of those,
and only all of those, strings that are given in genreTags, however we must
ensure that they match the Genre.raw field and not the normal Genre field.
I can get the script to run and can scaffold the logic in the MVEL shell,
but I cannot for the life of me figure out how to access the Genre.raw
field from the MVEL script. I've tried doc['Genre.raw'], doc['Genre'].raw,
doc['Genre'].values.raw and nothing seems to work.

Can someone point me in the right direction?

Here's a gist that contains a simple CURL recreation of the above, but,
in the end, I just need to figure out how to access each individual field
in a multi-field type in MVEL (and I can't find that in the docs either).

https://gist.github.com/hjc1710/e08d8e329f87ba942513

With the above script, ideally, after the query is run the document with
just item1 in it should get the boost and should be returned first, but the
one with both item1 and item2 is returned first.

Thanks in advance.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(hjc1710) #5

Awesome! That really is an excellent idea and such a better approach.
Luckily, the logic that formats the DB objects for elasticsearch is in a
central place and is easily changeable =). So I think we'll be using this,
far superior, approach.

Again, thank you so much for your input, you not only saved me time, but it
looks like you've improved the end state of our application =). The
elasticsearch community really is an amazing and helpful group!

On Fri, Oct 4, 2013 at 2:44 PM, Igor Motov imotov@gmail.com wrote:

You got it. Unfortunately, you will have to generate this string yourself,
elasticsearch doesn't have any facilities that would build such string for
you.

On Friday, October 4, 2013 2:10:04 PM UTC-4, Hayden Chudy wrote:

Thank you SOOOO much!!! That was what I was missing. The various types in
MVEL, especially having lists and arrays, really throws me for a loop.
I just didn't even know where to really start with this (I now know arrays
in elasticsearch are represented as lists, and not arrays, in MVEL). Given
that the MVEL documentation is just about terrible, posting some guidelines
as to what every type in elasticsearch becomes when its represented in MVEL
would be very useful. I assume that, for the most part, its a 1:1 mapping,
but knowing that arrays are turned into MVEL lists and not MVEL arrays is a
nice piece of knowledge to have.

Now, about your improvement to this script. This is how I read that: add
another field to the index type, we'll call it genreSortString for now.
genreSortString will consist of a string that is made by first sorting all
strings in alphabetical order in the Genre field, then by joining all of
those sorted strings into one larger string (that is possibly comma
delimited). This string is not analyzed and, when I go to write this query
and search for the genreSortString field, I rebuild the desired string with
the query information I have now and match on that. Is that a correct
interpretation? If so, it sounds like a damn good idea. Is there anyway I
can make it so elasticsearch will build this string for me every time the
Genre field is updated or inserted? Or does this need to be done
procedurally and inserted manually into elasticsearch?

Obviously, I would like to avoid running the script, so if this is easy
to implement, I will happily use it.

Thank you so much.

On Friday, October 4, 2013 1:59:14 PM UTC-4, Igor Motov wrote:

There are much faster ways to implement this behavior, such as indexing
sorted set of all elements of the field Genre as a single not analyzed
string and matching on this entire string in order to boost. But, to answer
your question about the script, you should simply replace .length with
.size() in it to make it work.

            "script": "res = true; foreach(t : genreTags) { res =

res && doc["Genre.raw"].values contains t }; res = res &&
genreTags.size() == doc["Genre.raw"].values.**size(); return res;",

On Friday, October 4, 2013 12:26:18 PM UTC-4, Hayden Chudy wrote:

So, in a nutshell, I have a mapping with a simple multi-field type in
an index type called track. It has two fields, one is Genre (which is the
actual field) and is a string analyzed with the standard analyzer. Another
is Genre.raw, which just inserts the string unindexed. I need to write a
script that takes one argument, an array of strings (we'll call it
genreTags), and returns true if the track document contains all of those,
and only all of those, strings that are given in genreTags, however we must
ensure that they match the Genre.raw field and not the normal Genre field.
I can get the script to run and can scaffold the logic in the MVEL shell,
but I cannot for the life of me figure out how to access the Genre.raw
field from the MVEL script. I've tried doc['Genre.raw'], doc['Genre'].raw,
doc['Genre'].values.raw and nothing seems to work.

Can someone point me in the right direction?

Here's a gist that contains a simple CURL recreation of the above, but,
in the end, I just need to figure out how to access each individual field
in a multi-field type in MVEL (and I can't find that in the docs either).

https://gist.github.com/**hjc1710/e08d8e329f87ba942513https://gist.github.com/hjc1710/e08d8e329f87ba942513

With the above script, ideally, after the query is run the document
with just item1 in it should get the boost and should be returned first,
but the one with both item1 and item2 is returned first.

Thanks in advance.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/Mu2c3Ao5SDo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(system) #6