Native script in ES


(Adrian M) #1

Hi,
I'm curious how scripts work in ES.
I've created a simple script which checks if a field in a record is equal
to a certain value then return 1 otherwise return 0. I used
AbstractDoubleSearchScript as the super class for this script.
My records contains some a data field that is an int and a date field
(long) (some other ones too that I don't care too much yet)

What I've noticed is that when I execute the script, each record in the ES
db goes through the script and is returned either w a score of 0 or 1.
What I would like to know is if it is possible to do a range query, say
given 2 date params, and select only the records that fall between those
dates to run through the script.

I know how to do this with the API but not in script format. For example
this non script code works:
SearchRequestBuilder s = client.prepareSearch("posts")

.setQuery(QueryBuilders.queryString(query).defaultField("post.body"))
.setSize(count)
.setFrom(start)

.setFilter(FilterBuilders.rangeFilter("post.creationDate").from(interval.getStart.toString()).to(interval.getEnd.toString()))
.setFilter(FilterBuilders.existsFilter("userId"));
s.setSearchType(SearchType.DFS_QUERY_THEN_FETCH);

But when extending AbstractDoubleSearchScript I have access only to a few
things like doc(), source(), fields(), lookup() and I'm not sure how to
filter records.
Also I can't find documentation on what these are ie: what is source()
other than a Map.

Any help on filtering is greatly appreciated!
Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/1673cee1-468e-462b-85a4-c8e45fa74156%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Jörg Prante) #2

Scripts are not an alternative to write filters or queries, they are part
of the filter or query computation. With scripts, you can access document
fields and perform evaluations depending on one or more field values, for
example, for custom scoring. Also, index term statistics can be accessed
from scripts. Script results are passed back to the query processing.

Jörg

On Wed, Jul 9, 2014 at 10:41 PM, Adrian eamocanu@gmail.com wrote:

Hi,
I'm curious how scripts work in ES.
I've created a simple script which checks if a field in a record is equal
to a certain value then return 1 otherwise return 0. I used
AbstractDoubleSearchScript as the super class for this script.
My records contains some a data field that is an int and a date field
(long) (some other ones too that I don't care too much yet)

What I've noticed is that when I execute the script, each record in the ES
db goes through the script and is returned either w a score of 0 or 1.
What I would like to know is if it is possible to do a range query, say
given 2 date params, and select only the records that fall between those
dates to run through the script.

I know how to do this with the API but not in script format. For example
this non script code works:
SearchRequestBuilder s = client.prepareSearch("posts")

.setQuery(QueryBuilders.queryString(query).defaultField("post.body"))
.setSize(count)
.setFrom(start)

.setFilter(FilterBuilders.rangeFilter("post.creationDate").from(interval.getStart.toString()).to(interval.getEnd.toString()))
.setFilter(FilterBuilders.existsFilter("userId"));
s.setSearchType(SearchType.DFS_QUERY_THEN_FETCH);

But when extending AbstractDoubleSearchScript I have access only to a few
things like doc(), source(), fields(), lookup() and I'm not sure how to
filter records.
Also I can't find documentation on what these are ie: what is source()
other than a Map.

Any help on filtering is greatly appreciated!
Thanks

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/1673cee1-468e-462b-85a4-c8e45fa74156%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/1673cee1-468e-462b-85a4-c8e45fa74156%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHAVe3_M5%2BuWhoZW2fxwmZKsu%3DM3ELij-Q8Ef1X6wmXYA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Adrian M) #3

Ok, thanks.

Though I've noticed that I can inject Node in the Factory method to get the
client in the script as shown here:
https://github.com/imotov/elasticsearch-native-script-example/blob/master/src/main/java/org/elasticsearch/examples/nativescript/script/LookupScript.java.

Couldn't I in this case perform a query through the client?

On Wednesday, 9 July 2014 19:03:49 UTC-4, Jörg Prante wrote:

Scripts are not an alternative to write filters or queries, they are part
of the filter or query computation. With scripts, you can access document
fields and perform evaluations depending on one or more field values, for
example, for custom scoring. Also, index term statistics can be accessed
from scripts. Script results are passed back to the query processing.

Jörg

On Wed, Jul 9, 2014 at 10:41 PM, Adrian <eamo...@gmail.com <javascript:>>
wrote:

Hi,
I'm curious how scripts work in ES.
I've created a simple script which checks if a field in a record is equal
to a certain value then return 1 otherwise return 0. I used
AbstractDoubleSearchScript as the super class for this script.
My records contains some a data field that is an int and a date field
(long) (some other ones too that I don't care too much yet)

What I've noticed is that when I execute the script, each record in the
ES db goes through the script and is returned either w a score of 0 or 1.
What I would like to know is if it is possible to do a range query, say
given 2 date params, and select only the records that fall between those
dates to run through the script.

I know how to do this with the API but not in script format. For example
this non script code works:
SearchRequestBuilder s = client.prepareSearch("posts")

.setQuery(QueryBuilders.queryString(query).defaultField("post.body"))
.setSize(count)
.setFrom(start)

.setFilter(FilterBuilders.rangeFilter("post.creationDate").from(interval.getStart.toString()).to(interval.getEnd.toString()))
.setFilter(FilterBuilders.existsFilter("userId"));
s.setSearchType(SearchType.DFS_QUERY_THEN_FETCH);

But when extending AbstractDoubleSearchScript I have access only to a few
things like doc(), source(), fields(), lookup() and I'm not sure how to
filter records.
Also I can't find documentation on what these are ie: what is source()
other than a Map.

Any help on filtering is greatly appreciated!
Thanks

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/1673cee1-468e-462b-85a4-c8e45fa74156%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/1673cee1-468e-462b-85a4-c8e45fa74156%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5b353881-4a58-48af-980b-7a0e5f2ae2cf%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Jörg Prante) #4

Yes, this is a lookup from one document of another document using a get
request. Note the result of the lookup is a single document source, whereas
a search request provides a search response with a document source list,
which is vastly oversized for use from a script, and takes even longer to
complete. Note also the comment "This is not very efficient" ... searches
would be even more harmful at that place in a script code. I do not
recommend it. Instead, use search requests from a client.

Jörg

On Thu, Jul 10, 2014 at 9:14 PM, Adrian eamocanu@gmail.com wrote:

Ok, thanks.

Though I've noticed that I can inject Node in the Factory method to get
the client in the script as shown here:
https://github.com/imotov/elasticsearch-native-script-example/blob/master/src/main/java/org/elasticsearch/examples/nativescript/script/LookupScript.java
.

Couldn't I in this case perform a query through the client?

On Wednesday, 9 July 2014 19:03:49 UTC-4, Jörg Prante wrote:

Scripts are not an alternative to write filters or queries, they are part
of the filter or query computation. With scripts, you can access document
fields and perform evaluations depending on one or more field values, for
example, for custom scoring. Also, index term statistics can be accessed
from scripts. Script results are passed back to the query processing.

Jörg

On Wed, Jul 9, 2014 at 10:41 PM, Adrian eamo...@gmail.com wrote:

Hi,
I'm curious how scripts work in ES.
I've created a simple script which checks if a field in a record is
equal to a certain value then return 1 otherwise return 0. I used
AbstractDoubleSearchScript as the super class for this script.
My records contains some a data field that is an int and a date field
(long) (some other ones too that I don't care too much yet)

What I've noticed is that when I execute the script, each record in the
ES db goes through the script and is returned either w a score of 0 or 1.
What I would like to know is if it is possible to do a range query, say
given 2 date params, and select only the records that fall between those
dates to run through the script.

I know how to do this with the API but not in script format. For example
this non script code works:
SearchRequestBuilder s = client.prepareSearch("posts")
.setQuery(QueryBuilders.queryString(query).
defaultField("post.body"))
.setSize(count)
.setFrom(start)
.setFilter(FilterBuilders.rangeFilter("post.
creationDate").from(interval.getStart.toString()).to(
interval.getEnd.toString()))
.setFilter(FilterBuilders.existsFilter("userId"));
s.setSearchType(SearchType.DFS_QUERY_THEN_FETCH);

But when extending AbstractDoubleSearchScript I have access only to a
few things like doc(), source(), fields(), lookup() and I'm not sure how to
filter records.
Also I can't find documentation on what these are ie: what is source()
other than a Map.

Any help on filtering is greatly appreciated!
Thanks

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/1673cee1-468e-462b-85a4-c8e45fa74156%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/1673cee1-468e-462b-85a4-c8e45fa74156%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/5b353881-4a58-48af-980b-7a0e5f2ae2cf%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/5b353881-4a58-48af-980b-7a0e5f2ae2cf%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGFFQNdRuj6PD5wL0g20pO4XzsPu5SLRJ753SELn7Rn7w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Adrian M) #5

I understand what you mean regarding efficiency/speed.
My reason for doing things this way was so that I could compare 2 time
series, ones coming from 1 query and another coming from another query.

On Thursday, 10 July 2014 15:27:28 UTC-4, Jörg Prante wrote:

Yes, this is a lookup from one document of another document using a get
request. Note the result of the lookup is a single document source, whereas
a search request provides a search response with a document source list,
which is vastly oversized for use from a script, and takes even longer to
complete. Note also the comment "This is not very efficient" ... searches
would be even more harmful at that place in a script code. I do not
recommend it. Instead, use search requests from a client.

Jörg

On Thu, Jul 10, 2014 at 9:14 PM, Adrian <eamo...@gmail.com <javascript:>>
wrote:

Ok, thanks.

Though I've noticed that I can inject Node in the Factory method to get
the client in the script as shown here:
https://github.com/imotov/elasticsearch-native-script-example/blob/master/src/main/java/org/elasticsearch/examples/nativescript/script/LookupScript.java
.

Couldn't I in this case perform a query through the client?

On Wednesday, 9 July 2014 19:03:49 UTC-4, Jörg Prante wrote:

Scripts are not an alternative to write filters or queries, they are
part of the filter or query computation. With scripts, you can access
document fields and perform evaluations depending on one or more field
values, for example, for custom scoring. Also, index term statistics can be
accessed from scripts. Script results are passed back to the query
processing.

Jörg

On Wed, Jul 9, 2014 at 10:41 PM, Adrian eamo...@gmail.com wrote:

Hi,
I'm curious how scripts work in ES.
I've created a simple script which checks if a field in a record is
equal to a certain value then return 1 otherwise return 0. I used
AbstractDoubleSearchScript as the super class for this script.
My records contains some a data field that is an int and a date field
(long) (some other ones too that I don't care too much yet)

What I've noticed is that when I execute the script, each record in the
ES db goes through the script and is returned either w a score of 0 or 1.
What I would like to know is if it is possible to do a range query, say
given 2 date params, and select only the records that fall between those
dates to run through the script.

I know how to do this with the API but not in script format. For
example this non script code works:
SearchRequestBuilder s = client.prepareSearch("posts")
.setQuery(QueryBuilders.queryString(query).
defaultField("post.body"))
.setSize(count)
.setFrom(start)
.setFilter(FilterBuilders.rangeFilter("post.
creationDate").from(interval.getStart.toString()).to(
interval.getEnd.toString()))
.setFilter(FilterBuilders.existsFilter("userId"));
s.setSearchType(SearchType.DFS_QUERY_THEN_FETCH);

But when extending AbstractDoubleSearchScript I have access only to a
few things like doc(), source(), fields(), lookup() and I'm not sure how to
filter records.
Also I can't find documentation on what these are ie: what is source()
other than a Map.

Any help on filtering is greatly appreciated!
Thanks

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/1673cee1-468e-462b-85a4-c8e45fa74156%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/1673cee1-468e-462b-85a4-c8e45fa74156%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/5b353881-4a58-48af-980b-7a0e5f2ae2cf%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/5b353881-4a58-48af-980b-7a0e5f2ae2cf%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/96487e27-b538-48c1-9840-42e75ea021f3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Jörg Prante) #6

The fastest way I know is to execute the two queries, scan/scroll over
result docs, save one result set, and compare values while iterating over
the other.

Jörg

On Thu, Jul 10, 2014 at 9:38 PM, Adrian eamocanu@gmail.com wrote:

I understand what you mean regarding efficiency/speed.
My reason for doing things this way was so that I could compare 2 time
series, ones coming from 1 query and another coming from another query.

On Thursday, 10 July 2014 15:27:28 UTC-4, Jörg Prante wrote:

Yes, this is a lookup from one document of another document using a get
request. Note the result of the lookup is a single document source, whereas
a search request provides a search response with a document source list,
which is vastly oversized for use from a script, and takes even longer to
complete. Note also the comment "This is not very efficient" ... searches
would be even more harmful at that place in a script code. I do not
recommend it. Instead, use search requests from a client.

Jörg

On Thu, Jul 10, 2014 at 9:14 PM, Adrian eamo...@gmail.com wrote:

Ok, thanks.

Though I've noticed that I can inject Node in the Factory method to get
the client in the script as shown here: https://github.com/imotov/
elasticsearch-native-script-example/blob/master/src/main/
java/org/elasticsearch/examples/nativescript/script/LookupScript.java.

Couldn't I in this case perform a query through the client?

On Wednesday, 9 July 2014 19:03:49 UTC-4, Jörg Prante wrote:

Scripts are not an alternative to write filters or queries, they are
part of the filter or query computation. With scripts, you can access
document fields and perform evaluations depending on one or more field
values, for example, for custom scoring. Also, index term statistics can be
accessed from scripts. Script results are passed back to the query
processing.

Jörg

On Wed, Jul 9, 2014 at 10:41 PM, Adrian eamo...@gmail.com wrote:

Hi,
I'm curious how scripts work in ES.
I've created a simple script which checks if a field in a record is
equal to a certain value then return 1 otherwise return 0. I used
AbstractDoubleSearchScript as the super class for this script.
My records contains some a data field that is an int and a date field
(long) (some other ones too that I don't care too much yet)

What I've noticed is that when I execute the script, each record in
the ES db goes through the script and is returned either w a score of 0 or
1.
What I would like to know is if it is possible to do a range query,
say given 2 date params, and select only the records that fall between
those dates to run through the script.

I know how to do this with the API but not in script format. For
example this non script code works:
SearchRequestBuilder s = client.prepareSearch("posts")
.setQuery(QueryBuilders.queryString(query).
defaultField("post.body"))
.setSize(count)
.setFrom(start)
.setFilter(FilterBuilders.rangeFilter("post.
creationDate").from(interval.getStart.toString()).to(interval.
getEnd.toString()))
.setFilter(FilterBuilders.existsFilter("userId"));
s.setSearchType(SearchType.DFS_QUERY_THEN_FETCH);

But when extending AbstractDoubleSearchScript I have access only to a
few things like doc(), source(), fields(), lookup() and I'm not sure how to
filter records.
Also I can't find documentation on what these are ie: what is source()
other than a Map.

Any help on filtering is greatly appreciated!
Thanks

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/1673cee1-468e-462b-85a4-c8e45fa74156%40goo
glegroups.com
https://groups.google.com/d/msgid/elasticsearch/1673cee1-468e-462b-85a4-c8e45fa74156%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/5b353881-4a58-48af-980b-7a0e5f2ae2cf%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/5b353881-4a58-48af-980b-7a0e5f2ae2cf%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/96487e27-b538-48c1-9840-42e75ea021f3%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/96487e27-b538-48c1-9840-42e75ea021f3%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEdphZhSAa6ypMoudokvq%3D2QrtnoOH_CM-6PHRRVQvjQw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(system) #7