Error when running mvel scripts


(Abhishek Andhavarapu) #1

Hi,

I get occasional errors when running mvel scripts. I found the similar
issue addressed at

I get the exception twice before it comes back with results. And I don't
get the execption any more. Since the above posts were 4 months old I was
hoping if there was a fix or workaround. Also is using JS slower than MVEL ?

My Query :

{

"facets": {

"facet1": {

  "statistical": {

    "script": "float val = 0; float val1 = 0; for (int i = 0; i < 

_source.rdata.size(); i++){if(_source.rdata[i].mapid == 87) {val =
_source.rdata[i].value} } return val"

  },

  "facet_filter": {

    "terms": {

      "mapids": [

        "87"

      ],

      "execution": "and"

    }

  }

}

}

}

Thanks,

Abhishek

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Zachary Tong) #2

If you are getting the exceptions referenced in issue #3021 ("*
java.lang.verifyError: *Expecting to find integer on stack"), then that is
just a problem with MVEL, I'm not aware of any workarounds. MVEL is,
unfortunately, not well maintained. It is only used as the default in
Elasticsearch because it is the fastest. We are hoping to move to
something more universal (e.g. Javascript) in the future, once Java 8
clears up some of the performance issues.

Javascript (and other language plugins) will be slightly slower than MVEL,
but the performance may be acceptable to you.

Regardless of language choice, your script is going to be massively slow.
You are using the "_source" variable, which loads data directly from the
original source. This requires random access to the disk for every single
document evaluated by the facet. Disk I/O is the slowest component in a
modern computer, so this will kill facet performance. Instead, use
in-memory doc values. E.g:

"script": "float val = 0; float val1 = 0; for (int i = 0; i <
doc['rdata'].size(); i++){if(doc['rdata'][i].mapid == 87) {val =
doc['rdata'][i].value} } return val"

Doc values are stored in memory and will be several orders of magnitude
faster than loading from _source.

Hope that helps!
-Zach

On Thursday, September 19, 2013 12:54:46 PM UTC-4, Abhishek Andhavarapu
wrote:

Hi,

I get occasional errors when running mvel scripts. I found the similar
issue addressed at

https://github.com/elasticsearch/elasticsearch/issues/3021

https://gist.github.com/imotov/5554678

I get the exception twice before it comes back with results. And I don't
get the execption any more. Since the above posts were 4 months old I was
hoping if there was a fix or workaround. Also is using JS slower than MVEL ?

My Query :

{

"facets": {

"facet1": {

  "statistical": {

    "script": "float val = 0; float val1 = 0; for (int i = 0; i < 

_source.rdata.size(); i++){if(_source.rdata[i].mapid == 87) {val =
_source.rdata[i].value} } return val"

  },

  "facet_filter": {

    "terms": {

      "mapids": [

        "87"

      ],

      "execution": "and"

    }

  }

}

}

}

Thanks,

Abhishek

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Abhishek Andhavarapu) #3

Thanks a lot for the reply Zach. I tried using doc before. I get this exception

nested: ElasticSearchIllegalArgumentException[No field found for [rdata] in mapping with types [response]]; }]
status: 500

The same works when I use _source. How do I rewrite the query to use doc ?

Regards,
Abhishek

On Saturday, September 21, 2013 at 10:58 AM, Zachary Tong wrote:

If you are getting the exceptions referenced in issue #3021 ("java.lang.verifyError: Expecting to find integer on stack"), then that is just a problem with MVEL, I'm not aware of any workarounds. MVEL is, unfortunately, not well maintained. It is only used as the default in Elasticsearch because it is the fastest. We are hoping to move to something more universal (e.g. Javascript) in the future, once Java 8 clears up some of the performance issues.

Javascript (and other language plugins) will be slightly slower than MVEL, but the performance may be acceptable to you.

Regardless of language choice, your script is going to be massively slow. You are using the "_source" variable, which loads data directly from the original source. This requires random access to the disk for every single document evaluated by the facet. Disk I/O is the slowest component in a modern computer, so this will kill facet performance. Instead, use in-memory doc values. E.g:

"script": "float val = 0; float val1 = 0; for (int i = 0; i < doc['rdata'].size(); i++){if(doc['rdata'][i].mapid == 87) {val = doc['rdata'][i].value} } return val"

Doc values are stored in memory and will be several orders of magnitude faster than loading from _source.

Hope that helps!
-Zach

On Thursday, September 19, 2013 12:54:46 PM UTC-4, Abhishek Andhavarapu wrote:

Hi,

I get occasional errors when running mvel scripts. I found the similar issue addressed at

https://github.com/elasticsearch/elasticsearch/issues/3021
https://gist.github.com/imotov/5554678
I get the exception twice before it comes back with results. And I don't get the execption any more. Since the above posts were 4 months old I was hoping if there was a fix or workaround. Also is using JS slower than MVEL ?
My Query :
{
"facets": {
"facet1": {
"statistical": {
"script": "float val = 0; float val1 = 0; for (int i = 0; i < _source.rdata.size(); i++){if(_source.rdata[i].mapid == 87) {val = _source.rdata[i].value} } return val"
},
"facet_filter": {
"terms": {
"mapids": [
"87"
],
"execution": "and"
}
}
}
}
}
Thanks,
Abhishek

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/vyhWsTj3UOM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscribe@googlegroups.com (mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Zachary Tong) #4

Can you post the mapping of your document? Did you index the rdata
field? It will only be loaded into Field Data (e.g. doc values) if it is
an indexed field.

-Zach

On Saturday, September 21, 2013 5:21:14 PM UTC-4, Abhishek Andhavarapu
wrote:

Thanks a lot for the reply Zach. I tried using doc before. I get this
exception

  • nested: ElasticSearchIllegalArgumentException[No field found for
    [rdata] in mapping with types [response]]; }]
  • status: 500

The same works when I use _source. How do I rewrite the query to use doc ?

Regards,
Abhishek

On Saturday, September 21, 2013 at 10:58 AM, Zachary Tong wrote:

If you are getting the exceptions referenced in issue #3021 ("*
java.lang.verifyError: *Expecting to find integer on stack"), then that
is just a problem with MVEL, I'm not aware of any workarounds. MVEL is,
unfortunately, not well maintained. It is only used as the default in
Elasticsearch because it is the fastest. We are hoping to move to
something more universal (e.g. Javascript) in the future, once Java 8
clears up some of the performance issues.

Javascript (and other language plugins) will be slightly slower than MVEL,
but the performance may be acceptable to you.

Regardless of language choice, your script is going to be massively slow.
You are using the "_source" variable, which loads data directly from the
original source. This requires random access to the disk for every single
document evaluated by the facet. Disk I/O is the slowest component in a
modern computer, so this will kill facet performance. Instead, use
in-memory doc values. E.g:

"script": "float val = 0; float val1 = 0; for (int i = 0; i <
doc['rdata'].size(); i++){if(doc['rdata'][i].mapid == 87) {val =
doc['rdata'][i].value} } return val"

Doc values are stored in memory and will be several orders of magnitude
faster than loading from _source.

Hope that helps!
-Zach

On Thursday, September 19, 2013 12:54:46 PM UTC-4, Abhishek Andhavarapu
wrote:

Hi,

I get occasional errors when running mvel scripts. I found the similar
issue addressed at

https://github.com/elasticsearch/elasticsearch/issues/3021

https://gist.github.com/imotov/5554678

I get the exception twice before it comes back with results. And I don't
get the execption any more. Since the above posts were 4 months old I was
hoping if there was a fix or workaround. Also is using JS slower than MVEL ?

My Query :

{

"facets": {

"facet1": {

  "statistical": {

    "script": "float val = 0; float val1 = 0; for (int i = 0; i < 

_source.rdata.size(); i++){if(_source.rdata[i].mapid == 87) {val =
_source.rdata[i].value} } return val"

  },

  "facet_filter": {

    "terms": {

      "mapids": [

        "87"

      ],

      "execution": "and"

    }

  }

}

}

}

Thanks,

Abhishek

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/vyhWsTj3UOM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Abhishek Andhavarapu) #5

Thanks Zach. Here is my mapping. Should I add _index : true in the
properties. What will be the affect on this on the memory ? Will it store
all the documents in the memory ?

mappings: {
response: {
properties: {
rdate: {
format: dateOptionalTimetype: date
}rid: {
type: integer
}rtext: {
properties: {
textvalue: {
type: string
}textmapid: {
type: integer
}
}type: nested
}rdays: {
type: string
}mapids: {
type: integer
}rdata: {
properties: {
mapid: {
type: integer
}value: {
type: float
}
}type: nested
}
}
}

Thanks,
Abhishek

On Sun, Sep 22, 2013 at 7:04 AM, Zachary Tong zacharyjtong@gmail.comwrote:

Can you post the mapping of your document? Did you index the rdata
field? It will only be loaded into Field Data (e.g. doc values) if it is
an indexed field.

-Zach

On Saturday, September 21, 2013 5:21:14 PM UTC-4, Abhishek Andhavarapu
wrote:

Thanks a lot for the reply Zach. I tried using doc before. I get this
exception

  • nested: ElasticSearchIllegalArgumentEx**ception[No field found for
    [rdata] in mapping with types [response]]; }]
  • status: 500

The same works when I use _source. How do I rewrite the query to use doc ?

Regards,
Abhishek

On Saturday, September 21, 2013 at 10:58 AM, Zachary Tong wrote:

If you are getting the exceptions referenced in issue #3021 ("*
java.lang.verifyError: *Expec**ting to find integer on stack"), then
that is just a problem with MVEL, I'm not aware of any workarounds. MVEL
is, unfortunately, not well maintained. It is only used as the default in
Elasticsearch because it is the fastest. We are hoping to move to
something more universal (e.g. Javascript) in the future, once Java 8
clears up some of the performance issues.

Javascript (and other language plugins) will be slightly slower than
MVEL, but the performance may be acceptable to you.

Regardless of language choice, your script is going to be massively slow.
You are using the "_source" variable, which loads data directly from the
original source. This requires random access to the disk for every single
document evaluated by the facet. Disk I/O is the slowest component in a
modern computer, so this will kill facet performance. Instead, use
in-memory doc values. E.g:

"script": "float val = 0; float val1 = 0; for (int i = 0; i <
doc['rdata'].size(); i++){if(doc['rdata'][i].mapid == 87) {val =
doc['rdata'][i].value} } return val"

Doc values are stored in memory and will be several orders of magnitude
faster than loading from _source.

Hope that helps!
-Zach

On Thursday, September 19, 2013 12:54:46 PM UTC-4, Abhishek Andhavarapu
wrote:

Hi,

I get occasional errors when running mvel scripts. I found the similar
issue addressed at

https://github.com/**elasticsearch/elasticsearch/**issues/3021https://github.com/elasticsearch/elasticsearch/issues/3021

https://gist.github.com/**imotov/5554678https://gist.github.com/imotov/5554678

I get the exception twice before it comes back with results. And I don't
get the execption any more. Since the above posts were 4 months old I was
hoping if there was a fix or workaround. Also is using JS slower than MVEL ?

My Query :

{

"facets": {

"facet1": {

  "statistical": {

    "script": "float val = 0; float val1 = 0; for (int i = 0; i <

_source.rdata.size(); i++){if(_source.rdata[i].mapid == 87) {val =
_source.rdata[i].value} } return val"

  },

  "facet_filter": {

    "terms": {

      "mapids": [

        "87"

      ],

      "execution": "and"

    }

  }

}

}

}

Thanks,

Abhishek

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/**
topic/elasticsearch/**vyhWsTj3UOM/unsubscribehttps://groups.google.com/d/topic/elasticsearch/vyhWsTj3UOM/unsubscribe
.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@**googlegroups.com.

For more options, visit https://groups.google.com/**groups/opt_outhttps://groups.google.com/groups/opt_out
.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/vyhWsTj3UOM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Abhishek Andhavarapu) #6

Zach,

Can you please look at the mappings I evem added "index" : "yes" to the
rdata mapping. I still get this expection

  • nested: ElasticSearchIllegalArgumentException[No field found for
    [rdata] in mapping with types [response]]; }]

when running "script": "float val = 0; float val1 = 0; for (int i = 0; i <
doc['rdata'].size(); i++){if(doc['rdata'][i].mapid == 87) {val =
doc['rdata'][i].value} } return val"

On Mon, Sep 23, 2013 at 2:24 PM, Abhishek abhishek376@gmail.com wrote:

Thanks Zach. Here is my mapping. Should I add _index : true in the
properties. What will be the affect on this on the memory ? Will it store
all the documents in the memory ?

mappings: {
response: {
properties: {
rdate: {
format: dateOptionalTimetype: date
}rid: {
type: integer
}rtext: {
properties: {
textvalue: {
type: string
}textmapid: {
type: integer
}
}type: nested
}rdays: {
type: string
}mapids: {
type: integer
}rdata: {
properties: {
mapid: {
type: integer
}value: {
type: float
}
}type: nested
}
}
}

Thanks,
Abhishek

On Sun, Sep 22, 2013 at 7:04 AM, Zachary Tong zacharyjtong@gmail.comwrote:

Can you post the mapping of your document? Did you index the rdata
field? It will only be loaded into Field Data (e.g. doc values) if it is
an indexed field.

-Zach

On Saturday, September 21, 2013 5:21:14 PM UTC-4, Abhishek Andhavarapu
wrote:

Thanks a lot for the reply Zach. I tried using doc before. I get this
exception

  • nested: ElasticSearchIllegalArgumentEx**ception[No field found for
    [rdata] in mapping with types [response]]; }]
  • status: 500

The same works when I use _source. How do I rewrite the query to use doc
?

Regards,
Abhishek

On Saturday, September 21, 2013 at 10:58 AM, Zachary Tong wrote:

If you are getting the exceptions referenced in issue #3021 ("*
java.lang.verifyError: *Expec**ting to find integer on stack"), then
that is just a problem with MVEL, I'm not aware of any workarounds. MVEL
is, unfortunately, not well maintained. It is only used as the default in
Elasticsearch because it is the fastest. We are hoping to move to
something more universal (e.g. Javascript) in the future, once Java 8
clears up some of the performance issues.

Javascript (and other language plugins) will be slightly slower than
MVEL, but the performance may be acceptable to you.

Regardless of language choice, your script is going to be massively
slow. You are using the "_source" variable, which loads data directly from
the original source. This requires random access to the disk for every
single document evaluated by the facet. Disk I/O is the slowest component
in a modern computer, so this will kill facet performance. Instead,
use in-memory doc values. E.g:

"script": "float val = 0; float val1 = 0; for (int i = 0; i <
doc['rdata'].size(); i++){if(doc['rdata'][i].mapid == 87) {val =
doc['rdata'][i].value} } return val"

Doc values are stored in memory and will be several orders of magnitude
faster than loading from _source.

Hope that helps!
-Zach

On Thursday, September 19, 2013 12:54:46 PM UTC-4, Abhishek Andhavarapu
wrote:

Hi,

I get occasional errors when running mvel scripts. I found the similar
issue addressed at

https://github.com/**elasticsearch/elasticsearch/**issues/3021https://github.com/elasticsearch/elasticsearch/issues/3021

https://gist.github.com/**imotov/5554678https://gist.github.com/imotov/5554678

I get the exception twice before it comes back with results. And I don't
get the execption any more. Since the above posts were 4 months old I was
hoping if there was a fix or workaround. Also is using JS slower than MVEL ?

My Query :

{

"facets": {

"facet1": {

  "statistical": {

    "script": "float val = 0; float val1 = 0; for (int i = 0; i <

_source.rdata.size(); i++){if(_source.rdata[i].mapid == 87) {val =
_source.rdata[i].value} } return val"

  },

  "facet_filter": {

    "terms": {

      "mapids": [

        "87"

      ],

      "execution": "and"

    }

  }

}

}

}

Thanks,

Abhishek

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/**
topic/elasticsearch/**vyhWsTj3UOM/unsubscribehttps://groups.google.com/d/topic/elasticsearch/vyhWsTj3UOM/unsubscribe
.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@**googlegroups.com.

For more options, visit https://groups.google.com/**groups/opt_outhttps://groups.google.com/groups/opt_out
.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/vyhWsTj3UOM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(system) #7