Cannot iterate arrays in couchdb river scripts

Trying this post again with a new clean thread (sorry, just hoping to
actually get a response from someone on this topic as it seems to be
getting pretty buried)...

Hi there. Thanks for an amazing product...

We're currently having some issues with scripts in a couchdb river. We have
most of our mutations all working, but there is a case where the incoming
doc has a property that is an array of strings. No matter what we've tried,
we simply cannot iterate this array to mutate the indexed document based in
its values.

For example:

"script": "ctx.doc.myArray.forEach(function(obj) { /* do something here */
})"

results in the following error in the ES logs:
org.mozilla.javascript.EcmaError: TypeError: Cannot call property forEach
in object null. It is not a function, it is "undefined". (Script12.js#1)
at
org.mozilla.javascript.ScriptRuntime.constructError(ScriptRuntime.java:3687)
at
org.mozilla.javascript.ScriptRuntime.constructError(ScriptRuntime.java:3665)
at
org.mozilla.javascript.ScriptRuntime.typeError(ScriptRuntime.java:3693)
at
org.mozilla.javascript.ScriptRuntime.typeError3(ScriptRuntime.java:3719)
at
org.mozilla.javascript.ScriptRuntime.notFunctionError(ScriptRuntime.java:3786)
at
org.mozilla.javascript.ScriptRuntime.getPropFunctionAndThisHelper(ScriptRuntime.java:2269)
at
org.mozilla.javascript.ScriptRuntime.getPropFunctionAndThis(ScriptRuntime.java:2251)
at
org.mozilla.javascript.gen.Script12_js_11._c_script_0(Script12.js:1)
at org.mozilla.javascript.gen.Script12_js_11.call(Script12.js)
at
org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:394)
at
org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3091)
at org.mozilla.javascript.gen.Script12_js_11.call(Script12.js)
at org.mozilla.javascript.gen.Script12_js_11.exec(Script12.js)
at
org.elasticsearch.script.javascript.JavaScriptScriptEngineService$JavaScriptExecutableScript.run(JavaScriptScriptEngineService.java:186)
at
org.elasticsearch.river.couchdb.CouchdbRiver.processLine(CouchdbRiver.java:241)
at
org.elasticsearch.river.couchdb.CouchdbRiver.access$500(CouchdbRiver.java:64)
at
org.elasticsearch.river.couchdb.CouchdbRiver$Indexer.run(CouchdbRiver.java:334)
at java.lang.Thread.run(Thread.java:722)

That was pseudocode, because our script is doing more of course, but know
that we are sanity checking before that line (e.g. "if (ctx.doc.myArray)
{}"), so the sanity checks pass (i.e. myArray is NOT null), but we still
cannot iterate the darn thing.

Also weird with this is if we change our sanity checks to be more explicit
(i.e. "if (typeof ctx.doc.myArray != 'undefined') {}"), we then get another
odd error:

org.mozilla.javascript.EvaluatorException: Invalid JavaScript value of type
java.util.ArrayList (Script13.js#1)
at
org.mozilla.javascript.DefaultErrorReporter.runtimeError(DefaultErrorReporter.java:77)
at
org.mozilla.javascript.Context.reportRuntimeError(Context.java:913)
at
org.mozilla.javascript.Context.reportRuntimeError(Context.java:969)
at
org.mozilla.javascript.Context.reportRuntimeError1(Context.java:932)
at
org.mozilla.javascript.ScriptRuntime.errorWithClassName(ScriptRuntime.java:3964)
at
org.mozilla.javascript.ScriptRuntime.typeof(ScriptRuntime.java:2530)
at
org.mozilla.javascript.gen.Script13_js_12._c_script_0(Script13.js:1)
at org.mozilla.javascript.gen.Script13_js_12.call(Script13.js)
at
org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:394)
at
org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3091)
at org.mozilla.javascript.gen.Script13_js_12.call(Script13.js)
at org.mozilla.javascript.gen.Script13_js_12.exec(Script13.js)
at
org.elasticsearch.script.javascript.JavaScriptScriptEngineService$JavaScriptExecutableScript.run(JavaScriptScriptEngineService.java:186)
at
org.elasticsearch.river.couchdb.CouchdbRiver.processLine(CouchdbRiver.java:241)
at
org.elasticsearch.river.couchdb.CouchdbRiver.access$500(CouchdbRiver.java:64)
at
org.elasticsearch.river.couchdb.CouchdbRiver$Indexer.run(CouchdbRiver.java:334)
at java.lang.Thread.run(Thread.java:722)

We have also tried testing general array iteration with tests like "['foo',
'bar'].forEach(function(obj) {}" and that works fine. So there's something
buggy about the incoming arrays on ctx.doc that is completely preventing us
from iterating them. We've tried everything we can think of at this point,
including using a regular old for..loop, assigning a new variable from the
ctx.doc.myArray reference and iterating that, etc... nothing seems to work.

Any ideas?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

On Monday, June 3, 2013 2:08:20 PM UTC-6, ry...@huterra.com wrote:

Trying this post again with a new clean thread (sorry, just hoping to
actually get a response from someone on this topic as it seems to be
getting pretty buried)...

Hi there. Thanks for an amazing product...

We're currently having some issues with scripts in a couchdb river. We
have most of our mutations all working, but there is a case where the
incoming doc has a property that is an array of strings. No matter what
we've tried, we simply cannot iterate this array to mutate the indexed
document based in its values.
[SNIP]

Did you ever figure this one out? We're trying to grab the first element of
an array field from CouchDB and assign that to a new field...

-Brian

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

No, it just doesn't work. Whatever vm is being used (I'm assuming Rhino but
haven't dug that deep) is being used very weirdly. Arrays exist as far as
standard javascript sanity checks are concerned (i.e. "if (obj.arr) {}"),
but no matter what you simply cannot iterate them, and more strict "typeof
obj.arr" checks completely bomb out. We are working around it by adding a
mutation step outside of ES, but it sucks because the workaround is code
and complexity we wouldn't have to have if the vm wasn't wonky.

On Wednesday, July 10, 2013 9:23:15 PM UTC-5, Brian Gadoury wrote:

On Monday, June 3, 2013 2:08:20 PM UTC-6, ry...@huterra.com wrote:

Trying this post again with a new clean thread (sorry, just hoping to
actually get a response from someone on this topic as it seems to be
getting pretty buried)...

Hi there. Thanks for an amazing product...

We're currently having some issues with scripts in a couchdb river. We
have most of our mutations all working, but there is a case where the
incoming doc has a property that is an array of strings. No matter what
we've tried, we simply cannot iterate this array to mutate the indexed
document based in its values.
[SNIP]

Did you ever figure this one out? We're trying to grab the first element
of an array field from CouchDB and assign that to a new field...

-Brian

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

On Thursday, July 11, 2013 8:26:33 AM UTC-6, ry...@huterra.com wrote:

No, it just doesn't work. Whatever vm is being used (I'm assuming Rhino
but haven't dug that deep) is being used very weirdly. Arrays exist as far
as standard javascript sanity checks are concerned (i.e. "if (obj.arr)
{}"), but no matter what you simply cannot iterate them, and more strict
"typeof obj.arr" checks completely bomb out. We are working around it by
adding a mutation step outside of ES, but it sucks because the workaround
is code and complexity we wouldn't have to have if the vm wasn't wonky.

Yep. I've found that toString() and join() do not work on array data,
either. That's pretty limiting, especially when you take into account how
Elasticsearch seems to store arrays of strings already sorted, so you can't
use a script sort to implement sorting on the first element of a given
field.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.