Java API Plainless script `indexOf` give wrong answer


(Zt Zeng) #1

Executor following script using rest way and Java API give different result:

  "script": {
    "inline": "ctx._source.t = params.users; ctx._source.i1= ctx._source.users.indexOf(params.users); ctx._source.i2= ctx._source.users.lastIndexOf(params.users);",
    "params": {
      "users": 540722
    }
  }


// the rest result
"t": 540722,
"i1": 0,
"i2": 105,
// the Java API result
"t1": 540722,
"i11": -1,
"i21": -1

what would be possible reason (version mismatch?)?

Server version: 5.6.1
Java client: 5.5.0


Update:

Using the Java client 5.6.1
give me the following result (still not right for indexOf):

"t1": 540722,
"i11": 105,
"i21": 105

(David Pilato) #2

I'm surprised. May be try to reproduce it and share the script with us?

  • One reproduction script that we can run in Kibana Dev Console.
  • One java class that we can use to replay the second behavior.

(Zt Zeng) #3

kibana:

PUT test/test/1
{
      "users": [
      540722,
      540722,
      540722,
      540722,
      540722,
      540722,
      540722,
      540722,
      540722,
      540722,
      540722,
      540722,
      540722,
      540722,
      540722,
      540722,
      540722,
      540722,
      540722,
      540722
    ]
}
POST test/test/1/_update
{
  "script": {
    "inline": "ctx._source.t = params.users; ctx._source.i1= ctx._source.users.indexOf(params.users); ctx._source.i2= ctx._source.users.lastIndexOf(params.users);",
    "params": {
      "users": 540722
    }
  }
}
GET test/test/1

Part of Java code:

HashMap<String, Object> params = new HashMap<>();
params.put("users", 540722L);
Script painless = new Script(ScriptType.INLINE, "painless",
    "ctx._source.t1 = params.users; ctx._source.i11= ctx._source.users.indexOf(params.users); ctx._source.i21= ctx._source.users.lastIndexOf(params.users);",
    params);
UpdateRequestBuilder updateRequestBuilder = client.prepareUpdate("test", "test", "1")
    .setScript(painless);

BulkRequestBuilder bulkRequest = client.prepareBulk();
bulkRequest.add(updateRequestBuilder.request());
BulkResponse bulkItemResponses = bulkRequest.execute().actionGet();

We found some clue:

params.put("users", 540722L); // will return -1, now

params.put("users", 540722); // will work

(Zt Zeng) #4

And even it give the index in range, ES still complains about ArrayIndexOutOfBound, which is my original intuitive:

POST task-0/task/13031005/_update
{
  "script": {
    "inline": "ctx._source.users.remove(ctx._source.users.indexOf(params.users))",
    "params": {
      "users": 540722
    }
  }
}

(Zt Zeng) #5

So any plan to fix this bug because my "users" is actually long list


(Jack Conradson) #6

Unfortunately this is not a simple issue to fix. someList.indexOf is based on the equivalent Java method which takes an Object and does comparisons based on that. So somewhere in the chain the users field is being parsed as both Integer and Long values (probably due to the values being unknown at the time of parsing, so anything smaller than Integer.MAX_VALUE becomes an Integer and anything greater becomes a Long.

As a workaround for now one thing to try could be the following:

def users = params.users = params.users < Integer.MAX_VALUE ? (int)params.users : (long)params.users; <yourlist>.indexOf(users)

This will ensure the appropriate reference type (Integer or Long) based on the incoming value for comparison against.


(Zt Zeng) #7

But my mappings is

    "users": {
      "type": "long"
    }

Shouldn't this ensure my list to be List<Long>?

By the way, can I set users as a hashset and directly use users.remove?


(Ryan Ernst) #8

Painless does not have generics, so to it, you only have List (of Object). In java, if you call Long.equals with an Integer, it will always return false (boxed types do not do promotions when comparing).


(Zt Zeng) #9

So, as you said, even I use a hashset, remove will still not work, right?


(Ryan Ernst) #10

Right, since remove takes Object, unless you pass in the exact type of the boxed value you are searching for.


(Zt Zeng) #11

One more question, the elastic array type can be any Java collection type?


(Ryan Ernst) #12

Arrays aren't a real type, they are a form that gets parsed into fields, whether they are sub fields (eg passing json objects) or concrete values in the array. As with all of a document passed to elasticsearch, it must be in json, so the limitation is whatever data types json has. See https://www.elastic.co/guide/en/elasticsearch/reference/current/array.html.


(system) #13

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.