Occasionally missing fields in custom plugin


(Roeland Van Batenburg) #1

I'm running into a nasty bug where the params HashMap provided to the newScript function is somewhat incomplete. I can't find any issue like this and considered filing a bug, but I don't know how to reproduce it.

Elasticsearch version: Version: 5.3.0, Build: 3adb13b/2017-03-23T03:31:50.652Z, JVM: 1.8.0_92-internal

Plugins installed: [CustomPlugin, analysis-icu, x-pack]

JVM version (java -version): openjdk version "1.8.0_92-internal"
OpenJDK Runtime Environment (build 1.8.0_92-internal-alpine-r1-b14)
OpenJDK 64-Bit Server VM (build 25.92-b14, mixed mode)

OS version (uname -a if on a Unix-like system):

  • Linux f973639e3af1 4.9.75-linuxkit-aufs #1 SMP Tue Jan 9 10:58:17 UTC 2018 x86_64 Linux (dockerized, local)
  • Linux telc1 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9 14:43:09 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux (server, no docker)

Description of the problem including expected versus actual behavior:
While processing a script_score or script_fields elasticsearch is not passing the full information to the newScript function of our custom plugin. This is appear to happen randomly, the same plugin is used in script_score and script_fields and often only one of those is missing some fields. More specifically it appears that the size of one of the HashMaps in the params is wrong:

From a debug on org/elasticsearch/elasticsearch/5.3.0/elasticsearch-5.3.0-sources.jar!/org/elasticsearch/script/NativeScriptEngineService.java:75 I see that the HashMap has 13 items (as expected), but the size is reported to be 11. It always seem to happen to either tireFront or tireRear. When calculating the score I might be using the budget value which is then missing (I might get lucky and the missing 2 fields might not be necessary).

vars.get("tireFront") = {HashMap@13585}  size = 11
 0 = {HashMap$Node@13593} "normal" -> "14"
 1 = {HashMap$Node@13594} "year" -> "31052017"
 2 = {HashMap$Node@13595} "id" -> "P"
 3 = {HashMap$Node@13596} "rimSize" -> "15"
 4 = {HashMap$Node@13597} "serviceLifeCorrection" -> "0.52"
 5 = {HashMap$Node@13598} "premium" -> "14"
 6 = {HashMap$Node@13599} "tireType" -> "R"
 7 = {HashMap$Node@13600} "width" -> "185"
 8 = {HashMap$Node@13601} "serviceLife" -> "65000.0"
 9 = {HashMap$Node@13602} "speedIndex" -> "T"
 10 = {HashMap$Node@13603} "tireSize" -> "P18560TR15"
 11 = {HashMap$Node@13604} "height" -> "60"
 12 = {HashMap$Node@13605} "budget" -> "14"

Steps to reproduce:

Unfortunately I can't share the plugin or the elasticsearch request due to IP restrictions. I'm not sure how to reproduce it any other way.


(Ivan Brusic) #2

Don't have much insight into your problem, but is perhaps threading an
issue? Can you use an immutable map and see if write exceptions after the
fact occur?


(Roeland Van Batenburg) #3

I'll look into that and see if it happens. Thanks


(system) #4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.