What is wrong with my JSON Input usage?

Hello,

I have an index where I store some server information I pull from an http API, I'm trying to get a pie chart with the number of vcpus each of the server has available. I have to multiply the 'totalProcessorCores' by 2 to get the vcpus (threads) but for some reason I'm getting either of this messages depending of the request I'm sending to elasticsearch.

Visualize: [parsing_exception] Unknown key for a START_OBJECT in [script]

{
  "size": 0,
  "query": {
    "bool": {
      "must": [
        {
          "query_string": {
            "analyze_wildcard": true,
            "query": "type: Server"
          }
        },
        {
          "range": {
            "@timestamp": {
              "gte": 1491195600000,
              "lte": 1491281999999,
              "format": "epoch_millis"
            }
          }
        }
      ],
      "must_not": []
    }
  },
  "_source": {
    "excludes": []
  },
  "aggs": {
    "2": {
      "terms": {
        "field": "id.name",
        "size": 20,
        "order": {
          "_term": "desc"
        }
      },
      "aggs": {
        "1": {
          "top_hits": {
            "docvalue_fields": [
              "totalProcessorCores"
            ],
            "_source": "totalProcessorCores",
            "size": 1,
            "sort": [
              {
                "@timestamp": {
                  "order": "desc"
                }
              }
            ],
            "script": {
              "lang": "groovy",
              "inline": "doc['totalProcessorCores'].value * 2"
            }
          }
        }
      }
    }
  }
}

Visualize: [parsing_exception] Unknown key for a VALUE_STRING in [script]

{
  .
  .
  .,
            "script": "doc['totalProcessorCores'].value * 2",
            "lang": "groovy"
          }
        }
      }
    }
  }
}

I also allowed the groovy scripts in my elasticsearch.yml and restarted the service.

script.engine.groovy.inline.aggs: true
script.engine.groovy.inline.search: true

In some posts I found googling this errors they were always related to some typo or bad use of the DSL but I don't seem to catch mine. I already checked it agains the current docs and also tried this example with no success.

Thank you,

N

Can you try the following?

GET YOUR_INDEX/YOUR_DOC/_search
{
  "script_fields": {
    "my_doubled_field": {
      "script": {
        "lang":   "expression",
        "inline": "doc['totalProcessorCores'] * multiplier",
        "params": {
          "multiplier": 2
        }
      }
    }
  }
} 

Hello jkuang,

Tried. Same "Unknown key for a START_OBJECT in [script]." error shows up.

N

I think it's the way the doc['totalProcessorCores'] field object is being reference. The code says it can't match this object.

Can you provide your index mapping and a line of json input?

Sure, this is the mapping for the server type.

    "mappings" : {
      "Server" : {
        "dynamic" : false,
        "properties" : {
          "biosVersion" : {
            "type" : "keyword"
          },
          "coresPerProcessorSocket" : {
            "type" : "integer"
          },
          "serverPoolId" : {
            "properties" : {
              "name" : {
                "type" : "keyword"
              },
              "type" : {
                "type" : "keyword"
              },
              "value" : {
                "type" : "keyword"
              },
              "uri" : {
                "type" : "keyword"
              }
            }
          },
          "memory" : {
            "type" : "long"
          },
          "processorType" : {
            "type" : "keyword"
          },
          "biosReleaseDate" : {
            "type" : "keyword"
          },
          "ethernetPortIds" : {
            "type" : "nested",
            "properties" : {
              "name" : {
                "type" : "keyword"
              },
              "type" : {
                "type" : "keyword"
              },
              "value" : {
                "type" : "keyword"
              },
              "uri" : {
                "type" : "keyword"
              }
            }
          },
          "threadsPerCore" : {
            "type" : "integer"
          },
          "serverRunState" : {
            "type" : "keyword"
          },
          "clusterId" : {
            "properties" : {
              "name" : {
                "type" : "keyword"
              },
              "type" : {
                "type" : "keyword"
              },
              "value" : {
                "type" : "keyword"
              },
              "uri" : {
                "type" : "keyword"
              }
            }
          },
          "processorSpeed" : {
            "type" : "double"
          },
          "productName" : {
            "type" : "keyword"
          },
          "manufacturer" : {
            "type" : "keyword"
          },
          "biosVendor" : {
            "type" : "keyword"
          },
          "hostname" : {
            "type" : "keyword"
          },
          "protected" : {
            "type" : "boolean"
          },
          "networkIds" : {
            "type" : "nested",
            "properties" : {
              "name" : {
                "type" : "keyword"
              },
              "type" : {
                "type" : "keyword"
              },
              "value" : {
                "type" : "keyword"
              },
              "uri" : {
                "type" : "keyword"
              }
            }
          },
          "enabledProcessorCores" : {
            "type" : "integer"
          },
          "managerUuid" : {
            "type" : "keyword"
          },
          "storageInitiatorIds" : {
            "type" : "nested",
            "properties" : {
              "name" : {
                "type" : "keyword"
              },
              "type" : {
                "type" : "keyword"
              },
              "value" : {
                "type" : "keyword"
              },
              "uri" : {
                "type" : "keyword"
              }
            }
          },
          "ovmVersion" : {
            "type" : "keyword"
          },
          "serialNumber" : {
            "type" : "keyword"
          },
          "repositoryExportIds" : {
            "type" : "nested",
            "properties" : {
              "name" : {
                "type" : "keyword"
              },
              "type" : {
                "type" : "keyword"
              },
              "value" : {
                "type" : "keyword"
              },
              "uri" : {
                "type" : "keyword"
              }
            }
          },
          "totalProcessorCores" : {
            "type" : "integer"
          },
          "populatedProcessorSockets" : {
            "type" : "integer"
          },
          "vmIds" : {
            "type" : "nested",
            "properties" : {
              "name" : {
                "type" : "keyword"
              },
              "type" : {
                "type" : "keyword"
              },
              "value" : {
                "type" : "keyword"
              },
              "uri" : {
                "type" : "keyword"
              }
            }
          },
          "usableMemory" : {
            "type" : "long"
          },
          "storageElementIds" : {
            "type" : "nested",
            "properties" : {
              "name" : {
                "type" : "keyword"
              },
              "type" : {
                "type" : "keyword"
              },
              "value" : {
                "type" : "keyword"
              },
              "uri" : {
                "type" : "keyword"
              }
            }
          },
          "maintenanceMode" : {
            "type" : "boolean"
          }
        }
      },

About the line of json input well I just simply want to multiply that value by 2, or if it's possible by the threadsPerCore value which is another integer in the same type mapping, so the script I wrote in the OP stands as an example.

Thank you very much!

I think something is wrong with your index. I couldn't reproduce your issue. Please see my sample below using your mapping:

PUT mymulti
{
 "mappings" : {
      "Server" : {
        "dynamic" : false,
        "properties" : {
          "biosVersion" : {
            "type" : "keyword"
          },
          "coresPerProcessorSocket" : {
            "type" : "integer"
          },
          "serverPoolId" : {
            "properties" : {
              "name" : {
                "type" : "keyword"
              },
              "type" : {
                "type" : "keyword"
              },
              "value" : {
                "type" : "keyword"
              },
              "uri" : {
                "type" : "keyword"
              }
            }
          },
          "memory" : {
            "type" : "long"
          },
          "processorType" : {
            "type" : "keyword"
          },
          "biosReleaseDate" : {
            "type" : "keyword"
          },
          "ethernetPortIds" : {
            "type" : "nested",
            "properties" : {
              "name" : {
                "type" : "keyword"
              },
              "type" : {
                "type" : "keyword"
              },
              "value" : {
                "type" : "keyword"
              },
              "uri" : {
                "type" : "keyword"
              }
            }
          },
          "threadsPerCore" : {
            "type" : "integer"
          },
          "serverRunState" : {
            "type" : "keyword"
          },
          "clusterId" : {
            "properties" : {
              "name" : {
                "type" : "keyword"
              },
              "type" : {
                "type" : "keyword"
              },
              "value" : {
                "type" : "keyword"
              },
              "uri" : {
                "type" : "keyword"
              }
            }
          },
          "processorSpeed" : {
            "type" : "double"
          },
          "productName" : {
            "type" : "keyword"
          },
          "manufacturer" : {
            "type" : "keyword"
          },
          "biosVendor" : {
            "type" : "keyword"
          },
          "hostname" : {
            "type" : "keyword"
          },
          "protected" : {
            "type" : "boolean"
          },
          "networkIds" : {
            "type" : "nested",
            "properties" : {
              "name" : {
                "type" : "keyword"
              },
              "type" : {
                "type" : "keyword"
              },
              "value" : {
                "type" : "keyword"
              },
              "uri" : {
                "type" : "keyword"
              }
            }
          },
          "enabledProcessorCores" : {
            "type" : "integer"
          },
          "managerUuid" : {
            "type" : "keyword"
          },
          "storageInitiatorIds" : {
            "type" : "nested",
            "properties" : {
              "name" : {
                "type" : "keyword"
              },
              "type" : {
                "type" : "keyword"
              },
              "value" : {
                "type" : "keyword"
              },
              "uri" : {
                "type" : "keyword"
              }
            }
          },
          "ovmVersion" : {
            "type" : "keyword"
          },
          "serialNumber" : {
            "type" : "keyword"
          },
          "repositoryExportIds" : {
            "type" : "nested",
            "properties" : {
              "name" : {
                "type" : "keyword"
              },
              "type" : {
                "type" : "keyword"
              },
              "value" : {
                "type" : "keyword"
              },
              "uri" : {
                "type" : "keyword"
              }
            }
          },
          "totalProcessorCores" : {
            "type" : "integer"
          },
          "populatedProcessorSockets" : {
            "type" : "integer"
          },
          "vmIds" : {
            "type" : "nested",
            "properties" : {
              "name" : {
                "type" : "keyword"
              },
              "type" : {
                "type" : "keyword"
              },
              "value" : {
                "type" : "keyword"
              },
              "uri" : {
                "type" : "keyword"
              }
            }
          },
          "usableMemory" : {
            "type" : "long"
          },
          "storageElementIds" : {
            "type" : "nested",
            "properties" : {
              "name" : {
                "type" : "keyword"
              },
              "type" : {
                "type" : "keyword"
              },
              "value" : {
                "type" : "keyword"
              },
              "uri" : {
                "type" : "keyword"
              }
            }
          },
          "maintenanceMode" : {
            "type" : "boolean"
          }
        }
      }
 }}
 
 POST mymulti/Server/
 {
   "totalProcessorCores" : 2
 }
 GET mymulti/Server/_search
{
  "script_fields": {
    "my_doubled_field": {
      "script": {
        "lang":   "expression",
        "inline": "doc['totalProcessorCores'] * multiplier",
        "params": {
          "multiplier": 2
        }
      }
    }
  }
}

Results:

{
  "took": 2,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 2,
    "max_score": 1,
    "hits": [
      {
        "_index": "mymulti",
        "_type": "Server",
        "_id": "AVs6uNoNnn3k1Vj753OV",
        "_score": 1,
        "fields": {
          "my_doubled_field": [
            4
          ]
        }
      },
      {
        "_index": "mymulti",
        "_type": "Server",
        "_id": "AVs6vWEjnn3k1Vj753OW",
        "_score": 1,
        "fields": {
          "my_doubled_field": [
            6
          ]
        }
      }
    ]
  }
}

I tried your code again in the dev tools tab in kibana and it shows the same results. However I still get the errors if I try to do the same in a visualization json input. For instance,

And in the input json I wrote:

{
"script": {
        "lang":   "expression",
        "inline": "doc['totalProcessorCores'] * multiplier",
        "params": {
          "multiplier": 2
        }
      }
}

But still got the Unknown key for a START_OBJECT in [script] error.

What version of ES and Kibana are you using?

I'm using 5.3 for everything.

I'm using 5.3 too and my sample works, but it doesn't work for you. Just to confirm, you're getting the error in dev tools or in the visualization?

In the visualization. Here is a bigger print screen.

I couldn't fit the json input field in the image but you can read what I wrote 2 comments ago.

I'm able to reproduce your error now. This is a problem with the JSON input for sure. I even used the example from the doc and I am getting your error.

https://www.elastic.co/guide/en/kibana/current/metric-chart.html

There is a workaround. You can go to Kibana > Management > Index Pattern > choose your index > click on the scripted field tab and put the script there. Then go back to visualization and choose the scripted field .

Hello jkuang,

Thank you very much for your help. Indeed making a new scripted field worked like a charm, do you consider a good idea to open an issue in the github page? Did you already do it by any chance?

Thank you very much again!

N

I would recommend you open an issue on github. ^^ I'm sure many people would encounter this too. Helping each other is what open source is about. :smiley:

All right, I just opened an issue to address this matter.

Thank you again jimmy!

N

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.