Max emits for creating runtime field

[robert-n-stewart (robert)]

Hello,

I'm trying to create a runtime field which takes a multi-value long field and converts it to a multi-value string field. This way I can perform a regex search on it. The following search works, but only for fields only with 100 values or less:

{"runtime_mappings": {
    "numbers_to_string": {
      "type": "keyword",
      "script": {
        "source": "if (doc.containsKey('numbers')) {int count = 0; for (item in doc['numbers']) { if (count < 100) {emit(item.toString()); count++} }}"
      }
    }
  },
     "fields": [
     "numbers_to_string"
   ],
   "query": {
     "bool": {"must": [
       {"regexp": {"numbers_to_string": {"value": ".*123.*"}}}
     ]}
   },
  "_source": ["numbers_to_string"]
}

if I increase the cap above 100 I get a max emits error. How can I get around this?

An alternative approach I considered was using a runtime "text" field and separating the numbers by a space. But, I get the following error:

{
        "type" : "mapper_parsing_exception",
        "reason" : "No handler for type [text] declared on runtime field [numbers_to_string]"
}

Any insight on this would be helpful as well

[nik9000 (Nik Everett)]

Runtime fields don't support text because it's have to analyze on the fly and we didn't build that. It'd be slower than lots of emits anyway.

I'd file an issue explaining that you want to work with lots of values. The emit limit isn't configurable now but it's worth talking with the folks who wrote it to see if it should be. Or if we should bump the limit up.

How many values do you commonly see? Runtime fields will have to check each on in sequence, so if you have zillions of values it'll get slow.

[robert-n-stewart (robert)]

Thank you for the quick reply Nik!

The number of values I'm working with are in the hundreds, not thousands or more. It would be nice if the emit count was configurable. Where can I file this issue?

For my use case, would you suggest using the script query if I want to search for numbers containing a certain sequence of digits?

Edit: to be more specific, the number of values actually ranges from 0-10,000, but the average is <100.