Update Script in Elasticsearch Output Parsing Array as String

Elastic Stack Logstash
1

I have a document with the field "name_servers" and "past_name_servers". If I have the following:

script_type => "inline"
				script => '
					ctx._source.name_servers = "%{name_servers}";
					ctx._source.past_name_servers.add("%{name_servers}");
				'

"past_name_servers" is properly updated and the new values are added into the array. However, "name_servers" gets the "stringified" value of the array...

I also tried removing the field first in case it made any difference to no avail.

I checked with "stdout{codec=>rubydebug}" and the value of "name_servers" is definitely an array.

This is the mapping for both fields:

                "name_servers": {
                    "type": "string",
                    "index": "analyzed"
                },
                "past_name_servers": {
                    "type": "string",
                    "index": "analyzed"
                }

If I were to use the following instead, it works:

ctx._source.name_servers = "%{name_servers}".split(",");

However, I believe I'm missing out on a way to take my formatted array and assign the value as such without having to go array=>string=>array each and every time.

2

Hi Edgar,

Did you find a solution to this? I believe I am facing a similar issue: Creating/updating array of objects in elasticsearch logstash output

Thanks,
Nandan

3

Hi Nandan,

I ended up just using split each and every time as there doesn't seem to be a fix for this issue

4

Hi Edgar,

Can you try using

event.name_servers

instead of

"%{name_servers}".split(",");
5

I updated my setup to v5 except for Logstash as I saw no need for it. However, all my scripts now explicitly say "painless" to ensure I won't be using groovy anymore. The same issue is still found. I'm having a hard time finding a way to recreate it as it only happens "sometimes". Meaning that out of thousands of records that go through the exact same config, only some of them are parsed badly.

I'm not sure if this is caused by the elasticsearch output plugin (my best guess) or groovy/painless as I've tried both at this point. Is this a known bug? Is there a way for the array object to be interpreted as a string in the output filter of Logstash in the inner workings of Logstash?