Accessing Event Metadata inside of Elasticsearch Output Script

Elastic Stack Logstash
1

Using the elasticsearch output module, I have doc_as_upsert set to true for many update actions. I'm using the @metadata capability to store the fields which I don't want upserting into ES.

However, I'm having trouble accessing fields inside of @metadata inside the Elasticsearch script function. The below script makes sure the urls array is less than 1,001 and also makes sure the new URL being added is unique to the array:

output {
elasticsearch{
    hosts => "***************"
    user => "****"
    index => "****"
    password => "*********"
    document_type => "document"
    document_id => "%{[@metadata][domain]}"
    action => "update"
    script => 'if(ctx._source.urls.length < 1001){ boolean match = false; for (url in ctx._source.urls){if (url == params.event.get("[@metadata][url]")){match = true;}} if(match==false){ctx._source.urls.add(params.event.get("[@metadata][url]")); ;}}'
    doc_as_upsert => true
}

}

Inside of ES, the URL value gets appended simply as NULL. I viewed the url metadata field inside of rubydebug and it is for sure being added. I can't reference the metadata directly "%{[@metadata][domain]}" because I run into script compilation errors for too many generated scripts.

Is there any way to access the metadata.url field inside of the script function for elasticsearch output plugin?

Thank you!

2

bump any ideas?

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.