How to export a scripted field in the elasticsearch-output-plugin

I've written a scripted field to standardise and create a shorthand for URLs.
However from what I understood in the docs we can't use it as a keyword field, also it works only on Kibana and not if I try and get data directly from Elasticsearch.

From my reading I've seen that we can add a script to the elasticsearch-output-plugin.
Again though the docs aren't very clear on how to implement the same.

This is my scripted field

def APIFull = doc['document.APIName.keyword'].value;
def APIFullIndex = APIFull.indexOf('/');
def APIFullColon = APIFull.indexOf(':');
def APIFullv = APIFull.indexOf('v');
def APIparams = APIFull.indexOf('?');
def APIFulla = APIFull.indexOf('a');

if (APIFulla == 1) {
    if (APIparams > 0) {
        return "cms/v1" + APIFull.substring(4,APIparams) 
    } else {
        return "cms/v1" + APIFull.substring(4)
    }  
} else if (APIFullIndex == 0 || APIFullIndex == 3) {
    if (APIparams > 0) {
        return "dxl/" + APIFull.substring(APIFullv, APIFullv+3) + APIFull.substring(APIFullv+3, APIparams) 
    } else {
        return "dxl/" + APIFull.substring(APIFullv, APIFullv+3) + APIFull.substring(APIFullv+3)
    }  
} else if (APIFullColon > 0) {
    return "cms/v1" + APIFull.substring(56, APIparams)
} else {
    return APIFull
}

I tried the following in logstash

output {
  elasticsearch {
    hosts => ["http://elastic:9200"]
    document_id => "%{[document][_id]}"
    #doc_as_upsert => true
    action => "update"
    codec => json
    script => 'def APIFull = ctx._source.APIName;
      def APIFullIndex = APIFull.indexOf("/");
      def APIFullColon = APIFull.indexOf(":");
      def APIFullv = APIFull.indexOf("v");
      def APIparams = APIFull.indexOf("?");
      def APIFulla = APIFull.indexOf("a");

      if (APIFulla == 1) {
         if (APIparams > 0) {
            return "cms/v1" + APIFull.substring(4,APIparams) 
        } else {
         return "cms/v1" + APIFull.substring(4)
        }  
      } else if (APIFullIndex == 0 || APIFullIndex == 3) {
          if (APIparams > 0) {
            return "dxl/" + APIFull.substring(APIFullv, APIFullv+3) + APIFull.substring(APIFullv+3, APIparams) 
          } else {
              return "dxl/" + APIFull.substring(APIFullv, APIFullv+3) + APIFull.substring(APIFullv+3)
          }  
      } else if (APIFullColon > 0) {
          return "cms/v1" + APIFull.substring(56, APIparams)
         } else {
          return APIFull
      }'
     scripted_upsert => true
   }
}

However I got the error -

Too many dynamic script compilations within, max: [75/5m]; please use indexed, or scripts with parameters instead; this limit can be changed by the [script.max_compilations_rate] setting", "bytes_wanted"=>0, "bytes_limit"=>0, "durability"=>"TRANSIENT"}}}}}}

Please help with the same.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.