I've written a scripted field to standardise and create a shorthand for URLs.
However from what I understood in the docs we can't use it as a keyword field, also it works only on Kibana and not if I try and get data directly from Elasticsearch.
From my reading I've seen that we can add a script to the elasticsearch-output-plugin.
Again though the docs aren't very clear on how to implement the same.
This is my scripted field
def APIFull = doc['document.APIName.keyword'].value;
def APIFullIndex = APIFull.indexOf('/');
def APIFullColon = APIFull.indexOf(':');
def APIFullv = APIFull.indexOf('v');
def APIparams = APIFull.indexOf('?');
def APIFulla = APIFull.indexOf('a');
if (APIFulla == 1) {
if (APIparams > 0) {
return "cms/v1" + APIFull.substring(4,APIparams)
} else {
return "cms/v1" + APIFull.substring(4)
}
} else if (APIFullIndex == 0 || APIFullIndex == 3) {
if (APIparams > 0) {
return "dxl/" + APIFull.substring(APIFullv, APIFullv+3) + APIFull.substring(APIFullv+3, APIparams)
} else {
return "dxl/" + APIFull.substring(APIFullv, APIFullv+3) + APIFull.substring(APIFullv+3)
}
} else if (APIFullColon > 0) {
return "cms/v1" + APIFull.substring(56, APIparams)
} else {
return APIFull
}
I tried the following in logstash
output {
elasticsearch {
hosts => ["http://elastic:9200"]
document_id => "%{[document][_id]}"
#doc_as_upsert => true
action => "update"
codec => json
script => 'def APIFull = ctx._source.APIName;
def APIFullIndex = APIFull.indexOf("/");
def APIFullColon = APIFull.indexOf(":");
def APIFullv = APIFull.indexOf("v");
def APIparams = APIFull.indexOf("?");
def APIFulla = APIFull.indexOf("a");
if (APIFulla == 1) {
if (APIparams > 0) {
return "cms/v1" + APIFull.substring(4,APIparams)
} else {
return "cms/v1" + APIFull.substring(4)
}
} else if (APIFullIndex == 0 || APIFullIndex == 3) {
if (APIparams > 0) {
return "dxl/" + APIFull.substring(APIFullv, APIFullv+3) + APIFull.substring(APIFullv+3, APIparams)
} else {
return "dxl/" + APIFull.substring(APIFullv, APIFullv+3) + APIFull.substring(APIFullv+3)
}
} else if (APIFullColon > 0) {
return "cms/v1" + APIFull.substring(56, APIparams)
} else {
return APIFull
}'
scripted_upsert => true
}
}
However I got the error -
Too many dynamic script compilations within, max: [75/5m]; please use indexed, or scripts with parameters instead; this limit can be changed by the [script.max_compilations_rate] setting", "bytes_wanted"=>0, "bytes_limit"=>0, "durability"=>"TRANSIENT"}}}}}}
Please help with the same.