Dynamic templates and transform script


(Rr Photosoft) #1

Hi,
I'm using logback to generate some logs from my java app and FILEBEAT to add them to elasticsearch All that is working fine.
I DO NOT want to use logstash because I have memory constraints, and currently no budget to buy more RAM.
So what I am trying to do is use a transform script in the dynamic template provided with filebeat, to transform the "msg" field using a Groovy Script. Unfortunately I have no idea, why my script is never invoked. I have stored the script in config/scripts , while dynamic_scripting is disabled. Can someone please help to correct my syntax.

{
"mappings": {
"_default_": {
  "_all": {
    "enabled": true,
    "norms": {
      "enabled": false
    }
  },
  "dynamic_templates": [
    {
      "template1": {
        "mapping": {
          "doc_values": true,
          "ignore_above": 1024,
          "index": "not_analyzed",
          "type": "{dynamic}",
          "transform":{
             "script":{
                  "file":"transformer"
             }
           }
        },
        "match": "message"
      }
    }
  ],
  "properties": {
    "message": {
      "type": "string",
      "index": "analyzed"
    },
   "test_field":{
     "type":"string',
    "index":"not_analyzed"
    }
  }
 }

Here is my transformer

transformer.groovy
ctx._test_field = "test";

I do not see any test_field in the log index documents, while the message field contains the usual log output. My final intention is to use a regex to parse out certain contents from the message field and append them to other fields.

Thanks.
RRphotosoft.


(system) #2