Problem with mustache on watcher action

Hi all

I'm trying to do this action (a webhook)

"hook-tornado2": {
  "webhook": {
    "scheme": "http",
    "host": "localhost",
    "port": 8080,
    "method": "post",
    "path": "/event/failed-passwords",
    "params": {
      "token": "abc"
    },
    "headers": {},
    "body": {
      "source": "{{#toJson}}Watcher Notification Encountered {{ctx.payload.hits.total}} failed logon from user {{#ctx.payload.hits.hits.0}}{{_source.winlog.event_data.TargetUserName}}{{/ctx.payload.hits.hits.0}}.{{/toJson}}",
      "lang": "mustache"
    }
  }
}

I have this error

"actions": [
  {
    "id": "hook-tornado2",
    "type": "webhook",
    "status": "failure",
    "error": {
      "root_cause": [
        {
          "type": "general_script_exception",
          "reason": "Failed to compile inline script [{{#toJson}}Watcher Notification Encountered {{ctx.payload.hits.total}} failed logon from user {{#ctx.payload.hits.hits.0}}{{_source.winlog.event_data.TargetUserName}}{{/ctx.payload.hits.hits.0}}.{{/toJson}}] using lang [mustache]"
        }
      ],
      "type": "general_script_exception",
      "reason": "Failed to compile inline script [{{#toJson}}Watcher Notification Encountered {{ctx.payload.hits.total}} {{/toJson}}{{#toJson}}failed logon from user {{#ctx.payload.hits.hits.0}}{{_source.winlog.event_data.TargetUserName}}{{/ctx.payload.hits.hits.0}}.{{/toJson}}] using lang [mustache]",
      "caused_by": {
        "type": "mustache_exception",
        "reason": "Mustache function [toJson] must contain one and only one identifier"
      }
    }
  },

I need to compose the message, in output of the body like an email action.

Is it possible?

Thank you
Franco

what are you trying to achieve by using toJson here? That function is able to convert a map to valid JSON, but you have been putting sentences in there - so I am not sure what your expected final result is.

Please show an example of the expected output. Thanks!

I'd like to do a JSON like I did with

"body": "{{#toJson}}ctx.payload{{/toJson}}"

but I want to create a JSON like this

{
    "message": "TEXT"
}    

with the parameter of ctx.pyaload

Thank you
Franco

what you would need to do is running a transform that returns a map return ['message': 'TEXT'], then you can do {{#toJson}}ctx.payload{{/toJson}}

--Alex

Hi

thank you for response. Could you give me an example of transformation on action. I tried to do this, but I not sure to have understand

    "hook-tornado2": {
      "webhook": {
        "transform" : {
            "script" : "return [ 'msg' : 'Watcher Notification Encountered {{ctx.payload.hits.total}} failed logon from user {{#ctx.payload.hits.hits.0}}{{_source.winlog.event_data.TargetUserName}}{{/ctx.payload.hits.hits.0}}.']" 
        },          
        "scheme": "http",
        "host": "localhost",
        "port": 8080,
        "method": "post",
        "path": "/event/failed-passwords",
        "params": {
          "token": "abc"
        },
        "headers": {},
        "body": "{{#toJson}}ctx.payload{{/toJson}}"
      }
    }

With this I have the parameter ctx.payload.msg, is it true?

Thank you
Franco

The JSON output of this will be

{
  "msg": "Watcher Notification..."
}

Hi @spinscale I have just simulate this webhook and I have a message of error

Preformatted text[parse_exception] could not parse http request template. unexpected object field [transform]

Why?

Thank you
Franco

please provide the full watch and the output of the execute watch API. Thanks.

Hi

the code of watcher is

{
  "trigger": {
    "schedule": {
      "interval": "5m"
    }
  },
  "input": {
    "search": {
      "request": {
        "search_type": "query_then_fetch",
        "indices": [
          "winlogbeat-*"
        ],
        "types": [],
        "body": {
          "query": {
            "bool": {
              "filter": [
                {
                  "range": {
                    "@timestamp": {
                      "from": "now-5m",
                      "to": "now"
                    }
                  }
                },
                {
                  "match": {
                    "winlog.event_id": "4625"
                  }
                }
              ]
            }
          },
          "aggs": {
            "users": {
              "terms": {
                "field": "winlog.event_data.TargetUserName.keyword",
                "size": 10
              }
            }
          }
        }
      }
    }
  },
  "condition": {
    "compare": {
      "ctx.payload.hits.total": {
        "gte": 10
      }
    }
  },
  "actions": {
    "my-logging-action": {
      "logging": {
        "level": "info",
        "text": "Watcher Notification Encountered {{ctx.payload.hits.total}} failed logon from user {{#ctx.payload.hits.hits.0}}{{_source.winlog.event_data.TargetUserName}}{{/ctx.payload.hits.hits.0}}."
      }
    },
    "hook-tornado": {
      "webhook": {
        "scheme": "http",
        "host": "localhost",
        "port": 8080,
        "method": "post",
        "path": "/event/webhook-elastic",
        "params": {
          "token": "123"
        },
        "headers": {},
        "body": "{{#toJson}}ctx.payload{{/toJson}}"
      }
    },
    "hook-tornado2": {
      "webhook": {
        "transform" : {
            "script" : "return [ 'msg' : 'Watcher Notification Encountered {{ctx.payload.hits.total}} failed logon from user {{#ctx.payload.hits.hits.0}}{{_source.winlog.event_data.TargetUserName}}{{/ctx.payload.hits.hits.0}}.']" 
        },               
        "scheme": "http",
        "host": "localhost",
        "port": 8080,
        "method": "post",
        "path": "/event/failed-passwords",
        "params": {
          "token": "abc"
        },
        "headers": {},
        "body": "{{#toJson}}ctx.payload{{/toJson}}"
      }
    }
  }
}

I simulate before to save the watcher. I have this error

Thank you
Franco

the transform needs to be placed outside of the webhook but inside of the hook-tornado field as a sibling to webhook

So I changed the watcher.

Now I have

{
  "trigger": {
    "schedule": {
      "interval": "5m"
    }
  },
  "input": {
    "search": {
      "request": {
        "search_type": "query_then_fetch",
        "indices": [
          "winlogbeat-*"
        ],
        "types": [],
        "body": {
          "query": {
            "bool": {
              "filter": [
                {
                  "range": {
                    "@timestamp": {
                      "from": "now-5m",
                      "to": "now"
                    }
                  }
                },
                {
                  "match": {
                    "winlog.event_id": "4625"
                  }
                }
              ]
            }
          },
          "aggs": {
            "users": {
              "terms": {
                "field": "winlog.event_data.TargetUserName.keyword",
                "size": 10
              }
            }
          }
        }
      }
    }
  },
  "condition": {
    "compare": {
      "ctx.payload.hits.total": {
        "gte": 10
      }
    }
  },
  "transform" : {
    "script" : "return [ 'msg' : 'Watcher Notification Encountered {{ctx.payload.hits.total}} failed logon from user {{#ctx.payload.hits.hits.0}}{{_source.winlog.event_data.TargetUserName}}{{/ctx.payload.hits.hits.0}}.']" 
  },   
  "actions": {
    "my-logging-action": {
      "logging": {
        "level": "info",
        "text": "Watcher Notification Encountered {{ctx.payload.hits.total}} failed logon from user {{#ctx.payload.hits.hits.0}}{{_source.winlog.event_data.TargetUserName}}{{/ctx.payload.hits.hits.0}}."
      }
    },
    "hook-tornado": {
      "webhook": {
        "scheme": "http",
        "host": "localhost",
        "port": 8080,
        "method": "post",
        "path": "/event/webhook-elastic",
        "params": {
          "token": "123"
        },
        "headers": {},
        "body": "{{#toJson}}ctx.payload{{/toJson}}"
      }
    },
    "hook-tornado2": {
      "webhook": {
        "scheme": "http",
        "host": "localhost",
        "port": 8080,
        "method": "post",
        "path": "/event/failed-passwords",
        "params": {
          "token": "abc"
        },
        "headers": {},
        "body": "{{#toJson}}ctx.payload{{/toJson}}"
      }
    }
  }
}

The simulation is ok.

At the moment I don't see the log message of the action because I think I change the ctx, is it correct?

I found in the pyalod only the message that I use in transform instead of ctx.payload

Could I pass all context and adding to it the msg?

Thank you
Franco

the transform you specified is done after the condition - which means that the logging action will not log anything. You can have a transform in your action as well, I think that should be the way to go here.

In order to include the original payload you could do something like this

def payload = ctx.payload;
payload.msg = 'foo'
return payload;

I tried with your suggestion but in the simulated I don't see the correct result in the msg

I attach my webhook configuration

"hook-tornado2": {
  "transform" : { 
  	"script" : "def payload = ctx.payload; payload.msg = 'Watcher Notification Encountered {{#payload.hits.total}} failed logon from user {{#payload.hits.hits.0}}{{_source.winlog.event_data.TargetUserName}}{{/.payload.hits.hits.0}}.'; return payload;"
  },        
  "webhook": {
    "scheme": "http",
    "host": "localhost",
    "port": 8080,
    "method": "post",
    "path": "/event/failed-passwords",
    "params": {
      "token": "abc"
    },
    "headers": {},
    "body": "{{#toJson}}ctx.payload{{/toJson}}"

And in the simulated I had

"msg": "Watcher Notification Encountered {{#payload.hits.total}} failed logon from user {{#payload.hits.hits.0}}{{_source.winlog.event_data.TargetUserName}}{{/.payload.hits.hits.0}}.",

How could I recall the value in payload?

Thank you
Franco

you need to use {{ctx.payload...

Hi @spinscale I use the ctx.payload but I have the same

"msg": "Watcher Notification Encountered {{#ctx.payload.hits.total}} failed logon from user {{#ctx.payload.hits.hits.0}}{{_source.winlog.event_data.TargetUserName}}{{/ctx.payload.hits.hits.0}}.",

Is it possible that in this msg the script doesn't see other parameter and/or variable?

Thyank you
Franco

You are only referring to the first element of the hits, instead of looping through all of them. You either need to use {{#ctx.payload.hits.hits}} or if you just want to access the first value you can go with {{ctx.payload.hits.hits.0._source.winlog.event_data.TargetUserName}}

Hello @spinscale

I changed webhook

"hook-tornado2": {
  "transform" : { 
  	"script" : "def payload = ctx.payload; payload.msg = 'Watcher Notification Encountered {{payload.hits.totals}} failed logon from user {{payload.hits.hits.0._source.winlog.event_data.TargetUserName}}.'; return payload;"
  },           
  "webhook": {
    "scheme": "http",
    "host": "localhost",
    "port": 8080,
    "method": "post",
    "path": "/event/failed-passwords",
    "params": {
      "token": "abc"
    },
    "headers": {},
    "body": "{{#toJson}}payload{{/toJson}}"
  }
} 

But the result is similar to the other results

"msg": "Watcher Notification Encountered {{payload.hits.totals}} failed logon from user {{payload.hits.hits.0._source.winlog.event_data.TargetUserName}}.",

please always share the whole watch plus the complete output of the execute watch API again in order to debug better. Note, the body needs to specify ctx.payload

The complete watch is the following

{
  "trigger": {
    "schedule": {
      "interval": "5m"
    }
  },
  "input": {
    "search": {
      "request": {
        "search_type": "query_then_fetch",
        "indices": [
          "winlogbeat-*"
        ],
        "types": [],
        "body": {
          "query": {
            "bool": {
              "filter": [
                {
                  "range": {
                    "@timestamp": {
                      "from": "now-5m",
                      "to": "now"
                    }
                  }
                },
                {
                  "match": {
                    "winlog.event_id": "4625"
                  }
                }
              ]
            }
          },
          "aggs": {
            "users": {
              "terms": {
                "field": "winlog.event_data.TargetUserName.keyword",
                "size": 10
              }
            }
          }
        }
      }
    }
  },
  "condition": {
    "compare": {
      "ctx.payload.hits.total": {
        "gte": 10
      }
    }
  },
  "actions": {
    "my-logging-action": {
      "logging": {
        "level": "info",
        "text": "Watcher Notification Encountered {{ctx.payload.hits.total}} failed logon from user {{#ctx.payload.hits.hits.0}}{{_source.winlog.event_data.TargetUserName}}{{/ctx.payload.hits.hits.0}}."
      }
    },
    "hook-tornado": {
      "webhook": {
        "scheme": "http",
        "host": "localhost",
        "port": 8080,
        "method": "post",
        "path": "/event/webhook-elastic",
        "params": {
          "token": "123"
        },
        "headers": {},
        "body": "{{#toJson}}ctx.payload{{/toJson}}"
      }
    },
    "hook-tornado2": {
      "transform" : { 
      	"script" : "def payload = ctx.payload; payload.msg = 'Watcher Notification Encountered {{payload.hits.totals}} failed logon from user {{payload.hits.hits.0._source.winlog.event_data.TargetUserName}}.'; return payload;"
      },           
      "webhook": {
        "scheme": "http",
        "host": "localhost",
        "port": 8080,
        "method": "post",
        "path": "/event/failed-passwords",
        "params": {
          "token": "abc"
        },
        "headers": {},
        "body": "{{#toJson}}payload{{/toJson}}"
      }
    }
  }
}

The simulate watch is the following

image

and I insert the output to this json blob
https://jsonblob.com/d3b15a81-8dea-11e9-b697-7bc43a3afcc5.

Thank you
Franco

Hi @spinscale do you see my last response?

Do you have an idea how could I solve this problem.

Thank you
Franco