Problems getting pipeline to run in filebeat module

hi folks
I'm having some trouble with GROK within a custom filebeat module
Runing filebeat 6.5 on windows

The mudule is ingesting & sending the logs to elasticserach without any issues
so i dont know if the module is calling the ingest pipeline correctly, debug on filebeat didnt show any obvious problems.

I've listed the contenst of my manifest.yml & the pipeline.json files.
i'm setting a new field in pipeline.json ifthe processor completes or fails but when i look at the record in elasticsearch the grok_status feild is missing & no groking has taken place

module_version: 1.0

  - name: paths
  - //*.xml

ingest_pipeline: ingest/pipeline.json
input: config/log.yml


    "description": "Pipeline for parsing Kana application logs",
    "processors": [
            "grok": {
                "field": "message",
                "patterns": [
                    "%{INT:kananode} %{DATA:parition} %{INT:kanaint}"
				"ignore_missing": true
			"set": {
                "field": "grok_status",
                "value": "GROK PARSE OK"
    "on_failure": [
            "set": {
                "field": "grok_status",
                "value": "GROK PARSE ERROR"

@John_Swift Can you check if your pipeline is installed on Elasticsearch?

& just like that it all clicks into place.....
Thanks Pier, thats exsaclty what i was missing !!!

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.