Filebeat pipeline not being used 500 internal server error

Elastic Stack Beats
1

ECE 2.3
v7.3.0
Filebeat 7.3

Were having trouble getting our Filebeat pipeline to work. We are currently able to establish a connection from Filebeat to our Elastic Cloud Enterprise cluster and send data when the pipeline line is commented out in the Filebeat yaml. Once the pipeline line is uncommented in the yaml we get a 500 internal server error, some other errors (below) but I'm also able to see the pipeline we are trying to use in the DevTools (below). An insight on what else to look at would be appreciated.

Caused by: java.lang.IllegalStateException: There are no ingest nodes in this cluster, unable to forward request to an ingest node.

[2019-10-25T12:33:01,290][WARN ][org.elasticsearch.xpack.monitoring.MonitoringService] [instance-0000000000] monitoring execution failed org.elasticsearch.xpack.monitoring.exporter.ExportException: failed to flush export bulks

GET _cat/pipelines in DevTools returns the following which would be correct.

{
  "acecsvs" : {
    "description" : "Convert ACE logs CSV data to indexed data",
    "version" : 1,
    "processors" : [
      {
        "grok" : {
          "field" : "message",
          "patterns" : [
            """%{DATA:xxx.applicationName},%{NUMBER:xxx.applicationVersion},%{DATA:xxx.hostName},%{TIMESTAMP_ISO8601:xxx.dateTime},%{DATA:xxx.userName},%{DATA:xxx.messageLevel},%{DATA:xxx.messageID},%{DATA:xxx.messageBody},%{DATA:xxx.messageSource},%{URI:xxx.messageURI},%{DATA:xxx.stackTrace},"%{DATA:xxx.request}",%{DATA:xxx.additionalMsg01},%{DATA:xxx.additionalMsg02},%{DATA:xxx.additionalMsg03},%{DATA:xxx.additionalMsg04}"""
          ]
        },
        "remove" : {
          "field" : "message"
        }
      }
    ],
    "on_failure" : [
      {
        "set" : {
          "field" : "error",
          "value" : " - Error processing message - "
        }
      }
    ]
  }
}
2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.