Filebeat Multiline Tutorial

Elastic Stack Beats
1

I'm trying out the filebeat tutorial ( https://www.elastic.co/guide/en/beats/filebeat/current/configuration-filebeat-options.html ) again after i was stuck with my own pattern.
Filebeat still sends the logs as different log events.
Does anyone know how to fix this?

The example log:

[beat-logstash-some-name-832-2015.11.28] IndexNotFoundException[no such index]
    at org.elasticsearch.cluster.metadata.IndexNameExpressionResolver$WildcardExpressionResolver.resolve(IndexNameExpressionResolver.java:566)
    at org.elasticsearch.cluster.metadata.IndexNameExpressionResolver.concreteIndices(IndexNameExpressionResolver.java:133)
    at org.elasticsearch.cluster.metadata.IndexNameExpressionResolver.concreteIndices(IndexNameExpressionResolver.java:77)
    at org.elasticsearch.action.admin.indices.delete.TransportDeleteIndexAction.checkBlock(TransportDeleteIndexAction.java:75)

Here is my filebeat config:

filebeat:
  prospectors:
    -
      paths:
        - /var/log/filebeatmultiline

      input_type: log

      document_type: appConsole

  registry_file: /var/lib/filebeat/registry555555

multiline:
    pattern: ^\[
    negate: true
    match: after

output:
  logstash:
    hosts: ["IP:5044"]
    bulk_max_size: 1024

    tls:
      certificate_authorities: ["/etc/pki/tls/certs/logstash-forwarder.crt"]

shipper:

logging:
  files:
    rotateeverybytes: 10485760 # = 10MB
2

Solved it was a formatting mismatch.

The correct config is:

 filebeat:
  prospectors:
    -
      paths:
        - /var/log/filebeatmultiline

      input_type: log

      document_type: appConsole
      multiline:
         pattern: ^\[
         negate: true
         match: after
  registry_file: /var/lib/filebeat/registry5555679

output:
  logstash:
    hosts: ["IP:5044"]
    bulk_max_size: 1024

    tls:
      certificate_authorities: ["/etc/pki/tls/certs/logstash-forwarder.crt"]

shipper:

logging:
  files:
    rotateeverybytes: 10485760 # = 10MB
1 Like