Multiline Codec Not Retrieving Any Data

Elastic Stack Logstash
1

I'm using the Multiline codec with the File input in hopes to retrieve multiline log files from a file that is being populated on the same box I'm running Logstash (v5.1.2) & Elasticsearch (v.2.4.2).

I've isolated just the input and output config in order to minimize the variables that could be affecting the desired performance.

input {
  file {
    path => "/var/log/cluster/cluster.log"
    start_position => beginning
    codec => multiline {
      pattern => "0x7fff"
      what => "previous"
      auto_flush_interval => 3
    }
  }
}

output {
  elasticsearch {
    hosts => ["localhost:9200", "remotehost:9200"]
#    sniffing => true
    manage_template => false
    index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
    document_type => "%{[@metadata][type]}"
  }
}

And an example of the multiline data populating the cluster.log file;

active node table
-------------------
pkey  vlan  master-s/n  gateway-s/n  gateway-prio      if-name        gateway name   advertised ip
  0x7fff     0     0xa0640      0xa0640           100          if0      Switch-22-io   10.191.144.48
  0x7fff     0     0xa0640      0xa0400           100          if0      Switch-28-io   10.191.144.50
  0x7fff     0     0xa0640      0xa0180           100          if0      Switch-21-io   10.191.144.44
  0x7fff     0     0xa0640      0xa0140           100          if0       Switch-3-io   10.191.144.40
  0x7fff     0     0xa0640      0xa0100           100          if0      Switch-27-io   10.191.144.46
  0x7fff     0     0xa0640      0xa0080           100          if0      Switch-12-io   10.191.144.43
  0x7fff     0     0xa0640      0xa0040           100          if0      Switch-30-io   10.191.144.47
  0x7fff     0     0xa0640      0xa0000           100          if0       Switch-6-io   10.191.144.41
  0x7fff     0     0xa0640      0x9d340           100          if0       Switch-6-io   10.191.144.28

Any thoughts on what might be stopping this from working? A similar config was working with the multiline filter, however in an attempt to get more-consistent results, I decided to start using the multiline codec, however I've not got any results since the change. Furthermore, the logs aren't populating any useful data, thus further complicating debugging.

Any issues anyone sees with my config? How would you go about debugging this issue? Thanks for any help.

2

I was able to get this filter working by starting logstash via the command:

sudo -u logstash /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d --path.settings /etc/logstash &

I had to use this exact syntax, found here, else I'd run into a whole slew of errors when attempting to start Logstash. Now I'm getting Grok timeouts but at least logstash is running and the multiline codec works, albeit appears as a duct tape and shoe string solution.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.