Repeated message

Elastic Stack Beats
1

Hi,
I have a simple filebeat configuration sending the output of a tomcat server to my logstash

- type: log
  paths:
    - '/home/appserver/logs/catalina.out'
  tags: [swtp]

  multiline.pattern: '^[[:space:]]+(at|\.{3})\b|^Caused by:'
  multiline.negate: false
  multiline.match: after

My logstash filter is also rather simple

filter {
  if "swtp" in [tags] {
    grok {
      pattern_definitions => { "SWTPGENERIC" => "%{YEAR}-%{MONTHNUM}-%{MONTHDAY} %{HOUR}:?%{MINUTE}(?::?%{SECOND}) %{LOGLEVEL} \[%{WORD:java.method}\.%{WORD:java.class}] - <%{GREEDYDATA:message}>" }
      pattern_definitions => { "SWTPMAIL" => "%{YEAR}-%{MONTHNUM}-%{MONTHDAY} %{HOUR}:?%{MINUTE}(?::?%{SECOND}) %{LOGLEVEL} \[%{WORD:java.method}\.%{WORD:java.class}] - <sendMail %{WORD:transmit_status} %{GREEDYDATA:message}>" }
      
      pattern_definitions => { "EMAILADDRESSPART" => "[a-zA-Z0-9_.+-=:]+" }
      match => { "message" => "%{SWTPMAIL}|%{SWTPGENERIC}" }
    }
  }
}

The problem is that the applications outputs to following line
2018-10-24 09:31:48,685 INFO [customer.mails] - <sendMail OK to: XXX.XXX@gmail.com subject: Bestätigung Ihres Eintrags >

But in ES I see the entry as

2018-10-24 09:31:48,685 INFO [customer.mails] - <sendMail OK to: XXX.XXX@gmail.com subject: Bestätigung Ihres Eintrags>, sendMail OK to: XXX:XXX@gmail.com subject: Bestätigung Ihres Eintrags

As logstash can not parse the message I suspect that filebeat is sending the log wrongly.
The question is, why a second "sendMail OK to: XXX:XXX@gmail.com subject: Bestätigung Ihres Eintrags" is appended to the message.

Any clues?

2

I think the multiline pattern is wrong in your case, with your example I am not sure how they are applied. Did you try to match on a date format instead as the delimiter of new events?

3

Multiline pattern was wrong, indeed :frowning:
Many thanks for rubbing my nose into it :slight_smile: :slight_smile:

1 Like
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.