Just started using the logstash-imap-input filter to pull logs from a UNIX IMAP server. Using Logstash 7.6.1 running on a Scientific Linux 6.7 server (don't ask).
The pipeline worked for 535 messages before it ran into this error that appears to be blocking further progress:
Can not decode an entire message, try calling #decoded on the various fields and body or parts if it is a multipart message.
Here's the full body of the message in /var/log/logstash/logstash-plain.log:
[2020-04-07T12:42:10,367][ERROR][logstash.inputs.imap ][root_mail] Encountered error NoMethodError {:message=>"Can not decode an entire message, try calling #decoded on the various fields and body or parts if it is a multipart message.", :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/mail-2.6.6/lib/mail/message.rb:1903:in `decoded'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-imap-3.0.7/lib/logstash/inputs/imap.rb:163:in `parse_mail'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-imap-3.0.7/lib/logstash/inputs/imap.rb:116:in `block in check_mail'", "org/jruby/RubyArray.java:1814:in `each'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-imap-3.0.7/lib/logstash/inputs/imap.rb:112:in `block in check_mail'", "org/jruby/RubyArray.java:1856:in `each_slice'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-imap-3.0.7/lib/logstash/inputs/imap.rb:110:in `check_mail'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-imap-3.0.7/lib/logstash/inputs/imap.rb:91:in `block in run'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/stud-0.0.23/lib/stud/interval.rb:20:in `interval'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-imap-3.0.7/lib/logstash/inputs/imap.rb:90:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:328:in `inputworker'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:320:in `block in start_input'"]}
Here's the input filter configuration with sanitized hostnames and account data:
input {
imap {
id => "root_mail_input"
host => "somehost"
port => 993
check_interval => 10
user => "someaccount"
password => "somepassword"
strip_attachments => true
delete => true
expunge => true
}
}
I added the delete and expunge flags in an attempt to clear the backlog that's evidently there, but it doesn't help. The data are sent to Kafka for further import into an Elasticsearch 7.6.1 cluster.
The input filter documentation is a little sparse (a lot sparse, actually) and the only other mentions of this kind of problem I've found on other sites and on Github were never answered. Any help would be appreciated.