Imap input gives weird error

Elastic Stack Logstash
1

Hello,

I want to run imap plugin to load some reporting mails into Elasticsearch.

I have it configured as the following:

input {
  imap {
    check_interval => "10"
    host => "host"
    port => "995"
    user => "email@domain.com"
    password => "password"
    save_attachments => "true"
    secure => "true"
  }
}

and I see the following in the logs:

Dec 01 08:23:06 fedora logstash[19425]: [2021-12-01T08:23:06,648][ERROR][logstash.inputs.imap     ][dmarc][e6720cf5bc6d63a76eeed29de1599d8053892e2638453f2549e5954462755d6b] Encountered error Net::IMAP::ResponseParseError {:message=>"unexpected token ATOM (expected CRLF)", :backtrace=>["uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/imap.rb:3493:in `parse_error'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/imap.rb:3345:in `match'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/imap.rb:2261:in `response'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/imap.rb:2179:in `parse'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/imap.rb:1242:in `get_response'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/imap.rb:1109:in `initialize'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-imap-3.1.0/lib/logstash/inputs/imap.rb:84:in `connect'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-imap-3.1.0/lib/logstash/inputs/imap.rb:99:in `check_mail'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-imap-3.1.0/lib/logstash/inputs/imap.rb:92:in `block in run'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/stud-0.0.23/lib/stud/interval.rb:20:in `interval'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-imap-3.1.0/lib/logstash/inputs/imap.rb:91:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:405:in `inputworker'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:396:in `block in start_input'"]}

Any idea how to make this work?

2

Nevermind I was using the wrong port. once changed to 993 it started to work just fine.

3

I understand this is not the problem here, but in case someone stumbles upon this issue in the future, all of the IMAP functionality is in the stdlib/net-imap library of ruby. That has meta-issues about meta-issues used to collect details of parts of the parser that need to be updated to support new protocol features.

4

Any suggestions how to update the stdlib/net-imap library?

5

@Badger any suggestion?

6

From certain clients when I recieve a mail I can see the following error message in logstash logs:

Dec 08 11:09:23 fedora logstash[123168]: [2021-12-08T11:09:23,299][ERROR][logstash.inputs.imap ][dmarc][d80c5d9cecefc5e8838feb1e77df01942ce3fd89a2675c7323a70f4ddf1fc06a] Encountered error NoMethodError {:message=>"Can not decode an entire message, try calling #decoded on the various fields and body or parts if it is a multipart message.", :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/mail-2.6.6/lib/mail/message.rb:1903:in `decoded'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-imap-3.2.0/lib/logstash/inputs/imap.rb:220:in `parse_mail'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-imap-3.2.0/lib/logstash/inputs/imap.rb:163:in `block in check_mail'", "org/jruby/RubyArray.java:1820:in `each'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-imap-3.2.0/lib/logstash/inputs/imap.rb:157:in `block in check_mail'", "org/jruby/RubyArray.java:1862:in `each_slice'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-imap-3.2.0/lib/logstash/inputs/imap.rb:155:in `check_mail'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-imap-3.2.0/lib/logstash/inputs/imap.rb:136:in `block in run'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/stud-0.0.23/lib/stud/interval.rb:20:in `interval'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-imap-3.2.0/lib/logstash/inputs/imap.rb:135:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:405:in `inputworker'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:396:in `block in start_input'"]}

Any idea how to fix it?

7

No, I have never used the input and am unfamiliar with the code and protocol.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.