Nmap result xml file send to elasticsearch using logstash - read_to_eof Error

Elastic Stack Logstash
1

I want to send to nmap xml result to elasticsearch using logstash.

So i wrote nmap-logstash conf file like below

input {
  file {
    path => "/home/elktest/elk/nmaptoelk/*.xml"
    start_position => "beginning"
    sincedb_path => "/dev/null"
    codec => nmap
    tags => [nmap]
  }
}
filter {
  if "nmap" in [tags] {
    # Don't emit documents for 'down' hosts
    if [status][state] == "down" {
      drop {}
    }
   mutate {
      # Drop HTTP headers and logstash server hostname
      remove_field => ["headers", "hostname"]
    }
   if "nmap_traceroute_link" == [type] {
      geoip {
        source => "[to][address]"
        target => "[to][geoip]"
      }
     geoip {
        source => "[from][address]"
        target => "[from][geoip]"
      }
    }
   if [ipv4] {
      geoip {
        source => ipv4
        target => geoip
      }
    }
 }
}
output {
  if "nmap" in [tags] {
    elasticsearch {
      document_type => "nmap-reports"
      document_id => "%{[id]}"
      # Nmap data usually isn't too bad, so monthly rotation should be fine
      index => "nmap-logstash-%{+YYYY.MM}"
      template => "/home/elktest/elk/nmaptoelk/elasticsearch_nmap_template.json"
      template_name => "logstash_nmap"
    }
   stdout {
      codec => json_lines
    }
  }
}

when i was execute logstash, success to install index template into elasticsearch, but when read xml file logstash send to error like below

[ERROR] 2019-11-11 17:20:22.596 [[main]<file] grow - read_to_eof: general error reading /home/elktest/elk/nmaptoelk/report2.xml {"error"=>"#<NoMethodError: undefined method `[]' for nil:NilClass>", "backtrace"=>["/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/ruby-nmap-0.9.3/lib/nmap/xml.rb:101:in `scanner'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-codec-nmap-0.0.21/lib/logstash/codecs/nmap.rb:40:in `decode'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.1.11/lib/logstash/inputs/file/patch.rb:6:in `accept'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/delegate.rb:83:in `method_missing'"]}
[ERROR] 2019-11-11 17:20:22.807 [[main]<file] grow - read_to_eof: general error reading /home/elktest/elk/nmaptoelk/report1.xml {"error"=>"#<NoMethodError: undefined method `[]' for nil:NilClass>", "backtrace"=>["/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/ruby-nmap-0.9.3/lib/nmap/xml.rb:101:in `scanner'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-codec-nmap-0.0.21/lib/logstash/codecs/nmap.rb:40:in `decode'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.1.11/lib/logstash/inputs/file/patch.rb:6:in `accept'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/delegate.rb:83:in `method_missing'"]}

How to solve this problem?

My ELK version is 7.4,
Nmap version 7.80,
ruby-nmap version is 0.9.3,
logstash-codec-nmap version is 0.0.21

Thanks.

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.