Unable to figure out how to get the 'scripts' output for a port into ES with the JSON template

[TheZeke]

I'm trying to get the script output added to elasticsearch when using --script.. on an nmap scan. I think I'm doing it in the right place but it's not adding anything whatsoever (or even generating an error in logstash).

Ultimately I'd like these added to the nmap_port type with the output in it's own field.

Also, on a side note, the indexed nmap_port type does not seem to indicate what port it is anywhere. Perhaps that should be added to the JSON template.

• Version:
  logstash 5.4.3
• Operating System:
  Linux logstash1-dev 4.8.0-58-generic #63-Ubuntu SMP Mon Jun 26 17:08:21 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
• Config File (if you have sensitive info, please remove it):

My elasticsearch_nmap_template.json TEMPLATE:

I added a script section within nmap_port->properties->port->properties
https://pastebin.com/aHvz9QQa

INPUT:

# {{ ansible_managed }}
input {
  http {
    port => 2626
    codec => nmap
    tags => [nmap]
  }
}

filter {
  if "nmap" in [tags] {
    # Don't emit documents for 'down' hosts
    if [status][state] == "down" {
      drop {}
    }
    mutate {
      # Drop HTTP headers and logstash server hostname
      remove_field => [ "headers", "hostname", "tags" ]
    }
    if "nmap_traceroute_link" == [type] {
      geoip {
        source => "[to][address]"
        target => "[to][geoip]"
      }
      geoip {
        source => "[from][address]"
        target => "[from][geoip]"
      }
    }
    if [ipv4] {
      geoip {
        source => ipv4
        target => geoip
      }
    }
  }
}

OUTPUT:

# {{ ansible_managed }}
output {
  if "nmap" in [tags] {
    elasticsearch {
      hosts => [ "{{ es_nodes|join('", "') }}" ]
      document_id => "%{id}"
      document_type => "%{type}"
      template => "/etc/logstash/elasticsearch_nmap_template.json"
      template_name => "logstash_nmap"
      index => "%{type}-%{+YYYY.MM}"
    }
  } else {
    elasticsearch {
      hosts => [ "{{ es_nodes|join('", "') }}" ]
      index => "%{type}-%{+YYYY.MM}"
    }
  }
}

• Sample Data:

nmap -oX - -T5 -sU -p 123 --script "ntp*" 10.10.217.113

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE nmaprun>
<?xml-stylesheet href="file:///usr/bin/../share/nmap/nmap.xsl" type="text/xsl"?>
<!-- Nmap 7.40 scan initiated Mon Jul 24 16:25:51 2017 as: nmap -oX - -T5 -sU -p 123 -&#45;script ntp* 10.10.217.113 -->
<nmaprun scanner="nmap" args="nmap -oX - -T5 -sU -p 123 -&#45;script ntp* 10.10.217.113" start="1500927951" startstr="Mon Jul 24 16:25:51 2017" version="7.40" xmloutputversion="1.04">
<scaninfo type="udp" protocol="udp" numservices="1" services="123"/>
<verbose level="0"/>
<debugging level="0"/>
<host starttime="1500927951" endtime="1500927956"><status state="up" reason="echo-reply" reason_ttl="240"/>
<address addr="10.10.217.113" addrtype="ipv4"/>
<hostnames>
</hostnames>
<ports><port protocol="udp" portid="123"><state state="open" reason="udp-response" reason_ttl="240"/><service name="ntp" ostype="cisco" method="probed" conf="10"/><script id="ntp-info" output="&#xa;  receive time stamp: 2017-07-24T20:26:01&#xa;  system: cisco&#xa;  leap: 0&#xa;  stratum: 3&#xa;  rootdelay: 2.91&#xa;  rootdispersion: 68.73&#xa;  peer: 48067&#xa;  refid: 208.122.25.26&#xa;  reftime: 0xDD20D2B7.B82B9D0D&#xa;  poll: 10&#xa;  clock: 0xDD20D64F.C8C11879&#xa;  phase: -0.361&#xa;  freq: -46.78&#xa;  error: 28.67\x0D&#xa;"><elem key="receive time stamp">2017-07-24T20:26:01</elem>
<elem key="system">cisco</elem>
<elem key="leap">0</elem>
<elem key="stratum">3</elem>
<elem key="rootdelay">2.91</elem>
<elem key="rootdispersion">68.73</elem>
<elem key="peer">48067</elem>
<elem key="refid">208.122.25.26</elem>
<elem key="reftime">0xDD20D2B7.B82B9D0D</elem>
<elem key="poll">10</elem>
<elem key="clock">0xDD20D64F.C8C11879</elem>
<elem key="phase">-0.361</elem>
<elem key="freq">-46.78</elem>
<elem key="error">28.67\x0D&#xa;</elem>
</script></port>
</ports>
<times srtt="10195" rttvar="7839" to="50000"/>
</host>
<runstats><finished time="1500927956" timestr="Mon Jul 24 16:25:56 2017" elapsed="5.58" summary="Nmap done at Mon Jul 24 16:25:56 2017; 1 IP address (1 host up) scanned in 5.58 seconds" exit="success"/><hosts up="1" down="0" total="1"/>
</runstats>
</nmaprun>