500 error when CURLing to Logstash with nmap codec enabled

Elastic Stack Logstash
1

I have the problem described in this linked (and closed) issue:

I have changed the CURL statement to --data-binary, but the error remains.

command:
curl -H "Content-Type:text/xml;charset=UTF-8" 'http://192.168.6.6:8008' --data-binary @/var/www/html/daily-port-scans/diff-2017-11-29.xml -vThis text will be hidden

output:

* Rebuilt URL to: http://192.168.6.6:8008/
* Trying 192.168.6.6...
* Connected to 192.168.6.6 (192.168.6.6) port 8008 (#0)
> POST / HTTP/1.1
> Host: 192.168.6.6:8008
> User-Agent: curl/7.50.1
> Accept: */*
> Content-Type:text/xml;charset=UTF-8
> Content-Length: 37078
> Expect: 100-continue
> 
* Done waiting for 100-continue
* We are completely uploaded and fine
< HTTP/1.1 500 Internal Server Error
< Content-Type: text/plain
< Content-Length: 14
< 
* Connection #0 to host 192.168.6.6 left intact
internal error

I am sending a diff file created with the script:

#!/bin/sh
TARGETS="/root/nmap-scans/lists/daily-target-list"
OPTIONS="-T4 -A -iL"
date=`date +%F`
cd /root/nmap-scans/daily
nmap $OPTIONS $TARGETS -oA scan-$date > /dev/null
if [ -e scan-baseline.xml ]; then
    ndiff scan-baseline.xml scan-$date.xml --xml > diffs/diff-$date.xml
fi
cd /root/nmap-scans/daily/diffs
awk 'NR==2{print "<?xml-stylesheet href=\"/xsl/ndiff_d.xsl\" type=\"text/xsl\"?>"} 1' diff-$date.xml > tmp.xml && mv tmp.xml diff-$date.xml 
cp diff-$date.xml /var/www/html/daily-port-scans/diff-$date.xml

Is it simply that the codec cannot handle ndiffs? A snippet of the XML is:

<?xml version="1.0" encoding="utf-8"?>
<?xml-stylesheet href="/xsl/ndiff_d.xsl" type="text/xsl"?>
<nmapdiff version="1"><scandiff><a><nmaprun args="nmap -sV -T2 -F -iL /root/nmap-scans/lists/daily-target-list -oA /root/nmap-scans/daily/scan-baseline" scanner="nmap" start="1464874947" startstr="Thu Jun 02 14:42:27 2016" version="7.12"/>
</a><b><nmaprun args="nmap -T4 -A -iL /root/nmap-scans/lists/daily-target-list -oA scan-2017-11-30" scanner="nmap" start="1512046801" startstr="Thu Nov 30 13:00:01 2017" version="6.47"/>
</b><hostdiff>
<host>
<address addr="xxx.xxx.xxx.xxx" addrtype="ipv4"/>
<hostnames>
<hostname name="xxx.xxx.com"/>
</hostnames>
<ports>
<a>
<extraports count="100" state="filtered"/>
</a>
<b>
<extraports count="994" state="filtered"/>
</b>
<portdiff>
<a>
<port portid="25" protocol="tcp"/>
</a>
<b>
<port portid="25" protocol="tcp">
<state state="open"/>
<service name="smtp" product="Alt-N xxx mail server" version="xxx.xxx.1"/>
<script id="smtp-commands" output="mail.xxx.com Hello mail.xxx.com [xxx.xxx.xxx.xxx], pleased to meet you, ETRN, AUTH LOGIN CRAM-MD5 PLAIN, 8BITMIME, ENHANCEDSTATUSCODES, STARTTLS, SIZE 10020000, "/>
<script id="ssl-cert" output="Subject: commonName=*.xxx.com/organizationName=xxx xxx Ltd/stateOrProvinceName=xxx/countryName=GB
Not valid before: 2015-xxx-xxx:00:00+00:00
Not valid after:  2018-xxx-xxx:59:59+00:00"/>
<script id="ssl-date" output="2017-xxx-xxx:14:38+00:00; +6m55s from local time."/>
</port>
</b>
</portdiff>
<portdiff>
<a>
<port portid="80" protocol="tcp"/>
</a>
<b>
<port portid="80" protocol="tcp">
<state state="open"/>
<service name="http" product="xxx/4.0"/>
<script id="http-methods" output="No Allow or Public header in OPTIONS response (status code 302)"/>
<script id="http-title" output="Did not follow redirect to https://mail.xxx.com/"/>
</port>
</b>
</portdiff>
<portdiff>
<a>
<port portid="443" protocol="tcp"/>
</a>
2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.