Parse XML nessus config to logstash

Elastic Stack Logstash
1

I have a nessus file that looks like this :

<ReportHost name="192.168.2.65"><HostProperties>
<tag name="traceroute-hop-1">192.168.2.65</tag>
<tag name="HOST_START">Wed Nov 10 12:48:53 2021</tag>
<tag name="HOST_START_TIMESTAMP">1636548533</tag>
<tag name="host-ip">192.168.2.65</tag>
<tag name="HOST_END">Wed Nov 10 12:50:22 2021</tag>
<tag name="HOST_END_TIMESTAMP">1636548622</tag>
<tag name="host-rdns">chuangmi.camera.ipc019.Home</tag>
<tag name="host-fqdn">chuangmi.camera.ipc019.Home</tag>
<tag name="traceroute-hop-0">?</tag>
<tag name="mac-address">5C:E5:0C:6D:82:73</tag>
<tag name="host-fqdns">[{&quot;FQDN&quot;:&quot;chuangmi.camera.ipc019.Home&quot;,&quot;sources&quot;:[&quot;get_host_fqdn()&quot;,&quot;determine_fqdn()&quot;]}]</tag>
<tag name="operating-system-method">Unknown</tag>
<tag name="operating-system-conf">-1</tag>
<tag name="os">other</tag>
<tag name="patch-summary-total-cves">1</tag>
<tag name="policy-used">Basic Network Scan</tag>
<tag name="Credentialed_Scan">false</tag>
<tag name="LastUnauthenticatedResults">1636548622</tag>
</HostProperties>
<ReportItem port="0" svc_name="general" protocol="tcp" severity="0" pluginID="12053" pluginName="Host Fully Qualified Domain Name (FQDN) Resolution" pluginFamily="General">
<description>Nessus was able to resolve the fully qualified domain name (FQDN) of the remote host.</description>
<fname>fqdn.nasl</fname>
<plugin_modification_date>2017/04/14</plugin_modification_date>
<plugin_name>Host Fully Qualified Domain Name

and i really need to parse this into logstash. i tried many options from other topics.

this is my config file :

input {
  file {
    path => "/home/rsc/scansNessus/*"
    start_position => "beginning"
    sincedb_path => "/dev/null"
    type => xml
    codec => multiline {
	pattern => "<?xml" 
        negate => "true"
        what => "previous"
    }
  }
}

filter {
    xml{
	source => "message"
	store_xml => false
	#target => "ReportItem"
      	xpath => [ "//NessusClientData_v2/Report/ReportHost/ReportItem/@pluginName","description" ]
    }
}

Anyone can help me? i've been struggling with this for 2 weeks now

1 Like
2

If that is true, which seems unlikely, then using an xpath that starts with //NessusClientData_v2 is not going to return anything. Please provide a reproducible example.

3

The nessus file has this format:

<NessusClientData_v2>
<Policy><policyName>My Policy</policyName>
[..]
</Policy>
<Report name="My Scan">
[..]
</Report>
</NessusClientData_v2>

You need to share the full file so people can try to reproduce your pipeline, you are sharing only part of the file.

4

Im sorry, you are right. here it is.
nessusfile

this is what's inside "ReportHost"

insidereporthost
insidereporthost1790×667 35.5 KB

The "Policy" part is huge thats why i didnt copy it to here.

5

So reduce it until you have something you can post. For example, with

<?xml version="1.0" ?>
<NessusClientData_v2>
<Policy><policyName>Vulnerability Scan Policy</policyName>
<Preferences><ServerPreferences>
<preference><name>max_simult_tcp_sessions</name> <value>unlimited</value> </preference>
<preference><name>scan.enable_utf8_output</name> <value>no</value> </preference>
<preference><name>bw_prevent_core_updates</name> <value>yes</value> </preference>
</ServerPreferences> </Preferences>
</Policy>
<Report name="Servers" xmlns:cm="http://www.nessus.org/cm">
<ReportHost name="155.19.37.21"><HostProperties>
<tag name="HOST_END_TIMESTAMP">1632974554</tag>
<tag name="HOST_END">Wed Sep 29 22:02:34 2021</tag>
<tag name="patch-summary-total-cves">0</tag>
<tag name="HOST_START_TIMESTAMP">1632974404</tag>
<tag name="HOST_START">Wed Sep 29 22:00:04 2021</tag>
</HostProperties>
<ReportItem port="623" svc_name="asf-rmcp" protocol="udp" severity="0" pluginID="72063" pluginName="IPMI Versions Supported" pluginFamily="General">
<description>This script detects which IPMI versions are supported by the remote service for managing the system, as well as additional settings.</description>
</ReportItem>
<ReportItem port="623" svc_name="asf-rmcp" protocol="udp" severity="0" pluginID="68932" pluginName="IPMI Cipher Suites Supported" pluginFamily="General">
<description>This script detects which IPMI cipher suites are supported by the remote service for the authentication, integrity, and confidentiality of communications.</description>
</ReportItem>
<ReportItem port="623" svc_name="asf-rmcp" protocol="udp" severity="0" pluginID="45555" pluginName="Alert Standard Format / Remote Management and Control Protocol Detection" pluginFamily="Service detection">
<asset_inventory>True</asset_inventory>
</ReportItem>
</ReportHost>
</Report>
</NessusClientData_v2>

and

xml { source => "message" store_xml => false xpath => { "/NessusClientData_v2/Report/ReportHost/ReportItem/@pluginName" => "description" } }

The sample XML that I started with was missing </ServerPreferences>, </Preferences>, and </ReportHost> tags so the xml filter did not do anything. Once the XML was valid I got

"description" => [
    [0] "IPMI Versions Supported",
    [1] "IPMI Cipher Suites Supported",
    [2] "Alert Standard Format / Remote Management and Control Protocol Detection"
]

If I remove the closing </Preferences> tag then I get no description field and no error message. If you set store_xml to be true then you will get error messages for invalid XML.

6

This is the scan reduced

<?xml version="1.0" ?>
	<NessusClientData_v2>
	<Policy><policyName>Scan Policy</policyName>
	<Preferences><ServerPreferences><preference><name>bw_prevent_core_updates</name>
	<value>yes</value>
	</preference>
	<preference><name>scan.enable_utf8_output</name>
	<value>no</value>
	</preference>
	</ServerPreferences>
	<PluginsPreferences><item><pluginName>ADSI Settings</pluginName>
	<pluginId>60024</pluginId>
	<fullName>ADSI Settings[entry]:Domain Controller :</fullName>
	<preferenceName>Domain Controller :</preferenceName>
	<preferenceType>entry</preferenceType>
	<preferenceValues></preferenceValues>
	<selectedValue></selectedValue>
	</item>
	</PluginsPreferences>
	</Preferences>
	<FamilySelection><FamilyItem><FamilyName>MacOS X Local Security Checks</FamilyName>
	<Status>enabled</Status>
	</FamilyItem>
	</FamilySelection>
	<IndividualPluginSelection><PluginItem><PluginId>34220</PluginId>
	<PluginName>Netstat Portscanner (WMI)</PluginName>
	<Family>Netstat Portscanner (WMI)</Family>
	<Status>enabled</Status>
	</PluginItem>
	</IndividualPluginSelection>
	</Policy>
	<Report name="teste 10LLLL" xmlns:cm="http://www.nessus.org/cm">
	<ReportHost name="192.168.2.64"><HostProperties>
	<tag name="HOST_START">Wed Nov 10 12:48:50 2021</tag>
	<tag name="HOST_START_TIMESTAMP">1636548530</tag>
	<tag name="host-ip">192.168.2.64</tag>
	<tag name="HOST_END">Wed Nov 10 12:53:54 2021</tag>
	<tag name="HOST_END_TIMESTAMP">1636548834</tag>
	<tag name="host-rdns">shelly1-40F5201DBCE0.Home</tag>
	<tag name="host-fqdn">shelly1-40F5201DBCE0.Home</tag>
	<tag name="sinfp-signature">
	   P1:B11113:F0x12:W2920:O0204ffff:M1460:
	   P2:B11113:F0x12:W2920:O0204ffff:M1460:
	   P3:B00000:F0x00:W0:O0:M0
	   P4:190000_7_p=80</tag>
	<tag name="sinfp-ml-prediction">[{&quot;predicted-os&quot;: &quot;Windows Server 2016 Standard&quot;, &quot;confidence&quot;: 1},{&quot;predicted-os&quot;: &quot;Microsoft Windows Server 2016 Standard&quot;, &quot;confidence&quot;: 1},{&quot;predicted-os&quot;: &quot;Microsoft Windows Server 2008 R2&quot;, &quot;confidence&quot;: 1},{&quot;predicted-os&quot;: &quot;Cisco NX-OS&quot;, &quot;confidence&quot;: 4},{&quot;predicted-os&quot;: &quot;Microsoft Windows Server 2008 R2 Enterprise.Service Pack 1&quot;, &quot;confidence&quot;: 1}]</tag>
	<tag name="operating-system">EthernetBoard OkiLAN 8100e</tag>
	<tag name="system-type">switch</tag>
	<tag name="traceroute-hop-0">192.168.2.64</tag>
	<tag name="mac-address">40:F5:20:1D:BC:E0</tag>
	<tag name="host-fqdns">[{&quot;FQDN&quot;:&quot;shelly1-40F5201DBCE0.Home&quot;,&quot;sources&quot;:[&quot;get_host_fqdn()&quot;,&quot;determine_fqdn()&quot;]}]</tag>
	<tag name="operating-system-method">SinFP</tag>
	<tag name="operating-system-conf">65</tag>
	<tag name="os">other</tag>
	<tag name="patch-summary-total-cves">0</tag>
	<tag name="policy-used">Basic Network Scan</tag>
	<tag name="Credentialed_Scan">false</tag>
	<tag name="LastUnauthenticatedResults">1636548834</tag>
	</HostProperties>
	<ReportItem port="80" svc_name="www" protocol="tcp" severity="0" pluginID="11219" pluginName="Nessus SYN scanner" pluginFamily="Port scanners">
	<description>This plugin is a SYN &apos;half-open&apos; port scanner.  It shall be reasonably quick even against a firewalled target. 

	Note that SYN scans are less intrusive than TCP (full connect) scans against broken services, but they might cause problems for less robust firewalls and also leave unclosed connections on the remote target, if the network is loaded.</description>
	<fname>nessus_syn_scanner.nbin</fname>
	<plugin_modification_date>2021/09/16</plugin_modification_date>
	<plugin_name>Nessus SYN scanner</plugin_name>
	<plugin_publication_date>2009/02/04</plugin_publication_date>
	<plugin_type>remote</plugin_type>
	<risk_factor>None</risk_factor>
	<script_version>1.40</script_version>
	<solution>Protect your target with an IP filter.</solution>
	<synopsis>It is possible to determine which TCP ports are open.</synopsis>
	<plugin_output>Port 80/tcp was found to be open</plugin_output>
	</ReportItem>
	<ReportItem port="0" svc_name="general" protocol="tcp" severity="0" pluginID="19506" pluginName="Nessus Scan Information" pluginFamily="Settings">
	<agent>all</agent>
	<always_run>1</always_run>
	<description>This plugin displays, for each tested host, information about the scan itself :

	  - The version of the plugin set.
	  - The type of scanner (Nessus or Nessus Home).
	  - The version of the Nessus Engine.
	  - The port scanner(s) used.
	  - The port range scanned.
	  - The ping round trip time 
	  - Whether credentialed or third-party patch management     checks are possible.
	  - Whether the display of superseded patches is enabled
	  - The date of the scan.
	  - The duration of the scan.
	  - The number of hosts scanned in parallel.
	  - The number of checks done in parallel.</description>
	<fname>scan_info.nasl</fname>
	<plugin_modification_date>2021/09/27</plugin_modification_date>
	<plugin_name>Nessus Scan Information</plugin_name>
	<plugin_publication_date>2005/08/26</plugin_publication_date>
	<plugin_type>summary</plugin_type>
	<risk_factor>None</risk_factor>
	<script_version>1.106</script_version>
	<solution>n/a</solution>
	<synopsis>This plugin displays information about the Nessus scan.</synopsis>
	<plugin_output>Information about this scan : 

	Nessus version : 10.0.0
	Nessus build : 20283
	Plugin feed version : 202110160410
	Scanner edition used : Nessus Home
	Scanner OS : LINUX
	Scanner distribution : ubuntu1110-x86-64
	Scan type : Normal
	Scan name : teste 10
	Scan policy used : Basic Network Scan
	Scanner IP : 192.168.2.82
	Port scanner(s) : nessus_syn_scanner 
	Port range : default
	Ping RTT : 38.467 ms
	Thorough tests : no
	Experimental tests : no
	Paranoia level : 1
	Report verbosity : 1
	Safe checks : yes
	Optimize the test : yes
	Credentialed checks : no
	Patch management checks : None
	Display superseded patches : yes (supersedence plugin launched)
	CGI scanning : disabled
	Web application tests : disabled
	Max hosts : 30
	Max checks : 4
	Recv timeout : 5
	Backports : None
	Allow post-scan editing: Yes
	Scan Start Date : 2021/11/10 12:48 WET
	Scan duration : 304 sec
	</plugin_output>
	</ReportItem>
	</ReportHost>
</Report>
</NessusClientData_v2>

Im sorry im new to all this

The file keeps giving error that no end tag foung for any tag.
the file has all the end tags

this is the conf file


input {
  file {
    path => "/home/rsc/scansNessus/*"
    start_position => "beginning"
    sincedb_path => "/dev/null"
    type => xml
    #codec => multiline {
	#pattern => "<?xml" 
     	#negate => "true"
        #what => "previous"
   # }
  }
}

filter {
    xml{
	source => "message"
	store_xml => true
	target => NessusClientData_v2
      	xpath => { "/NessusClientData_v2/Report/ReportHost/ReportItem/@pluginName" => "description" }
    }
}



output {
	stdout {
	codec => rubydebug
}

Since i disabled multiline :

imagem
imagem1482×513 51.1 KB

with multiline it gave me error for "not founding end tag /nessusclientdata_v2"

7

You will definitely need the multiline codec. I see you are not using auto_flush_interval on the codec, so an event will not get created until another <?xml is written to the file, the last XML object will not get read.

8

This was the output now.

{
               "@version" => "1",
             "@timestamp" => 2021-12-09T11:45:05.442Z,
                   "host" => "soc-vulnwhisper01.serv.redsoc.local",
    "NessusClientData_v2" => {
        "Policy" => [
            [0] {
                               "policyName" => [
                    [0] "Scan Policy"
                ],
                "IndividualPluginSelection" => [
                    [0] {
                        "PluginItem" => [
                            [0] {
                                "PluginName" => [
                                    [0] "Netstat Portscanner (WMI)"
                                ],
                                    "Status" => [
                                    [0] "enabled"
                                ],
                                  "PluginId" => [
                                    [0] "34220"
                                ],
                                    "Family" => [
                                    [0] "Netstat Portscanner (WMI)"
                                ]
                            }
                        ]
                    }
                ],
                          "FamilySelection" => [
                    [0] {
                        "FamilyItem" => [
                            [0] {
                                    "Status" => [
                                    [0] "enabled"
                                ],
                                "FamilyName" => [
                                    [0] "MacOS X Local Security Checks"
                                ]
                            }
                        ]
                    }
                ],
                              "Preferences" => [
                    [0] {
                         "ServerPreferences" => [
                            [0] {
                                "preference" => [
                                    [0] {
                                        "value" => [
                                            [0] "yes"
                                        ],
                                         "name" => [
                                            [0] "bw_prevent_core_updates"
                                        ]
                                    },
                                    [1] {
                                        "value" => [
                                            [0] "no"
                                        ],
                                         "name" => [
                                            [0] "scan.enable_utf8_output"
                                        ]
                                    }
                                ]
                            }
                        ],
                        "PluginsPreferences" => [
                            [0] {
                                "item" => [
                                    [0] {
                                            "pluginName" => [
                                            [0] "ADSI Settings"
                                        ],
                                              "pluginId" => [
                                            [0] "60024"
                                        ],
                                        "preferenceType" => [
                                            [0] "entry"
                                        ],
                                              "fullName" => [
                                            [0] "ADSI Settings[entry]:Domain Controller :"
                                        ],
                                        "preferenceName" => [
                                            [0] "Domain Controller :"
                                        ]
                                    }
                                ]
                            }
                        ]
                    }
                ]
            }
        ],
        "Report" => [
            [0] {
                      "name" => "teste 10LLLL",
                  "xmlns:cm" => "http://www.nessus.org/cm",
                "ReportHost" => [
                    [0] {
                                  "name" => "192.168.2.64",
                        "HostProperties" => [
                            [0] {
                                "tag" => [
                                    [ 0] {
                                           "name" => "HOST_START",
                                        "content" => "Wed Nov 10 12:48:50 2021"
                                    },
                                    [ 1] {
                                           "name" => "HOST_START_TIMESTAMP",
                                        "content" => "1636548530"
                                    },
                                    [ 2] {
                                           "name" => "host-ip",
                                        "content" => "192.168.2.64"
                                    },
                                    [ 3] {
                                           "name" => "HOST_END",
                                        "content" => "Wed Nov 10 12:53:54 2021"
                                    },
                                    [ 4] {
                                           "name" => "HOST_END_TIMESTAMP",
                                        "content" => "1636548834"
                                    },
                                    [ 5] {
                                           "name" => "host-rdns",
                                        "content" => "shelly1-40F5201DBCE0.Home"
                                    },
                                    [ 6] {
                                           "name" => "host-fqdn",
                                        "content" => "shelly1-40F5201DBCE0.Home"
                                    },
                                    [ 7] {
                                           "name" => "sinfp-signature",
                                        "content" => "\n\t   P1:B11113:F0x12:W2920:O0204ffff:M1460:\n\t   P2:B11113:F0x12:W2920:O0204ffff:M1460:\n\t   P3:B00000:F0x00:W0:O0:M0\n\t   P4:190000_7_p=80"
                                    },
                                    [ 8] {
                                           "name" => "sinfp-ml-prediction",
                                        "content" => "[{\"predicted-os\": \"Windows Server 2016 Standard\", \"confidence\": 1},{\"predicted-os\": \"Microsoft Windows Server 2016 Standard\", \"confidence\": 1},{\                                                                                    "predicted-os\": \"Microsoft Windows Server 2008 R2\", \"confidence\": 1},{\"predicted-os\": \"Cisco NX-OS\", \"confidence\": 4},{\"predicted-os\": \"Microsoft Windows Server 2008 R2 Enterprise.Service Pack 1\",                                                                                     \"confidence\": 1}]"
                                    },
                                    [ 9] {
                                           "name" => "operating-system",
                                        "content" => "EthernetBoard OkiLAN 8100e"
                                    },
                                    [10] {
                                           "name" => "system-type",
                                        "content" => "switch"
                                    },
                                    [11] {
                                           "name" => "traceroute-hop-0",
                                        "content" => "192.168.2.64"
                                    },
                                    [12] {
                                           "name" => "mac-address",
                                        "content" => "40:F5:20:1D:BC:E0"
                                    },
                                    [13] {
                                           "name" => "host-fqdns",
                                        "content" => "[{\"FQDN\":\"shelly1-40F5201DBCE0.Home\",\"sources\":[\"get_host_fqdn()\",\"determine_fqdn()\"]}]"
                                    },
                                    [14] {
                                           "name" => "operating-system-method",
                                        "content" => "SinFP"
                                    },
                                    [15] {
                                           "name" => "operating-system-conf",
                                        "content" => "65"
                                    },
                                    [16] {
                                           "name" => "os",
                                        "content" => "other"
                                    },
                                    [17] {
                                           "name" => "patch-summary-total-cves",
                                        "content" => "0"
                                    },
                                    [18] {
                                           "name" => "policy-used",
                                        "content" => "Basic Network Scan"
                                    },
                                    [19] {
                                           "name" => "Credentialed_Scan",
                                        "content" => "false"
                                    },
                                    [20] {
                                           "name" => "LastUnauthenticatedResults",
                                        "content" => "1636548834"
                                    }
                                ]
                            }
                        ],

continues the next post, has too many characters

9 
"ReportItem" => [
                            [0] {
                                                    "port" => "80",
                                "plugin_modification_date" => [
                                    [0] "2021/09/16"
                                ],
                                 "plugin_publication_date" => [
                                    [0] "2009/02/04"
                                ],
                                                "severity" => "0",
                                             "plugin_type" => [
                                    [0] "remote"
                                ],
                                           "plugin_output" => [
                                    [0] "Port 80/tcp was found to be open"
                                ],
                                          "script_version" => [
                                    [0] "1.40"
                                ],
                                                "protocol" => "tcp",
                                                "pluginID" => "11219",
                                             "plugin_name" => [
                                    [0] "Nessus SYN scanner"
                                ],
                                                "solution" => [
                                    [0] "Protect your target with an IP filter."
                                ],
                                            "pluginFamily" => "Port scanners",
                                              "pluginName" => "Nessus SYN scanner",
                                                   "fname" => [
                                    [0] "nessus_syn_scanner.nbin"
                                ],
                                             "risk_factor" => [
                                    [0] "None"
                                ],
                                             "description" => [
                                    [0] "This plugin is a SYN 'half-open' port scanner.  It shall be reasonably quick even against a firewalled target. \n\tNote that SYN scans are less intrusive than TCP (full c                                                                                    onnect) scans against broken services, but they might cause problems for less robust firewalls and also leave unclosed connections on the remote target, if the network is loaded."
                                ],
                                                "synopsis" => [
                                    [0] "It is possible to determine which TCP ports are open."
                                ],
                                                "svc_name" => "www"
                            },
                            [1] {
                                                    "port" => "0",
                                "plugin_modification_date" => [
                                    [0] "2021/09/27"
                                ],
                                 "plugin_publication_date" => [
                                    [0] "2005/08/26"
                                ],
                                                "severity" => "0",
                                                   "agent" => [
                                    [0] "all"
                                ],
                                             "plugin_type" => [
                                    [0] "summary"
                                ],
                                           "plugin_output" => [
                                    [0] "Information about this scan : \n\tNessus version : 10.0.0\n\tNessus build : 20283\n\tPlugin feed version : 202110160410\n\tScanner edition used : Nessus Home\n\tScanner O                                                                                    S : LINUX\n\tScanner distribution : ubuntu1110-x86-64\n\tScan type : Normal\n\tScan name : teste 10\n\tScan policy used : Basic Network Scan\n\tScanner IP : 192.168.2.82\n\tPort scanner(s) : nessus_syn_scanner \                                                                                    n\tPort range : default\n\tPing RTT : 38.467 ms\n\tThorough tests : no\n\tExperimental tests : no\n\tParanoia level : 1\n\tReport verbosity : 1\n\tSafe checks : yes\n\tOptimize the test : yes\n\tCredentialed che                                                                                    cks : no\n\tPatch management checks : None\n\tDisplay superseded patches : yes (supersedence plugin launched)\n\tCGI scanning : disabled\n\tWeb application tests : disabled\n\tMax hosts : 30\n\tMax checks : 4\n\                                                                                    tRecv timeout : 5\n\tBackports : None\n\tAllow post-scan editing: Yes\n\tScan Start Date : 2021/11/10 12:48 WET\n\tScan duration : 304 sec\n\t"
                                ],
                                          "script_version" => [
                                    [0] "1.106"
                                ],
                                                "protocol" => "tcp",
                                                "pluginID" => "19506",
                                             "plugin_name" => [
                                    [0] "Nessus Scan Information"
                                ],
                                                "solution" => [
                                    [0] "n/a"
                                ],
                                              "always_run" => [
                                    [0] "1"
                                ],
                                            "pluginFamily" => "Settings",
                                              "pluginName" => "Nessus Scan Information",
                                                   "fname" => [
                                    [0] "scan_info.nasl"
                                ],
                                             "risk_factor" => [
                                    [0] "None"
                                ],
                                             "description" => [
                                    [0] "This plugin displays, for each tested host, information about the scan itself :\n\t  - The version of the plugin set.\n\t  - The type of scanner (Nessus or Nessus Home).\                                                                                    n\t  - The version of the Nessus Engine.\n\t  - The port scanner(s) used.\n\t  - The port range scanned.\n\t  - The ping round trip time \n\t  - Whether credentialed or third-party patch management     checks ar                                                                                    e possible.\n\t  - Whether the display of superseded patches is enabled\n\t  - The date of the scan.\n\t  - The duration of the scan.\n\t  - The number of hosts scanned in parallel.\n\t  - The number of checks d                                                                                    one in parallel."
                                ],
                                                "synopsis" => [
                                    [0] "This plugin displays information about the Nessus scan."
                                ],
                                                "svc_name" => "general"
                            }
                        ]
                    }
                ]
            }
        ]
    },
                   "tags" => [
        [0] "multiline"
    ],
                   "type" => "xml",
                   "path" => "/home/rsc/scansNessus/scan.xml"
}

but still gave me this error when trying to send it to Elasticsearch

imagem
imagem1477×380 73.6 KB

this is my config file

input {
  file {
    path => "/home/rsc/scansNessus/*"
    start_position => "beginning"
    sincedb_path => "/dev/null"
    type => xml
    codec => multiline {
        pattern => "<?xml*?>"
        negate => "true"
        what => "previous"
        auto_flush_interval => 20
    }
  }
}

filter {
    xml{
        source => "message"
        store_xml => true
        target => NessusClientData_v2
        xpath => { "NessusClientData_v2/Report/ReportHost/ReportItem/@description" => "description" }
    }

    mutate{
        remove_field => ["message"]
    }
}



output {
        stdout {
        codec => rubydebug
}

    elasticsearch {

      hosts => [ "ip:port" ]
      index => "logstash-vulnwhisperer-%{+YYYY.MM}"
      user => "username"
      password => "password"
      ssl => true
      cacert => '/etc/logstash/certs/ca.crt'
    }
}

i wanted to see every tag in Elasticsearch. in order to see like X number of vulnerabilities, for one IP.

10

The Elasticsearch error is a mapping_parsing_exception, this means that you are trying to index a field with a different mapping from the one that was indexed before.

In this case the field NessusClientData_v2 was indexed before as a text, and now you want to index it as a json object.

Did you create a mapping for the logstash-vulnwhisperer-* index? If not, you just need to delete your index and try again, if you create a mapping, you will need to fix the mapping for the NessusClientData_v2 field.

11

The [NessusClientData_v2] field is an object on that event. elasticsearch is expecting it to be text. A field cannot be an object on some documents and text on others, it has to be one or the other. Once you have the XML filter working the exception may disappear once you roll over to a new index.

12

IT worked, i could upload it to vulnwhisperer, how can i separate all the fields now?
i wanted it to look like the image below

22
221003×800 46.3 KB

13

Like i want to have a document for each report item with the corresponding hostname
But want to have all the tags inside the report item seperated.

14

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.