How To receive a SOAP Response message from web service

[s1m0ne]

Hi Folks,

NOTE: Logstash version 5.6.2

Currently, I receive events in (json format) from apache kafka and I use logstash with HTTP Output plugin for sending SOAP Request message to Web Service, via WSDL.

output {
http {
url => "ABC.NET"
http_method => "post"
content_type => "text/xml;charset=UTF-8"
headers => {
"SOAPAction" => "ABC.NET"
}
format => "message"
message => '<?xml version="1.0" encoding="UTF-8"?><soapenv:Envelope xmlns:ent="ABC.NET" xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"> xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">soapenv:Headerpwduid</soapenv:Header>soapenv:Body>ent:mySOAPAction</ent:mySOAPAction></soapenv:Body></soapenv:Envelope>'
}
...other output plugin..
}

my goal is to receive the SOAP Response message in Logstash and forward to kafka in json format but I do not find any official plugin that support this functionality.

Do you have any experience/suggestion which plugin or strategy I can use to receive those SOAP Response?

Many thanks in advance for your response.
Cheers,
s1m0ne

[guyboertje]

As I don't have access to a SOAP endpoint I can't test any suggestions.

Some folks on here have tried the http_poller plugin. However you said you want XML to JSON conversion and that is a problem because we don't have a XML codec to decode the XML to a LS Event.

The XML filter uses XPath to "pluck" values out of the XML and add them with keys to an event and that may work for you.

http_poller -> xml filter -> kafka output + json codec

[s1m0ne]

Hi @guyboertje,

thank you for your response.

Actually, I am thinking to use HTTP_Poller Input Plugin for this purpose but the http_poller performes async request to wdsl (setting up polling interval attribute) and I concern about the performance and resilience of LS plugin.

Regarding the XML Filter, I would forward the soap response as I receive, to kafka topic in json format.

Cheers,
s1m0ne

[Nonow53 (Arnaud Giffard)]

Hi @s1m0ne,

Could you post your logstash pipeline please ?
I try to work with http_poll to call SOAP Service but I don't understand why my conf not works...

thanks

@Nonow53

[Nonow53 (Arnaud Giffard)]

My pipeline

input {
http_poller {
urls => {
testVersion => {
method => post
url => "https://forge.douane/mantis/api/soap/mantisconnect.php"
headers => {
"Content-Type" => "text/xml; charset=UTF-8"
"SOAPAction" => "urn:mc_version"
}
body => '<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:man="http://futureware.biz/mantisconnect">
soapenv:Header/
soapenv:Body
<man:mc_version soapenv:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"/>
</soapenv:Body>
</soapenv:Envelope>'
}
}
request_timeout => 30
schedule => @every 30s
codec => "plain"
# A hash of request metadata info (timing, response headers, etc.) will be sent here
metadata_target => "http_poller_metadata"
}
}
output {
stdout {
codec => rubydebug
}
}

[Nonow53 (Arnaud Giffard)]

Hi,

I find the problem, it is my property "schedule".
It must be a hash.