Wsdl in logstash input file

Elastic Stack Logstash
1

Hey Guys,

I have been trying to fetch data from BMC Remedy using logstash http poller method, as i was not able to get the restapi's but managed to get a wsdl link, the link was working properly when i test using soap UI.

If i give the WSDL directly in the http poller config file i'm not getting any data in elasticsearch, do i need to add any additional data in the config file? or is there any other method available for this type of integration? Please advice.

My config File for reference:
input {
http_poller {
urls => {
url => "http://1.12.15.9:8080/arsys/services/ARService?server=remedyappsrv&webService=nk"
}
request_timeout => 60
proxy => { host => "1.1.1.5" port => "9090" scheme => "http"}
user => ""
password => ""
schedule => { cron => " * * * *"}
codec => "json"
metadata_target => "http_poller_metadata"
}
}
filter
{
split
{
field => "result"
}
}
output {
elasticsearch {
hosts => ["1.1.1.1:9200"]
index => "bmcremedy"
}

Thanks
Gauti

2

Comment out your elasticsearch and use a stdout { codec => rubydebug } output to debug the http_poller input. Once the event looks fine you can look into getting the data into ES.

3

Had commented out the elasticsearch host, thanks @magnusbaeck...

Anybody have any suggestion for this issue....

Thanks
Gautham

4

Right now it's not clear what the issue even is. Are you getting the expected data from the stdout output but nothing to ES? Whenever possible give concrete examples of what happens (including copy/pasting from terminal output) rather than describing what you think happens.

5

hey @magnusbaeck the issue here is i'm trying to fetch data from BMC Remedy through logstash http poller method, using the config file which i have mentioned in my previous post.

In general http poller works properly for rest API's , my situation here is i'm using WSDL for which i'm not able to get any data.

Here is the output from stdout
"tags" => [
[0] "_jsonparsefailure"
],
"@timestamp" => 2018-06-25T13:58:00.580Z,
"@version" => "1",
"http_poller_metadata" => {
"request" => {
"url" => "http://1.12.15.9:8080/arsys/services/ARService?server=remedyappsrv&webService=nk",
"method" => "get"
},
"response_headers" => {
"date" => "Mon, 25 Jun 2018 13:51:01 GMT",
"x-xss-protection" => "1;mode=block",
"connection" => "Keep-Alive",
"cache-control" => "proxy-revalidate",
"server" => "Apache-Coyote/1.1",
"content-length" => "133",
"content-type" => "text/html;charset=utf-8"
},
"response_message" => "OK",
"times_retried" => 0,
"code" => 200,
"runtime_seconds" => 0.409728,
"host" => "elasticprd02",
"name" => "url"
},
"message" => "

ARService

\r\n

Hi there, this is an AXIS service!

\r\nPerhaps there will be a form for invoking the service here...\r\n"
}

DO i need to add additional fields when i use WSDL?

Thanks
Gautham

6

Always format configuration snippets and logs as preformatted text.

The http_poller defaults to the json codec which isn't appropriate here. Set it to "plain" and add an xml filter that parses the message field in whatever way you want (I'm assuming that message actually contains XML but as you can see the XML tags have been stripped).

7

hey @magnusbaeck thanks for the ideas.
Actually the output which i'm looking from the input URL : http://1.12.15.9:8080/arsys/services/ARService?server=remedyappsrv&webService=nk

is provided properly through SOAP UI, my question here is, is there a way where i can include the SOAP function and decode the output and ingest tat data in to elasticsearch.

Mainly looking for an example or syntax config file to include SOAP function to resolve the WSDL link

Thanks
Gauti

8

I'm not sure I understand enough of SOAP to answer that question, but it sounds like what you really need is a SOAP plugin. It should be pretty easy to make one.

9

@magnusbaeck Can you guide me where i can get some steps for making a SOAP plugin

Thanks
Gauti

10

The Logstash documentation has rather extensive documentation about plugin creation.

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.