# Http\_poller error when building XML/SOAP request

**URL:** https://discuss.elastic.co/t/http-poller-error-when-building-xml-soap-request/210698
**Category:** Logstash
**Created:** [December 5, 2019, 12:05pm UTC](https://discuss.elastic.co/t/http-poller-error-when-building-xml-soap-request/210698 "2019-12-05T12:05:33Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![sgreszcz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sgreszcz/32/46322_2.png) [@sgreszcz](https://discuss.elastic.co/u/sgreszcz)
#### Post date: [December 5, 2019, 12:05pm UTC](https://discuss.elastic.co/t/http-poller-error-when-building-xml-soap-request/210698/1 "2019-12-05T12:05:33Z")

</div>

Hi there, I'm trying to use the http\_poller input to periodically generate an HTTP/XML/SOAP requests but I'm getting "Invalid URL post" back - Logstash doesn't even seem to be building the request:

```auto
Pipeline aborted due to error {:pipeline_id=>"ucm-get-perfmon", :exception=>#<LogStash::ConfigurationError: Invalid URL post>, :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-http_poller-5.0.1/lib/logstash/inputs/http_poller.rb:103:in `validate_request!'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-http_poller-5.0.1/lib/logstash/inputs/http_poller.rb:95:in `normalize_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-http_poller-5.0.1/lib/logstash/inputs/http_poller.rb:55:in `block in setup_requests!'", "org/jruby/RubyHash.java:1417:in `each'", "org/jruby/RubyEnumerable.java:851:in `map'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-http_poller-5.0.1/lib/logstash/inputs/http_poller.rb:55:in `setup_requests!'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-http_poller-5.0.1/lib/logstash/inputs/http_poller.rb:45:in `register'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:199:in `block in register_plugins'", "org/jruby/RubyArray.java:1800:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:198:in `register_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:303:in `start_inputs'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:259:in `start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:153:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:108:in `block in start'"], "pipeline.sources"=>["/usr/share/logstash/pipelines/ucm-get-perfmon.conf"], :thread=>"#<Thread:0x7653662d run>"}

```

My config for logstash looks like this:

```auto
input {
    http_poller {
        urls => {
            method => post
            url => "https://server.com:8443/logcollectionservice2/services/LogCollectionPortTypeService?wsdl"
            auth => {
                user => "user"
                password => "password"
            }
            headers => {
                "Content-Type" => "text/xml"
                "SOAPAction" => "CUCM:DB ver=12.5"
            }
            body => '<!--LogCollection API - SelectLogFiles - Request--><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:soap="http://schemas.cisco.com/ast/soap"><soapenv:Header/><soapenv:Body><soap:selectLogFiles><soap:FileSelectionCriteria><soap:ServiceLogs><soap:item></soap:item></soap:ServiceLogs><soap:SystemLogs><soap:item>Cisco RIS Data Collector PerfMonLog</soap:item></soap:SystemLogs><soap:SearchStr></soap:SearchStr><soap:Frequency>OnDemand</soap:Frequency><soap:JobType>PushtoSFTPServer</soap:JobType><soap:ToDate></soap:ToDate><soap:FromDate></soap:FromDate><soap:TimeZone></soap:TimeZone><soap:RelText>Days</soap:RelText><soap:RelTime>5</soap:RelTime><soap:Port>22</soap:Port><soap:IPAddress>labserver.com</soap:IPAddress><soap:UserName>username</soap:UserName><soap:Password>password</soap:Password><soap:ZipInfo>false</soap:ZipInfo><soap:RemoteFolder>/data/tracelogger/test/</soap:RemoteFolder></soap:FileSelectionCriteria></soap:selectLogFiles></soapenv:Body></soapenv:Envelope>'
        }
        request_timeout => 60
        # Supports "cron", "every", "at" and "in" schedules by rufus scheduler
        schedule => { cron => "* * * * * UTC" }
    }
}

output {
  stdout {
    codec => rubydebug
  }
} 

```

I can get the request working OK using postman and it looks like this:

```auto
POST /logcollectionservice2/services/LogCollectionPortTypeService?wsdl HTTP/1.1
Host: hostname.com:8443
SOAPAction: CUCM:DB ver=12.5 
Content-Type: text/xml
Authorization: Basic dHJhY2Vsb2dnZXIuYXhsLXJvLmFkbWluOnp5bU9pRGVybk1vcg==
User-Agent: PostmanRuntime/7.20.1
Accept: */*
Cache-Control: no-cache
Postman-Token: 65b97d05-1e4f-44cd-af48-935989a062ef,47702e32-3cf2-4668-9a79-8c82e26a5e9f
Host: hostname.com:8443
Accept-Encoding: gzip, deflate
Content-Length: 1422
Cookie: SoapSession.id=88B86D797786C68C381575460218344; JSESSIONID=A474438A26B671FEBE53C0DF917B0A9E; JSESSIONIDSSO=DC7D5E497DC11A2B3EF86107207F278B
Connection: keep-alive
cache-control: no-cache

<!--LogCollection API - SelectLogFiles - Request-->
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:soap="http://schemas.cisco.com/ast/soap">
    <soapenv:Header/>
    <soapenv:Body>
        <soap:selectLogFiles>
            <soap:FileSelectionCriteria>
                <soap:ServiceLogs>
                    <soap:item></soap:item>
                </soap:ServiceLogs>
                <soap:SystemLogs>
                    <soap:item>Cisco RIS Data Collector PerfMonLog</soap:item>
                </soap:SystemLogs>
                <soap:SearchStr></soap:SearchStr>
                <soap:Frequency>OnDemand</soap:Frequency>
                <soap:JobType>PushtoSFTPServer</soap:JobType>
                <soap:ToDate></soap:ToDate>
                <soap:FromDate></soap:FromDate>
                <soap:TimeZone></soap:TimeZone>
                <soap:RelText>Days</soap:RelText>
                <soap:RelTime>5</soap:RelTime>
                <soap:Port>22</soap:Port>
                <soap:IPAddress>server.com</soap:IPAddress>
                <soap:UserName>username</soap:UserName>
                <soap:Password>password</soap:Password>
                <soap:ZipInfo>false</soap:ZipInfo>
                <soap:RemoteFolder>/data/tracelogger/test/</soap:RemoteFolder>
            </soap:FileSelectionCriteria>
        </soap:selectLogFiles>
    </soapenv:Body>
</soapenv:Envelope>

```

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [December 5, 2019, 5:31pm UTC](https://discuss.elastic.co/t/http-poller-error-when-building-xml-soap-request/210698/2 "2019-12-05T17:31:15Z")

</div>

> [@sgreszcz](#):
>
> input { http\_poller { urls =\> { method =\> post url =\>

That looks wrong to me. It should be

```
input {
    http_poller {
        urls => {
            randomName => {
                method => post
                url => [...]

```

---

<div class="post-metadata">

### Author: ![sgreszcz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sgreszcz/32/46322_2.png) [@sgreszcz](https://discuss.elastic.co/u/sgreszcz)
#### Post date: [December 9, 2019, 12:24pm UTC](https://discuss.elastic.co/t/http-poller-error-when-building-xml-soap-request/210698/3 "2019-12-09T12:24:10Z")

</div>

Thanks @Badger, the syntax was broken having to do with the urls =\> and then nesting specific urls with "randomName" below them.

Now I have another problem, it seems as though the HTTPs request is failing due to the self-signed certificates:

```auto
    "http_request_failure" => {
              "backtrace" => nil,
                  "error" => "PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target",

```

From the documentation, it looks like I have to create a JKS and download the certificate in advance. How does this actually work with Logstash in the official Docker containers? Having to manually download certificates and manage a Java Key Store with passwords is not really easy to scale/automate? Is there some way to automatically accept the certificate?

[https://www.elastic.co/guide/en/logstash/current/plugins-inputs-http\_poller.html](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-http_poller.html)

---

<div class="post-metadata">

### Author: ![sgreszcz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sgreszcz/32/46322_2.png) [@sgreszcz](https://discuss.elastic.co/u/sgreszcz)
#### Post date: [December 9, 2019, 1:24pm UTC](https://discuss.elastic.co/t/http-poller-error-when-building-xml-soap-request/210698/4 "2019-12-09T13:24:41Z")

</div>

OK, it looks like there is no easy way to do this as the following "ignore" switches were depreciated.

```auto
ssl => true
ssl_certificate_verification => false

```

There are several outstanding issues on http\_poller input. Is there anyway we can get the "ignore" switch back on to handle trusted SSL/TLS HTTPs endpoints? What can we do to get this back in the input?

> <https://github.com/logstash-plugins/logstash-input-http_poller/issues/115>
>
> There is no simple way to accept TLS/SSL certificates using http\_poller input. Using something like python's "requests" library or postman allows...

  

> <https://github.com/logstash-plugins/logstash-input-http_poller/issues/96>
>
> Hi,
> I'm getting the following error when I try to poll a website running a self signed certificate.
> "Host name '192.168.1.35' does not...

  

> <https://github.com/logstash-plugins/logstash-input-http_poller/issues/73>
>
> looking at the latest docs about ssl\_certificate\_verification the feature is described without mention of being deprecated even though it is

---

<div class="post-metadata">

### Author: ![raged](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/raged/32/45722_2.png) [@raged](https://discuss.elastic.co/u/raged)
#### Post date: [December 9, 2019, 5:44pm UTC](https://discuss.elastic.co/t/http-poller-error-when-building-xml-soap-request/210698/5 "2019-12-09T17:44:35Z")

</div>

I had issues with certificates when using the http\_poller input. I kept getting SSL Errors for handshake alert: unrecognized\_name.

I ended up having to put:

`-Djsse.enableSNIExtension=false` in the /etc/logstash/jvm.options

After I restarted logstash, I was up and running.

---

<div class="post-metadata">

### Author: ![sgreszcz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sgreszcz/32/46322_2.png) [@sgreszcz](https://discuss.elastic.co/u/sgreszcz)
#### Post date: [December 9, 2019, 10:10pm UTC](https://discuss.elastic.co/t/http-poller-error-when-building-xml-soap-request/210698/6 "2019-12-09T22:10:37Z")

</div>

@raged Thanks for this hint, but unfortunately it isn't working. I passed in the switch using the LS\_JAVA\_OPT environment parameter, and I can see it in the Docker container, but I get this in the logs:

```auto
jcmd 1 VM.system_properties
1:
#Mon Dec 09 22:02:30 UTC 2019
awt.toolkit=sun.awt.X11.XToolkit
ls.log.format=plain
java.specification.version=11
jsse.enableSNIExtension=false

```

```auto
 "error" => "PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target",

```

Maybe I'm still doing something wrong? I tried to find another switch to skip certificate checking (as this is just a lab/test server with self-signed certificate) but didn't find anything supported by java.

---

<div class="post-metadata">

### Author: ![Intissar\_Ayadi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/intissar_ayadi/32/59264_2.png) [@Intissar\_Ayadi](https://discuss.elastic.co/u/Intissar_Ayadi)
#### Post date: [December 12, 2019, 10:30am UTC](https://discuss.elastic.co/t/http-poller-error-when-building-xml-soap-request/210698/7 "2019-12-12T10:30:44Z")

</div>

Hello,

Please any help.  
I have soap http logstash output and i would like to post some grok fields output using soap webservice.  
I used the logstash sprintf syntax ${fieldname} but i doesn't work.(error message: cannot evaluate environnement variable)  
Any help please

Thnx in advance

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 9, 2020, 10:31am UTC](https://discuss.elastic.co/t/http-poller-error-when-building-xml-soap-request/210698/8 "2020-01-09T10:31:03Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.