# POST method http\_poller

**URL:** https://discuss.elastic.co/t/post-method-http-poller/141940
**Category:** Logstash
**Created:** [July 27, 2018, 12:11pm UTC](https://discuss.elastic.co/t/post-method-http-poller/141940 "2018-07-27T12:11:37Z")
**Posts on this page:** 20
**Page:** 1

<div class="post-metadata">

### Author: ![m.sereda](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/m.sereda/32/48232_2.png) [@m.sereda](https://discuss.elastic.co/u/m.sereda)
#### Post date: [July 27, 2018, 12:11pm UTC](https://discuss.elastic.co/t/post-method-http-poller/141940/1 "2018-07-27T12:11:37Z")

</div>

Trying to make a POST method to login but smth is going wrong. To make a login I have to fill "login" and "password" poles which is in JSON. Where is a mistake?

```
input {
  http_poller {
    urls => {
      soap_request => {
        method => post
        url => "http://localhost:8080/api/ses/login"
        headers => {
          Content-Type => "application/json"
        }
        body => '{"login": "login", "password": "password"}'
      }
    }
    request_timeout => 10
    interval => 10
    codec => "plain"
  }
}

output {
  stdout {
    codec => rubydebug
  }
}

```

After I need to take a cookie and send it to GET method `http://localhost:8080/api/my/destination` , how to do that?

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [July 27, 2018, 6:49pm UTC](https://discuss.elastic.co/t/post-method-http-poller/141940/2 "2018-07-27T18:49:13Z")

</div>

> Trying to make a POST method to login but smth is going wrong.

Well, what happens?

> After I need to take a cookie and send it to GET method [http://localhost:8080/api/my/destination](http://localhost:8080/api/my/destination) , how to do that?

I don't think there's any convenient way of doing that.

---

<div class="post-metadata">

### Author: ![m.sereda](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/m.sereda/32/48232_2.png) [@m.sereda](https://discuss.elastic.co/u/m.sereda)
#### Post date: [July 27, 2018, 6:52pm UTC](https://discuss.elastic.co/t/post-method-http-poller/141940/3 "2018-07-27T18:52:17Z")

</div>

Hmm... is it possible to write any script or somehow to do that?

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [July 27, 2018, 6:53pm UTC](https://discuss.elastic.co/t/post-method-http-poller/141940/4 "2018-07-27T18:53:36Z")

</div>

Sure, you could make the HTTP call in a separate script that you invoke with an exec plugin.

---

<div class="post-metadata">

### Author: ![m.sereda](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/m.sereda/32/48232_2.png) [@m.sereda](https://discuss.elastic.co/u/m.sereda)
#### Post date: [July 27, 2018, 6:55pm UTC](https://discuss.elastic.co/t/post-method-http-poller/141940/5 "2018-07-27T18:55:53Z")

</div>

Could you please help me to do that or is there any tutorial?

How this script should looks like? what type ?

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [July 27, 2018, 9:25pm UTC](https://discuss.elastic.co/t/post-method-http-poller/141940/6 "2018-07-27T21:25:34Z")

</div>

You can write such a script in any language. I don't have an example at hand.

---

<div class="post-metadata">

### Author: ![m.sereda](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/m.sereda/32/48232_2.png) [@m.sereda](https://discuss.elastic.co/u/m.sereda)
#### Post date: [July 28, 2018, 8:13am UTC](https://discuss.elastic.co/t/post-method-http-poller/141940/7 "2018-07-28T08:13:27Z")

</div>

> Well, what happens?

Here is my logstash.conf with POST method and it is not working. Do I put login /password in a good way? The format which I should put is JSON

```
input {
  http_poller {
    urls => {
      test1 => {
        url => "http://localhost:8080/api/sess/login"
        method => "post"
        headers => {
          Accept => "application/json"
        }
        body => '{"login":"login", 
        "password":"pass"}'
      
    }

```

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [July 28, 2018, 11:11am UTC](https://discuss.elastic.co/t/post-method-http-poller/141940/8 "2018-07-28T11:11:29Z")

</div>

> Here is my logstash.conf with POST method and it is not working.

That doesn't answer the question "what happens?". Are you getting any response at all from the server? If yes, what does it look like?

---

<div class="post-metadata">

### Author: ![m.sereda](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/m.sereda/32/48232_2.png) [@m.sereda](https://discuss.elastic.co/u/m.sereda)
#### Post date: [July 29, 2018, 8:06am UTC](https://discuss.elastic.co/t/post-method-http-poller/141940/9 "2018-07-29T08:06:49Z")

</div>

I got a 415 error

`@version:1 message:<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>Payara Micro #badassfish - Error report</title><style type="text/css"><!--H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} H3 {font-family:Tahoma,Arial,sans-serif;color:whit`

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [July 29, 2018, 6:46pm UTC](https://discuss.elastic.co/t/post-method-http-poller/141940/10 "2018-07-29T18:46:00Z")

</div>

415 is "Unsupported media type". Perhaps you need to send "Content-type: application/json" as an additional HTTP header in your request? Otherwise I don't know. Try using e.g. Wireshark to capture a successful request and compare it to the one Logstash makes.

---

<div class="post-metadata">

### Author: ![m.sereda](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/m.sereda/32/48232_2.png) [@m.sereda](https://discuss.elastic.co/u/m.sereda)
#### Post date: [July 30, 2018, 10:31am UTC](https://discuss.elastic.co/t/post-method-http-poller/141940/11 "2018-07-30T10:31:56Z")

</div>

OK, I've made a python script with saving a cookie in a file and it is working

```
import os
from cookielib import LWPCookieJar

import requests

s = requests.Session()
s.cookies = LWPCookieJar('cookiejar')
if not os.path.exists('cookiejar'):
    # Create a new cookies file and set our Session's cookies
    print('setting cookies')
    s.cookies.save()
    r = s.post('http://localhost:8080/api/ses/login', json={'login': 'login', 'password': 'pass'})
else:
    # Load saved cookies from the file and use them in a request
    print('loading saved cookies')
    s.cookies.load(ignore_discard=True)
    r = s.post('http://localhost:8080/api/ses/login', json={'login': 'login', 'password': 'pass'})
#print(r.text)
# Save the session's cookies back to the file
s.cookies.save(ignore_discard=True)

print(r.status_code)
#print(r.json())
print(r.cookies)

```

Cookie file

```
#LWP-Cookies-2.0
Set-Cookie3: JSESSIONID=aae34175e7cea354b77b29020288; path="/"; domain="localhost.local"; path_spec; discard; HttpOnly=None; version=0

```

and trying to use an exec plugin. Don't understand how to send cookies from that scrypt to logstash

```
input {
  exec {
        command => "python /home/maksym/PathFolder/pythonpractice/postsample.py"
        interval => 20
        id => "plugin_id"
        }
  http_poller {
    urls => {
       test2 => {
       url => "http://localhost:8080/api/dash/space"
       method => "get"
       headers => {
         Accept => "application/json"
         Cookie => "/home/maksym/PathFolder/pythonpractice/cookiejar"
       }
    }
   }
   request_timeout => 60
   # Supports "cron", "every", "at" and "in" schedules by rufus scheduler
   schedule => { cron => "* * * * * UTC"}
   codec => "json"
   # A hash of request metadata info (timing, response headers, etc.) will be sent here
   metadata_target => "http_poller_metadata"
   }
}

output {
   stdout {
     codec => rubydebug
   }
}

```

For now the output is

 ![Screenshot%20from%202018-07-30%2012-30-52](https://us1.discourse-cdn.com/elastic/original/3X/d/b/db6a488d67c1e88959953fac5299dd4ad2565ba6.png)

Appreciate any help. Thanks

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [July 30, 2018, 8:24pm UTC](https://discuss.elastic.co/t/post-method-http-poller/141940/12 "2018-07-30T20:24:31Z")

</div>

Do everything from the Python script. Don't try to pass the cookies from the Python script to the http\_poller input.

---

<div class="post-metadata">

### Author: ![m.sereda](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/m.sereda/32/48232_2.png) [@m.sereda](https://discuss.elastic.co/u/m.sereda)
#### Post date: [July 31, 2018, 8:03am UTC](https://discuss.elastic.co/t/post-method-http-poller/141940/13 "2018-07-31T08:03:17Z")

</div>

OK, now I think I understand. I changed my logstash.conf as the following

```
input {
  exec {
        command => "python /home/maksym/PathFolder/pythonpractice/postsample.py"
        interval => 20
        id => "plugin_id"  
        }
   request_timeout => 60
   schedule => { cron => "* * * * * UTC"}
   codec => "json"
}
      
output {
   stdout { 
     codec => rubydebug
   }
}

```

and python.py like this:

```
import os
os.chdir("/home/maksym/PathFolder")
from cookielib import LWPCookieJar

import requests

s = requests.Session()
s.cookies = LWPCookieJar('cookiejar')
if not os.path.exists('cookiejar'):
    # Create a new cookies file and set our Session's cookies
    print('setting cookies')
    s.cookies.save()
    r = s.post('http://localhost:8080/api/sess/login', json={'login': 'login', 'password': 'pass'})
    print(r.text)
else:
    # Load saved cookies from the file and use them in a request
    print('loading saved cookies')
    s.cookies.load(ignore_discard=True)
    r = s.get('http://localhost:8080/api/dash/space')
print(r.text)
# Save the session's cookies back to the file
s.cookies.save(ignore_discard=True)

print(r.status_code)
#print(r.json())
print(r.cookies)

```

and when I running the script from the console it is working, but from logstash no - nothing showing in Kibana. Could you tell me where I have made a mistake?

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [August 1, 2018, 6:00am UTC](https://discuss.elastic.co/t/post-method-http-poller/141940/14 "2018-08-01T06:00:00Z")

</div>

Does your stdout output produce the expected results?

---

<div class="post-metadata">

### Author: ![m.sereda](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/m.sereda/32/48232_2.png) [@m.sereda](https://discuss.elastic.co/u/m.sereda)
#### Post date: [August 1, 2018, 7:19am UTC](https://discuss.elastic.co/t/post-method-http-poller/141940/15 "2018-08-01T07:19:34Z")

</div>

no, could you tell me how to fix it?

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [August 1, 2018, 9:04am UTC](https://discuss.elastic.co/t/post-method-http-poller/141940/16 "2018-08-01T09:04:15Z")

</div>

Well, until the stdout output produces the desired results there's no point in looking in Kibana (and right now you don't even have an elasticsearch output). What output are you getting from Logstash? Does the log indicate that the script is being run? Is the cookie file being created?

---

<div class="post-metadata">

### Author: ![m.sereda](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/m.sereda/32/48232_2.png) [@m.sereda](https://discuss.elastic.co/u/m.sereda)
#### Post date: [August 1, 2018, 9:28am UTC](https://discuss.elastic.co/t/post-method-http-poller/141940/17 "2018-08-01T09:28:52Z")

</div>

I add an elasticsearch output

```
input {
  exec {
        command => "/usr/bin/python /home/maksym/PathFolder/pythonpractice/postsample.py"
        interval => 20
        data_format => "json"

        }
   request_timeout => 60
   schedule => { cron => "* * * * * UTC"}
}
   
output {
   elasticsearch {
        hosts => ["localhost:9200"]
        index => "logstash-%{+YYYY.MM.dd}"
}
   stdout {
     codec => rubydebug 
   }  
}

```

but still no result

 ![Screenshot%20from%202018-08-01%2011-18-42](https://us1.discourse-cdn.com/elastic/original/3X/9/a/9a97579fcf07a915a6185207a6dfa7125974ffea.png)

when I run the python script for the first and second time the outputs are like those and it is the same as in Postman

 ![Screenshot%20from%202018-08-01%2011-27-51](https://us1.discourse-cdn.com/elastic/original/3X/4/a/4a7184171ae471f31f046ec2b33451888e9dc052.png)

Cookie file created after the first running the script

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [August 1, 2018, 9:37am UTC](https://discuss.elastic.co/t/post-method-http-poller/141940/18 "2018-08-01T09:37:45Z")

</div>

Make sure the script output only contains the JSON data you want to load into Logstash. I repeat: What output are you getting from Logstash? Does the log indicate that the script is being run?

---

<div class="post-metadata">

### Author: ![m.sereda](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/m.sereda/32/48232_2.png) [@m.sereda](https://discuss.elastic.co/u/m.sereda)
#### Post date: [August 1, 2018, 9:50am UTC](https://discuss.elastic.co/t/post-method-http-poller/141940/19 "2018-08-01T09:50:35Z")

</div>

due to logstash-plain.log I have an error

```
[2018-08-01T11:43:14,097][INFO][logstash.runner] Starting Logstash {"logstash.version"=>"6.3.2"}
[2018-08-01T11:43:15,633][ERROR][logstash.agent] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of #, { at line 130, column 20 (byte 2388) after input {\n exec {\n\tcommand => \"/usr/bin/python /home/maksym/PathFolder/pythonpractice/postsample.py\"\n interval => 20\n\t#id => \"myid\"\n\tdata_format => \"json\"\n\n\t}\n request_timeout ", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:42:in `compile_imperative'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:50:in `compile_graph'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:12:in `block in compile_sources'", "org/jruby/RubyArray.java:2486:in `map'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:11:in `compile_sources'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:49:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:167:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:40:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:305:in `block in converge_state'"]}
[2018-08-01T11:43:16,100][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=>9600}
[2018-08-01T11:43:56,166][INFO][logstash.runner] Starting Logstash {"logstash.version"=>"6.3.2"}
[2018-08-01T11:43:58,223][ERROR][logstash.agent] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of #, { at line 130, column 20 (byte 2388) after input {\n exec {\n\tcommand => \"/usr/bin/python /home/maksym/PathFolder/pythonpractice/postsample.py\"\n interval => 20\n\t#id => \"myid\"\n\tdata_format => \"json\"\n\n\t}\n request_timeout ", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:42:in `compile_imperative'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:50:in `compile_graph'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:12:in `block in compile_sources'", "org/jruby/RubyArray.java:2486:in `map'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:11:in `compile_sources'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:49:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:167:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:40:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:305:in `block in converge_state'"]}
[2018-08-01T11:43:58,744][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=>9600}
[2018-08-01T11:44:37,468][INFO][logstash.runner] Starting Logstash {"logstash.version"=>"6.3.2"}
[2018-08-01T11:44:39,641][ERROR][logstash.agent] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of #, { at line 130, column 20 (byte 2388) after input {\n exec {\n\tcommand => \"/usr/bin/python /home/maksym/PathFolder/pythonpractice/postsample.py\"\n interval => 20\n\t#id => \"myid\"\n\tdata_format => \"json\"\n\n\t}\n request_timeout ", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:42:in `compile_imperative'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:50:in `compile_graph'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:12:in `block in compile_sources'", "org/jruby/RubyArray.java:2486:in `map'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:11:in `compile_sources'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:49:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:167:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:40:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:305:in `block in converge_state'"]}
[2018-08-01T11:44:40,182][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=>9600}

```

 ![Screenshot%20from%202018-08-01%2011-51-21](https://us1.discourse-cdn.com/elastic/original/3X/e/7/e7a893117f63c2252e471fec3cec840cbd0d5f87.png)

Please tell me what do you mean/how to do that, I thought that I could see an output in Kibana

> What output are you getting from Logstash?

And I see `message=>"Expected one of #, { at line 130, column 20 (byte 2388)`but I don't have 130 lines

---

<div class="post-metadata">

### Author: ![tgaudin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tgaudin/32/32583_2.png) [@tgaudin](https://discuss.elastic.co/u/tgaudin)
#### Post date: [August 1, 2018, 10:05am UTC](https://discuss.elastic.co/t/post-method-http-poller/141940/20 "2018-08-01T10:05:17Z")

</div>

Your `input` configuration is wrong, so the pipeline doesn't start at all.

The `exec` filter takes _either_ `interval` or `schedule` configuration, and doesn't have a `data_format` option.  
Also, you have pieces of your old `http_poller` configuration left, which you should remove.

A correct `input` block would be

```auto
input {
  exec {
    command => "/usr/bin/python /home/maksym/PathFolder/pythonpractice/postsample.py"
    interval => 20
    codec => "json"
  }
}

```

Also, regarding

> [@m.sereda](#):
>
> And I see `message=>"Expected one of #, { at line 130, column 20 (byte 2388)` but I don't have 130 lines

Logstash combines all of your configuration files, so line numbers aren't accurate. You should look at the text of the error, `"after input ... request_timeout "` to know where the error is.

[Next page](https://discuss.elastic.co/t/post-method-http-poller/141940.md?page=2)