Sending post request via nodejs to logstash

Elastic Stack Logstash
1

Hi everyone!

I want to sent http post requests via nodejs code to a logstash with a basic configuration file :

input { http { } }
output {
elasticsearch { hosts => ["localhost:9200"] }
stdout { codec => rubydebug }
}

and here is my nodejs script

var querystring = require('querystring');
var http = require ('http');

var options = {
  host: "192.168.69.112",
  port: 8080,
  path: "/my-path/my-device/1",
  method: "POST",
  headers: {
    "Content-type": "application/x-www-form-urlencoded;charset=UTF-8'",
    "Accept": "text/plain",
  }
};

var postData = querystring.stringify({
  "message":"test"
});  

var post_option = http.request(options, function(res){
  res.setEncoding("utf8");
  res.on("data", function(chunk){
    console.log("BODY: ${chunk}");
  });
  res.on('end', function(finish) {
    console.log('No more data in response.');
    console.log(postData.toString());
  });
}); 
post_option.write(postData);
post_option.end();

Each time I send a request, It is recognized get an error on logstash :

2017-03-20 14:33:47 +0100: HTTP parse error, malformed request (): #<Puma::HttpParserError: Invalid HTTP format, parsing fails.>
2017-03-20 14:33:47 +0100: ENV: {"rack.version"=>[1, 3], "rack.errors"=>#<IO:fd 2>, "rack.multithread"=>true, "rack.multiprocess"=>false, "rack.run_once"=>false, "SCRIPT_NAME"=>"", "QUERY_STRING"=>"", "SERVER_PROTOCOL"=>"HTTP/1.1", "SERVER_SOFTWARE"=>"2.16.0", "GATEWAY_INTERFACE"=>"CGI/1.2"}
---
{
       "headers" => {
                   "http_accept" => "text/plain",
                  "content_type" => "application/x-www-form-urlencoded;charset=UTF-8'",
                  "request_path" => "/my-path/my-device/1",
                  "http_version" => "HTTP/1.1",
               "http_connection" => "keep-alive",
        "http_transfer_encoding" => "chunked",
                "request_method" => "POST",
                     "http_host" => "192.168.69.112:8080",
                   "request_uri" => "/my-path/my-device/1"
    },
    "@timestamp" => 2017-03-20T13:33:47.467Z,
      "@version" => "1",
          "host" => "192.168.69.241",
       "message" => ""
}

plus the message field is empty. I don't understand since my code is able to send post requests to a server.

I've tried this example :

And the curl commands are working, so I guess the problem comes from the nodejs script but I can't find it... Can you help me?

Thanks

2

Could you use Wireshark or a similar tool to record exactly what your NodeJS code is sending?

1 Like
3

Hi,

Here is what I get :

nislam@fabrice-HP-EliteDesk-800-G1-USDT:~/logstash-5.2.2$ nc -l 8080

PUT /my-path/my-device/1 HTTP/1.1

Content-type: application/json

Accept: text/plain

Host: 192.168.69.112:8080

Connection: keep-alive

Transfer-Encoding: chunked

3c

{"data":"connected, subscribing\n","log":3,"message":"test"}

0

with my program. for some reason I can't get rid of the chunked transfer encoding. Also, will it work if I keep alive my connection?

when I'm using the curl of the link i get :

nislam@fabrice-HP-EliteDesk-800-G1-USDT:~/logstash-5.2.2$ nc -l 8080

PUT /twitter/tweet/1 HTTP/1.1

User-Agent: curl/7.35.0

Host: 192.168.69.112:8080

Accept: /

content-type: application/json

Content-Length: 110

{

"user" : "kimchy",

"post_date" : "2009-11-15T14:12:12",

"message" : "trying out Elasticsearch"

}

Thanks

4

Maybe it's the chunking that isn't supported by the http input's HTTP library? I don't know what's going on here.

5

Hi,

I've finally solved my problem, in nc the http data were sended chunked and logstash couldn't parse it correctly, i just added the following header :
"Transfer-Encoding":""

thanks

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.