Post data to logstash using http input

Elastic Stack Logstash
1

I am current trying to do a JavaScript post to Logstash by using a tcp input.

JavaScript Post

 xhr = new XMLHttpRequest();
 var url = "http://localhost:5043";
 xhr.open("POST", url, true);
 xhr.setRequestHeader("Content-type", "application/json");
 var data = JSON.stringify({"test" : hello});
 xhr.send(data);

Logstash config file

input {
	tcp {
		port => 5043
	}
}

filter{
}

output {
  stdout  { 
		codec => rubydebug
	}
}

Output in console

{
       "message" => "OPTIONS / HTTP/1.1\r",
      "@version" => "1",
    "@timestamp" => "2016-12-15T09:58:54.611Z",
          "host" => "0:0:0:0:0:0:0:1",
          "port" => 55867,
  
}
{
       "message" => "Host: localhost:5043\r",
      "@version" => "1",
    "@timestamp" => "2016-12-15T09:58:54.620Z",
          "host" => "0:0:0:0:0:0:0:1",
          "port" => 55867,
 
}
{
       "message" => "Connection: keep-alive\r",
      "@version" => "1",
    "@timestamp" => "2016-12-15T09:58:54.621Z",
          "host" => "0:0:0:0:0:0:0:1",
          "port" => 55867,
  
}
{
       "message" => "Access-Control-Request-Method: POST\r",
      "@version" => "1",
    "@timestamp" => "2016-12-15T09:58:54.622Z",
          "host" => "0:0:0:0:0:0:0:1",
          "port" => 55867,
    
}
{
       "message" => "Origin: http://atgdev11\r",
      "@version" => "1",
    "@timestamp" => "2016-12-15T09:58:54.623Z",
          "host" => "0:0:0:0:0:0:0:1",
          "port" => 55867,
      
}
{
       "message" => "User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36\r",
      "@version" => "1",
    "@timestamp" => "2016-12-15T09:58:54.626Z",
          "host" => "0:0:0:0:0:0:0:1",
          "port" => 55867,
      
}
{
       "message" => "Access-Control-Request-Headers: content-type\r",
      "@version" => "1",
    "@timestamp" => "2016-12-15T09:58:54.634Z",
          "host" => "0:0:0:0:0:0:0:1",
          "port" => 55867,
    
}
{
       "message" => "Accept: */*\r",
      "@version" => "1",
    "@timestamp" => "2016-12-15T09:58:54.651Z",
          "host" => "0:0:0:0:0:0:0:1",
          "port" => 55867,
   
}
{
       "message" => "Referer: http://test/Welcome.jsp\r",
      "@version" => "1",
    "@timestamp" => "2016-12-15T09:58:54.653Z",
          "host" => "0:0:0:0:0:0:0:1",
          "port" => 55867,
  
}
{
       "message" => "Accept-Encoding: gzip, deflate, sdch, br\r",
      "@version" => "1",
    "@timestamp" => "2016-12-15T09:58:54.719Z",
          "host" => "0:0:0:0:0:0:0:1",
          "port" => 55867,

}
{
       "message" => "Accept-Language: en-US,en;q=0.8\r",
      "@version" => "1",
    "@timestamp" => "2016-12-15T09:58:54.720Z",
          "host" => "0:0:0:0:0:0:0:1",
          "port" => 55867,
   
}

I cant seem to see my json data {"test" : hello} passing into logstash could there be something wrong with my logstash.config file ? Please help

2 Likes
2

The TCP input allows data to be sent across a TCP connection. It does not implement or parse the HTTP protocol, so the results you are seeing are expected. Try using the http input plugin instead.

3

@Christian_Dahlqvist thanks for your fast response. I changed it to

input {
http {
port => 5043
}
}

I get the following error is broswser : XMLHttpRequest cannot load http://localhost:5043/. Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://test/' is therefore not allowed access.

1 Like
4

do i need to use :

response_headers {
"Content-Type" => ""Content-type", "application/json"",
"Access-Control-Allow-Origin" => "*",
}

5

I am sorry, but I do unfortunately not have time to dig into this at the moment.

6

Its ok @Christian_Dahlqvist , Thank you for your help. If i do find a way to make this work i will update this thread.

7

@adhikz can you post the request you're doing to logstash so I can replicate and investigate?

8

@jsvd thanks for your response. Here is the POST request sent from my browser sent to logstash :

 xhr = new XMLHttpRequest();
 var url = "http://localhost:5043";
 xhr.open("POST", url, true);
 xhr.setRequestHeader("Content-type", "application/json");
 var data = JSON.stringify({"test" : hello});
 xhr.send(data);

I basically expect to see "hello" pass into the message in logstash

9

This is a classic CORS issue where it's the client that's directly executing a request on a different domain of the website..on the logstash side you can work around it with:

input {
  http {
    port => 5043
    response_headers => {
      "Access-Control-Allow-Origin" => "*"
      "Content-Type" => "text/plain"
      "Access-Control-Allow-Headers" => "Origin, X-Requested-With, Content-Type, Accept"
    }
  }
}

the (recommended) alternative is to set up the flow so that the client will request some endpoint on the server side which in turn calls the logstash endpoint. This avoid exposing logstash directly to end users/clients.

more on this issue: https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS, http://stackoverflow.com/questions/35588699/response-to-preflight-request-doesnt-pass-access-control-check-angularjs

2 Likes
10

Thank you @jsvd , im going to try and do this now and see if it works, I will get back to you with an update. I really appreciate your time and effort.

11

Yay! :grin: its working now.

Heres what i was doing wrong:

I had the follwoing :

response_headers {
"Content-Type" => ""Content-type", "application/json"",
"Access-Control-Allow-Origin" => "*",
}

turns out i was missing "Access-Control-Allow-Headers" => "Origin, X-Requested-With, Content-Type, Accept" in the response header.

"Access-Control-Allow-Headers" => "Origin, X-Requested-With, Content-Type, Accept"

Thanks so much @jsvd :slight_smile:

2 Likes
12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.