Not able to execute http request in Filter with JSON Payload

Elastic Stack Logstash
1

Hi

Not able to execute http request in Filter with JSON Payload
Here how my conf file looks like

rest {
   request => {
     url =>"https://abc.com"       
     method => "post"                  
  headers => {
   	"Authorization" => "%{access}"
   	"Content-Type" => "application/json"
   	}

     params => '{
   "reportRequests" :
   [{
   		"checkid" : "2345678",
   		"dateRanges" : [{
   				"endDate" : "2018-11-12",
   				"startDate" : "2018-11-01"
   			}
   		]
   	}
   ]
}'
   }
   json => true                         
   target => "my_key"                  

 }

This is the error i am getting

`

fieldViolations": [\n {\n "description": "Invalid JSON payload received. Unknown name \"\": Root element must be a message."

The exact body works perfectly in POSTMAN, but here it does not work. Looks like something is missing params/body. Please suggest the right syntax or such feature is not supported?

`

2

Hi

Did anyone try this kind of payload in the params or body?

Thanks

3

Can you post the full raw input?

4

Hi, Thanks for your response. Here is the full sample conf file

input {
http_poller{
urls => {
http_request => {
method => POST
url =>"https://www.xyz.com"
}
}

    }

}

filter {

rest {
request => {
url =>"https://abc.com"
method => "post"
headers => {
"Authorization" => "%{access}"
"Content-Type" => "application/json"
}

 params => '{

"reportRequests" :
[{
"checkid" : "2345678",
"dateRanges" : [{
"endDate" : "2018-11-12",
"startDate" : "2018-11-01"
}
]
}
]
}'
}
json => true
target => "my_key"

}

}

output {

elasticsearch....

}

5

Sorry I mean can you post some raw input lines? Can we see the JSON message you are trying to input?

6

Hi

From the input, i am hitting the URL, wherein i get the token for authorization of REST URL used in filter.

There response from input looks like this

{
    "access": "ya29.GlxdBqDTNQNJ4qHBFQVr3byMTmFofKKrsoZ3tM6XCDQ-W3ysrjOL9bl_4hGwAhirD-YHgtTtFbcUjt6jlsR-dpTxsIn4Hw6Fpnt_r0IAi9wq-",
    "scope": "https://abc.com",
}

This access i go and pass on to the filter section as %{access}

I hope you are looking for the same

Thanks!

7

Your error mentions the field "description" - where does that field come from?

8

Hi

Below in code quote there is the complete error message from the log file.I do not have as such description field/event in source, that description is given by the API service to describe the error.

I think it will be nicer if you have any working conf file with REST Filter which uses method as 'post" and while posting it uses body to post the message to get the response.

Here the body should act as complete json, if you have any working example for REST filter then i think it can solve my problem

Thanks

{
: request = > [ : post, "https://abc.com", {
: headers = > {
"Authorization" = > "ya29.GlxdBqDTNQNJ4qHBFQVr3byMTmFofKKrsoZ3tM6XCDQ-W3ysrjOL9bl_4hGwAhirD-YHgtTtFbcUjt6jlsR-dpTxsIn4Hw6Fpnt_r0IAi9wq-"
},
: body = > ""{\n\t'reportRequests' :\n\t[{\n\t\t\t'checkid' : '2345678',\n\t\t\t'dateRanges' : [{\n\t\t\t\t\t'endDate' : '2018-11-12',\n\t\t\t\t\t'startDate' : '2018-11-01'\n\t\t\t\t}\n\t\t\t],}""
}
],
: json = > true,
: code = > 400,
: body = > "{\n "error": {\n "code": 400,\n "message": "Invalid JSON payload received. Unknown name \"\": Root element must be a message.",\n "status": "INVALID_ARGUMENT",\n "details": [\n {\n "@type": "abc.com.rpc.BadRequest",\n "fieldViolations": [\n {\n "description": "Invalid JSON payload received. Unknown name \"\": Root element must be a message."\n }\n ]\n }\n ]\n }\n}\n",
: client_error = > nil
}

9

Ah, it looks like your body is incorrect, are you manually specifying the line breaks in the body?

10

Yes from the error response i am getting that body is wrong.
But what is the correct way to define the body with rest filter?

As asked in my previous message, do you have any working rest filter conf file,which helps to define the right body...

Thanks!

11

This is how you do a HTTP-Poller request - Note your header will not work because you do not have a field called %{access} yet - you need to input this manually

input {
  http_poller {
    urls => {
      "request" => {
        method => POST
        url => "https://abc.com"
        headers => {
"Authorization" => "%{access}"
"Content-Type" => "application/json"
        }
      body => '{"password":"myPassword","userName":"yourUsername"}'
     }
    }
    request_timeout => 10
    codec => "json"
    metadata_target => "http_poller_metadata"
  }
}
12

I think there is a huge disconnect.
My http_poller have no issues, i have used this input in many cases.

I have problem with rest filter, which is not a official filter, i got it from Github. I am looking for example on rest filter

Please let me know if you have a working example of rest filter with method as post and message as json body

Thanks

13

Ah I see, I do not have any experience of this plugin unfortunately. I see that you are trying to do a nested body which does not look to be supported, I think this is the issue.

14

okay. I feel somehow it should be possible. By looking the filter plugin description, looks like it is possible. But not sure why its not working

I am going to take a different route by pinging the URL with ruby filter.
Thanks for your time

1 Like
15

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.