Good Day,
How do I send an API KEY ID and an API KEY with http_poller. I am trying to replicate the following curl command which retrieves valid data but with http_poller.
-H 'Accept: application/json' \
-H 'Content-Type: application/json' \
-H 'Authorization: <API_KEY>' -H 'x-abc-auth-id: 2' -H 'Accept-Encoding: br' \
'https://api-test.com/public_api/v1/audits/test_events' \
-d '{
"request_data": {
"search_from": 0,
"search_to": 100,
"sort": {
"field": "timestamp",
"keyword": "asc"
}
}
}'
I have tried a few different variations but basically my logstash code look as a follows:
input {
http_poller {
urls => {
xdr_logs => {
method => post
url => "https://api-test.com/public_api/v1/audits/test_events"
headers => {
"authorization" => "apikey <API-KEY-VALUE>"
"x-abc-auth-id" => "2"
"Accept" => "application/json"
"Content-Type" => "application/json"
"Accept-Encoding" => "*"
}
body => '{
"request_data": {
"search_from": 0,
"search_to": 100,
"sort": {
"field": "timestamp",
"keyword": "asc"
}
}
}'
}
}
request_timeout => 60
schedule => { every => "60s" }
codec => "json"
}
}
It always comes back with a 401 error from the server.
[DEBUG] 2025-03-25 13:49:51.213 [pool-6-thread-1] MainClientExec - Proxy auth state: UNCHALLENGED
[DEBUG] 2025-03-25 13:49:51.215 [pool-6-thread-1] headers - http-outgoing-0 >> POST /api-test.com/public_api/v1/audits/test_events HTTP/1.1
[DEBUG] 2025-03-25 13:49:51.216 [pool-6-thread-1] headers - http-outgoing-0 >> Connection: Keep-Alive
[DEBUG] 2025-03-25 13:49:51.216 [pool-6-thread-1] headers - http-outgoing-0 >> Authorization: API-KEY-VALUE-REMOVED)
[DEBUG] 2025-03-25 13:49:51.216 [pool-6-thread-1] headers - http-outgoing-0 >> Accept: application/json
[DEBUG] 2025-03-25 13:49:51.216 [pool-6-thread-1] headers - http-outgoing-0 >> x-abc-auth-id: 2
[DEBUG] 2025-03-25 13:49:51.216 [pool-6-thread-1] headers - http-outgoing-0 >> Content-Type: application/json
[DEBUG] 2025-03-25 13:49:51.216 [pool-6-thread-1] headers - http-outgoing-0 >> Content-Length: 214
[DEBUG] 2025-03-25 13:49:51.216 [pool-6-thread-1] headers - http-outgoing-0 >> Host: api-test.com
[DEBUG] 2025-03-25 13:49:51.216 [pool-6-thread-1] headers - http-outgoing-0 >> User-Agent: Manticore 0.8.0
[DEBUG] 2025-03-25 13:49:51.216 [pool-6-thread-1] headers - http-outgoing-0 >> Accept-Encoding: gzip,deflate
[DEBUG] 2025-03-25 13:49:51.216 [pool-6-thread-1] wire - http-outgoing-0 >> "POST /api-test.com/public_api/v1/audits/test_events HTTP/1.1[\r][\n]"
[DEBUG] 2025-03-25 13:49:51.216 [pool-6-thread-1] wire - http-outgoing-0 >> "Connection: Keep-Alive[\r][\n]"
[DEBUG] 2025-03-25 13:49:51.217 [pool-6-thread-1] wire - http-outgoing-0 >> "Authorization: <API-KEY-VALUE-REMOVED>)[\r][\n]"
[DEBUG] 2025-03-25 13:49:51.217 [pool-6-thread-1] wire - http-outgoing-0 >> "Accept: application/json[\r][\n]"
[DEBUG] 2025-03-25 13:49:51.218 [pool-6-thread-1] wire - http-outgoing-0 >> "x-abc-auth-id: 2[\r][\n]"
[DEBUG] 2025-03-25 13:49:51.218 [pool-6-thread-1] wire - http-outgoing-0 >> "Content-Type: application/json[\r][\n]"
[DEBUG] 2025-03-25 13:49:51.218 [pool-6-thread-1] wire - http-outgoing-0 >> "Content-Length: 214[\r][\n]"
[DEBUG] 2025-03-25 13:49:51.218 [pool-6-thread-1] wire - http-outgoing-0 >> "Host: api-test.com[\r][\n]"
[DEBUG] 2025-03-25 13:49:51.218 [pool-6-thread-1] wire - http-outgoing-0 >> "User-Agent: Manticore 0.8.0[\r][\n]"
[DEBUG] 2025-03-25 13:49:51.218 [pool-6-thread-1] wire - http-outgoing-0 >> "Accept-Encoding: gzip,deflate[\r][\n]"
[DEBUG] 2025-03-25 13:49:51.218 [pool-6-thread-1] wire - http-outgoing-0 >> "[\r][\n]"
[DEBUG] 2025-03-25 13:49:51.218 [pool-6-thread-1] wire - http-outgoing-0 >> "{[\n]"
[DEBUG] 2025-03-25 13:49:51.218 [pool-6-thread-1] wire - http-outgoing-0 >> " "request_data": {[\n]"
[DEBUG] 2025-03-25 13:49:51.218 [pool-6-thread-1] wire - http-outgoing-0 >> " "search_from": 0,[\n]"
[DEBUG] 2025-03-25 13:49:51.218 [pool-6-thread-1] wire - http-outgoing-0 >> " "search_to": 100,[\n]"
[DEBUG] 2025-03-25 13:49:51.218 [pool-6-thread-1] wire - http-outgoing-0 >> " "sort": {[\n]"
[DEBUG] 2025-03-25 13:49:51.218 [pool-6-thread-1] wire - http-outgoing-0 >> " "field": "timestamp",[\n]"
[DEBUG] 2025-03-25 13:49:51.218 [pool-6-thread-1] wire - http-outgoing-0 >> " "keyword": "asc"[\n]"
[DEBUG] 2025-03-25 13:49:51.219 [pool-6-thread-1] wire - http-outgoing-0 >> " }[\n]"
[DEBUG] 2025-03-25 13:49:51.219 [pool-6-thread-1] wire - http-outgoing-0 >> " }[\n]"
[DEBUG] 2025-03-25 13:49:51.219 [pool-6-thread-1] wire - http-outgoing-0 >> " }"
[DEBUG] 2025-03-25 13:49:51.223 [pool-6-thread-1] wire - http-outgoing-0 << "HTTP/1.1 401 Unauthorized[\r][\n]"
[DEBUG] 2025-03-25 13:49:51.224 [pool-6-thread-1] wire - http-outgoing-0 << "Date: Tue, 25 Mar 2025 13:49:51 GMT[\r][\n]"
[DEBUG] 2025-03-25 13:49:51.224 [pool-6-thread-1] wire - http-outgoing-0 << "Content-Length: 0[\r][\n]"
[DEBUG] 2025-03-25 13:49:51.224 [pool-6-thread-1] wire - http-outgoing-0 << "Connection: keep-alive[\r][\n]"
[DEBUG] 2025-03-25 13:49:51.224 [pool-6-thread-1] wire - http-outgoing-0 << "X-Public-Api-Body: 1[\r][\n]"
[DEBUG] 2025-03-25 13:49:51.224 [pool-6-thread-1] wire - http-outgoing-0 << "Strict-Transport-Security: max-age=31536000; includeSubDomains[\r][\n]"
[DEBUG] 2025-03-25 13:49:51.224 [pool-6-thread-1] wire - http-outgoing-0 << "[\r][\n]"
[DEBUG] 2025-03-25 13:49:51.227 [pool-6-thread-1] headers - http-outgoing-0 << HTTP/1.1 401 Unauthorized
[DEBUG] 2025-03-25 13:49:51.227 [pool-6-thread-1] headers - http-outgoing-0 << Date: Tue, 25 Mar 2025 13:49:51 GMT
[DEBUG] 2025-03-25 13:49:51.227 [pool-6-thread-1] headers - http-outgoing-0 << Content-Length: 0
[DEBUG] 2025-03-25 13:49:51.227 [pool-6-thread-1] headers - http-outgoing-0 << Connection: keep-alive
[DEBUG] 2025-03-25 13:49:51.227 [pool-6-thread-1] headers - http-outgoing-0 << X-Public-Api-Body: 1
[DEBUG] 2025-03-25 13:49:51.227 [pool-6-thread-1] headers - http-outgoing-0 << Strict-Transport-Security: max-age=31536000; includeSubDomains
[DEBUG] 2025-03-25 13:49:51.231 [pool-6-thread-1] MainClientExec - Connection can be kept alive indefinitely
[DEBUG] 2025-03-25 13:49:51.231 [pool-6-thread-1] HttpAuthenticator - Authentication required
[DEBUG] 2025-03-25 13:49:51.231 [pool-6-thread-1] HttpAuthenticator - api-test.com:443 requested authentication