I got your point and referring Billing API documentation,
Now I am testing billing API for "Get costs to overview for the organization"
Config file and Output attached. Getting error of Invalid URL or root unauthenticated.
Although I search for HTTP poller authentication, got this link, but didn't get much idea how to process it in URL.
https://discuss.elastic.co/t/http-poller-authentication-encode-token/143532
This config setting is just for testing purpose-
input{
http_poller {
type => "billing"
urls => {
url => "https://api.elastic-cloud.com/api/v1/billing/costs/<orgid>-HAuthorization:ApiKey<my api key>"
}
request_timeout => 60
schedule => { every => "5s"}
codec => "json"
}
}
output {
stdout {
codec => rubydebug
}
#elasticsearch {
#host ip to be changed to new test system
# hosts => ["<hostaname>"]
#index => "<Billing>"
# user => "<username>"
# password => "<password>"
#}
}
Output
-bash-4.2$ sudo bin/logstash -f /etc/logstash/conf.d/billing1.conf
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in ve rsion 9.0 and will likely be removed in a future release.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.headius.backport9.modules.Modules (fil e:/usr/share/logstash/logstash-core/lib/jars/jruby-complete-9.2.11.1.jar) to met hod sun.nio.ch.NativeThread.signal(long)
WARNING: Please consider reporting this to the maintainers of com.headius.backpo rt9.modules.Modules
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflect ive access operations
WARNING: All illegal access operations will be denied in a future release
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/conf ig or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.pr operties. Using default config which logs errors to the console
[WARN ] 2021-08-10 07:49:26.800 [LogStash::Runner] multilocal - Ignoring the 'pi pelines.yml' file because modules or command line options are specified
[INFO ] 2021-08-10 07:49:26.828 [LogStash::Runner] runner - Starting Logstash {" logstash.version"=>"7.8.0", "jruby.version"=>"jruby 9.2.11.1 (2.5.7) 2020-03-25 b1f55b1a40 OpenJDK 64-Bit Server VM 11.0.12+7-LTS on 11.0.12+7-LTS +indy +jit [l inux-x86_64]"}
[INFO ] 2021-08-10 07:49:35.000 [Converge PipelineAction::Create<main>] Reflections - Reflections took 328 ms to scan 1 urls, producing 21 keys and 41 values
[INFO ] 2021-08-10 07:49:37.791 [[main]-pipeline-manager] javapipeline - Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>500, "pipeline.sources"=>["/etc/logstash/conf.d/billing1.conf"], :thread=>"#<Thread:0x54177e5c run>"}
[INFO ] 2021-08-10 07:49:41.357 [[main]-pipeline-manager] http_poller - Registering http_poller Input {:type=>"billing", :schedule=>{"every"=>"5s"}, :timeout=>nil}
[INFO ] 2021-08-10 07:49:41.459 [[main]-pipeline-manager] javapipeline - Pipeline started {"pipeline.id"=>"main"}
[INFO ] 2021-08-10 07:49:41.769 [Agent thread] agent - Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[INFO ] 2021-08-10 07:49:43.077 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9600}
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/awesome_print-1.7.0/lib/awesome_print/formatters/base_formatter.rb:31: warning: constant ::Fixnum is deprecated
{
"type" => "billing",
"@version" => "1",
"@timestamp" => 2021-08-10T07:49:51.423Z,
"errors" => [
[0] {
"message" => "The resource requires authentication, which was not supplied with the request",
"code" => "root.unauthenticated"
}
]
}