Is there a way to develop my owner filebeat to collect data via calling Restful API

Elastic Stack Beats
1

I want to develop my own filebeat which receives data by regularly calling Restful API provided by other services. Similar to the way AWS module pulls data from S3 or VPC flows.
It'd be deeply appreciated if anyone can point me to a development guide. Or let me know how Elastic otherwise carries out this type of job differently.

Thanks

2

Hi @Eorchid,

I achieved this kind of job using logstash, and here is the guide for it.

The way it works, logstash will call the designed API endpoint depending on the schedule to get the data. It also support basic authentication.

to be note, if the endpoint is using certificate, logstash need to recognize it first and you need to configure that in the logstash "jvm.options" using this line

-Djavax.net.ssl.trustStore=/your/path/to/certs.jks
-Djavax.net.ssl.trustStorePassword=yourtruststorepassword
3

Hi, I would recommend looking at the CEL Common Expression Language input | Filebeat Reference [8.15] | Elastic and HTTPJSON HTTP JSON input | Filebeat Reference [8.15] | Elastic inputs