Configure logstash input from database values?

Elastic Stack Logstash
1

I've got a dynamic number of sites. There is a RESTful api on each site that is used to consume their logs. The RESTful endpoint for each site is stored in a single DB on server x. I want to create a logstash input plugin to consume the logs from each of these sites on server x, filter them and output to elasticsearch.
I cant have a logstash forwarder on the sites.

Is it possible for an input plugin to read the endpoint data from a db or service or whatever, and then consume all of the logs for the sites.

2

Sure, a custom input plugin can read its configuration from any source. However, you can't pass that information to other input plugins. What you can do is wrap/subclass the input plugin(s) you want to use and thereby pass the dynamic configuration to the underlying code. How easy this actually is depends on the input plugin.

3

Thanks Magnus. So for example a single instance of logstash can have a single custom input plugin that reads its configuration from a DB, and then consume 20 logs from 20 different endpoints? I don't think I'd need to use any other input plugins although I'm new to logstash so maybe I don't understand enough.

4

You're on the right track.