Logstash-input-reqsample, a plugin for randomly generating log data


(Tyler Langlois) #1

I've been setting up some stacks lately for testing/example dashboards and couldn't easily come up with sample data, so I wrote a tiny Ruby library called reqsample and accompanying Logstash input plugin to create randomly generated webserver access logs with statistically (normally distributed) weighted sampling for country of origin IP source addressing, HTTP User-Agent, and a few other fields.

What this effectively means is that you can easily load lots of appropriately-timestamped webserver logs into the stack to test logstash pipelines and kibana visualizations. It's probably easiest to demonstrate this with a short gif:

reqsample-demo

The plugin and library and still pretty basic, but do support some simple configurability such as log count/volume, simulated log streaming, and tweaking the normal distribution curve.

Hopefully this is useful for others, I can see some utility in cases such as short demos to quickly come up with dummy log data with a simple plugin install without needing to dig up/download a log corpus from somewhere (and still get usefully diverse data).

Both the Ruby library and the logstash plugin are on GitHub and I'd love to add features, so any PRs are welcome! :elastic: