CICD pipeline for testing the logstash config


I have my logstash config in gitlab, from where I deploy it on my production machines. As the config grows, and others contribute to it, the risk of getting errors is becoming bigger.
Therefore, I would like to implement a gitlab pipeline which does 2 things (not counting the deploy):

  1. Test the syntax
  2. test all the filters by passing some sample log data to it, and check the filter by looking to what it prints to stdout.

For the first point, I have already a solution, but it's the second one which causes me problems.
I run my pipeline in a logstash container, but since I have in filters some Elasticsearch things, like adding tags, it tries to connect to an eleasticsearch instance. This of course fails.

Now the questions I have:

  • Is there a way to avoid that logstash tries to reach out to Elasticsearch (I did not add Elasticsearch in the output section)?
  • Since in production I use filebeat to ship my logs to logstash, can I take a sample of the filebeat output (which I printed to stdout, and saved as mock data in my git repo), and use the http plugin to pass it to logstash?

Tnx in advance!


This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.