# Same config file for test and development

**URL:** <https://discuss.elastic.co/t/same-config-file-for-test-and-development/279263>\
**Category:** Logstash\
**Created:** [July 21, 2021, 12:47pm UTC](https://discuss.elastic.co/t/same-config-file-for-test-and-development/279263 "2021-07-21T12:47:14Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Loulou](https://avatars.discourse-cdn.com/v4/letter/l/82dd89/32.png) [@Loulou](https://discuss.elastic.co/u/Loulou)\
**Post date:** [July 21, 2021, 12:47pm UTC](https://discuss.elastic.co/t/same-config-file-for-test-and-development/279263/1 "2021-07-21T12:47:14Z")

</div>

Hi,  
What I am trying to do is having the same logstash.conf file for my test and deployment/shipping. However for testing I want the output to go to my local elastic search, and for deployment to a http address.  
I tried using an environment variable into a condition to check if the file is used for testing or not, and then redirect the output accordingly, but environment variables don't seem to be working inside condition.  
I also tried using metadata with environment variables, see this example : [https://github.com/elastic/logstash/issues/5115#issuecomment-214283550](https://github.com/elastic/logstash/issues/5115#issuecomment-214283550)  
but it did'nt work either.  
Do you know if there is a solution for this or do I have to make 2 separate configuration files?

Thanks

---

<div class="post-metadata">

**Author:** ![AquaX](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aquax/32/92006_2.png) [@AquaX](https://discuss.elastic.co/u/AquaX)\
**Post date:** [July 21, 2021, 2:35pm UTC](https://discuss.elastic.co/t/same-config-file-for-test-and-development/279263/2 "2021-07-21T14:35:58Z")

</div>

I would do this by having a separate configuration files for just your output block.

output\_test.conf

```auto
output {
     elasticsearch {
      ...define parameters elasticsearch
     }
}

```

output\_deployment.conf

```auto
output {
     http {
      ...define parameters
     }
}

```

This way your test system can have just the test output configuration file and your production system can have your other output file. Just need to define this in the pipelines.yml

---

<div class="post-metadata">

**Author:** ![Loulou](https://avatars.discourse-cdn.com/v4/letter/l/82dd89/32.png) [@Loulou](https://discuss.elastic.co/u/Loulou)\
**Post date:** [July 21, 2021, 3:01pm UTC](https://discuss.elastic.co/t/same-config-file-for-test-and-development/279263/3 "2021-07-21T15:01:51Z")

</div>

Thanks I'll give it a try. I never worked with pipelines.yml before, do you have an idea of what it would look like to define those .conf files?

---

<div class="post-metadata">

**Author:** ![AquaX](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aquax/32/92006_2.png) [@AquaX](https://discuss.elastic.co/u/AquaX)\
**Post date:** [July 21, 2021, 3:03pm UTC](https://discuss.elastic.co/t/same-config-file-for-test-and-development/279263/4 "2021-07-21T15:03:21Z")

</div>

You should read this article.

> **[Multiple Pipelines | Logstash Reference \[7.13\] | Elastic](https://www.elastic.co/guide/en/logstash/current/multiple-pipelines.html)**

Essentially you can define any number of pipelines (flow) for your data to take.

---

<div class="post-metadata">

**Author:** ![Loulou](https://avatars.discourse-cdn.com/v4/letter/l/82dd89/32.png) [@Loulou](https://discuss.elastic.co/u/Loulou)\
**Post date:** [July 21, 2021, 3:46pm UTC](https://discuss.elastic.co/t/same-config-file-for-test-and-development/279263/5 "2021-07-21T15:46:32Z")

</div>

Thanks for the article, I have a better understanding of pipelines now!  
However I still don't understand how I can use them to define only the output block. They are running at the same time am I correct?  
Because what I would need is that the data would go first into the common logstash.conf file (used for test and development) and the go to either of output files, depending on the situation. Is there a way to define that inside the pipelines.yml? Or maybe I should have 2 pipelines.yml file for each situation (test & deployment)?

Thanks for your time!

---

<div class="post-metadata">

**Author:** ![AquaX](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aquax/32/92006_2.png) [@AquaX](https://discuss.elastic.co/u/AquaX)\
**Post date:** [July 21, 2021, 5:20pm UTC](https://discuss.elastic.co/t/same-config-file-for-test-and-development/279263/6 "2021-07-21T17:20:10Z")

</div>

A pipeline will not start if the configuration file does not exist.  
So on your test system you can have only the output\_test.conf file and on your production system you can only have your output\_production.conf file.  
Or you can comment out the pipeline in the pipelines.yml file depending on the system.

Alternatively, can you put something in your incoming data that says

deployment.type:"test"  
or  
deployment.type:"prod" ?

Then you can include an if statement in your output to check if that field exists and output to the right place based on the incoming data.

---

<div class="post-metadata">

**Author:** ![Loulou](https://avatars.discourse-cdn.com/v4/letter/l/82dd89/32.png) [@Loulou](https://discuss.elastic.co/u/Loulou)\
**Post date:** [July 22, 2021, 8:15pm UTC](https://discuss.elastic.co/t/same-config-file-for-test-and-development/279263/7 "2021-07-22T20:15:50Z")

</div>

I finally found a simpler way to do this. I stayed with your idea of having 2 output files, and then I used logstash \*.conf to concatenate my common config file with either of the output files!  
Thanks for your help!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 19, 2021, 8:16pm UTC](https://discuss.elastic.co/t/same-config-file-for-test-and-development/279263/8 "2021-08-19T20:16:45Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
