# How to show the pipeline conf source filename of error in logs

**URL:** <https://discuss.elastic.co/t/how-to-show-the-pipeline-conf-source-filename-of-error-in-logs/302201>\
**Category:** Logstash\
**Created:** [April 12, 2022, 9:24am UTC](https://discuss.elastic.co/t/how-to-show-the-pipeline-conf-source-filename-of-error-in-logs/302201 "2022-04-12T09:24:31Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![Mahf](https://avatars.discourse-cdn.com/v4/letter/m/50afbb/32.png) [@Mahf](https://discuss.elastic.co/u/Mahf)\
**Post date:** [April 12, 2022, 9:24am UTC](https://discuss.elastic.co/t/how-to-show-the-pipeline-conf-source-filename-of-error-in-logs/302201/1 "2022-04-12T09:24:31Z")

</div>

Hello,  
We have multiple logstash conf files that we run simultaneously, the problem is that when we have many errors come from several pipelines, so we can't tell which one is the source of the error, sometimes it's easy to tell but with time it is becoming more complicated.

Example of logs

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/5/4/543c5b7c2dc012d4471a58fb5b7fc8df3adc09fd.png)

Can you tell me if its possible to show the name (or id) of the pipeline as info in logs? and how to do it?  
Thanks in advance.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 12, 2022, 2:39pm UTC](https://discuss.elastic.co/t/how-to-show-the-pipeline-conf-source-filename-of-error-in-logs/302201/2 "2022-04-12T14:39:39Z")

</div>

Use the pipeline.separate\_logs option in logstash.yml.

---

<div class="post-metadata">

**Author:** ![zx8086](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zx8086/32/94917_2.png) [@zx8086](https://discuss.elastic.co/u/zx8086)\
**Post date:** [April 12, 2022, 2:44pm UTC](https://discuss.elastic.co/t/how-to-show-the-pipeline-conf-source-filename-of-error-in-logs/302201/3 "2022-04-12T14:44:22Z")

</div>

Logstash.yml

```auto
# ------------ Other Settings --------------
#
# Where to find custom plugins
# path.plugins: []
#
# Flag to output log lines of each pipeline in its separate log file. Each log filename contains the pipeline.name
# Default is false
pipeline.separate_logs: true

```

---

<div class="post-metadata">

**Author:** ![Mahf](https://avatars.discourse-cdn.com/v4/letter/m/50afbb/32.png) [@Mahf](https://discuss.elastic.co/u/Mahf)\
**Post date:** [April 12, 2022, 4:17pm UTC](https://discuss.elastic.co/t/how-to-show-the-pipeline-conf-source-filename-of-error-in-logs/302201/4 "2022-04-12T16:17:39Z")

</div>

I tried that but I got this error:

```auto
[2022-04-12T18:13:31,966+02:00][FATAL][logstash.runner] An unexpected error occurred! {:error=>#<ArgumentError: Setting "pipeline.separate_logs" hasn't been registered>, :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/settings.rb:36:in `get_setting'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:69:in `set_value'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:88:in `block in merge'", "org/jruby/RubyHash.java:1343:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:88:in `merge'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:137:in `validate_all'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:278:in `execute'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/clamp-0.6.5/lib/clamp/command.rb:67:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:237:in `run'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/clamp-0.6.5/lib/clamp/command.rb:132:in `run'", "/usr/share/logstash/lib/bootstrap/environment.rb:73:in `<main>'"]}
[2022-04-12T18:13:32,046+02:00][ERROR][org.logstash.Logstash] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit

```

is it the version of logstash that doesn't support this or an extra config that I should add, if it's the version, are there any other options to show the name / id of the pipeline in the global log file,  
Thanks!

---

<div class="post-metadata">

**Author:** ![zx8086](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zx8086/32/94917_2.png) [@zx8086](https://discuss.elastic.co/u/zx8086)\
**Post date:** [April 12, 2022, 4:26pm UTC](https://discuss.elastic.co/t/how-to-show-the-pipeline-conf-source-filename-of-error-in-logs/302201/5 "2022-04-12T16:26:47Z")

</div>

Can you share the full logstash.yml?

---

<div class="post-metadata">

**Author:** ![Mahf](https://avatars.discourse-cdn.com/v4/letter/m/50afbb/32.png) [@Mahf](https://discuss.elastic.co/u/Mahf)\
**Post date:** [April 12, 2022, 4:30pm UTC](https://discuss.elastic.co/t/how-to-show-the-pipeline-conf-source-filename-of-error-in-logs/302201/6 "2022-04-12T16:30:15Z")

</div>

```auto
---
# ------------ Node identity ------------
node.name: "host-logstash"

# ------------ Data path ------------------
# Which directory should be used by logstash and its plugins
# for any persistent needs. Defaults to LOGSTASH_HOME/data
path.data: /var/lib/logstash

# ------------ Queuing Settings --------------
queue.type: memory

# ------------ Metrics Settings --------------
# Bind address for the metrics REST endpoint
http.host: 0.0.0.0
#
# Bind port for the metrics REST endpoint, this option also accept a range
# (9600-9700) and logstash will pick up the first available ports.
http.port: 9600-9700
#
# ------------ Debugging Settings --------------
# Options for log.level:
# * fatal
# * error
# * warn
# * info (default)
# * debug
# * trace
#
log.level: trace
path.logs: /var/log/logstash

pipeline.separate_logs: true
#
# ------------ Other Settings --------------
xpack.monitoring.enabled: false
xpack.monitoring.elasticsearch.url: ["host-elasticsearch.intra:9200"]
xpack.monitoring.elasticsearch.sniffing: false
xpack.monitoring.elasticsearch.username: "username"
xpack.monitoring.elasticsearch.password: "password"
xpack.monitoring.elasticsearch.ssl.ca: /etc/logstash/ssl/ca/Bundle.crt
xpack.monitoring.elasticsearch.ssl.keystore.path: /etc/logstash/ssl/logstash/logstash.p12
xpack.monitoring.elasticsearch.ssl.keystore.password: "ssl_password"

```

---

<div class="post-metadata">

**Author:** ![zx8086](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zx8086/32/94917_2.png) [@zx8086](https://discuss.elastic.co/u/zx8086)\
**Post date:** [April 12, 2022, 4:35pm UTC](https://discuss.elastic.co/t/how-to-show-the-pipeline-conf-source-filename-of-error-in-logs/302201/7 "2022-04-12T16:35:26Z")

</div>

Was the service restarted or just the config reloaded? There should be more info in the logs.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 12, 2022, 4:52pm UTC](https://discuss.elastic.co/t/how-to-show-the-pipeline-conf-source-filename-of-error-in-logs/302201/8 "2022-04-12T16:52:01Z")

</div>

> [@Mahf](#):
>
> is it the version of logstash that doesn't support this

Per-pipeline logging was added in [7.5.0](https://www.elastic.co/guide/en/logstash/7.5/logstash-7-5-0.html). If you are running something older you cannot use it.

---

<div class="post-metadata">

**Author:** ![Mahf](https://avatars.discourse-cdn.com/v4/letter/m/50afbb/32.png) [@Mahf](https://discuss.elastic.co/u/Mahf)\
**Post date:** [April 12, 2022, 4:52pm UTC](https://discuss.elastic.co/t/how-to-show-the-pipeline-conf-source-filename-of-error-in-logs/302201/9 "2022-04-12T16:52:19Z")

</div>

We restart the service each time we change the config

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 10, 2022, 4:53pm UTC](https://discuss.elastic.co/t/how-to-show-the-pipeline-conf-source-filename-of-error-in-logs/302201/10 "2022-05-10T16:53:09Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
