# Stdout cause syslog file to be to big

**URL:** https://discuss.elastic.co/t/stdout-cause-syslog-file-to-be-to-big/360107
**Category:** Logstash
**Created:** [May 23, 2024, 10:48pm UTC](https://discuss.elastic.co/t/stdout-cause-syslog-file-to-be-to-big/360107 "2024-05-23T22:48:39Z")
**Posts on this page:** 9
**Page:** 1

<div class="post-metadata">

### Author: ![Honestabe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/honestabe/32/122763_2.png) [@Honestabe](https://discuss.elastic.co/u/Honestabe)
#### Post date: [May 23, 2024, 10:48pm UTC](https://discuss.elastic.co/t/stdout-cause-syslog-file-to-be-to-big/360107/1 "2024-05-23T22:48:39Z")

</div>

sometime I have to use

`stdout { codec => rubydebug { metadata => true } } `

to do some troubleshooting on a pipeline.

I noticed that when I do the syslog file is taking up GB of storage.

is there a way to have it write to a separate file instead so that I can easy delete it when I am done with it?

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [May 23, 2024, 11:37pm UTC](https://discuss.elastic.co/t/stdout-cause-syslog-file-to-be-to-big/360107/2 "2024-05-23T23:37:20Z")

</div>

Instead of using a stdout output, use a file output

```
file { codec => rubydebug { metadata => true } path => "/path/to/log.log" }

```

If the output keeps the file open then deleting it will not free up disk space.

---

<div class="post-metadata">

### Author: ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)
#### Post date: [May 24, 2024, 9:30am UTC](https://discuss.elastic.co/t/stdout-cause-syslog-file-to-be-to-big/360107/3 "2024-05-24T09:30:04Z")

</div>

Just to add...

> path =\> "/path/to/log.log"

Since you have the output files in GB, maybe is useful to use files on daily base.  
`path => "/path/to/log_%{+YYYY-MM-dd}.log"`

---

<div class="post-metadata">

### Author: ![Honestabe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/honestabe/32/122763_2.png) [@Honestabe](https://discuss.elastic.co/u/Honestabe)
#### Post date: [May 24, 2024, 4:20pm UTC](https://discuss.elastic.co/t/stdout-cause-syslog-file-to-be-to-big/360107/4 "2024-05-24T16:20:21Z")

</div>

That helped a lot. Thanks ! Is there a way to name based on the pipeline and host it is coming from? Or have it auto generate sub folders based on them?

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [May 24, 2024, 5:10pm UTC](https://discuss.elastic.co/t/stdout-cause-syslog-file-to-be-to-big/360107/5 "2024-05-24T17:10:05Z")

</div>

> [@Honestabe](#):
>
> Is there a way to name based on the pipeline and host it is coming from?

You can reference any field on the event in the path option of the file output, although there are [a couple of restrictions](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-file.html#plugins-outputs-file-path).

You could use a ruby filter to add the [pipeline\_id](https://discuss.elastic.co/t/logstash-access-name-of-current-pipeline-within-cfg/195786/2) and [hostname](https://discuss.elastic.co/t/adding-logstash-ip-address-field-when-processing-filter/270055/3) to the event. I would add them inside the [@metadata] field, which you can reference in the output using %{}, but they will not be sent to the destination with the rest of the fields on the event.

Something like

```
ruby {
    init => 'require "socket"'
    code => '
        # Or save in @class variable in init to avoid repeated call
        event.set("[@metadata][hostname]", Socket.gethostname)

        event.set("[@metadata][pipeline_id]", execution_context.pipeline.pipeline_id)"
    '
 }

```

...

```
file {
    path => "/TheFixedBit/%{[@metadata][hostname]/%{@metadata][pipeline_id]}-%{+YYYY-MM-dd}.log"
    ...

```

---

<div class="post-metadata">

### Author: ![Honestabe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/honestabe/32/122763_2.png) [@Honestabe](https://discuss.elastic.co/u/Honestabe)
#### Post date: [May 24, 2024, 9:47pm UTC](https://discuss.elastic.co/t/stdout-cause-syslog-file-to-be-to-big/360107/6 "2024-05-24T21:47:23Z")

</div>

will this give me the host name of the logstash server or the sever the beats agent is send data from?

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [May 24, 2024, 10:01pm UTC](https://discuss.elastic.co/t/stdout-cause-syslog-file-to-be-to-big/360107/7 "2024-05-24T22:01:15Z")

</div>

It would be the hostname of the logstash server. If you want the hostname of the beat then have the beat add [host metadata](https://www.elastic.co/guide/en/beats/filebeat/current/add-host-metadata.html) to the event.

---

<div class="post-metadata">

### Author: ![Honestabe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/honestabe/32/122763_2.png) [@Honestabe](https://discuss.elastic.co/u/Honestabe)
#### Post date: [May 24, 2024, 10:34pm UTC](https://discuss.elastic.co/t/stdout-cause-syslog-file-to-be-to-big/360107/8 "2024-05-24T22:34:30Z")

</div>

I ended up using this

```auto
file { codec => rubydebug { metadata => true } path => "/logstash/logs/file_output/%{[@metadata][beat]}/%{[host.hostname]}.log" }

```

It worked but it returned {[host.hostname]}.log as the file name and not the actual host name

---

<div class="post-metadata">

### Author: ![Honestabe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/honestabe/32/122763_2.png) [@Honestabe](https://discuss.elastic.co/u/Honestabe)
#### Post date: [May 24, 2024, 10:38pm UTC](https://discuss.elastic.co/t/stdout-cause-syslog-file-to-be-to-big/360107/9 "2024-05-24T22:38:24Z")

</div>

```auto
file { codec => rubydebug { metadata => true } path => "/logstash/logs/file_output/%{[@metadata][beat]}/%{[host][hostname]}.log" }

```

This one worked.
