# Logstash output and multiple destinations (elasticsearch and local file)

**URL:** <https://discuss.elastic.co/t/logstash-output-and-multiple-destinations-elasticsearch-and-local-file/58895>\
**Category:** Logstash\
**Created:** [August 25, 2016, 6:58am UTC](https://discuss.elastic.co/t/logstash-output-and-multiple-destinations-elasticsearch-and-local-file/58895 "2016-08-25T06:58:15Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![somerandomguy](https://avatars.discourse-cdn.com/v4/letter/s/e19b73/32.png) [@somerandomguy](https://discuss.elastic.co/u/somerandomguy)\
**Post date:** [August 25, 2016, 6:58am UTC](https://discuss.elastic.co/t/logstash-output-and-multiple-destinations-elasticsearch-and-local-file/58895/1 "2016-08-25T06:58:15Z")

</div>

Hello.  
I have a requirement to send beats to multiple locations, ES (which is workning fine) and to a local file (that will be processed by another system).  
For the local file I might need to format it out of JSON format back to basic syslog style.  
Is this possible?  
Below is an extract of my output file with the relevant section.

output {  
if [type] == "beats" {  
elasticsearch {  
hosts =\> ["es-node"]  
sniffing =\> true  
manage\_template =\> false  
index =\> "beats-%{+YYYY.MM.dd}"  
document\_type =\> "beats"  
file {  
path =\> ["/var/log/beat"]  
codec =\> plain {  
charset =\> "ISO-8859-1"  
}  
}  
}  
}

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 25, 2016, 7:24am UTC](https://discuss.elastic.co/t/logstash-output-and-multiple-destinations-elasticsearch-and-local-file/58895/2 "2016-08-25T07:24:18Z")

</div>

What you currently have should work fairly well. What are you currently getting in /var/log/beat and what would you like to see instead?

---

<div class="post-metadata">

**Author:** ![somerandomguy](https://avatars.discourse-cdn.com/v4/letter/s/e19b73/32.png) [@somerandomguy](https://discuss.elastic.co/u/somerandomguy)\
**Post date:** [August 25, 2016, 11:49pm UTC](https://discuss.elastic.co/t/logstash-output-and-multiple-destinations-elasticsearch-and-local-file/58895/3 "2016-08-25T23:49:35Z")

</div>

Getting nothing at all.  
even stripped back output config to be basic:

But stil no output to a file. I am hoping to output my windows event logs to a flat file == /var/log/beats.

Anything else we can try?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 26, 2016, 7:28am UTC](https://discuss.elastic.co/t/logstash-output-and-multiple-destinations-elasticsearch-and-local-file/58895/4 "2016-08-26T07:28:23Z")

</div>

Um, wait. You're not closing the elasticsearch output before the file output is opened. This is quite clear if the configuration is properly indented:

```nohighlight
output {
  if [type] == "beats" {
    elasticsearch {
      hosts => ["es-node"]
      sniffing => true
      manage_template => false
      index => "beats-%{+YYYY.MM.dd}"
      document_type => "beats"
      file {
        path => ["/var/log/beat"]
        codec => plain {
        charset => "ISO-8859-1"
      }
    }
  }
}

```

Fix this first.

---

<div class="post-metadata">

**Author:** ![somerandomguy](https://avatars.discourse-cdn.com/v4/letter/s/e19b73/32.png) [@somerandomguy](https://discuss.elastic.co/u/somerandomguy)\
**Post date:** [August 27, 2016, 3:22am UTC](https://discuss.elastic.co/t/logstash-output-and-multiple-destinations-elasticsearch-and-local-file/58895/5 "2016-08-27T03:22:19Z")

</div>

Hi done but still no file output.  
I have noticed that this output config file it not working now either?

else if [type] == "wineventlog" {  
elasticsearch {  
hosts =\> ["els03","els04"]  
sniffing =\> true  
manage\_template =\> false  
index =\> "xyz-wineventlog-%{+YYYY.MM.dd}"  
document\_type =\> "wineventlog-log01"  
}  
file {  
path =\> "/var/log/wineventlog-log01"  
}  
}

Do I need to touch the file or something like that?

\*\* I pasted it as indented but its not showing it correctly.. I can place it in pastebin if required.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 27, 2016, 9:46am UTC](https://discuss.elastic.co/t/logstash-output-and-multiple-destinations-elasticsearch-and-local-file/58895/6 "2016-08-27T09:46:53Z")

</div>

> Hi done but still no file output.

How do you know Logstash is getting input to process?

> I pasted it as indented but its not showing it correctly..

Use the Preformatted text button on the toolbar.

---

<div class="post-metadata">

**Author:** ![jamesl](https://avatars.discourse-cdn.com/v4/letter/j/c0e974/32.png) [@jamesl](https://discuss.elastic.co/u/jamesl)\
**Post date:** [August 28, 2016, 4:49am UTC](https://discuss.elastic.co/t/logstash-output-and-multiple-destinations-elasticsearch-and-local-file/58895/7 "2016-08-28T04:49:56Z")

</div>

I have made some adjustments to my output file.  
I have an input of TCP/1514 and output that currently goes to ES. When I have LS running I can Telnet to log01 on 1514 and then write something which LS then feeds to ES which I can then search for it with no problems.  
I adjusted my output to be bare --\>

```
} else if [type] == "esxi-log01" {
  file {
    path => "/var/log/esxi.log"

```

And running foreground mode it works!! (Enabled --verbose, so I could see what's going on).  
I used this command:

/opt/logstash/bin/logstash -f /etc/logstash/conf.d/ --verbose

Opening file {:path=\>"/var/log/esxi.log", :level=\>:info}  
Starting stale files cleanup cycle {:files=\>{"/var/log/esxi.log"=\>#\<IOWriter:0x36bb512e @active=true, @io=#\<File:/var/log/esxi.log\>\>}, :level=\>:info}  
Starting stale files cleanup cycle {:files=\>{"/var/log/esxi.log"=\>#\<IOWriter:0x36bb512e @active=true, @io=#\<File:/var/log/esxi.log\>\>}, :level=\>:info}  
Starting stale files cleanup cycle {:files=\>{"/var/log/esxi.log"=\>#\<IOWriter:0x36bb512e @active=true, @io=#\<File:/var/log/esxi.log\>\>}, :level=\>:info}  
Starting stale files cleanup cycle {:files=\>{"/var/log/esxi.log"=\>#\<IOWriter:0x36bb512e @active=true, @io=#\<File:/var/log/esxi.log\>\>}, :level=\>:info}  
Starting stale files cleanup cycle {:files=\>{"/var/log/esxi.log"=\>#\<IOWriter:0x36bb512e @active=true, @io=#\<File:/var/log/esxi.log\>\>}, :level=\>:info}

I can see stuff in /var/log/esxi.log.

I then did a test of running it with 'systemctl start logstash' to see whats going on...  
nothing.  
I then chmod 777 esxi.log and then telneted into port 1514 and I can see my output.

So I think I have a permission issue but for now my workaround is to touch the file beforehand and 'fixup' the permission then start LS.. will also adjust my logrotate so when a new file is created it has the right permission.

I have put the file output with elasticsearch as per following:

```
} else if [type] == "esxi-log01" {
  file {
    path => "/var/log/esxi.log"
  }
  elasticsearch {
    hosts => ["els03","els04"]
    sniffing => true
    manage_template => false
    index => "esxi-%{+YYYY.MM.dd}"
    document_type => "esxi-log01"
  }

```

performed a config test -  
/opt/logstash/bin/logstash --configtest -f /etc/logstash/conf.d/  
came back with configuration OK.  
restart LS and now I can see my output in both the log file and in ES 🙂

Happy !

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 28, 2016, 1:12pm UTC](https://discuss.elastic.co/t/logstash-output-and-multiple-destinations-elasticsearch-and-local-file/58895/8 "2016-08-28T13:12:06Z")

</div>

> I then chmod 777 esxi.log

The file should not be world-writable and there's no point in setting the executable bit.

If you don't want to give Logstash write permissions to /var/log, why not create a logstash-owned subdirectory where you store the file? That would be less fragile then relying on the file to be pre-created.

---

<div class="post-metadata">

**Author:** ![somerandomguy](https://avatars.discourse-cdn.com/v4/letter/s/e19b73/32.png) [@somerandomguy](https://discuss.elastic.co/u/somerandomguy)\
**Post date:** [September 1, 2016, 11:52pm UTC](https://discuss.elastic.co/t/logstash-output-and-multiple-destinations-elasticsearch-and-local-file/58895/9 "2016-09-01T23:52:41Z")

</div>

Thanks Magnus. Will create a new folder, assign permissions accordingly and go from there.

Thank you again.

---

<div class="post-metadata">

**Author:** ![somerandomguy](https://avatars.discourse-cdn.com/v4/letter/s/e19b73/32.png) [@somerandomguy](https://discuss.elastic.co/u/somerandomguy)\
**Post date:** [September 3, 2016, 1:19am UTC](https://discuss.elastic.co/t/logstash-output-and-multiple-destinations-elasticsearch-and-local-file/58895/10 "2016-09-03T01:19:54Z")

</div>

Things are easy once you know what to do 🙂

Created a dir in /var/log/logstashfileoutput/  
permissions of 775  
like this:  
drwxrwxr-x. 2 logstash root

Fixed up my output file ...

file {  
path =\> "/var/log/logstashfileoutput/output.log"  
}

And I kicked logstash.. once restarted the 'output.log' file was created !!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:40am UTC](https://discuss.elastic.co/t/logstash-output-and-multiple-destinations-elasticsearch-and-local-file/58895/11 "2017-07-06T04:40:06Z")

</div>