# Getting error while running logstash with multiple beats data

**URL:** https://discuss.elastic.co/t/getting-error-while-running-logstash-with-multiple-beats-data/146401
**Category:** Logstash
**Created:** [August 28, 2018, 4:38pm UTC](https://discuss.elastic.co/t/getting-error-while-running-logstash-with-multiple-beats-data/146401 "2018-08-28T16:38:01Z")
**Posts on this page:** 17
**Page:** 1

<div class="post-metadata">

### Author: ![ravikt](https://avatars.discourse-cdn.com/v4/letter/r/dfb087/32.png) [@ravikt](https://discuss.elastic.co/u/ravikt)
#### Post date: [August 28, 2018, 4:38pm UTC](https://discuss.elastic.co/t/getting-error-while-running-logstash-with-multiple-beats-data/146401/1 "2018-08-28T16:38:01Z")

</div>

Currently I am sending metricbeat and winlogbeat data from same source to same logstash instance A. Now I would like to send filebeat data from another source to same logstash instance A.

When I restart logstash, am getting error  
Error- [FATAL][logstash.runner] Logstash could not b  
e started because there is already another instance using the configured data di  
rectory. If you wish to run multiple instances, you must change the "path.data"  
setting

Here is my logstash configuration:  
input {  
beats {  
port =\> 5044  
}  
}

filter {  
if [beat][name] == "metricbeat" {  
mutate {  
add\_field =\> { "indice" =\> "metricbeats"}  
}  
}  
if [beat][name] == "winlogbeat" {  
mutate {  
add\_field =\> { "indice" =\> "winbeats"}  
}  
}  
if [beat][name] == "filebeat" {  
mutate {  
add\_field =\> { "indice" =\> "filebeats"}  
}  
}  
}

output {  
elasticsearch  
{  
hosts =\> ["172.31.1.10:9200"]  
index =\> "%{indice}-%{+YYYY.MM.dd}"  
}  
}

Please advise.

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [August 28, 2018, 6:37pm UTC](https://discuss.elastic.co/t/getting-error-while-running-logstash-with-multiple-beats-data/146401/2 "2018-08-28T18:37:36Z")

</div>

The error message indicates that you're trying to run two concurrent instances of Logstash. How are you restarting Logstash? After you get the error message, is there an old Logstash instance running?

---

<div class="post-metadata">

### Author: ![ravikt](https://avatars.discourse-cdn.com/v4/letter/r/dfb087/32.png) [@ravikt](https://discuss.elastic.co/u/ravikt)
#### Post date: [August 28, 2018, 6:39pm UTC](https://discuss.elastic.co/t/getting-error-while-running-logstash-with-multiple-beats-data/146401/3 "2018-08-28T18:39:54Z")

</div>

I have one logstash instance running.

I use the below command once I make the changes to configuration file.

sudo systemctl restart logstash.service

cd /usr/share/logstash  
sudo bin/logstash -f /etc/logstash/conf.d/logstash.conf

---

<div class="post-metadata">

### Author: ![ravikt](https://avatars.discourse-cdn.com/v4/letter/r/dfb087/32.png) [@ravikt](https://discuss.elastic.co/u/ravikt)
#### Post date: [August 28, 2018, 6:40pm UTC](https://discuss.elastic.co/t/getting-error-while-running-logstash-with-multiple-beats-data/146401/4 "2018-08-28T18:40:38Z")

</div>

Also, if logstash configuration looks good?

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [August 28, 2018, 6:49pm UTC](https://discuss.elastic.co/t/getting-error-while-running-logstash-with-multiple-beats-data/146401/5 "2018-08-28T18:49:36Z")

</div>

I... don't understand. You run `systemctl restart logstash.service` but you _also_ start Logstash in your terminal with `bin/logstash -f ...`?

---

<div class="post-metadata">

### Author: ![ravikt](https://avatars.discourse-cdn.com/v4/letter/r/dfb087/32.png) [@ravikt](https://discuss.elastic.co/u/ravikt)
#### Post date: [August 28, 2018, 7:01pm UTC](https://discuss.elastic.co/t/getting-error-while-running-logstash-with-multiple-beats-data/146401/6 "2018-08-28T19:01:01Z")

</div>

I did restart but I found no changes to the output. so then I did bin/logstash -f ...

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [August 28, 2018, 7:40pm UTC](https://discuss.elastic.co/t/getting-error-while-running-logstash-with-multiple-beats-data/146401/7 "2018-08-28T19:40:33Z")

</div>

Well, if Logstash is running you can't start another instance with the same data directory. Stop the daemonized Logstash instance or configure either instance to use another data directory.

---

<div class="post-metadata">

### Author: ![ravikt](https://avatars.discourse-cdn.com/v4/letter/r/dfb087/32.png) [@ravikt](https://discuss.elastic.co/u/ravikt)
#### Post date: [August 28, 2018, 7:55pm UTC](https://discuss.elastic.co/t/getting-error-while-running-logstash-with-multiple-beats-data/146401/8 "2018-08-28T19:55:42Z")

</div>

I am not sure if I understood. Do I need to stop the logstash service and then update the logstash.conf file?

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [August 28, 2018, 9:01pm UTC](https://discuss.elastic.co/t/getting-error-while-running-logstash-with-multiple-beats-data/146401/9 "2018-08-28T21:01:13Z")

</div>

No. Just don't run two Logstash instances at the same time. If you have to do that they need to use different data directories.

---

<div class="post-metadata">

### Author: ![ravikt](https://avatars.discourse-cdn.com/v4/letter/r/dfb087/32.png) [@ravikt](https://discuss.elastic.co/u/ravikt)
#### Post date: [August 29, 2018, 10:12am UTC](https://discuss.elastic.co/t/getting-error-while-running-logstash-with-multiple-beats-data/146401/10 "2018-08-29T10:12:04Z")

</div>

I am currently running only one logstash instance and I want to run only one instance of logstash.

Initially we were sending data for metricbeat and winlogbeat to this logstash instance. Later we decided to also send data for filebeat to the same logstash instance.

Wondering if I want to update the logstash configuration file to include filebeat with separate index, can I do it without stopping the logstash service and if I can use this configuration?

input {  
beats {  
port =\> 5044  
}  
}

filter {  
if [beat][name] == "metricbeat" {  
mutate {  
add\_field =\> { "indice" =\> "metricbeats"}  
}  
}  
if [beat][name] == "winlogbeat" {  
mutate {  
add\_field =\> { "indice" =\> "winbeats"}  
}  
}  
if [beat][name] == "filebeat" {  
mutate {  
add\_field =\> { "indice" =\> "filebeats"}  
}  
}  
}

output {  
elasticsearch  
{  
hosts =\> ["172.31.1.10:9200"]  
index =\> "%{indice}-%{+YYYY.MM.dd}"  
}  
}

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [August 29, 2018, 11:26am UTC](https://discuss.elastic.co/t/getting-error-while-running-logstash-with-multiple-beats-data/146401/11 "2018-08-29T11:26:17Z")

</div>

> Wondering if I want to update the logstash configuration file to include filebeat with separate index, can I do it without stopping the logstash service

Yes, if you have automatic config reloading enabled it'll get picked up automatically. Otherwise you need to restart Logstash.

> and if I can use this configuration?

That configuration looks reasonable.

---

<div class="post-metadata">

### Author: ![ravikt](https://avatars.discourse-cdn.com/v4/letter/r/dfb087/32.png) [@ravikt](https://discuss.elastic.co/u/ravikt)
#### Post date: [September 3, 2018, 4:50pm UTC](https://discuss.elastic.co/t/getting-error-while-running-logstash-with-multiple-beats-data/146401/12 "2018-09-03T16:50:24Z")

</div>

I tried sending metricbeat and winlogbeat data from source A to logstash instance B and filebeat data from source C to logstash instance B. I cannot see the winlogbeat and filebeat data sent to elasticsearch through logstash instance B.

Here is my configuration on logstash instance B:

input {  
beats {  
port =\> 5044  
}  
}

filter {  
if [beat][name] == "metricbeat" {  
mutate {  
add\_field =\> { "indice" =\> "metricbeat"}  
}  
}  
if [beat][name] == "winlogbeat" {  
mutate {  
add\_field =\> { "indice" =\> "winlogbeat"}  
}  
}  
if [beat][name] == "filebeat" {  
mutate {  
add\_field =\> { "indice" =\> "filebeat"}  
}  
}  
}

output {  
elasticsearch  
{  
hosts =\> ["172.31.1.10:9200"]  
index =\> "%{indice}-%{+YYYY.MM.dd}"  
}  
}

what shall I be doing to fix this?

---

<div class="post-metadata">

### Author: ![ravikt](https://avatars.discourse-cdn.com/v4/letter/r/dfb087/32.png) [@ravikt](https://discuss.elastic.co/u/ravikt)
#### Post date: [September 3, 2018, 6:56pm UTC](https://discuss.elastic.co/t/getting-error-while-running-logstash-with-multiple-beats-data/146401/13 "2018-09-03T18:56:38Z")

</div>

Now I can see the output with index for metricbeat and filebeat as  
metricbeat-2018.09.03  
filebeat-2018.09.03  
the above looks fine.

But for winlogbeat, I can see output with index as  
%{indice}-2018.09.03

Any idea why I am getting the output as this?

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [September 3, 2018, 7:14pm UTC](https://discuss.elastic.co/t/getting-error-while-running-logstash-with-multiple-beats-data/146401/14 "2018-09-03T19:14:30Z")

</div>

What does the `[beat][name]` field contain in the Winlogbeat case? The `indice` field is clearly not being set.

---

<div class="post-metadata">

### Author: ![ravikt](https://avatars.discourse-cdn.com/v4/letter/r/dfb087/32.png) [@ravikt](https://discuss.elastic.co/u/ravikt)
#### Post date: [September 3, 2018, 7:20pm UTC](https://discuss.elastic.co/t/getting-error-while-running-logstash-with-multiple-beats-data/146401/15 "2018-09-03T19:20:29Z")

</div>

I am trying to filter the data sending from winlogbeat. Shall I use [type] == wineventlog?

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [September 3, 2018, 8:58pm UTC](https://discuss.elastic.co/t/getting-error-while-running-logstash-with-multiple-beats-data/146401/16 "2018-09-03T20:58:59Z")

</div>

Use whatever is unique to the Winlogbeat messages. I don't know what your events looks like.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [October 1, 2018, 9:09pm UTC](https://discuss.elastic.co/t/getting-error-while-running-logstash-with-multiple-beats-data/146401/17 "2018-10-01T21:09:48Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
