# Unable to differentiate logs

**URL:** <https://discuss.elastic.co/t/unable-to-differentiate-logs/83017>\
**Category:** Elasticsearch\
**Created:** [April 20, 2017, 8:56am UTC](https://discuss.elastic.co/t/unable-to-differentiate-logs/83017 "2017-04-20T08:56:02Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![Raja1](https://avatars.discourse-cdn.com/v4/letter/r/85f322/32.png) [@Raja1](https://discuss.elastic.co/u/Raja1)\
**Post date:** [April 20, 2017, 8:56am UTC](https://discuss.elastic.co/t/unable-to-differentiate-logs/83017/1 "2017-04-20T08:56:02Z")

</div>

Hi All,

I have configured ELK in my server and able to see the logs in Kibana but i am unable to find out the environment wise logs in Kibana.I have different environments like DEV,INT,UAT and PROD. All environments has the logs.

how can be the logs be differentiated. For example one log entry is from DEV, one log entry is from INT. How can be marked the logs that it is DEV or INT or UAT or PROD environment.

Regards  
Raja

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [April 20, 2017, 9:01am UTC](https://discuss.elastic.co/t/unable-to-differentiate-logs/83017/2 "2017-04-20T09:01:53Z")

</div>

You need to attach a field in there that does that.

---

<div class="post-metadata">

**Author:** ![Raja1](https://avatars.discourse-cdn.com/v4/letter/r/85f322/32.png) [@Raja1](https://discuss.elastic.co/u/Raja1)\
**Post date:** [April 20, 2017, 9:04am UTC](https://discuss.elastic.co/t/unable-to-differentiate-logs/83017/3 "2017-04-20T09:04:10Z")

</div>

Hi Mark,

Thank you

can you please provide me some example config to refer.

Regards  
Raja

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [April 20, 2017, 9:05am UTC](https://discuss.elastic.co/t/unable-to-differentiate-logs/83017/4 "2017-04-20T09:05:28Z")

</div>

That depends entirely on how you send data to ES.

---

<div class="post-metadata">

**Author:** ![Raja1](https://avatars.discourse-cdn.com/v4/letter/r/85f322/32.png) [@Raja1](https://discuss.elastic.co/u/Raja1)\
**Post date:** [April 20, 2017, 9:07am UTC](https://discuss.elastic.co/t/unable-to-differentiate-logs/83017/5 "2017-04-20T09:07:36Z")

</div>

I am not filtering anything as of now. I want the environment wise logs as it is from server to Kibana.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [April 20, 2017, 9:09am UTC](https://discuss.elastic.co/t/unable-to-differentiate-logs/83017/6 "2017-04-20T09:09:34Z")

</div>

How do those logs get to ES?

---

<div class="post-metadata">

**Author:** ![Raja1](https://avatars.discourse-cdn.com/v4/letter/r/85f322/32.png) [@Raja1](https://discuss.elastic.co/u/Raja1)\
**Post date:** [April 20, 2017, 9:14am UTC](https://discuss.elastic.co/t/unable-to-differentiate-logs/83017/7 "2017-04-20T09:14:30Z")

</div>

I have logstash where i am doing the configuration to ship the logs to [ES.My](http://ES.My) configuration looks like below. Here i have all environment logs under /etc/logs

input  
{  
file  
{  
path =\> "/etc/logs/\*.log"  
}  
}  
output  
{  
elasticsearch  
{  
hosts =\> "localhost:9200"  
}  
}

---

<div class="post-metadata">

**Author:** ![pablosan](https://avatars.discourse-cdn.com/v4/letter/p/e19adc/32.png) [@pablosan](https://discuss.elastic.co/u/pablosan)\
**Post date:** [April 20, 2017, 10:11am UTC](https://discuss.elastic.co/t/unable-to-differentiate-logs/83017/8 "2017-04-20T10:11:05Z")

</div>

I imagine you have a logstash in each different environment.  
Add a field with the value of the environment.  
[https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html#plugins-inputs-file-add\_field](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html#plugins-inputs-file-add_field)

```
input
{
  file
  {
    path => "/etc/logs/*.log"
    add_field => {"environment": "production"}
  }
}
output
{
  elasticsearch
  {
    hosts => "localhost:9200"
  }
}
```

---

<div class="post-metadata">

**Author:** ![A\_B](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/a_b/32/17104_2.png) [@A\_B](https://discuss.elastic.co/u/A_B)\
**Post date:** [April 20, 2017, 10:24am UTC](https://discuss.elastic.co/t/unable-to-differentiate-logs/83017/9 "2017-04-20T10:24:24Z")

</div>

Hi Raja,

it looks like you just have one server and collect logs from only one location...

Personally I would maybe use Filebeat to read the log files although you can do it with Logstash as well.

You will probably have to define more than one path of you have all logs in the same location, something like

```
input
{
  file
  {
    path => "/etc/logs/*.prod.log"
    add_field => {"environment": "production"}
  }
  file
  {
    path => "/etc/logs/*.dev.log"
    add_field => {"environment": "DEV"}
  }
  file
  {
    path => "/etc/logs/*.int.log"
    add_field => {"environment": "INT"}
  }
}

```

Adjust according to your naming convention of log files 🙂

Cheers,  
AB

---

<div class="post-metadata">

**Author:** ![Raja1](https://avatars.discourse-cdn.com/v4/letter/r/85f322/32.png) [@Raja1](https://discuss.elastic.co/u/Raja1)\
**Post date:** [April 20, 2017, 10:44am UTC](https://discuss.elastic.co/t/unable-to-differentiate-logs/83017/10 "2017-04-20T10:44:24Z")

</div>

Thanks Ab and Pablo

---

<div class="post-metadata">

**Author:** ![Raja1](https://avatars.discourse-cdn.com/v4/letter/r/85f322/32.png) [@Raja1](https://discuss.elastic.co/u/Raja1)\
**Post date:** [April 27, 2017, 4:51am UTC](https://discuss.elastic.co/t/unable-to-differentiate-logs/83017/11 "2017-04-27T04:51:24Z")

</div>

Hi Ab,

This is not working. I have done the similar way but its throwing an invalid configuration.

---

<div class="post-metadata">

**Author:** ![A\_B](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/a_b/32/17104_2.png) [@A\_B](https://discuss.elastic.co/u/A_B)\
**Post date:** [May 2, 2017, 8:59am UTC](https://discuss.elastic.co/t/unable-to-differentiate-logs/83017/12 "2017-05-02T08:59:00Z")

</div>

Hi Raja,  
sorry, I haven't used logstash this way... Looking at [https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html) looks like I left out the input type from the second and third path. I edited my previous post to fix that.

---

<div class="post-metadata">

**Author:** ![Raja1](https://avatars.discourse-cdn.com/v4/letter/r/85f322/32.png) [@Raja1](https://discuss.elastic.co/u/Raja1)\
**Post date:** [May 2, 2017, 9:27am UTC](https://discuss.elastic.co/t/unable-to-differentiate-logs/83017/13 "2017-05-02T09:27:54Z")

</div>

I fixed finally. I have created different configuration files referring same elasticsearch output so that i was able to make it possible.

Anyway Thank you

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 30, 2017, 9:38am UTC](https://discuss.elastic.co/t/unable-to-differentiate-logs/83017/14 "2017-05-30T09:38:34Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.