# Someone please clarify me on my configuration **URL:** \ **Category:** Logstash\ **Created:** [April 27, 2017, 3:59am UTC](https://discuss.elastic.co/t/someone-please-clarify-me-on-my-configuration/83789 "2017-04-27T03:59:32Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Raja1](https://avatars.discourse-cdn.com/v4/letter/r/85f322/32.png) [@Raja1](https://discuss.elastic.co/u/Raja1)\ **Post date:** [April 27, 2017, 3:59am UTC](https://discuss.elastic.co/t/someone-please-clarify-me-on-my-configuration/83789/1 "2017-04-27T03:59:32Z")
Hi All, I am getting bit confusion on type,field. Could you please someone confirm me by checking below configuration. My intention is to create the index by environment wise and remove some fields from the output. Please have a look and clarify me if any mistakes. input { file { path =\> "/etc/logs/POClogs/_.log" add\_field =\> {"environment": "POC"} start\_position =\> "beginning" } { path =\> "//etc/logstash/Devlogs/_.log" add\_field =\> {"environment": "DEV"} start\_position =\> "beginning" } } filter { grok { match =\> {"message" =\> "[%WARNING,%INFO]%{GREEDYDATA}[%WARNING]"} add\_field =\> { "ApplicationName" =\> "Devapp" } add\_field =\> { "ApplicationName" =\> "POCapp" } remove\_tag =\> ["\_grokparsefailure"] remove\_field =\> ["clientHostname"] remove\_field =\> ["\_type"] remove\_field =\> ["\_score"] } } output { if [type] == "Devlog" { elasticsearch { hosts =\> ["x.x.x.x:9200"] index =\> "Devlog-%{+YYYY.MM.dd}" } } else if [type] == "POClog" { elasticsearch { hosts =\> ["x.x.x.x:9200"] index =\> "POClog-%{+YYYY.MM.dd}" } } else { elasticsearch { hosts =\> ["x.x.x.x:9200"] } stdout { codec =\> rubydebug } } } Regards Raja ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [April 27, 2017, 5:22am UTC](https://discuss.elastic.co/t/someone-please-clarify-me-on-my-configuration/83789/2 "2017-04-27T05:22:50Z")
I don't see you ever setting the `type` field to anything so your conditions won't work as expected. > add\_field =\> {"environment": "POC"} Wrong syntax; use `=>` and not `:`. > grok > { > match =\> {"message" =\> "[%WARNING,%INFO]%{GREEDYDATA}[%WARNING]"} > add\_field =\> { "ApplicationName" =\> "Devapp" } > add\_field =\> { "ApplicationName" =\> "POCapp" } > remove\_tag =\> ["\_grokparsefailure"] > remove\_field =\> ["clientHostname"] > remove\_field =\> ["\_type"] > remove\_field =\> ["\_score"] > } What's the point of this filter? You're not capturing any fields in the grok expression and you're deleting a number of fields that'll never exist at that point in the pipeline. ---
**Author:** ![Raja1](https://avatars.discourse-cdn.com/v4/letter/r/85f322/32.png) [@Raja1](https://discuss.elastic.co/u/Raja1)\ **Post date:** [April 27, 2017, 5:31am UTC](https://discuss.elastic.co/t/someone-please-clarify-me-on-my-configuration/83789/3 "2017-04-27T05:31:41Z")
Thank you magnus. I did the changes in my configuration and now it is working as expected. But i have different folders having logs so how would i configure and also how can i make the index for each. Ex: /etc/dev/_.log /etc/uat/_.log /etc/int/\*.log I want to set the index for each folder location so that i can able to find environment wise logs. Write now my configuration for only one file path like below. Please guide me to complete this. input { file { path =\> ["/etc/Dev/\*.log"] codec =\> multiline { pattern =\> "[%WARNING,%INFO]%{GREEDYDATA}[%WARNING]" negate =\> "true" what =\> "previous" } sincedb\_path =\> "/dev/null" start\_position =\> beginning ignore\_older =\> 0 } } filter { grok { match =\> {"message" =\> "[%WARNING,%INFO]%{GREEDYDATA}[%WARNING]"} } } output { elasticsearch { hosts =\> ["x.x.x.x:9200"] action =\> "index" } stdout {codec =\> rubydebug } } ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [April 27, 2017, 5:38am UTC](https://discuss.elastic.co/t/someone-please-clarify-me-on-my-configuration/83789/4 "2017-04-27T05:38:57Z")
> I want to set the index for each folder location so that i can able to find environment wise logs. One index per environment is most likely a bad idea. If you're doing this only to be able to distinguish different kinds of logs just use another field for that. Like the `environment` field in the configuration you posted previously. > Write now my configuration for only one file path like below. Why not follow the pattern from the previous post? Apart from what I wrote it looked fine. > grok > { > match =\> {"message" =\> "[%WARNING,%INFO]%{GREEDYDATA}[%WARNING]"} > } Again, what's the purpose of this filter? ---
**Author:** ![Raja1](https://avatars.discourse-cdn.com/v4/letter/r/85f322/32.png) [@Raja1](https://discuss.elastic.co/u/Raja1)\ **Post date:** [April 27, 2017, 5:47am UTC](https://discuss.elastic.co/t/someone-please-clarify-me-on-my-configuration/83789/5 "2017-04-27T05:47:43Z")
Sorry it should be removed from my conf. Actually my logs are starting with some status message like WARNING,INFO,FATAL i.e., i used multiline pattern. How could i run mutiple conf files in single shot. Will that recommended to create multiple configuration files? ---
**Author:** ![Raja1](https://avatars.discourse-cdn.com/v4/letter/r/85f322/32.png) [@Raja1](https://discuss.elastic.co/u/Raja1)\ **Post date:** [April 27, 2017, 7:08am UTC](https://discuss.elastic.co/t/someone-please-clarify-me-on-my-configuration/83789/6 "2017-04-27T07:08:42Z")
I am getting below error after adding the add\_field. Please help me to resolve. Cannot load an invalid configuration {:reason=\>"Expected one of #, =\> at line 7, column 36 (byte 124) after input\n{\nfile\n{\n path =\> ["/etc/logstash/INTlogs/\*.log"]\n type =\> "INTlog"\n add\_field =\> {"environment"", :backtrace=\>["/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:47:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:139:in`initialize'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:277:in `create_pipeline'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:95:in`register\_pipeline'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:264:in `execute'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:67:in`run'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:183:in `run'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:132:in`run'", "/usr/share/logstash/lib/bootstrap/environment.rb:71:in `(root)'"]} ---
**Author:** ![Raja1](https://avatars.discourse-cdn.com/v4/letter/r/85f322/32.png) [@Raja1](https://discuss.elastic.co/u/Raja1)\ **Post date:** [April 27, 2017, 7:12am UTC](https://discuss.elastic.co/t/someone-please-clarify-me-on-my-configuration/83789/7 "2017-04-27T07:12:05Z")
resolved this issue...small mistake in configuration. Now i need to know executing multiple configuration files. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 25, 2017, 7:17am UTC](https://discuss.elastic.co/t/someone-please-clarify-me-on-my-configuration/83789/8 "2017-05-25T07:17:56Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.