# Logstash Parsing for tags **URL:** \ **Category:** Logstash\ **Created:** [July 26, 2017, 8:26pm UTC](https://discuss.elastic.co/t/logstash-parsing-for-tags/94703 "2017-07-26T20:26:29Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![jayden.kim](https://avatars.discourse-cdn.com/v4/letter/j/87869e/32.png) [@jayden.kim](https://discuss.elastic.co/u/jayden.kim)\ **Post date:** [July 26, 2017, 8:26pm UTC](https://discuss.elastic.co/t/logstash-parsing-for-tags/94703/1 "2017-07-26T20:26:29Z")
I am using rsyslog to send selected logs to a server with ELK. It is hard to manage the list of logs through rsyslog.d conf, thus I want to send all logs from a certain directory in /var/log. The problem I encounter is that I cannot select the Tag Name for each log if I use asterisk to select all logs from a directory. I.E. (etc/rsyslog.d/example.conf) ####HOW IT IS RIGHT NOW#### $InputFileName /var/log/example/example1.log $InputFileTag example1 $InputFileStateFile example1-status $InputRunFileMonitor $InputFileName /var/log/example/example2.log $InputFileTag example2 $InputFileStateFile example2-status $InputRunFileMonitor ####HOW I WANT IT LATER#### $InputFileName /var/log/example/\*.log $InputFileTag example $InputFileStateFile example-status $InputRunFileMonitor with the new configuration in rsyslog.d, Is there a way in logstash.conf to parse each log with a certain tag to be distinguished in Kibana? In example, if example1.log is parsed, it will have example1 tag as syslog\_program. ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [July 26, 2017, 8:51pm UTC](https://discuss.elastic.co/t/logstash-parsing-for-tags/94703/2 "2017-07-26T20:51:59Z")
There are a few ways to do this in Logstash, for example you can add tags using the grok filter or filter the inputs based in some condition and send the filtered result to different outputs. What is your Logstash configuration? How are you ingesting the log files? With logstash or using Filebeat? ---
**Author:** ![jayden.kim](https://avatars.discourse-cdn.com/v4/letter/j/87869e/32.png) [@jayden.kim](https://discuss.elastic.co/u/jayden.kim)\ **Post date:** [July 27, 2017, 2:34pm UTC](https://discuss.elastic.co/t/logstash-parsing-for-tags/94703/3 "2017-07-27T14:34:25Z")
I am using logstash to ingest the logs into elasticsearch. Below is the logstash.conf that we have. Thanks. input { tcp { port =\> 5000 type =\> syslog } udp { port =\> 5000 type =\> syslog } } filter { if [type] == "syslog" { grok { match =\> [ "message","^\<%{POSINT:syslog\_pri}\>%{SYSLOGTIMESTAMP:syslog\_timestamp} %{SYSLOGHOST:syslog\_hostname} %{SYSLOGPROG:syslog\_program}:? %{GREEDYDATA:syslog\_message}$", "message","^\<%{POSINT:syslog\_pri}\>%{SYSLOGTIMESTAMP:syslog\_timestamp} %{SYSLOGHOST:syslog\_hostname} %{DATA:syslog\_program}:? %{GREEDYDATA:syslog\_message}$", "message","^\<%{POSINT:syslog\_pri}\>%{SYSLOGTIMESTAMP:syslog\_timestamp} %{SYSLOGHOST:syslog\_hostname} %{GREEDYDATA:syslog\_message}$" ] add\_field =\> { "received\_at" =\> "%{@timestamp}" "received\_from" =\> "%{host}" } } mutate { gsub =\> ["syslog\_program", ":", ""] remove\_field =\> ["program"] } grok { match =\> [ "syslog\_hostname","^%{WORD:server\_rack}-%{WORD:server\_dc}-%{INT:server\_pos}$", "syslog\_hostname","^%{INT}-%{INT}-%{INT}-%{WORD:server\_rack}-%{WORD:server\_pos}$", "syslog\_hostname","%{GREEDYDATA}" ] } syslog\_pri { } date { match =\> ["syslog\_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"] } ``` if [syslog_program] == "apache_access" { grok { match => [ "syslog_message", "%{COMBINEDAPACHELOG} %{POSINT:resp_micros}", "syslog_message", "%{COMBINEDAPACHELOG}" ] } date { match => ["timestamp", "dd/MMM/YYYY:HH:mm:ss,SSS Z", "dd/MMM/YYYY:HH:mm:ss Z"] } geoip { source => "clientip" target => "geoip" add_field => ["[geoip][coordinates]", "%{[geoip][longitude]}" ] add_field => ["[geoip][coordinates]", "%{[geoip][latitude]}" ] } mutate { gsub => ["resp_micros", "^$", "-1"] remove_field => ["syslog_timestamp", "syslog_program", "syslog_facility", "syslog_facility_code", "syslog_message", "syslog_severity", "syslog_severity_code", "syslog_timestamp"] replace => ["type", "apacheaccess"] convert => ["[geoip][coordinates]", "float", "bytes", "integer", "resp_micros", "integer" ] } # Temporary add post work because lots of fields are inconsistant #if [beans_user] { # filter { # mutate { # uppercase => ["beans_user"] # } # } #} } ``` } output { #stdout { codec =\> rubydebug elasticsearch { hosts =\> "{{ansible\_bond0.ipv4.address}}:9200" } ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [July 27, 2017, 8:51pm UTC](https://discuss.elastic.co/t/logstash-parsing-for-tags/94703/4 "2017-07-27T20:51:39Z")
Your reply is not properly formatted, so maybe I didn't understood it correctly. But to do what you want you need to add tags, for example: ```auto if [syslog_program] == "apache_access" { grok { match => [ "syslog_message", "%{COMBINEDAPACHELOG} %{POSINT:resp_micros}", "syslog_message", "%{COMBINEDAPACHELOG}" ] add_tag => "apache-access" } } ``` The `add_tag => "apache-access"` parameter above will add a tag called apache-access to each entry that grok parses with success for the program apache\_access, is something like that that you want to do? ---
**Author:** ![jayden.kim](https://avatars.discourse-cdn.com/v4/letter/j/87869e/32.png) [@jayden.kim](https://discuss.elastic.co/u/jayden.kim)\ **Post date:** [July 28, 2017, 2:39pm UTC](https://discuss.elastic.co/t/logstash-parsing-for-tags/94703/5 "2017-07-28T14:39:14Z")
Yes, I want to parse it like it but if I use asterisk on the rsyslog conf, I should have same syslog\_program tag for all the logs under same directory. that is my only concern because I want all logs to have different tag for it. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 25, 2017, 2:39pm UTC](https://discuss.elastic.co/t/logstash-parsing-for-tags/94703/6 "2017-08-25T14:39:15Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.