# The unknown output information in logstash

**URL:** <https://discuss.elastic.co/t/the-unknown-output-information-in-logstash/109334>\
**Category:** Logstash\
**Created:** [November 28, 2017, 8:22am UTC](https://discuss.elastic.co/t/the-unknown-output-information-in-logstash/109334 "2017-11-28T08:22:05Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![antony.y](https://avatars.discourse-cdn.com/v4/letter/a/91b2a8/32.png) [@antony.y](https://discuss.elastic.co/u/antony.y)\
**Post date:** [November 28, 2017, 8:22am UTC](https://discuss.elastic.co/t/the-unknown-output-information-in-logstash/109334/1 "2017-11-28T08:22:05Z")

</div>

I have used some output plugin in my logstash and conduct the different actions through the condition i've defined.

```
if "cisco" in [tags] {
    elasticsearch {
        hosts => ["ph71v-esn01.ae007.com:9200", "ph71v-esn02.ae007.com:9200"]
        index => "network-log-%{+YYYY.MM.dd}"
                    document_type => "Network Device"
                document_id => "%{[@metadata][fingerprint]}"
    }
}

if "_grokparsefailure" in [tags] {
    file {
        path => "/var/log/logstash/fail-%{type}-%{+YYYY.MM.dd}.log"
    }
}
else {
    file {
        path => "/var/log/logstash/unknown_msg.log"
    }
}

```

i have some logs have been tagged by "cisco",but i can still find it in the unknow\_msg.log,i do not know why, is there any logic mistake i've made ?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [November 28, 2017, 8:44am UTC](https://discuss.elastic.co/t/the-unknown-output-information-in-logstash/109334/2 "2017-11-28T08:44:58Z")

</div>

With your configuration all messages that don't have a `_grokparsefailure` tag will get routed to the unknown\_msg.log file. In addition, if they have a `cisco` tag they will get routed to elasticsearch.

---

<div class="post-metadata">

**Author:** ![antony.y](https://avatars.discourse-cdn.com/v4/letter/a/91b2a8/32.png) [@antony.y](https://discuss.elastic.co/u/antony.y)\
**Post date:** [November 28, 2017, 10:03am UTC](https://discuss.elastic.co/t/the-unknown-output-information-in-logstash/109334/3 "2017-11-28T10:03:57Z")

</div>

the strange thing is that there should not be any unknown information ,you can see the input configuration  
input {  
tcp {  
port =\> "8514"  
type =\> "syslog-cisco"  
add\_field =\> {"Location" =\> "RCBC"}  
}  
udp {  
port =\> "514"  
type =\> "syslog-cisco"  
add\_field =\> {"Location" =\> "RCBC"}  
}  
}

filter {  
if [type] == "syslog-cisco" {

```
# The switches are sending the same message to all syslog servers for redundancy, this allows us to
## only store the message in elasticsearch once by generating a hash of the message and using that as
## the document_id.
fingerprint {
  source => ["message"]
  method => "SHA1"
  key => "0123"
  concatenate_sources => true
  target => "[@metadata][fingerprint]"
}

grok {

    # There are a couple of custom patterns associated with this filter.
	patterns_dir => ["/opt/logstash/patterns"]
	
	match => [
    "message", "<%{NUMBER:message_type_id}>%{NUMBER:internal_id}:%{SPACE}%{SYSLOGHOST:origin_hostname}:%{SPACE}%{CISCOTIMESTAMPTZ:cisco_timestamp}: \%%{WORD:facility}-%{INT:severity}-%{WORD:mnemonic}: %{GREEDYDATA:msg}",
	"message", "<%{NUMBER:message_type_id}>%{NUMBER:internal_id}:%{SPACE}%{SYSLOGHOST:origin_hostname}:%{SPACE}%{CISCOTIMESTAMP:cisco_timestamp}.*: \%%{WORD:facility}-%{INT:severity}-%{WORD:mnemonic}: %{GREEDYDATA:msg}",
	"message", "<%{NUMBER:message_type_id}>%{NUMBER:internal_id}:%{SPACE}%{SYSLOGHOST:origin_hostname}:%{SPACE}.*%{CISCOTIMESTAMP:cisco_timestamp}.*: \%%{WORD:facility}-%{INT:severity}-%{WORD:mnemonic}: %{GREEDYDATA:msg}",
    "message", "<%{NUMBER:message_type_id}>%{NUMBER:internal_id}:%{SPACE}%{SYSLOGHOST:origin_hostname}:.*\]:%{SPACE}%{CISCOTIMESTAMPTZ:cisco_timestamp}.*\%%{WORD:facility}-%{INT:severity}-%{WORD:mnemonic}: %{GREEDYDATA:msg}",
	"message", "<%{NUMBER:message_type_id}>%{NUMBER:internal_id}:%{SPACE}%{SYSLOGHOST:origin_hostname}:.*\]: %{NUMBER:sequencenumber}:%{SPACE}%{CISCOTIMESTAMPTZ:cisco_timestamp}.*\%%{WORD:facility}-%{INT:severity}-%{WORD:mnemonic}: %{GREEDYDATA:msg}",
	"message", "%{CISCOTIMESTAMPTZ:cisco_timestamp}%{SPACE}%{SYSLOGHOST:origin_hostname}%{SPACE}\%%{WORD:facility}-%{INT:severity}-%{WORD:mnemonic}: %{GREEDYDATA:msg}"
    ]
	add_tag => ["cisco"]
	remove_field => ["cisco_timestamp"]
	remove_field => ["message_type_id"]
	remove_field => ["internal_id"]
	remove_field => ["sequencenumber"]
}

if "cisco" in [tags] {

translate {
  field => "facility"
  destination => "facility_full"
```

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [November 28, 2017, 10:27am UTC](https://discuss.elastic.co/t/the-unknown-output-information-in-logstash/109334/4 "2017-11-28T10:27:07Z")

</div>

You're missing my point. With your configuration **all** messages that don't have a \_grokparsefailure tag will get routed to the unknown\_msg.log file **even if they have a `cisco` tag**. **In addition** , if they have a cisco tag they will get routed to elasticsearch.

I suspect the configuration you're looking for is this:

````nohighlight
if "cisco" in [tags] {
    elasticsearch {
        hosts => ["ph71v-esn01.ae007.com:9200", "ph71v-esn02.ae007.com:9200"]
        index => "network-log-%{+YYYY.MM.dd}"
                    document_type => "Network Device"
                document_id => "%{[@metadata][fingerprint]}"
    }
}
else if "_grokparsefailure" in [tags] {
    file {
        path => "/var/log/logstash/fail-%{type}-%{+YYYY.MM.dd}.log"
    }
}
else {
    file {
        path => "/var/log/logstash/unknown_msg.log"
    }
}```
````

---

<div class="post-metadata">

**Author:** ![antony.y](https://avatars.discourse-cdn.com/v4/letter/a/91b2a8/32.png) [@antony.y](https://discuss.elastic.co/u/antony.y)\
**Post date:** [November 29, 2017, 1:01am UTC](https://discuss.elastic.co/t/the-unknown-output-information-in-logstash/109334/5 "2017-11-29T01:01:50Z")

</div>

so it is! thank you so much ,it does work !

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 27, 2017, 1:02am UTC](https://discuss.elastic.co/t/the-unknown-output-information-in-logstash/109334/6 "2017-12-27T01:02:03Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
