# Why forwarding logs by Logstash to Elasticsearch is not working

**URL:** https://discuss.elastic.co/t/why-forwarding-logs-by-logstash-to-elasticsearch-is-not-working/125058
**Category:** Logstash
**Created:** [March 21, 2018, 7:07pm UTC](https://discuss.elastic.co/t/why-forwarding-logs-by-logstash-to-elasticsearch-is-not-working/125058 "2018-03-21T19:07:41Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![apetrovYa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/apetrovya/32/13954_2.png) [@apetrovYa](https://discuss.elastic.co/u/apetrovYa)
#### Post date: [March 21, 2018, 7:07pm UTC](https://discuss.elastic.co/t/why-forwarding-logs-by-logstash-to-elasticsearch-is-not-working/125058/1 "2018-03-21T19:07:41Z")

</div>

Hi all,

I am playing with Elastic stack through Docker containers (Filebeat -\> Logstash Reciever -\> Elasticsearch \<- Kibana). Everything works pretty fine, but I have one problem.

Assuming the following logstash.conf file:

```
input {
    beats {
      port => 5044
    }
  }
 
  filter {
     if [docker][container][name] =~ /ucp/ {
        mutate {
          add_tag => ["system"]
        }
     }
     else if [docker][container][name] =~ /dtr/ {
        mutate {
          add_tag => ["system"]
        }
     } else {
        mutate {
          add_tag => ["application"]
        }
     }
  }

  output {
   if "system" in [tags] {
    elasticsearch {
      hosts => ["elasticsearch:9200"]
      index => "infrastructure-%{+YYYY.MM.dd}"
      } 
    
    } else if "application" in [tags] {
       elasticsearch {
        hosts => ["elasticsearch:9200"]
        index => "application-%{+YYYY.MM.dd}" 
      }
    } else {
      elasticsearch {
        hosts => ["elasticsearch:9200"]
        index => "trash-%{+YYYY.MM.dd}" 
      }
    }
  }

```

I can not see containers logs in the " **application-%{+YYYY.MM.dd}**" Elasticsearch index.  
The "infrastructure-%{+YYYY.MM.dd}" index recieves data normally !

Can anyone help me to understand what I am missing?

Thanks in advance!

---

<div class="post-metadata">

### Author: ![yaauie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yaauie/32/23363_2.png) [@yaauie](https://discuss.elastic.co/u/yaauie)
#### Post date: [March 22, 2018, 7:14am UTC](https://discuss.elastic.co/t/why-forwarding-logs-by-logstash-to-elasticsearch-is-not-working/125058/2 "2018-03-22T07:14:23Z")

</div>

The configuration looks fine to me.

Do you have evidence that logs are being sent that match neither of the patterns given, and therefore should fall into the `else` bucket to be tagged as application?

---

<div class="post-metadata">

### Author: ![apetrovYa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/apetrovya/32/13954_2.png) [@apetrovYa](https://discuss.elastic.co/u/apetrovYa)
#### Post date: [March 22, 2018, 10:50am UTC](https://discuss.elastic.co/t/why-forwarding-logs-by-logstash-to-elasticsearch-is-not-working/125058/3 "2018-03-22T10:50:46Z")

</div>

So, using the [docker][container][name] as a string and =~ operator for pattern matching against /dtr/ or /ucp/ I am sure that logs to infrastructure index are sent. I can see them all with the **codec =\> rubydebug** on stdout of logstash container. But I continue to not see application logs. It seems to me like a passing high filter 🙂 only infrastructure logs are passing. Weird!

---

<div class="post-metadata">

### Author: ![yaauie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yaauie/32/23363_2.png) [@yaauie](https://discuss.elastic.co/u/yaauie)
#### Post date: [March 22, 2018, 8:06pm UTC](https://discuss.elastic.co/t/why-forwarding-logs-by-logstash-to-elasticsearch-is-not-working/125058/4 "2018-03-22T20:06:49Z")

</div>

in the `codec => rubydebug` output, can you tell what the value is for `[docker][container][name]` for the events that don't make it through?

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [April 19, 2018, 8:06pm UTC](https://discuss.elastic.co/t/why-forwarding-logs-by-logstash-to-elasticsearch-is-not-working/125058/5 "2018-04-19T20:06:50Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
