# Error logs for nginx and apache not working

**URL:** <https://discuss.elastic.co/t/error-logs-for-nginx-and-apache-not-working/105494>\
**Category:** Logstash\
**Created:** [October 27, 2017, 1:12am UTC](https://discuss.elastic.co/t/error-logs-for-nginx-and-apache-not-working/105494 "2017-10-27T01:12:31Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![joperr](https://avatars.discourse-cdn.com/v4/letter/j/bc79bd/32.png) [@joperr](https://discuss.elastic.co/u/joperr)\
**Post date:** [October 27, 2017, 1:12am UTC](https://discuss.elastic.co/t/error-logs-for-nginx-and-apache-not-working/105494/1 "2017-10-27T01:12:31Z")

</div>

Hi there!

I'm adding some conf files to my logstash based on [https://www.elastic.co/guide/en/logstash/current/logstash-config-for-filebeat-modules.html](https://www.elastic.co/guide/en/logstash/current/logstash-config-for-filebeat-modules.html)  
So I have two config files working great:

file1-

```
input {
  beats {
    port => 5044
    ssl => false
  }
}

filter {
  if "access_log" in [tags] {
    grok {
          match => { "message" => "%{COMBINEDAPACHELOG}" }

    }
    date {
          match => ["timestamp" , "dd/MMM/yyyy:HH:mm:ss Z"]
    }
    geoip {
          source => "clientip"
    }
    useragent {
          source => "agent"
    }
  }
}

output {
  elasticsearch {
        hosts => ["localhost:9200"]
        sniffing => true
        manage_template => true
        index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
        document_type => "%{[@metadata][type]}"
  }
}

```

file 2-

```
filter {
  if "nginx" in [tags] and "error_log" in [tags] {
    grok {
      match => { "message" => ["%{DATA:[nginx][error][time]} \[%{DATA:[nginx][error][level]}\] %{NUMBER:[nginx][error][pid]}#%{NUMBER:[nginx][error][tid]}: (\*%{NUMBER:[nginx][error][connection_id]} )?%{GREEDYDATA:[nginx][error][message]}"] }
      remove_field => "message"
    }
    mutate {
      rename => { "@timestamp" => "read_timestamp" }
    }
    date {
      match => ["[nginx][error][time]", "YYYY/MM/dd H:m:s" ]
      remove_field => "[nginx][error][time]"
    }
  }
}

```

And when I want to add another file to match apache errors, with this file:

```
filter {
  if "apache" in [tags] and "error_log" in [tags] {
   grok {
      match => { "message" => ["\[%{APACHE_TIME:[apache2][error][timestamp]}\] \[%{LOGLEVEL:[apache2][error][level]}\]( \[client %{IPORHOST:[apache2][error][client]}\])? %{GREEDYDATA:[apache2][error][message]}",
        "\[%{APACHE_TIME:[apache2][error][timestamp]}\] \[%{DATA:[apache2][error][module]}:%{LOGLEVEL:[apache2][error][level]}\] \[pid %{NUMBER:[apache2][error][pid]}(:tid %{NUMBER:[apache2][error][tid]})?\]( \[client %{IPORHOST:[apache2][error][client]}\])? %{GREEDYDATA:[apache2][error][message1]}" ] }
      pattern_definitions => {
        "APACHE_TIME" => "%{DAY} %{MONTH} %{MONTHDAY} %{TIME} %{YEAR}"
      }
      remove_field => "message"
   }
   mutate {
      rename => { "[apache2][error][message1]" => "[apache2][error][message]" }
   }
   date {
      match => ["[apache2][error][timestamp]", "EEE MMM dd H:m:s YYYY", "EEE MMM dd H:m:s.SSSSSS YYYY" ]
      remove_field => "[apache2][error][timestamp]"
   }
  }
}

```

Every apache error log is getting ignored and not logged into Kibana.

What am I doing wrong?

Thanks in advance

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [October 27, 2017, 3:36am UTC](https://discuss.elastic.co/t/error-logs-for-nginx-and-apache-not-working/105494/2 "2017-10-27T03:36:48Z")

</div>

Standard advice: Use a `stdout { codec => rubydebug }` output until you're sure that events are being processed as expected and that they look good. Only then do you add more complexity by enabling an elasticsearch output.

I'd also read the Logstash log and look for clues.

---

<div class="post-metadata">

**Author:** ![joperr](https://avatars.discourse-cdn.com/v4/letter/j/bc79bd/32.png) [@joperr](https://discuss.elastic.co/u/joperr)\
**Post date:** [October 27, 2017, 2:45pm UTC](https://discuss.elastic.co/t/error-logs-for-nginx-and-apache-not-working/105494/3 "2017-10-27T14:45:14Z")

</div>

Hi Magnus, I followed your advice, but nothing wrong came out on Logstash log.

But using stdout (I used "file output") I see this logged

```
{"@timestamp":"2017-10-27T17:00:17.352Z","offset":1298823,"apache2":{"error":{"pid":"58839","message":"[client 201.148.293.147:60848] PHP Stack trace:, referer: https://www.google.com.ar/","level":"error"}},"@version":"1","input_type":"log","beat":{"name":"troya","hostname":"troya","version":"5.6.2"},"host":"troya","source":"/var/log/apache2/error.log","type":"log","fields":{"serverName":"troya","datacenter":"azure","appName":"apache","serverType":"virtual"},"tags":["apache","error_log","beats_input_codec_plain_applied"]}

```

and when I change back to elasticsearch output that line it's being ignored.

If you see the line tags, "error\_log" is present, so why is it being parsed?

What am I doing wrong?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [October 27, 2017, 5:23pm UTC](https://discuss.elastic.co/t/error-logs-for-nginx-and-apache-not-working/105494/4 "2017-10-27T17:23:22Z")

</div>

How are you verifying that nothing gets into ES? Have you checked _all_ indexes?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 24, 2017, 5:23pm UTC](https://discuss.elastic.co/t/error-logs-for-nginx-and-apache-not-working/105494/5 "2017-11-24T17:23:32Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.